Automatic selection of parallel upload transfers
|Reported by:||David||Owned by:|
|Component version:||Operating system type:||Linux|
|Operating system version:|
I had been testing different scenarios of uploading a bunch of files with different network connections (wlan and eth) and have a good idea for improving upload speed.
- Lots of small files: If transferred in parallel, the whole proccess is completed in less time than transferring them one by one.
- Lots of large files: Transferring them one by one is *faster* than transferring two or more at the same time (the sum of all individual speeds did not equal than the individual upload speed of only one upload).
So with this data I suggest to have a configuration that would enable "automatic" parallel handling that would work as follows:
1.-Threshold to consider a file small (called A)
2.-Maximum number of threads to enable if procceeds (called B, can be taken from global config)
Suggested behaviour WITH this new option enabled:
1.-Sort all upload list by size (asc or desc)
3.-If current filesize is less or equal than threshold A
- Upload them in PARALLEL using B connections
- IGNORE parallel options and upload files ONE BY ONE
I tested this scenario locally connected directly by ethernet cable, and by wireless lan, and in both cases, speed improvement was really really significant.
So what about having it in config so this can be done automatically?