A server-friendly downloader implemented with Ruby
GentleDownloader has throttling mechanism so that it will cause neither overload problems nor performance issues on remote machines.
You should install the following libraries as prerequisite to GentleDownloader.
Start GentleDownloader.
# ./downloader.rb
GentleDownloader will start as a daemon.
Push a download item to GentleDownloader.
# ./addItem.rb [URL] [output] (--nowait) (--uid uniqueID)
-
URL
: A URL that you want to download -
output
: Specify file or directory that GentleDownloader stores the file to -
(optional) --nowait
: Specify the option if you do not want GentleDownloader to wait -
(optional) --uid uniqueID
: GentleDownloader restrains intensive accesses with the host of URL in default. If you specify uniqueID, GentleDownloader will treat all URL that has same uniqueID as a same host.
GentleDownloader does not access a same host within 15 seconds in default. You can change the value by modifying download.rb.
INTERVAL = 15
If GentleDownloader cannot download any files, remove box.kct and add.kct and restart GentleDownloader. This step will reset the download queue.