[Kget] KGet2 multithreaded plugin

[anthony l. bryan]

> Very cool.
> 
> There are several ways to implement this in kget:
> 1) The easy way: we don't consider the torrent links at all, 
> but use only the 
> http and ftp ones and grab them inside the multithreaded 
> plugin. This would 
> be really really simple to implement, but not as powerful as 
> it could be..
> 
> Anthony: do the other download managers allow to download the 
> same file with 
> torrents along with ftp and http?

That is the plan, but currently no other download managers support ftp/http
& BitTorrent. As you have said, it makes the whole thing that much more
complicated. Some are close to there, but not quite yet.

The only "requirement" for Metalink is multi-threaded ftp/http downloads.
P2P & checksum checking are optional. The other clients drop the parts they
do not support, like BitTorrent, ed2k, or magnet. I realize not all programs
may have built in MD5/SHA-1 checking, so to add that would add complexity.
So it they don't support it, its just dropped. If they do, it gets used. But
that information is nice to have when a magical client that supports
everything comes along.
 
> 2) The difficult way: in order to use at the same time and in 
> the same file 
> different tecnologies like bittorrent, http, ftp or whatever, 
> we have to:
> 	a) allow a transfer to have child transfers. In this 
> case, for example, the 
> metalink transfer could have more than one child transfer (a 
> multithread 
> transfer to handle all the http and ftp links, plus a 
> bittorrent transfer to 
> handle the bittorrent link)
> 	b) abstract the concept of file source, each one 
> providing informations like 
> speed, for example. This makes it possible to centrally 
> handle the whole 
> downloading process, switching to faster sources and so on.
> 	c) this is complex: we should extend the transfer 
> interface in order to let 
> the transfer share their concept of "chunk". This would make 
> it possible for 
> a transfer to cooperate with another without downloading the 
> same data.

Yes, it is much simpler to focus on ftp & http links. Those are in use
EVERYWHERE. P2P support would be great, but it complicates things quite a
bit. & BT isn't ubiquitous yet.
 
> I think that the first implementation could be done in a 
> little time. It's 
> just a matter of handling the link directly from the 
> multithread plugin.

I hope so. It took my friend a few hrs to write his patch enabling Metalink
(that I attached before), most of the time was getting his head into the
existing code. I have captured a video showing Metalink in use, but it is
around 100 megs if you would like to see in it action (let me know a place
to ftp it to).

I would suggest Solution 1 with minimal effort & investment. I think you
will like what that alone produces, and if you do then go on from there. If
not, then not much has been lost.

thanks!

ant