[Kget] kget BIG next changes

[Dario Massarin]

Hi all!

Ok. KGet now seems in a good shape.. But I also think it's time to pass to stage 2 and make it ready to really become the best downloading application on earth ;-)

There are a number of things the current code doesn't allow, and I thought about it for quite a long time.. The request to implement Metalink made me start thinking about the further steps we should do in order to improve things even more.

Ok. Let's start from the beginning.
As you know Metalink potentially allows one user to download a single file from multiple sources and, at the same time, multiple protocols. This means that it gives us a list of ftp, http, torrent links, etc. and we should theoretically be able to do a multisegmented download with segments composed of those kind of transfers. The really good work that Manolo had done was all centered on the creation of a Transfer which handles by his own the segments. But when it comes to handling torrent segments, its plugin can do nothing other than ignoring those torrent links. At this point his plugin could start handling by himself even torrent links, but taking this step would mean that the architecture doesn't fit anymore our needs. If we start building all on top of a transfer is because the functionalities it tries to achieve aren't available in the main kget structure. This means that we *need* to take a further step ahead.

Well.. what I want to do is:

1) make it possible for a Transfer to implement or not some interfaces we provide, for example
a) Container (still looking for a better name..)
b) Segment
c) BandwidthControl (in the future..)
d) ...

A container transfer is a transfer that can own other transfers and that never really downloads data, just uses other (non-container) transfers to download it. This is a key point. A multisegment download would be composed of segments, which are transfers themselves.

A segment transfer, instead, allows the user to specify the specific segment of data it should download (offset and size bytes). Each transfer should allow just the download of one segment.

A bandwidthControl transfer, instead, allows the control of the bandwidth, but this (maybe) will be done in the future..

2) Allow each transfer to have more than one source (url), whereas now we allow a transfer to have just one. 

3) Build on top of point #2 a search framework where each search engine would be provided as a plugin. In this way we could convert Manolo search idea in a plugin that could be used with all the available transfers plugins (if the particular protocols fits). The final result of a Search plugin will be of adding more sources to a transfer.

4) The gui would automatically display for each plugin the transfers, where available, that belong to a certain container transfer. This means that, for example, in a Metalink download we could automatically see all the "segments" we are downloading, despite they are http, ftp, torrents, etc.

With this in place, in order to potentially fully support Metalink (which is kind of hard, as you can see ;-) ), it will be able enough just to build one container Transfer that will transparently query kget for the existence of plugins that handle the list of urls Metalink gives. If no torrent plugin is there, we will not use it, but it would be sufficient to add a bittorrent plugin (or a eDonkey one) to use it, without any more troubles and changes in kget core code.

I know. It's frustrating to change things over and over again, but I prefer taking these decisions now than having to change everything tomorrow..

Manolo, your plugin is good and all these changes shouldn't make it necessary for you to do any change on it. My aim is to introduce all this stuff allowing us to keep the transfers we have today. With these changes we will gradually migrate to the new architecture, that means that part of your code, more specifically the part that creates the Kio Job, will become another plugin, whereas part of your segment-related transfer will be transfeered to the famous new container transfer.

What do you think?

(I don't know how much time I'll have to work on this, but maybe it's not really so hard as it seems..)

Bye Bye!!
    Dario




------------------------------------------------------
Passa a Infostrada. ADSL e Telefono senza limiti e senza canone Telecom
http://click.libero.it/infostrada07gen07