Review Request: New KIO::http_post and KIO::StoredHttpPost APIs that accept a QIODevice as input...

[Dawit Alemayehu]

> On Feb. 9, 2011, 9:59 a.m., David Faure wrote:
> > kioslave/http/http.cpp, line 3809
> > <http://git.reviewboard.kde.org/r/100516/diff/3/?file=8819#file8819line3809>
> >
> >     Hmm, yeah, doesn't this make this all useless, if we end up making a QByteArray with the whole data in it anyway? I thought the whole point was for kio_http to read in chunks and send along the way, rather than read it all upfront.

Well it is not completely useless since it fixes one portion of this issue. There are a couple of ways to solve the problem in kio_http as well. It is just that both of the solutions I could think of have their own particular downsides.

The first solution is to simply save the POST data into a temporary file and re-post from there if necessary or delete the file when the request is done. However, besides being inefficient (saving to file), this approach also has a potential to leave behind sensitive data in a temporary file if the kio_http happens to crash or exit abnormally. I doubt we want to take this risk, specially if the information we saved to a temporary file is for example bank account or credit card information.

The second potential solution does not rely on saving information to disk. Instead we add a new message the ioslave can send to the job to reset and start over (MSG_DATA_REQ_RESET). The ioslave then would simply call dataReq() again to obtain the same data from scratch. The problem with this solution is that it not only requires changes in slavebase.*, slaveinterface.* to send the new message and emit the necessary signal, but also modifications to how the current KIO::http_post API works. The slotDataReq function can no longer delete the data to be POST'd once it sends it to the ioslave. Instead it would have to keep it until it receives the result of the POST request. The most glaring down side however is the fact that with the new http_post API. If the QIODevice passed to it is sequential, i.e. a network socket, then it simply cannot be reset to restart to process all over again.

I personally think the second solution is the way to go since it absolves the ioslave from having to do anything other than calling something like SlaveBase::dataReqReset() to start over. The downside of course is whether or not we care about sequential devices. If not, then this is our solution.


- Dawit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://git.reviewboard.kde.org/r/100516/#review1324
-----------------------------------------------------------


On Feb. 9, 2011, 1:28 a.m., Dawit Alemayehu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://git.reviewboard.kde.org/r/100516/
> -----------------------------------------------------------
> 
> (Updated Feb. 9, 2011, 1:28 a.m.)
> 
> 
> Review request for kdelibs.
> 
> 
> Summary
> -------
> 
> The attached patch is the first portion a set of patches to make uploading data through HTTP more efficient without affecting the existing implementation. Right now the amount of memory consumed when uploading large files through http or webdav is really not acceptable because only a QByteArray based API is available. That means if you want to upload a file of say 50 or 100 MB to a server, then you have to read the entire thing first before you can call KIO::http_post!
> 
> 
> This addresses bug 34578.
>     http://bugs.kde.org/show_bug.cgi?id=34578
> 
> 
> Diffs
> -----
> 
>   kio/kio/job_p.h daac895 
>   kio/kio/jobclasses.h e9bd191 
>   kioslave/http/http.h be08d7b 
>   kioslave/http/http.cpp 187e3c7 
>   kio/kio/accessmanager.cpp 722203f 
>   kio/kio/job.h 632dfc8 
>   kio/kio/job.cpp 7d4a849 
> 
> Diff: http://git.reviewboard.kde.org/r/100516/diff
> 
> 
> Testing
> -------
> 
> Used by changing kdewebkit to use the new API. 
> 
> 
> Thanks,
> 
> Dawit
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-core-devel/attachments/20110209/172a5cec/attachment.htm>