[Nepomuk] Review Request 111513: Use parameterized ODBC queries for adding statements

[Vishesh Handa]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://git.reviewboard.kde.org/r/111513/#review38150
-----------------------------------------------------------

Ship it!


Preliminary tests show that this actually does help. Pushing a large amount of text now no longer results in a big memory jump for virtuoso. Though virtuoso still seems to be some minor increases over time, but this is a MAJOR improvement.

Unfortunately, most of nepomuk no longer uses addStatement calls and prepares the insert calls on their own. Oh well, I can at least change that for the plain text.

Thanks for doing this Sebastian. Please commit it to the 2.9 branch, that way I can make a new bug fix release and push distros to adopt it.

- Vishesh Handa


On July 15, 2013, 8:08 a.m., Sebastian Trueg wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://git.reviewboard.kde.org/r/111513/
> -----------------------------------------------------------
> 
> (Updated July 15, 2013, 8:08 a.m.)
> 
> 
> Review request for Nepomuk and Vishesh Handa.
> 
> 
> Description
> -------
> 
> Soprano uses only plain query strings for all operations via ODBC (Virtuoso). However, parameterized queries can provide a much better performance, especially for long string values. This patch (available in branch feature/odbcParameterizedQueries) makes Soprano use parameterized queries for addStatement at least.
> 
> This should fix the memory leaking problem with adding large plain text blobs given that addStatement is used (AFAIK the Nepomuk indexer currently uses custom queries to add the data).
> 
> Should this patch proove to be effective it should be expanded to use parameterized queries wherever possible like removeStatement. Also using prepared ODBC queries could improve the performance considerably, especially when adding a large amount of statements.
> 
> 
> Diffs
> -----
> 
>   backends/virtuoso/odbcconnection.h c87bcb7 
>   backends/virtuoso/odbcconnection.cpp 754c76d 
>   backends/virtuoso/virtuosomodel.cpp a884703 
>   backends/virtuoso/virtuosomodel_p.h d5a86d8 
> 
> Diff: http://git.reviewboard.kde.org/r/111513/diff/
> 
> 
> Testing
> -------
> 
> Soprano tests run through smoothly. More performance and mem leakage testing needs to be done using Nepomuk.
> 
> 
> Thanks,
> 
> Sebastian Trueg
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/nepomuk/attachments/20130819/487f5be6/attachment-0001.html>