<br><br><div class="gmail_quote">On Wed, Jun 27, 2012 at 1:22 AM, Sebastian Trüg <span dir="ltr"><<a href="mailto:trueg@kde.org" target="_blank">trueg@kde.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On 06/26/2012 10:42 AM, Vishesh Handa wrote:<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
<br>
<br>
On Tue, Jun 26, 2012 at 1:17 PM, Sebastian Trüg <<a href="mailto:trueg@kde.org" target="_blank">trueg@kde.org</a><br></div><div class="im">
<mailto:<a href="mailto:trueg@kde.org" target="_blank">trueg@kde.org</a>>> wrote:<br>
<br>
On 06/25/2012 10:50 AM, Vishesh Handa wrote:<br>
<br>
Hey Sebastian<br>
<br>
The reviewboard has gone insane and it refuses to let me upload<br>
my patch<br></div>
- "The file 'nepomuk/kioslaves/search/kio_<u></u>__nepomuksearch.cpp'<div class="im"><br>
(r2311b7a)<br>
could not be found in the repository".<br>
<br>
The auto updates donot seem to work any more, so that really<br>
needs to be<br>
<br>
><br>
<br>
fixed or thrown away. I would prefer throwing it away, but it<br>
does have<br>
a valid use-case, though the implementation is horrible.<br>
<br>
<br>
Why is the implementation horrible? I always thought it to be very<br>
clean. The concept is simple: the kio slave lists a folder which<br>
results in a KDirNotify signal which is used by the kded module to<br>
list the query in the query service. Since before that would<br>
<br>
have reused the same query folder there would have been no<br>
re-querying. The kded would simply subscribe to updates from the<br>
query service.<br>
<br>
<br>
I was referring to how the query service does the updates - It runs the<br>
queries every time any data has changed. The current implementation<br>
makes Nepomuk completely unusable. Mainly cause I have the<br>
telepathy-nepomuk-service running which updates the nepomuk repo any<br>
time the status of any of my contacts changes.<br>
<br>
Right now, the query will be run both in the kio-slave, and in the<br>
queryservice. Maybe we could abstract away the queryservice code, and<br>
make it run in the kioslave itself. That way the kded will listen to the<br>
kioslave for updates, and the queries will not have to be run twice.<br>
</div></blockquote>
<br>
The kio slave does not run forever. So you cannot do updates that way.<br>
But remember that we had the idea to use the resource watcher to do some heuristics.<br></blockquote><div><br>Hmm. So now what? Should I commit this patch?<br><br>I remember the plan of using some heuristics, I just don't think that would help a lot. For simple queries like listing all your music files - On my desktop that is over 20,000 files. Doing that query and processing 20000 files again, just cause one music file was added seems to be an overkill.<br>
<br>I rather not have this feature of automatic updates, if it means such a heavy performance penalty.<br> <br></div><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
<br>
commit 1fba20495fddfc596591f6ea8535d5<u></u>__fc6427c62b<div class="im"><br>
Author: Vishesh Handa<<a href="mailto:me@vhanda.in" target="_blank">me@vhanda.in</a> <mailto:<a href="mailto:me@vhanda.in" target="_blank">me@vhanda.in</a>><br></div>
<mailto:<a href="mailto:me@vhanda.in" target="_blank">me@vhanda.in</a> <mailto:<a href="mailto:me@vhanda.in" target="_blank">me@vhanda.in</a>>>><div class="im"><br>
<br>
Date: Sat Jun 16 02:57:57 2012 +0530<br>
<br>
Avoid using the query service to run queries in the kioslave<br>
<br>
Avoid creating a new thread in the query service, and<br>
parsing the<br>
results over dbus. Instead just run the query manually in<br>
the kioslave.<br>
<br>
We can do this cause kioslaves run in a separate process<br>
and are allowed<br>
to be blocking.<br>
<br></div>
diff --git a/nepomuk/kioslaves/common/__<u></u>resourcestat.cpp<br>
b/nepomuk/kioslaves/common/__<u></u>resourcestat.cpp<br>
index a259ba5..8f3088e 100644<br>
--- a/nepomuk/kioslaves/common/__<u></u>resourcestat.cpp<br>
+++ b/nepomuk/kioslaves/common/__<u></u>resourcestat.cpp<div class="im"><br>
@@ -349,6 +349,7 @@ KUrl Nepomuk2::redirectionUrl( const<br>
Nepomuk2::Resource& res )<br>
<br>
<br>
namespace {<br>
+<br>
/**<br>
* Check if the resource represents a local file with an<br>
existing nie:url property.<br>
*/<br></div>
diff --git a/nepomuk/kioslaves/search/__<u></u>kio_nepomuksearch.cpp<br>
b/nepomuk/kioslaves/search/__<u></u>kio_nepomuksearch.cpp<br>
index 2311b7a..3c82796 100644<br>
--- a/nepomuk/kioslaves/search/__<u></u>kio_nepomuksearch.cpp<br>
+++ b/nepomuk/kioslaves/search/__<u></u>kio_nepomuksearch.cpp<br>
@@ -182,7 +182,7 @@ void Nepomuk2::SearchProtocol::__<u></u>listDir(<div class="im"><br>
const KUrl& url )<br>
else {<br>
SearchFolder folder( url, this );<br>
updateQueryUrlHistory( url );<br>
- folder.waitForListing();<br>
+ folder.list();<br>
listEntry( KIO::UDSEntry(), true );<br>
finished();<br>
}<br></div>
@@ -315,7 +315,7 @@ void Nepomuk2::SearchProtocol::__<u></u>listRoot()<div class="im"><br>
if ( query.isValid() ) {<br>
// FIXME: Avoid this useless conversion to searchUrl<br>
and back<br>
SearchFolder folder( query.toSearchUrl(), this );<br>
- folder.waitForListing();<br>
+ folder.list();<br>
}<br>
<br>
listEntry( KIO::UDSEntry(), true );<br></div>
diff --git a/nepomuk/kioslaves/search/__<u></u>searchfolder.cpp<br>
b/nepomuk/kioslaves/search/__<u></u>searchfolder.cpp<br>
index 3cd098a..38e4b1d 100644<br>
--- a/nepomuk/kioslaves/search/__<u></u>searchfolder.cpp<br>
+++ b/nepomuk/kioslaves/search/__<u></u>searchfolder.cpp<div class="im"><br>
@@ -32,15 +32,19 @@<br>
#include<Nepomuk2/Thing><br>
#include<Nepomuk2/Types/Class><br>
#include<Nepomuk2/Query/Query><br></div>
-#include<Nepomuk2/Query/__<u></u>QueryParser><br>
+#include<Nepomuk2/Query/__<u></u>Result><br>
#include<Nepomuk2/Query/__<u></u>ResourceTypeTerm><br>
-#include<Nepomuk2/Query/__<u></u>QueryServiceClient><br>
#include<Nepomuk2/Vocabulary/_<u></u>_NFO><br>
#include<Nepomuk2/Vocabulary/_<u></u>_NIE><br>
#include<Nepomuk2/Vocabulary/_<u></u>_PIMO><br>
<br>
+#include<Nepomuk2/__<u></u>ResourceManager><div class="im"><br>
+#include<Nepomuk2/Resource><br>
+<br>
#include<QtCore/QMutexLocker><br>
#include<QTextDocument><br></div>
+#include<Soprano/__<u></u>QueryResultIterator><div class="im"><br>
+#include<Soprano/Model><br>
<br>
#include<KUrl><br>
#include<KDebug><br>
@@ -51,6 +55,8 @@<br>
#include<KConfig><br>
#include<KConfigGroup><br>
<br>
+using namespace Nepomuk2::Vocabulary;<br>
+using namespace Soprano::Vocabulary;<br>
<br></div>
Nepomuk2::SearchFolder::__<u></u>SearchFolder( const KUrl& url,<div class="im"><br>
KIO::SlaveBase* slave )<br>
: QObject( 0 ),<br></div>
@@ -60,23 +66,9 @@ Nepomuk2::SearchFolder::__<u></u>SearchFolder( const<div class="im"><br>
KUrl& url, KIO::SlaveBase* slave )<br>
// parse URL (this may fail in which case we fall back to<br>
pure SPARQL below)<br>
Query::parseQueryUrl( url, m_query, m_sparqlQuery );<br>
<br></div>
- m_client = new Nepomuk2::Query::__<u></u>QueryServiceClient();<div class="im"><br>
-<br>
- connect( m_client, SIGNAL( newEntries( const<br></div>
QList<Nepomuk2::Query::Result><u></u>__& ) ),<div class="im"><br>
- this, SLOT( slotNewEntries( const<br></div>
QList<Nepomuk2::Query::Result><u></u>__& ) ) );<div class="im"><br>
- connect( m_client, SIGNAL( resultCount(int) ),<br>
- this, SLOT( slotResultCount(int) ) );<br>
- connect( m_client, SIGNAL( finishedListing() ),<br>
- this, SLOT( slotFinishedListing() ) );<br>
- connect( m_client, SIGNAL( error(QString) ),<br>
- this, SLOT( slotFinishedListing() ) );<br>
- connect( m_client, SIGNAL( finishedListing() ),<br>
- m_client, SLOT( deleteLater() ) );<br>
-<br>
- if ( m_query.isValid() )<br>
- m_client->query( m_query );<br>
- else<br>
- m_client->sparqlQuery( m_sparqlQuery );<br>
+ if ( m_query.isValid() ) {<br>
+ m_sparqlQuery = m_query.toSparqlQuery();<br>
+ }<br>
}<br>
<br>
<br></div>
@@ -84,36 +76,20 @@ Nepomuk2::SearchFolder::~__<u></u>SearchFolder()<br>
{<br>
}<br>
<br>
-void Nepomuk2::SearchFolder::__<u></u>waitForListing()<div class="im"><br>
-{<br>
- m_eventLoop.exec();<br>
-}<br>
-<br></div>
-void Nepomuk2::SearchFolder::__<u></u>slotNewEntries( const<br>
QList<Nepomuk2::Query::Result><u></u>__& results )<div class="im"><br>
+void Nepomuk2::SearchFolder::list()<br>
{<br>
- KIO::UDSEntryList entryList;<br>
- foreach(const Query::Result& result, results ) {<br>
+ //FIXME: Do the result count as well?<br>
+ Soprano::Model* model =<br></div>
ResourceManager::instance()->_<u></u>_mainModel();<div class="im"><br>
+ Soprano::QueryResultIterator it = model->executeQuery(<br></div>
m_sparqlQuery, Soprano::Query::__<u></u>QueryLanguageSparql );<div class="im"><br>
+ while( it.next() ) {<br>
+ Query::Result result = extractResult( it );<br>
KIO::UDSEntry uds = statResult( result );<br>
if ( uds.count() ) {<br>
- //kDebug()<< "listing" <<<br></div>
result.resource().resourceUri(<u></u>__);<br>
m_slave->listEntry(uds, false);<br>
}<br>
}<br>
}<br>
<br>
-<br>
-void Nepomuk2::SearchFolder::__<u></u>slotResultCount( int count )<div class="im"><br>
-{<br>
- m_slave->totalSize( count );<br>
-}<br>
-<br>
-<br></div>
-void Nepomuk2::SearchFolder::__<u></u>slotFinishedListing()<div class="im"><br>
-{<br>
- m_eventLoop.exit();<br>
-}<br>
-<br>
-<br>
namespace {<br>
bool statFile( const KUrl& url, const KUrl& fileUrl,<br>
KIO::UDSEntry& uds )<br>
{<br>
@@ -146,7 +122,7 @@ KIO::UDSEntry<br></div>
Nepomuk2::SearchFolder::__<u></u>statResult( const Query::Result& result )<div class="im"><br>
{<br>
Resource res( result.resource() );<br>
const KUrl uri( res.resourceUri() );<br>
- KUrl nieUrl(<br></div>
result[Nepomuk2::Vocabulary::_<u></u>_NIE::url()].uri() );<div class="im"><br>
+ KUrl nieUrl( result[NIE::url()].uri() );<br>
<br>
// the additional bindings that we only have on unix systems<br>
// Either all are bound or none of them.<br>
@@ -263,3 +239,36 @@ KIO::UDSEntry<br></div>
Nepomuk2::SearchFolder::__<u></u>statResult( const Query::Result& result )<div class="im"><br>
<br>
return uds;<br>
}<br>
+<br>
+// copied from the QueryService<br>
+Nepomuk2::Query::Result<br></div>
Nepomuk2::SearchFolder::__<u></u>extractResult(const<div class="im"><br>
Soprano::QueryResultIterator& it) const<br>
+{<br>
+ Query::Result result( Resource::fromResourceUri(<br>
it[0].uri() ) );<br>
+ const Query::RequestPropertyMap map =<br>
m_query.requestPropertyMap();<br></div>
+ for( Query::RequestPropertyMap::__<u></u>const_iterator rit =<div><div class="h5"><br>
map.begin(); rit != map.constEnd(); rit++ ) {<br>
+ result.addRequestProperty( rit.value(), it.binding(<br>
rit.key() ) );<br>
+ }<br>
+<br>
+ // make sure we do not store values twice<br>
+ QStringList names = it.bindingNames();<br>
+ names.removeAll( QLatin1String("r" ) );<br>
+<br>
+ static const char* s_scoreVarName ="_n_f_t_m_s_";<br>
+ static const char* s_excerptVarName ="_n_f_t_m_ex_";<br>
+<br>
+ Soprano::BindingSet set;<br>
+ int score = 0;<br>
+ Q_FOREACH( const QString& var, names ) {<br>
+ if ( var == QLatin1String( s_scoreVarName ) )<br>
+ score = it[var].literal().toInt();<br>
+ else if ( var == QLatin1String( s_excerptVarName ) )<br>
+ result.setExcerpt( it[var].toString() );<br>
+ else<br>
+ set.insert( var, it[var] );<br>
+ }<br>
+<br>
+ result.setAdditionalBindings( set );<br>
+ result.setScore( ( double )score );<br>
+<br>
+ return result;<br>
+}<br>
\ No newline at end of file<br></div></div>
diff --git a/nepomuk/kioslaves/search/__<u></u>searchfolder.h<br>
b/nepomuk/kioslaves/search/__<u></u>searchfolder.h<br>
index 794baaa..b475402 100644<br>
--- a/nepomuk/kioslaves/search/__<u></u>searchfolder.h<br>
+++ b/nepomuk/kioslaves/search/__<u></u>searchfolder.h<div class="im"><br>
@@ -36,6 +36,9 @@<br>
#include<Nepomuk2/Resource><br>
#include<KUrl><br>
<br>
+namespace Soprano {<br>
+ class QueryResultIterator;<br>
+}<br>
<br>
namespace Nepomuk2 {<br>
namespace Query {<br>
@@ -76,17 +79,7 @@ namespace Nepomuk2 {<br>
/**<br>
* List the results directly on the parent slave.<br>
*/<br>
- void waitForListing();<br>
-<br>
- private Q_SLOTS:<br>
- /// connected to the QueryServiceClient in the search<br>
thread<br>
- void slotNewEntries( const<br></div>
QList<Nepomuk2::Query::Result><u></u>__& );<div class="im"><br>
-<br>
- /// connected to the QueryServiceClient in the search<br>
thread<br>
- void slotResultCount( int );<br>
-<br>
- /// connected to the QueryServiceClient in the search<br>
thread<br>
- void slotFinishedListing();<br>
+ void list();<br>
<br>
private:<br>
/**<br>
@@ -94,6 +87,8 @@ namespace Nepomuk2 {<br>
*/<br>
KIO::UDSEntry statResult( const Query::Result& result );<br>
<br>
+ Query::Result extractResult( const<br>
Soprano::QueryResultIterator& it ) const;<br>
+<br>
// folder properties<br>
KUrl m_url;<br>
<br>
<br>
<br>
<br>
--<br>
Vishesh Handa<br>
<br>
<br>
<br>
<br>
--<br>
Vishesh Handa<br>
<br>
</div></blockquote>
</blockquote></div><br><br clear="all"><br>-- <br><span style="color:rgb(192,192,192)">Vishesh Handa</span><br><br>