[Nepomuk] Re: Presentation, queries and other stuff

[Ignacio Serantes]

On Mon, Mar 7, 2011 at 12:33 PM, Ignacio Serantes <kde at aynoa.net> wrote:

>
>> As a first hacky query optimization you might try to use
>>
>> ?r a ?y .
>> ?y nao:userVisible ?visiblity .
>> filter(regex(?visibility),'true')) .
>>
>> looks weird but should do wonders because almost all resources are
>> visible.
>>
>
> I'm not following you I'm in my job so I can't test this but are you told
> me that your query is faster than this one:
>
> ?r a ?y .
> ?y nao:userVisible "true"^^<http://www.w3.org/2001/XMLSchema#boolean>
>
> Looks really weird :).
>

Umm, probably I misunderstand you :(. I do this group of tests:

Test #01 [TestUsingFilter]:  23292 results in   5.98952699 seconds
Test #02 [TestUsingFilter]:  23292 results in   6.63734818 seconds
Test #03 [TestUsingFilter]:  23292 results in   5.96942902 seconds
Test #04 [TestUsingFilter]:  23292 results in   6.54467010 seconds
Test #05 [TestUsingFilter]:  23292 results in   6.11729908 seconds
Test #06 [ TestUsingEqual]:  23292 results in   4.62860608 seconds
Test #07 [ TestUsingEqual]:  23292 results in   4.75821495 seconds
Test #08 [ TestUsingEqual]:  23292 results in   5.15577602 seconds
Test #09 [ TestUsingEqual]:  23292 results in   5.10496998 seconds
Test #10 [ TestUsingEqual]:  23292 results in   5.12364006 seconds

and Filter() is slow as expected, 2 seconds between the quickest using equal
and the slowest using Filter(). Here are the test cases I used:

<SPARQL>
Enabled = No
Name = TestUsingFilter
DisplayResultSets = No
<QUERY>
SELECT DISTINCT *
WHERE {
    ?r a ?y .
    ?y nao:userVisible ?z .
        FILTER(REGEX(?z, "true")) .
}

<SPARQL>
Enabled = No
Name = TestUsingEqual
DisplayResultSets = No
<QUERY>
SELECT DISTINCT *
WHERE {
    ?r a ?y .
    ?y nao:userVisible "true"^^<http://www.w3.org/2001/XMLSchema#boolean> .
}

Could you explain better because I'm misunderstand you.


> The optimization I was talking about is inference on client level. (As I
>> mentioned I am not doing much on the DB level yet.) It creates a
>> nao:userVisible value for each resource. Thus, one can test the
>> visibility directly on the resource. That combined with the filter above
>> is a big performance improvement. Obviously it is hacky and a view as
>> you mention seems a much better solution.
>>
>
> The view system could be a better solution but only several test
> will demonstrate it.
>
>>
>> Actually there is no dedicated index for nao:userVisible yet. The DB is
>> totally generic. We are only beginning with the optimization. Your input
>> is very very helpful.
>
>
> And you don't need to create it because boolean indexes are a lost of time
> and even could degrade db performance. You must check this in Virtuoso
> documentation.
>

Sorry, I think that I explain bad this. With b-tree indexes columns with low
cardinality could not be beneficed by an index but index must be maintained
by dbms but, with compound indexes or with bitmapped indexes, like Oracle
ones, you obviously could obtain better performance with and index in many
cases. So, with "check this" I am meaning that is necessary consult
Virtuoso behavior with this kind of indexes.

Sorry when I sounds rude but when I write in English sometimes I have not
much time to write and less to recheck :).


-- 
Cheers,
Ignacio
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.kde.org/pipermail/nepomuk/attachments/20110307/7db72d7f/attachment.htm