Hi Darren<br><br><div class="gmail_quote">On Mon, Jan 3, 2011 at 12:24 AM, Darren Cruse <span dir="ltr"><<a href="mailto:darren.cruse@gmail.com">darren.cruse@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hi guys sorry for some basic questions but I've been considering use<br>
of Nepomuk for a project gathering meta data from web pages...<br>
<br>
FWIW An initial cut at the project was done using java and xquery that<br>
spidered the web pages and downloaded them prior to creating RDF/XML<br>
that drove a web based UI for searching. And I'll spare the details,<br>
but some parts of that worked well, and some not so well, and I was<br>
trying to understand if something like Nepomuk would bring more "off<br>
the shelf" help for doing what we'd done, yet be open enough for us to<br>
enhance the generated meta data where needed.<br>
<br>
My questions:<br>
<br>
1. Is the Mandriva Linux distro the one most likely to get me the<br>
latest/greatest goodies for using Nepomuk KDE?<br>
<br>
The initial effort happens to be on Ubuntu, but reading the archives,<br>
just installing kubuntu-desktop won't get me all that Mandriva has -<br>
correct?<br>
<br></blockquote><div><br></div><div>Nepomuk just like any other project, has stable code and experimental stuff. The experimental stuff is lying in the playground. All distros package the stable stuff, but Mandriva additionally packages some of the experimental stuff.</div>
<div><br></div><div>This is because 'Sebastian Trueg' ( the main Nepomuk developer ) is employed by Mandriva.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
(hope it's not a dumb question btw - old time Solaris guy here still a<br>
little green with Linux).<br>
<br>
2. I assume html files are indexed? But is anything more than basic<br>
meta data (file size, etc.) gotten?<br></blockquote><div><br></div><div>I just checked. The entire file's content is indexed as plain-text along with the basic metadata.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
In particular, the project requires that triples are created that<br>
refer to the other resources linked to by an html page. i.e. The uris<br>
of images the page may use, other web pages it links to, flash files<br>
it might embed, etc. have to wind up as triples in the meta data.<br></blockquote><div><br></div><div>This sounds interesting. How would the triples be stored? RDF+XML? Or would the webpages contain a link to the turtle file? </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
3. To add to the fun, the project also wants entities that are more<br>
conceptual. e.g. if the html pages represent a book broken down into<br>
volumes and sections and chapters etc. the meta data must include the<br>
names of the volume, section, chapter, etc. that the html page refers<br>
to. i.e. This is more in the realm of "entity extraction"/"NLP" kind<br>
of stuff.<br>
<br>
Are there examples of something like that around? Where an app would<br>
customize the meta data being extracted?<br>
<br>
Does this mean I'm using "Scribo" for it's NLP extraction features?<br>
Or that I'm customizing how the "Strigi" indexing works?<br>
<br></blockquote><div><br></div><div>Hmm. This is difficult to answer. We currently use Strigi just to index files. Since webpages are HTML files, this does fall into that category a little bit.</div><div><br></div><div>
But since you're doing something more than what Strigi does, I think it falls more into the line of Scribo and NLP. A lot of work has been done in Scribo/NLP in Nov/Dec. All of it is in the playground.</div><div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Are such things a part of the current Mandriva distro or are these<br>
only in the playground?<br></blockquote><div><br></div><div>They are in the playground, but Madriva shops some of them. If you want the latest stuff, it's best to manually compile the playground.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
4. Barring anything real specific for #3, do I understand that<br>
Virtuoso is now the default/preferred triple store?<br>
<br></blockquote><div><br></div><div>Yes, that is correct, but what is the specific stuff in #3. We store ALL our triples in Virtuoso.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
And that I should be able to write software that adds/updates triples<br>
in Virtuoso directly if I choose to?<br></blockquote><div><br></div><div>The Nepomuk architecture is, well, quite structured. For storing Triples ( actually quadruples ) we use Soprano [1] We only use the Soprano API to add/update triples.</div>
<div><br></div><div>Soprano, however, has a plugin based backend-framework. One of its jobs is to provide a nice consistent API over any RDF store. The current backends are - Virtuoso, Redland, and Sesame2. </div><div><br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
Or that hits a SPARQL endpoint, e.g. to display the info using a custom web app?<br></blockquote><div><br></div><div>AFAIK we currently do NOT provide a SPARQL endpoint. Sebastian would be better at answering this question. He wrote most (all?) of Soprano.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
(part of this question also relates to the project using java btw so<br>
solutions that avoid the C++ api are a better fit for my work-mates -<br>
though I wouldn't mind :).<br>
<br></blockquote><div><br></div><div>The Nepomuk-Kde project doesn't use Java at all. It's all C++. But since we are a part of KDE, you can use Nepomuk with any of the language bindings provided by kde [2] </div>
<div><br></div><div>I don't think the original Nepomuk project in Java is maintained at all. Again, Sebastian!</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
5. Not a show stopper but just curious: Is Sesame still supported as<br>
an alternative backend store?<br></blockquote><div><br></div><div>Soprano supports it, but Nepomuk doesn't use it. We only use Virtuoso. Quite a bit of the Nepomuk APIS ( specifically the Query API ) have been tailored to use virtuoso specific stuff. Additionally, there is some code in Nepomuk that detects if any other backend is being used, and converts it to Virtuoso.</div>
<div><br></div><div>We used to use Redland as the preferred backend till Dec 2009.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
Apologize for all the newbie questions.<br></blockquote><div><br></div><div>No need to apologize. Keep the questions coming.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
But so far Nepomuk looks like the bees knees btw. :)<br></blockquote><div><br></div><div>Heh. I actually had to look up the expression 'bees knees'. Thanks :) </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
Darren<br>
_______________________________________________<br>
Nepomuk mailing list<br>
<a href="mailto:Nepomuk@kde.org">Nepomuk@kde.org</a><br>
<a href="https://mail.kde.org/mailman/listinfo/nepomuk" target="_blank">https://mail.kde.org/mailman/listinfo/nepomuk</a><br>
</blockquote></div><br><br clear="all">[1] <meta http-equiv="content-type" content="text/html; charset=utf-8"><a href="http://sourceforge.net/projects/soprano/">http://sourceforge.net/projects/soprano/</a><div><a href="http://sourceforge.net/projects/soprano/"></a>[2] <meta http-equiv="content-type" content="text/html; charset=utf-8"><a href="http://techbase.kde.org/Development/Languages">http://techbase.kde.org/Development/Languages</a><br>
-- <br><font color="#999999">Vishesh Handa</font><br>
</div>