Slow response, due to virtuoso-t, "kdeinit4: plasma-desktop"? HowTo fix? ; jor KDE

[Duncan]

giovanni_re posted on Sat, 26 Jun 2010 00:34:08 -0700 as excerpted:

> About every 10 minutes, lasting about 1.5 minutes, virtuoso-t is running
> (maybe also "kdeinit4: plasma-desktop"), doing about 100 reads per
> second, & slowing things down right then, & I think also in the minutes
> following.
> 
> I think this might be the source of my system running so slow at times.

Very possibly, particularly if you're running < 1 gig memory, which would 
force the system to decide between swapping apps out to cache data, or 
keeping apps and having less swap.

I see below that you're running kubuntu (and note -- after writing the 
above, that you're running 4 gig RAM.  That should be enough not to be 
doing the swap thing, unless you have something strange going on...  I'm 
actually running 6 gig RAM here, amd64, but four would be plenty).  That 
means Linux.  There's a kernel swappiness parameter that you can tweak 
that could help the system better decide between swap and cache, but let's 
deal with your questions, first.

> I think it might be causing other applications (ex, firefox, or a switch
> on the desktop to a different window, when I click to some other
> window), to have to do many disk reads to read in cached information,
> which has been lost from RAM when virtuoso-t just ran.
> 
> 
> ==  What is v-t doing?
> In the service of what other sw?  Where can this be learned about?  I
> didn't find a man page for v-t.
> 
> Where can settings for v-t, or the other sw, be viewed, & perhaps
> modified?  In the "System Settings" program someplace?

In general, kde apps (tho virtuoso is kde related but not direct kde) 
don't have traditional manpages, unfortunately.  I've often wished they 
did.  Most of them do offer the usual short description and command line 
parameter info if run from konsole or the like, with the --help parameter, 
and many of them have HTML based GUI help (kde handbook, see khelpcenter), 
but that doesn't replace a decent manpage!

Also, there's user oriented help at userbase on the kde website, and 
similarly, much more technical mostly dev oriented help, at techbase.  
There's also the sysadmin section in the kde handbook, available both 
online and in khelpcenter, but as with too much of the rest of the 
khelpcenter documentation, some of it hasn't been updated /well/ since the 
kde3 era (and not late in the kde3 era at that!).  Never-the-less, I've 
found the sysadmin section, particularly the info on default directories 
and environmental variables, quite helpful indeed while adminning my own 
systems.  A lot of that info transferred over directly from kde3, and much 
of what didn't is pretty close and/or has had at least some updates done 
to it.

In the case of virtuoso, most of the info you'll find will be on the user- 
and techbases, since it's quite new.  But I'll post some basics.

Virtuoso is database software, the backend for strigi/nepomuk's semantic 
desktop stuff.  Only with kde 4.4 did it get good enough to be the default 
strigi/nepomuk backend.  Before that, the default was the Java based 
Sesame2, with a waayyy slooowww fallback to something else, I forgot what).

What going on is that nepomuk/strigi is setup to index the files on your 
disk, thus the reads you're seeing, to enable better semantic searches, 
etc.  However, in practice, kde4.4 doesn't yet have the UI to make 
particularly good use of those indexes, so a lot of folks simply turn off 
that indexing, at least for now.

Perhaps with 4.5, but more likely it'll be 4.6 or 4.7 (so next year), the 
front-ends will actually give you usable access to all this indexed data 
giving you all sorts of new ways to access your stuff.  For instance, you 
could do a search listing all the images you have of a particular 
resolution, or if you're a photographer with a number of digital cameras, 
with a particular camera, or taken on specific dates, or using tagging, of 
a particular location, or perhaps all those you've edited with a 
particular photo editor.  Combining those could give you some really 
interesting flexibility in image search.  Text documents, email, instant-
messaging-logs, all sorts of stuff can be indexed, and correlated so that 
you can pull up say the instant message log discussing the photo someone 
sent you in email, along with the email, and the photo, all together.

But, as I said, the front-ends really aren't there to do much with the 
indexes at this point.

And really, truth be told, a lot of technically minded folks probably 
won't find such things /that/ interesting anyway, at least compared to the 
cost of the indexing.  Consider how long MS Office has had its indexer, 
for instance, and that all that time, one of the biggest speed-up hints 
for it and one of the first things many tech inclined people do is disable 
it.  Why?  Because tech oriented people intuitively understand the 
computer directory structure and generally have a good idea where they put 
something (or where a package is likely to install it) in the first place, 
so can either go directly to it, or find it with a minimal real-time grep/
file-search.  The cost of running that indexer doesn't give them enough 
benefit to be worth it.   OTOH, the folks who benefit the most from such 
tools are the non-tech oriented folks who have thousands of files dumped 
on their desktop, the folks who download stuff, and then can't find where 
they saved it, even tho they did the save just 30 seconds ago.  These 
folks simply have no clue about these black- boxes (umm, beige boxes) 
called computers, or how they work, nor do they really care, they just 
want to do whatever it is they're doing and not worry about it, and they 
need all the help they can get.  It's this type of folks that really find 
such file indexing and search tools worthwhile, even if it does slow down 
their system, because slow and being able to find that file they just 
downloaded 30 seconds ago, beats fast and can't find it to use, any day.

So depending on how much you need fancy searches, etc, perhaps with kde 
4.6 or so, you may want indexing on.  But meanwhile, chances are you 
aren't going to use it that much in kde 4.4, because the tools to really 
make use of it simply aren't there or aren't mature enough to do so, yet, 
so you might as well turn it off, for now.

Or alternatively, confine it to searching only part of your disk.  Perhaps 
you only care about your homedir, or the documents dir in your homedir, 
and can tell it to turn off indexing for the rest of the system, thereby 
dramatically reducing the time it takes doing that indexing.

As you suspected, system settings (which really should be called kde or 
user settings, as few of them apply to the system as a whole, only to the 
specific user while using kde, kde3's name, kcontrol, was *FAR* more 
accurate in that regard, and *FAR* more effectively googlable as well!), 
Advanced User Settings, Desktop Search.  You control what it indexes on 
the File Indexing tab, or simply turn off indexing entirely, on the Basic 
Settings tab.  (This applies to kde 4.4.  I don't track what versions 
various distributions ship with their various releases, as I run Gentoo, 
with its rolling releases, and update kde from the gentoo/kde overlay 
generally the day, certainly within the week, a new kde release comes out.)


> Perhaps one thing that might improve system response times is to run v-t
> less often - maybe once per hour, or per day.  Is this possible? Useful?
> A good idea?  What would need to be done/modified to achieve that?
> 
> Is this KDE specific sw?  Does GNOME also use this?

Nepomuk/strigi/virtuoso are pretty new, and generally kde-only at this 
point.  However, they're designed to be reasonably generic, usable by 
other software as it's developed to do so.  Also note that gnome, Apple, 
and MS, have similar technologies at various stages of advancement.  But 
these happen to be what KDE's using and being new, pretty specific to kde, 
ATM.

> ==  What is "kdeinit4: plasma-desktop" doing? In the service of what
> other sw?
> 
> Where can settings for "k4 p-d", or the other sw, be viewed, & perhaps
> modified?  In the "System Settings" program someplace?

Plasma-desktop is the desktop "shell", including the wallpaper and widgets 
on the desktop, any panels (one by default) you have configured, the 
kickoff menu (by default, the lancelot menu widget can replace kickoff, 
and there's also the classic kmenu), etc.

It can be noted that while krunner (the kde run dialog) shares many of the 
same base libraries, it's deliberately a separate app, so if either krunner 
or plasma-desktop crash, you can still work (and restart the crashed bit) 
using the other one.  Similarly, khotkeys operates independent of plasma-
desktop, so hot-key launching still works when plasma-desktop is 
crashed.   Thus, it's quite possible to operate kde without plasma-
desktop, but as it's the shell that most folks will know as the "face" of 
kde, it's not something most folks would find pleasant at all.

The kdeinit4 bit is somewhat of a technical trick.  By starting all the kde 
base apps (plasma-desktop, krunner, kwin, ksycoca4 (which stands for Kde 
SYstem COnfig CAche 4) from kdeinit4, it allows them to share memory 
resources and startup faster than they would otherwise.  If an app crashes 
or you shut it down manually, like I do with plasmadesktop occasionally 
(issuing the usual SIGTERM with killall, to let it exit gracefully), you 
can restart it, and it won't show the kdeinit4: bit in front, but it'll be 
duplicating certain memory resources that were shared with the other kde 
apps, when it was launched with kdeinit4.

So obviously, since plasma-desktop /is/ the desktop, changing the desktop 
settings, desktop background, number of activities, adding and deleting 
panels and widgets, all the stuff that you do to configure the desktop and 
panels, is the config for plasma-desktop. =:^)

> ==  System tray "Search Service" - does what? Note: In the system tray,
> when this disk IO had been going on (starting at 1146 PM) for several
> minutes, I left clicked on the "Search Service" icon, selected "suspend
> file indexing", but the IO kept going for several minutes more, & from
> iotop it was v-t & k4 p-d running.

Disabling file indexing as above, should take care of that.

FWIW, the plasma-desktop activity you noted was due to updating the systray 
status info, because the systray is one of the widgets running in plasma-
desktop.  If there's nothing to update...  But you'll almost certainly 
always have /some/ plasma-desktop activity, particularly if you have any 
system-monitor or other dynamically updating plasmoids running.

> Does that suspend mean "don't start it again", or "immediately"? Because
> it didn't stop immediately, but continued for several more minutes.

That I'm not actually sure.  I would have guessed immediately, but 
obviously, that's not what you're seeing.  Given that, I'd say it may be 
delayed, but that it should disable it for the rest of the session (or 
until unsuspended, not permanently, like disabling it thru kcontrol does), 
once it stops.

> ==  Where is the "file search" function? In KDE 3, IIRC, there was an
> item in the KDE start button to do a file search.
> 
> I don't see a file search function anywhere under the K-Start button
> menu now.

If you're using the kde4 default kickoff menu, there should be a "Find 
Files and Folders" menu item at the bottom of the Applications tab.  
Lancelot also has that menu item, again on the applications tab, and so 
does the classic menu (note that on it, the applications menu can be 
disabled, and you'd lose the find dialog item on it as well).  I just 
checked all three.  Of course, it's also possible kubuntu uses its own 
customized launcher widget, in which case I wouldn't know.

Dolphin also has a menu item to activate the find dialog, and of course, 
typing "find" in the run dialog pops up the find files and folder option, 
likely among others.

Actually, one of krunner's runner applets (configurable by clicking on the 
wrench icon) is a nepomuk based find, one of the places it *IS* already 
enabled.  So for instance, when I just typed "find" in krunner here, to 
check that it did offer the find dialog, it also popped up a whole list of 
various pdfs, shell scripts, etc, that it indexed, that happen to have the 
word "find" in them. =:^)

So that's one real-world use, right there.  Whether it's actually worth 
the hassle of the indexing is of course your decision, but that's one 
use.  Why use the find files and folders dialog when typing the content in 
question brings it up as an option in the run dialog? =:^)

> If v-t & "Search Service" are doing an indexing, what sw _uses_ that
> index to let the user do a search of the files on the disk?

> ===================================================================== ==
>  System: KUbuntu 10.4 64 bit version, 4 core processor, 4GB RAM, TB hard
> disk

FWIW, as I mentioned above, 6 gigs here (main system, I also have a 
netbook).  I'm running a now aging but still very capable dual Opteron 290 
(so dual-dual-core, 2.8 GHz), and at one point had two, two-gig sticks, 
hanging off each processor.  But one stick went bad and I've not replaced 
it yet, so it's 6 gigs RAM ATM.  But if I had it to do over, especially at 
what those sticks of registered/ecc RAM that the Opterons of that era 
required cost, I'd have stuck with four 1-gig sticks, 4 gig total.  I do 
run Gentoo, so build from source, and have the temporary build dir setup 
as a tmpfs so it's in RAM, which makes having the 8 (now 6) gig nice, but 
4 gig is really the sweet spot.  At what RAM costs now, sure, 8 gig, but 
IIRC I paid nearly $1000 a stick, and while it's nice to have the extra 
memory, at that price, there were better ways to spend my money.

Meanwhile, even accounting for the effect of that extra memory, I've 
noticed that I seem to have less problems with disk activity than most, 
even under heavy I/O.  And I simply don't have the slowdowns others seem 
to have with Virtuoso, etc, even with full indexing enabled.

I've come to the conclusion that there's several factors involved in this.

1) This is the latest one I discovered.  As I mentioned my system's now 
aging.  It still has PCI buses, no PCI-E (tho Opteron being a server CPU, 
and with dual-socket especially, I'm running on a server board, so they're 
64-bit PCI-X not the old 32-bit PCI, but it's still bus architecture, not 
the point-to-point of PCI-E).  As such, I'm not sure if this applies to 
newer PCI-E systems or not, but it certainly made a difference on my PCI/
PCI-X system:  In the BIOS, there's a PCI Latency Timer setting.  This is 
a characteristic latency vs. thruput tradeoff, with higher values meaning 
less overhead and thus better thruput, by allowing each device to use the 
bus for a longer period.  But with multiple devices sharing the bus, the 
longer any one gets to use it, the longer other devices have to wait to 
use it.

Under high I/O, higher values allow latency to get high enough to cause 
sound stuttering and even UI freezes.  I recently discovered that by 
setting the PCI Latency Timer to the lowest possible value, I could avoid  
audio skips even under extreme I/O pressure.  At a notch higher, I get 
music stuttering, but the UI's fine.  As I go higher still, the UI can lag 
out as well.

What was interesting to me, is that setting PCI Latency Timer low enough, 
I could lower my kernel tick rate to server-class 100 Hz and similarly set 
server class no preemptivity, and STILL have better responsiveness, than I 
did with even a single notch higher PCI Latency, with standard kernel 
desktop preemptivity and 300 Hz clocking.

2) I've been running 4-disk md/kernel RAID for a number of years now.  
I've tried both RAID-6 and 4-way RAID-1 for the main system.  The Linux 
kernel I/O scheduler is VERY impressive in its ability to handle multiple 
concurrent I/O tasks with md/RAID-1 (which works far better than RAID-6 
for my purposes).  I've been EXTREMELY pleased with md/RAID-1, and believe 
the kernel's ability to efficiently multi-task I/O on it is a major factor 
in why I literally don't notice disk access issues that are major problems 
for a more typical single-disk install.

So yes, it's as a longer term recommendation -- I don't expect people to 
jump up and go RAID right now -- but having seen the benefits of RAID-1 in 
system interactivity (not to mention its primary purpose, data protection 
thru redundant data storage), I wouldn't want to go back, personally, and 
IMO it's a cryin' shame that at the cost of disks these days, 2-way RAID-1 
minimum isn't standard.

3) I mentioned swappiness, above.  The swappiness knob is located at
/proc/sys/vm/swappiness.  This controls how the kernel balances 
applications vs. disk cache.  To change the value temporarily (as for 
testing), write (as root) a value between 0 and 100 to the file.  To make 
your change permanent, either setup a script to trigger such a write at 
boot, or set it up via the sysctl service that most distributions run at 
boot.  Here on Gentoo, that means adding a vm.swappiness = <number> line 
to /etc/sysctl.conf.  (FWIW, if you compare the files under /proc/sys/ to 
the entries in your sysctl config, you should quickly see the 
relationship.  Now you have a whole list of other knobs to google on and 
experiment with! =:^)

A swappiness value of 0 tells the kernel to ALWAYS favor running 
applications over cache -- the only time an app gets swapped out is if 
there's no disk cache left to dump in its place.  A value of 100 is the 
reverse -- the kernel will always try to keep cache if at all possible, 
dumping apps into swap in ordered to do so.  The kernel default (ubuntu 
may change it) is 60, slightly favoring cache over apps.

Now I mentioned that I have four disks and am striping swap over all four 
(using equal swap priority settings), so in theory, fetching applications 
back from swap should be four times as fast as reading them back in off 
the RAID-1 main system filesystem.  As such, I'm a bit of a special case, 
and have swappiness set to 100.

But from what I've read, many people with only one disk, or multiple disks 
but not in RAID, find that a a swappiness value of around 20 works far 
better for them than the default 60.  A value of 20 will reasonably 
strongly favor keeping apps in memory, at the expense of dumping disk 
cache when necessary, but will start swapping out apps before it dumps all 
cache, unlike the extreme value of 0, mentioned above.

So if you suspect you're having responsiveness issues due to cached data 
forcing apps out to swap, do experiment with swappiness.  You may well 
find, as have many others, that a value rather lower than the default 60 
works better for you.

Hope at least some of that's useful! =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

___________________________________________________
This message is from the kde mailing list.
Account management:  https://mail.kde.org/mailman/listinfo/kde.
Archives: http://lists.kde.org/.
More info: http://www.kde.org/faq.html.