[Nepomuk] Review Request 106748: Create a taskbar notification window when we run out of inotify watches

[Vishesh Handa]

> On Oct. 13, 2012, 1:20 a.m., David Edmundson wrote:
> > From a usability perspective you can't show random popups telling the user to run commands as root.
> > 
> > 1) It's putting users in a bad habbit, any app can trigger a popup telling someone to run "sudo rm * -Rf". 
> > 2) The vast majority of users aren't in a sys-admin position. Consider all the big KDE deployments to schools, universities and offices. The person using this computer will not have root powers, it's just a pointless helpless error message
> > 3) It looks scary and confusing to a non nerd-user, and looking like a virus to those who know a medium amount.
> > 
> > (from a code POV you're also triggering this every time the limit hits, which assuming the user can't or decides not to change it means this notification will be going off /constantly/.
> 
> Simeon Bird wrote:
>     Ok, would the text:
>     "Please ask your system administrator to increase the inotify watch limit, or new files will not be indexed."
>     be better?
>     
>     We're only triggering the notification once for every time nepomukserver runs and installs the watches, which is usually once per boot.
>     I take your point about the user not being sysadmin - the sysadmin doesn't even see the notification, so they don't even necessarily know the limit is hit.
>     
>     How about adding a checkbox to the notification (I personally don't know how to do that, but I imagine it's pretty easy to work out)
>     to disable it ("Don't show me this again"), and also send a message to syslog (which can't be turned off).
>     
>     What do you think? Or do you think the notification popup is a bad idea in general?
>     
>     [Aside, to Vishesh: Would it be sensible to put logic in place to make nepomuk somehow prioritise directories 
>     with metadata/indexed directories if we do run out of watches? That would be slightly more graceful, no? 
>     We would still have the possibility of losing data, but it would be somewhat less likely.]
> 
> Thomas Pfeiffer wrote:
>     I totally agree with David here. This looks like a bad solution to a correctly identified problem to me. The current situation that metadata is lost without users noticing is certainly a big problem. However asking users to fiddle with parts of their system they likely have never heard of (even if they have admin privileges) is not the solution.
>     Your suggestion for the new text is better than the original one, but it probably still scares users more than it helps (I don't expect many users to actually call their sysadmin after reading a notification they probably don't understand anyway).
>     Personally, I think we'll have to tackle the problem at its core: The solution must be to prevent running out of inodes in the first place, period.  We can do this by
>     a) Thinking hard about a solution which requires less inodes or doesn't lose data if there are not enough left (to me this is the "proper" solution, but one which might require lots of effort and big changes to Nepomuk)
>     b) Convince those distros shipping KDE by default to increase the inodes to a sensible number by default and those that do not ship it by default but offer it for installation to put a hint to increase the number to X (sensible value) into the post-installation messages (so that the admin who installs KDE sees the message and can act immediately)
> 
> Simeon Bird wrote:
>     Yes, ideally one wants to avoid losing metadata altogether. But, alas, we do not live in an ideal world, and either of the options you propose will take some time, even if they were possible. Better reporting of the error makes the situation better *now*, and does not prevent or delay a future better solution.
>     
>     Increasing the defaults is problematic, because there isn't really a sensible value. There's a tradeoff depending on how many watches a person might need vs how much memory they have. eg, you increase it to 16k. That's fine, until some joker has a homedir with 50k directories in it. You increase it to 500k and suddenly a local user can DoS with a simple mkdir loop.
>     
>     As to the other; so far we have three options to use fewer inotify watches (not inodes):
>     - Change inotify so that rename events include the destination path as well as the source, so we only need to watch the indexed/tagged folders, rather than all folders. (The kernel people won't make inotify recursive because it could easily eat memory, apparently).
>     - Change fanotify so it generates rename events and also fix it so it actually works.
>     - Store metadata alongside the file in xattrs. This is a bad idea because it is slow and the metadata is very easy to lose (eg, 'cp' by default loses xattrs); for this reason both mac and windows have in recent years moved away them.
>     
>     The problem is that the good solutions require kernel changes. In my opinion, the kernel is likely to look more favourably on requests from software which is undeniably awesome in all respects except the one which requires kernel changes. So we should make nepomuk as good as we possibly can on its own, and *then* go convince lkml.
> 
> Simeon Bird wrote:
>     Ok, how's this for an idea:
>     
>     If we run out of inotify watches, go into "semi-blind mode", where we only install watches on indexed folders (probably need a warning somewhere that metadata may be lost in this mode). 
>     
>     If (as is likely, since the default is to index and watch all of $HOME) we still don't have enough watches,
>     give a notification which reads:
>     
>     ==Ran out of folder watches==
>     "Too many folders to be indexed for instant search. Please reduce the number of indexed folders, 
>     or ask your system administrator to increase the inotify watch limit."
>     and then a button which opens the relevant kcm panel (Maybe this is the place for the warning?).
>     
>     That way the user is not scared, because they can control the situation by searching fewer directories, 
>     and if they want more they can bug the sysadmin. Also it connects the confusing bit of the system they haven't heard of into something concrete they can understand: 'too many directories to search', even if this link is in 
>     reality more tenuous than they might think.
>     
>     What do you think? Semi-blind mode has several traps for the unwary, especially wrt tags, but could be ok as an emergency fallback.
> 
> Simeon Bird wrote:
>     Bah, I came up with that idea less than an hour ago and I already hate it. It's too complicated, it introduces vast new and interesting possibilities for bugs, and its not even true; we need the watches primarily for tags, not indexing.
>     
>     How about:
>     "You have too many folders in your homedir to watch all of them for instant search. 
>     If you want to be sure to index new files, you should ask your system administrator to raise the inotify watch limit."
>     
>     That sounds (at least to me) less scary. It also explains why the watches need to be raised, so users can ignore it if they aren't interested. But I don't see a way around asking them to fiddle with their system, sorry, unless we just say "You have too many folders in your homedir to watch all of them for instant search."
> 
> David Edmundson wrote:
>     Another solution is to use KAuth and make the change ourselves, rather than telling the user to.
>     
>     That way instead of telling the user to type something, they click the notification, get a message and then the regular polkit prompt.
>     
>     The bonus beauty of this is that we can tell before showing the notification if the user does in fact have suitable permissions for this change. If the user can't do anything about the issue, there's not a lot of point notifying them, so we can just log to logfile instead. 
>     
>     We can just set the limit to current_limit*2 each time it's hit. 
>     
>     I've done some KAuth stuff before, so I can write the action file, and helper if needed.
> 
> Simeon Bird wrote:
>     That's a great idea. I have no experience with KAuth at all, so any help with it you could give would be very much appreciated.
>

Ping?

It would be awesome if we could get this into 4.11


- Vishesh


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://git.reviewboard.kde.org/r/106748/#review20241
-----------------------------------------------------------


On Oct. 11, 2012, 5:54 p.m., Simeon Bird wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://git.reviewboard.kde.org/r/106748/
> -----------------------------------------------------------
> 
> (Updated Oct. 11, 2012, 5:54 p.m.)
> 
> 
> Review request for Nepomuk, KDE Usability, Sebastian Trueg, and Vishesh Handa.
> 
> 
> Description
> -------
> 
> Currently, if we happen to run out of inotify watches in the filewatch service, all we get is a debug message, which is easily lost.
> This patch additionally creates a popup notification to warn people they need to raise the limit.
> 
> I worried that a popup might be a bit too invasive - I considered also just logging to syslog, but that seemed not invasive enough. 
> I figured that since metadata can get lost if the watches are not all installed, being fairly invasive is a good idea. 
> 
> What think you?
> 
> 
> Diffs
> -----
> 
>   services/filewatch/nepomukfilewatch.cpp 83045da 
>   services/filewatch/nepomukfilewatch.notifyrc bfa88a9 
> 
> Diff: http://git.reviewboard.kde.org/r/106748/diff/
> 
> 
> Testing
> -------
> 
> > sudo sysctl fs.inotify.max_user_watches=10 
> > killall nepomukserver && nepomukserver
> 
> Yup, there's a notification.
> 
> 
> Thanks,
> 
> Simeon Bird
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/nepomuk/attachments/20130204/f0f0e182/attachment-0001.html>