Application usage statistics and targeted user surveys

[Volker Krause]

Hi,

we have talked about the above topics a couple of times in the past, from what 
I remember usually agreeing it would be nice to have some more statistical 
information about our users, so we know what our applications are used for, 
and to measure impact of changes. Similarly, it would be nice to be able to 
actually ask our users questions without statistical bias by using out-of-band 
channels like blogs or social media. This can be obviously addressed by adding 
this into application code, but that raises some valid privacy concerns.

Wanting this for GammaRay I attempted to implement a generic framework for 
this, with the goal to make this fully transparent, and give the user full 
control over what data is shared, and how often they want to participate in 
surveys, ie. make this solid enough on the privacy side that even I would 
enable it myself. You'll find the code in Git (kde:kuserfeedback).

Feature-wise it so far contains:
- a set of built-in data sources (app version, Qt version, platform, 
application usage time, screen setup, etc) that applications can choose to 
enable
- generic data sources for tracking the time ratio a Q_PROPERTY has a specific 
value, allowing to track e.g. which application view is used how much
- the ability to add custom/application-specific data sources
- reference widgets for customizing what data you want to share, and showing 
exactly what that means, in human readable translated text and if you insists 
also all the way down to the raw JSON sent to the server.
- survey targeting using simple C++/JS-like expressions that can access all 
the data sources (ie. you can target e.g. only users with high DPI multi-
screen setups)
- configurable encouragement of users to contribute (ie. after X starts and/or 
Y hours of usage, repeated after Z months, suggest the user to participate if 
they aren't already doing so).
- a management and analytic tool that allows you to manage products and survey 
campaigns, and view recorded data using configurable aggregations
- the entire thing works without unique user ids. Fingerprinting can still be 
an issue on too small user sets and/or when using too much detail in the data.
- by default all of this is opt-in of course, although technically the API 
doesn't prevent applications to change this
- it can deal with multiple products, each product can have different data 
sources and survey campaigns

Technically, this consists of the following parts:
- a library that goes into the target application, backward compatible all the 
way to Qt4.8/MSVC2010 (needed for my GammaRay use-case), depending only on 
QtGui
- a library with the reference widgets, also with extended backward 
compatibility
- the server, written in PHP5 and supporting sqlite/mysql/postgresql. Not the 
most fun technology, but that stuff is available almost anywhere, and easy to 
deploy and maintain
- the management tool, recent Qt5/recent C++, using QtCharts for the data 
analysis
- a command line tool for data import/export, useful for eg. automated backups

All of this is LGPLv2+ licensed.

Feedback obviously very welcome, in particular around privacy concerns, or 
reasons that would make you enable/disable such a feature.

Regards,
Volker
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/kde-core-devel/attachments/20170423/f77e4ead/attachment.sig>