Application usage statistics and targeted user surveys

[Aleix Pol]

On Tue, May 23, 2017 at 6:31 PM, Aleix Pol <aleixpol at kde.org> wrote:
> On Sun, Apr 23, 2017 at 12:52 PM, Volker Krause <vkrause at kde.org> wrote:
>> Hi,
>>
>> we have talked about the above topics a couple of times in the past, from what
>> I remember usually agreeing it would be nice to have some more statistical
>> information about our users, so we know what our applications are used for,
>> and to measure impact of changes. Similarly, it would be nice to be able to
>> actually ask our users questions without statistical bias by using out-of-band
>> channels like blogs or social media. This can be obviously addressed by adding
>> this into application code, but that raises some valid privacy concerns.
>>
>> Wanting this for GammaRay I attempted to implement a generic framework for
>> this, with the goal to make this fully transparent, and give the user full
>> control over what data is shared, and how often they want to participate in
>> surveys, ie. make this solid enough on the privacy side that even I would
>> enable it myself. You'll find the code in Git (kde:kuserfeedback).
>>
>> Feature-wise it so far contains:
>> - a set of built-in data sources (app version, Qt version, platform,
>> application usage time, screen setup, etc) that applications can choose to
>> enable
>> - generic data sources for tracking the time ratio a Q_PROPERTY has a specific
>> value, allowing to track e.g. which application view is used how much
>> - the ability to add custom/application-specific data sources
>> - reference widgets for customizing what data you want to share, and showing
>> exactly what that means, in human readable translated text and if you insists
>> also all the way down to the raw JSON sent to the server.
>> - survey targeting using simple C++/JS-like expressions that can access all
>> the data sources (ie. you can target e.g. only users with high DPI multi-
>> screen setups)
>> - configurable encouragement of users to contribute (ie. after X starts and/or
>> Y hours of usage, repeated after Z months, suggest the user to participate if
>> they aren't already doing so).
>> - a management and analytic tool that allows you to manage products and survey
>> campaigns, and view recorded data using configurable aggregations
>> - the entire thing works without unique user ids. Fingerprinting can still be
>> an issue on too small user sets and/or when using too much detail in the data.
>> - by default all of this is opt-in of course, although technically the API
>> doesn't prevent applications to change this
>> - it can deal with multiple products, each product can have different data
>> sources and survey campaigns
>>
>> Technically, this consists of the following parts:
>> - a library that goes into the target application, backward compatible all the
>> way to Qt4.8/MSVC2010 (needed for my GammaRay use-case), depending only on
>> QtGui
>> - a library with the reference widgets, also with extended backward
>> compatibility
>> - the server, written in PHP5 and supporting sqlite/mysql/postgresql. Not the
>> most fun technology, but that stuff is available almost anywhere, and easy to
>> deploy and maintain
>> - the management tool, recent Qt5/recent C++, using QtCharts for the data
>> analysis
>> - a command line tool for data import/export, useful for eg. automated backups
>>
>> All of this is LGPLv2+ licensed.
>>
>> Feedback obviously very welcome, in particular around privacy concerns, or
>> reasons that would make you enable/disable such a feature.
>>
>> Regards,
>> Volker
>
> Hi volker,
> I've been looking into how it works, I wanted to test the tests/orwell
> application but I keep getting this error:
> ./bin/orwell: symbol lookup error: ./bin/orwell: undefined symbol:
> _ZN12UserFeedback18CompilerInfoSourceC1Ev
>
> I'm seeing a similar error when running autotests:
> ********* Start testing of DataSourceTest *********
> Config: Using QtTest library 5.9.0, Qt 5.9.0
> (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC
> 6.3.1 20170306)
> PASS   : DataSourceTest::initTestCase()
> PASS   : DataSourceTest::testPlatformInfoSource()
> PASS   : DataSourceTest::testScreenInfoSource()
> PASS   : DataSourceTest::testPropertyRatioSource()
> PASS   : DataSourceTest::testMultiPropertyRatioSource()
> PASS   : DataSourceTest::testApplicationVersionSource()
> PASS   : DataSourceTest::testQtVersionSource()
> PASS   : DataSourceTest::testStartCountSource()
> PASS   : DataSourceTest::testUsageTimeSource()
> PASS   : DataSourceTest::testCpuInfoSource()
> PASS   : DataSourceTest::testLocaleInfoSource()
> ./bin/datasourcetest: symbol lookup error: ./bin/datasourcetest:
> undefined symbol: _ZN12UserFeedback16OpenGLInfoSourceC1Ev
>
> I would have looked into fixing it, but I'm not sure I understand why
> there's all the RPATH logic in place, so I'd prefer to hear from you
> first.
>
> A good next step would also be to get it running on build.kde.org, so
> we can identify these issues.
>
> Aleix

Hey Volker, I figured out this one. Never mind.

I've done a proof of concept integrating it in Discover, here's 2 patches:
https://phabricator.kde.org/D5960
https://phabricator.kde.org/D5961

Now to proceed I'd like to give a try to whole system including the
server. Do you have documented how to set it up anywhere? Would make
it easier.

Thanks!
Aleix