Plasma 5.7 Test Days prep meeting #2 notes & log

Tue Apr 19 13:10:56 BST 2016

attendees: eike, jensreu, bshah, d_ed, riddell, kbroulik, notmart, 
sitter, mgraesslin
chair: eike

(a) ISOs for testing
- Riddell from Neon promises to get us test ISOs in time for test days, 
will poke him well ahead of time to plan more
- No comm yet with other distros
- einar77 may be able to interface with openSUSE for us

(b) Test cases
- Jens wrote down some ideas, currently on the etherpad 
(https://notes.kde.org/p/plasma-57-testday)
- d_ed ran some tests as part of GCI, his testcases can be found at: 
https://community.kde.org/Get_Involved/Quality/Beta/4.10/Plasma
- Jens will bring of some ideas into d_ed's format for next week's meeting
- Bhushan will add some desktop/panel related test cases
- Discussed splitting tests into seperate lists for VM vs. live USB vs. 
hdd install to account for environmental conditions and what sort of 
persistence support there is
- Decided to limit ourselves to test cases assuming clean config (fresh 
ISO boot / new user) for now to keep complexity low
- Decided to keep the number as well areas of test cases small for now; 
we still need to learn how many we can handle during Test Days
- Decided to focus test cases on core desktop shell (panels, launchers, 
desktop, task management, window management, related config)
- Decided to avoid writing test cases involving kdeplasma-addons for now
- Eike will look into writing locale-specific test cases later

(c) Dates
- Pondered rescheduling test days due to conflict with Randa sprint 
ending on the 19th and people being tired or travelling
- Not much wiggle room to move, scheduled dates (June 19th/20th) are 
soon after beta (16th) and soon before final release (30th)
- Core Test Days team (bshah, jensreu, eike) not impacted by Randa
- Decided not to reschedule
- Next prep meeting Tuesday 12:00 UTC in #plasma

--- snip ---

Previous week's notes: 
https://mail.kde.org/pipermail/plasma-devel/2016-April/051776.html


Cheers,
Eike
-------------- next part --------------
[19:30] <Sho_> bshah, jensreeuu: ping, meeting time
[19:30] <bshah> my tea is coming soon... but o/
[19:30] <jensreeuu> pongity!
[19:30] <Sho_> ok, this is the second prep meeting for the upcoming plasma 5.7 test days
[19:30] <d_ed> pong too
[19:31] <Sho_> for recap, this is last week's notes: https://mail.kde.org/pipermail/plasma-devel/2016-April/051776.html
[19:31] <jensreeuu> I've added to them Sho_
[19:31] <jensreeuu> For this week
[19:31] <Sho_> and this is our meeting etherpad: https://notes.kde.org/p/plasma-57-testday
[19:31] <Sho_> i have three agenda points i'm aware of for today:
[19:31] <Sho_> - check on progress re distro communication re isos
[19:31] <jensreeuu> I have two I havent done at all though
[19:31] <Sho_> - discuss sample test cases if we have any yet
[19:31] <jensreeuu> Ah
[19:31] <Sho_> - someone suggested we might want to change the test days date due to conflicting with randa schedule-wise
[19:32] <d_ed> I wrote some for Plasma 4.9/4.10 for some bug days I used to run:  https://community.kde.org/Get_Involved/Quality/Beta/4.10/Plasma
[19:32] <Riddell> kbroulik: published! https://dot.kde.org/kde-augsburger-linux-tag (well be on front page when cache clears)
[19:32] <Sho_> let's go through the agenda point by point, so first would be the ISO stuff
[19:32] <jensreeuu> d_ed: MAN your testcases are waaaay more well done than my wild scribbles
[19:32] <jensreeuu> Yesd
[19:33] <Sho_> jensreeuu: have you had any time to talk to any of the distros you listed last time? (kaos, opensuse, fedora, neon)
[19:33] <jensreeuu> Well I haven't talked to the distro people yet, so not much reply.
[19:33] <jensreeuu> Kaos, an email, fedora an email - Neon well Riddell?
[19:33] * Sho_ looks too Riddell as well
[19:33] <jensreeuu> Someone else need to take Opensuse though
[19:34] <-- JunkHunk (~ivan at 203.red-83-54-4.dynamicip.rima-tde.net) has left this server (Quit: Saliendo).
[19:34] <Sho_> on the opensuse front we can try to rope einar77_work into interfacing for us
[19:34] * Riddell looks around
[19:34] *** ochurlaud is now known as ochurlaud--away.
[19:34] <jensreeuu> Hello! Do you wanna have neon used as a testbase for the bug testing in June Riddell?
[19:34] <Sho_> Riddell: if we do plasma 5.7 test days on June 19th and 20th (or around that time in case we move them a bit), can we work with Neon to get us ISO images with testable git code in time?
[19:34] <Riddell> jensreeuu: it would be an honour
[19:35] <kbroulik> Riddell: <3
[19:35] <jensreeuu> Riddell: sucha  gentleman :)
[19:35] <Riddell> Sho_: yes indeed
[19:35] <Sho_> Riddell: last time i checked though neon was a repo people add to a kubuntu or something, are there actual ISOs with fresh coden ow?
[19:35] <Sho_> (apologies i am totally and miserably out of loop)
[19:35] <Riddell> Sho_: certainly are http://files.kde.org/neon/
[19:35] <Sho_> (i am a mere code monkey)
[19:35] <Sho_> awesome
[19:36] <Riddell> and user edition ones (released tars) still hidden away at http://images.neon.kde.org/neon-useredition-proposed/
[19:36] <jensreeuu> With Kaos I have yet to get a reply, and with Fedora I dont know if my nagging contacts worked
[19:36] <Sho_> that means we have at least one testing ISO locked in
[19:36] --> mgraesslin (~martin at kde/mgraesslin) has joined this channel.
[19:36] <Sho_> Riddell: we'll poke you well ahead of time to plan this out more
[19:36] <jensreeuu> if einar77_work can handle Opensuse I would be much appreciative
[19:36] <Riddell> that sounds rude
[19:36] <jensreeuu> :)
[19:37] <jensreeuu> d_ed: When you did the 4.10 testing how did it go? What was the turnout and what methods did you guys use to reach users?
[19:37] <Sho_> ok, moving on to test cases then
[19:38] * Sho_ takes a look at d_ed's link
[19:38] <jensreeuu> Sho_: my wild scribbles are in the etherpad
[19:38] <-- boombatower (~boombatow at drupal.org/user/214218/view) has left this server (Quit: Konversation terminated!).
[19:39] <jensreeuu> d_ed's is waaaay more organized and nicely done (will copy, steal and grab)
[19:39] <mgraesslin> Sho_: in future please stash commits - no need to have one revision with isShadable and one with isShadeable
[19:39] <Sho_> mgraesslin: k
[19:39] <bshah> I've some test cases for typical panel/desktops
[19:39] <Sho_> jensreeuu: I'm not sure the "does it stutter?" is something we can reasonably test for at this time
[19:40] <d_ed> jensreeuu: turnout was maybe 3-4 committed people
[19:40] <Sho_> machine configurations create too much variablity there
[19:40] <bshah> (not documented yet, but I do use it to find tests)
[19:40] <d_ed> with 10 people or so turning up for a bit
[19:40] <jensreeuu> Fair enough. Stuttering is one of those complaints we get so wanted to include
[19:40] <Sho_> it's true that "stutters on no machine" would be a quality level that would be nice to aim for
[19:40] <Sho_> but right now it means we'd fail all your testcases and get a lot of noise
[19:40] <d_ed> jensreeuu: I just used PlanetKDE and a blog post
[19:40] <jensreeuu> :)
[19:40] <Sho_> ok, so
[19:41] <Sho_> i like d_ed's format and i like many of jens idea's, so i think we need to combine the two
[19:41] <jensreeuu> Yeah d_eds format is golden
[19:41] <Sho_> because the goal is to nail down a bit of a format for test cases here so we have a formula we can use to distribute the task of writing more ones
[19:42] <Sho_> jens, do you want to bring some of the stuff from your list into d_ed-style format for next week?
[19:42] <jensreeuu> Sho_: I will rewrite stuff for the tests (this was just a doodle list of wanted things) - and will use Davids format
[19:42] <Sho_> sweet
[19:42] <jensreeuu> :) so "yes"
[19:42] <jensreeuu> I have a few questions concerning hardware issues
[19:42] <jensreeuu> Especially multiple monitors
[19:43] <jensreeuu> The issue is that there is no point testing pre Qt 5.6 things
[19:43] <jensreeuu> Right?
[19:43] <notmart> yep
[19:43] <sKreamer> notmart, you have notes! [Apr-19 03:35] <mck182> hey, with latest kirigami master, this error now comes up --> "Error loading QML file.\n140: Type NewConversationPage unavailable\n28: Type Kirigami.Page unavailable\n153: PageActionPropertyGroup is not a type\n"...looks like something was left out [Apr-19 04:04] <mck182> also, for components not present in kirigami, like Button, is one expected to use PlasmaComponents or plain QQC?
[19:43] <bshah> mostly right
[19:43] <bshah> but Neon have qt5.6
[19:44] <notmart> mck182: yeah, i noticed the problem this morning, it should be fixed now
[19:44] <Sho_> jensreeuu: "Is it there on reboot?" might also be a problem when people test the ISO in VMs and don't do a hw install
[19:44] <notmart> (also, just use qqc button)
[19:44] <Sho_> *hd
[19:44] <Sho_> jensreeuu: and i don't know we want to make a hd install a test requirement
[19:44] <jensreeuu> Sho_: ok will remove. But should we split tests into "WM", "Live USB", "New Installation"?
[19:44] <Sho_> jensreeuu: yeah that could make sense
[19:45] <jensreeuu> Just so that we have like a clear indicator of what test is relevant where?
[19:45] <notmart> WM has a lot of kwin issues that make many things invalid tough
[19:45] <jensreeuu> notmart: yeah so we could sort tests based on what method is used to test them instead of all together
[19:45] <notmart> yeah
[19:45] <jensreeuu> So you have one list for WM, one for Live USB etc
[19:46] <-- mbohlender (~mike at aftr-185-17-204-85.dynamic.mnet-online.de) has left this server (Remote host closed the connection).
[19:46] <Sho_> true, on the other hand VMs are definitely a huge part of our user demographic now (people simply use VMs for a lot of reasons, from testing to dev work to, ..)
[19:46] <Sho_> so working in VMs is certainly a config we're expected to support
[19:46] <jensreeuu> Plus I am sure the distro maintainers would love to have some testing based on that live USB thing too
[19:46] <jensreeuu> Sho_: true
[19:46] <jensreeuu> So focus on WM, but available tests for Live USB and Install?
[19:47] --> apol (~apol at 62.57.48.198.dyn.user.ono.com) has joined this channel.
[19:47] <Sho_> yep
[19:47] <bshah> should we also check for migration bugs? e.g, upgrade from 5.6 to 5.7
[19:48] <bshah> (people with already installed plasma can help with that)
[19:48] <jensreeuu> We might as well add them as fringe testing?
[19:48] <Sho_> not sure we can, i think we're currently aiming tests at the first-use experience
[19:48] <jensreeuu> Ah ok
[19:48] <bshah> okay
[19:48] <Sho_> consider e.g. a testcase like "Change the colour theme in your system settings, does the desktop theme change? What are the experiences with that, how long did it take? Did you have to log in and out?"
[19:48] <Sho_> recoloring is only known to work with Breeze, so this testcase assumes clean config, i.e. the user didn't switch away from Breeze
[19:48] <jensreeuu> Good point...
[19:48] <Sho_> if the testcases are written to assume pristine config, then you can't test migration
[19:49] <Sho_> and i think we should probably focus on testcases assuming pristine config for now (which also trickles down to the VM vs live usb vs hd thing)
[19:50] <Sho_> just to keep the complexity down
[19:50] <jensreeuu> Yeah...
[19:50] <bshah> one point I want to add is instead of having say 50ish test cases, we should start with smaller number of testcases with pre-defined areas (say desktop containment and plasmoids, instead of spreading to whole plasma stack for first time)
[19:50] <Sho_> bshah: agreed
[19:50] <jensreeuu> I mean if it works golden we can ramp stuff up for next time
[19:50] <Sho_> especially since this is the first test days and we have a lot to learn
[19:50] <jensreeuu> bshah: good point
[19:50] <Sho_> if we find out "yeah we can handle more test cases", that's something to learn too
[19:51] <Sho_> we don't need a massive amount right now
[19:51] <Sho_> ok, so should we focus on test cases assuming clean config for now?
[19:51] <jensreeuu> +1
[19:51] <Sho_> and thus also require clean iso boots?
[19:51] <jensreeuu> +1 again I think it would save us a ton of issues
[19:51] <bshah> I personally won't restrict to clean boot, probably new user?
[19:52] <Sho_> adding a new user on the iso though ...?
[19:52] <Sho_> never done that, i mean if it works ...
[19:52] <bshah> no, well for instance thanks to uefi, neon doesn't boot for me yet
[19:52] <Sho_> any mechanism to get a clean config is ok with me, though ;)
[19:53] <bshah> so.. I will have to go with pre-installed stuffs
[19:53] <Sho_> bshah: yeah but i mean we we do want people to run the test cases against the same packages though, right?
[19:53] <Sho_> hmm
[19:53] <jensreeuu> Wait are you guys sure that is a good safe way to get a clean config? (I could have sworn someone talked about this a while ago)
[19:53] <jensreeuu> Although old age may play tricks on me
[19:53] <Sho_> jensreeuu: in practice there are sometimes differences between install/iso user and new user, yeha, due to distros messing things up
[19:54] <jensreeuu> Ah so that is one of those "talk closely with distros so we know before hand" things?
[19:54] <bshah> I'm fine with either way... iso/clean boot/whatever
[19:54] <d_ed> on the other hand, if there are "distro mess ups" we need to know about them anyway
[19:54] <bshah> ^ what d_ed said
[19:54] <jensreeuu> d_ed: yeah and I suppose that is one of the drives for distros to be a part of it, since that would be a good test for them too
[19:54] <Sho_> true
[19:54] <bshah> you can't say 100% bugs we will get will be from neon.. we need to test different configurations
[19:55] <Sho_> ok, let's go with "any mechanism to get a clean config for now"
[19:55] <bshah> +1
[19:55] <Sho_> ok, i think that's enough to make progress on test cases until next week
[19:55] <Sho_> final topic on the agenda is dates
[19:55] <Sho_> we scheduled for june 19th and 20th, but someone noted this might conflict with randa
[19:55] <Sho_> so the question is whether to move the dates, and whether we even can
[19:56] <bshah> when is plasma release?
[19:56] <bshah> or beta or tag day?
[19:56] <jensreeuu> Or if that is actually bad considering the fact that Randa is a collection of devs in one place for a time
[19:56] <Sho_> so beta freeze is on the 16th
[19:56] <Sho_> which means on the 19th we have the beta available, which is what we wanted
[19:56] <Sho_> and final release is on the 30th
[19:57] <Sho_> which gives us 10 days to get test days results into .0
[19:57] <Sho_> i don't see much wiggle room to move stuff
[19:57] <-- jgrulich (~jgrulich at 2a00:1028:83ce:4fe6:830f:9572:74a0:8d6b) has left this server (Remote host closed the connection).
[19:57] --> jgrulich (~jgrulich at 2a00:1028:83ce:4fe6:830f:9572:74a0:8d6b) has joined this channel.
[19:57] <-- jgrulich (~jgrulich at 2a00:1028:83ce:4fe6:830f:9572:74a0:8d6b) has left this server (Remote host closed the connection).
[19:57] <Sho_> randa starts on the 12th and ends on the 19th
[19:57] --> jgrulich (~jgrulich at 2a00:1028:83ce:4fe6:830f:9572:74a0:8d6b) has joined this channel.
[19:57] <Sho_> so the worry is people will be tired or travelling
[19:58] <jensreeuu> ooooh I thought the other way around, yeah that is an issue
[19:58] <Sho_> but i think there's nothing much we can do about that - we probably just have to try and do a good job collecting the test data, and get it into the hands of devs recovered from randa for the 10 days after
[19:58] <jensreeuu> +1
[19:59] <Sho_> i won't be at randa, so i am free on the scheduled days
[19:59] <Sho_> how about you guys?
[19:59] * bshah neither
[19:59] <bshah> sadly.. :(
[19:59] <bshah> because I want to go to QtCon
[19:59] * notmart will be elsewhere, returning monday evening
[20:00] <jensreeuu> free
[20:00] <sitter> I am not going randa so I won't be tried
[20:00] <Sho_> ok, that means the core test days gang is not impacted by randa at least
[20:00] <sitter> muhahah
[20:00] <Sho_> and sitter just volunteered to do most of the work
[20:00] <Sho_> so i think we good on dates?
[20:00] <sitter> wow
[20:00] <jensreeuu> yay sitter = testgroups personal manservant!
[20:00] <Sho_> let's +1 this before he reconsiders
[20:00] <bshah> yep...
[20:00] <jensreeuu> +1
[20:01] <bshah> +2
[20:01] <Sho_> yay
[20:01] <Sho_> that concludes the agenda i was aware of
[20:01] * notmart won't be there but you guys have fun
[20:01] <jensreeuu> so 4 votes to one abstaining...
[20:01] <sitter> jensreeuu: I'll have you know that slavery is illegal in most of europe
[20:01] <Sho_> anyone got more points? :)
[20:01] <Sho_> notmart: will miss you :(
[20:01] <bshah> as I said earlier
[20:01] <bshah> can we decide area of plasma where test cases will be based on?
[20:02] <bshah> say desktop? system settings? etc?
[20:03] <jensreeuu> Desktop would be the better of the two since that has more direct impact on users, and system settings is going through a gazillion and one changes anyway...
[20:03] <jensreeuu> I mean if you guys feel like focus should be there - then I am all for
[20:03] <Sho_> agreed, let's focus on the core desktop shell chrome
[20:04] <bshah> I am all with starting from desktop.. because thats thing "in face" when user starts computer
[20:04] <Sho_> panels, launchers, task management, krunner, window management, desktop surface
[20:04] <Sho_> and related config
[20:04] <bshah> okay
[20:04] <Sho_> it's a good point to start from for now i thinl
[20:04] <jensreeuu> okay
[20:05] <Sho_> ok!
[20:05] <notmart> possibly at first even excluding stuff in kdeplasma-addonsto stay a bit more focused
[20:05] <bshah> +1
[20:05] <mgraesslin> 19th I most likely won't have time, but 20th is fine
[20:05] <Sho_> aye, i think we can do that by not writing test cases that specifically involve kpa for now
[20:06] <jensreeuu> okidokey
[20:06] <Sho_> one thing i'd like to see is locale-specific test cases, but
[20:06] <Sho_> i'll try to pen something there once we have a test case format locked down
[20:06] <jensreeuu> +1 can we add to the etherpad?
[20:07] <jensreeuu> I mean during the week
[20:07] <Sho_> sure
[20:07] <-- eliasp_ (~quassel at HSI-KBW-046-005-044-147.hsi8.kabel-badenwuerttemberg.de) has left this server (Remote host closed the connection).
[20:07] --> eliasp (~quassel at HSI-KBW-046-005-044-147.hsi8.kabel-badenwuerttemberg.de) has joined this channel.
[20:07] <Sho_> ok, so if no one else has anything, i'll keep the etherpad meeting notes editing window open till :15, then send out the notes to plasma-devel and the distro list
[20:07] <Sho_> along with the log
[20:07] <Sho_> :)
[20:07] <bshah> next meeting on monday?
[20:07] * sitter read leg for some reason
[20:07] <Sho_> Tuesday as usual?
[20:08] <bshah> err
[20:08] <bshah> right
[20:08] <Sho_> 12:00 UTC in #plasma
[20:08] * bshah have lost sense of weekdays..
[20:08] <Sho_> :)
[20:08] <jensreeuu> +1
[20:08] <notmart> when every day is a monday, things get ugly ;)
[20:08] <jensreeuu> :)
[20:09] <sebas> mgraesslin: https://bugs.kde.org/show_bug.cgi?id=361953 is this a kwin thing? (Windows opening in a not-quite-right position)
[20:09] <Sho_> ok, /meeting!
[20:09] <bugbot> KDE bug 361953 in KScreen (common) "When monitors are not vertically aligned, some windows are wrongly vertically placed" [Minor,Unconfirmed: ]
[20:09] <Sho_> thanks for attending everyone