Plasma 5.7 Test Days prep meeting #4 notes & log

[Eike Hein]

attendees: eike, jensreu, bshah
chair: eike

a) ISO status
- No further communication with Fedora yet (Eike, Jens) due to vacations
- Bhushan says Arch ISOs will be trivial

b) Dates
- Future meetings at 13:00 instead of 12:00 to avoid conflict with 
Kirigami meetings

c) Test cases
- Bhushan expanded his test cases a bit
- No movement on wikifying test cases, Eike & Jens & Bhushan doing 
sitdown on Wednesday to hammer it out, about 10 AM CET
- Once we got wikified sample test cases, have to trial them to see if 
they make sense

d) Promo
- Jens says looking good for first week of June @ VDG assets

e) Wiki
- Jens plans to do wiki work after tomorrow's testcase writing sitdown

f) Test day planning
- Might need to draw up schedule assigning dev handlers to shifts, who 
talk to users running the test cases and collect notes
- Alternatively, survey website where you pick testcase, answer 
questions like "did you find the instructions easy to follow?" and "was 
the testcase a success?" along with a field for freeform notes
-- Could collect some additional metrics along the way, like "How much 
experience do you have using Plasma?" and "How long did you need to 
complete the testcase?" to find out how long users take to do what we 
ask them to do
-- Might be able to use survey.kde.org as platform?

Last week's meeting was cancelled due to conflict with a Kirigami
meeting. Notes from two weeks ago: 
https://mail.kde.org/pipermail/plasma-devel/2016-April/052292.html

The next meeting is next Tuesday at 13:00 (!) in #plasma.


Cheers,
Eike
-------------- next part --------------
[20:01] <Sho_> meeting time
[20:01] <Sho_> jensreu back? :)
[20:01] <Sho_> while we wait for pong, here are the notes from two weeks ago: https://mail.kde.org/pipermail/plasma-devel/2016-April/052292.html
[20:01] <-- alexxy[home] (~alexxy at gentoo/developer/alexxy) has left this server (Ping timeout: 260 seconds).
[20:02] <Sho_> last week we cancelled due to a schedule conflict with the kirigami meeting
[20:02] <Sho_> but we've had https://community.kde.org/Plasma/TestDays to look at during the week
[20:02] <Sho_> etherpad for today's meeting: https://notes.kde.org/p/plasma-57-testday
[20:03] <jensreu> Sho_: I am le back
[20:03] <Sho_> wonderful
[20:04] <Sho_> alright-o
[20:04] <bshah> o/
[20:04] <jensreu> I can go through mine since it will ppainfully short
[20:04] <Sho_> starting with ISOs as usual: I forgot to talk to Fedora
[20:04] <jensreu> :)
[20:04] <Sho_> but I also had a short last week due to vacatiob
[20:04] <Sho_> so i have a bad excuse
[20:04] <Sho_> :)
[20:04] <jensreu> Sho_: the guy I was going to talk to is on vacation
[20:04] <Sho_> heh
[20:04] <jensreu> the person who told him to check my email missed it by two days
[20:05] <Sho_> anybody got anything else on the iso front? bshah, your arch stuff?
[20:05] <bshah> well.. about arch stuff it is matter of running one command...
[20:05] --> alexxy (~alexxy at gentoo/developer/alexxy) has joined this channel.
[20:05] <Sho_> right
[20:05] <jensreu> so nothing there - ankke was game it seems (need to talk to here directly) but she thought the idea would be blocked by "plasma devs"
[20:05] <jensreu> Dont know why thought
[20:05] <jensreu> *though
[20:05] <jensreu> (ankke = KaOS)
[20:06] <Sho_> at least this plasma dev does not intend to block anything
[20:06] <-- Xeno__ (~Xeno at 159-205-129-63.adsl.inetia.pl) has left this server (Ping timeout: 244 seconds).
[20:06] <Sho_> ok, moving on from isos let's quickly get dates talk out of the way
[20:06] <jensreu> Kinda what I thought "What dev would block that?"
[20:06] <Sho_> do we move these meetings from 12:00 to 13:00 in general to avoid the kirigami conflict?
[20:06] <jensreu> sure
[20:06] <jensreu> fair enough
[20:06] <Sho_> kk
[20:06] <bshah> fine with that...
[20:07] <Sho_> ok, which brings us to the meat ... test cases
[20:07] *** bcooksley is now known as bcooksley-away.
[20:08] <Sho_> jens, did you have any time to wikify bhushan's stuff?
[20:08] <bshah> I've added two more test cases...
[20:08] <jensreu> No :/
[20:08] <bshah> more to come.. but sadly there is no easy braindump command.. or interface
[20:08] <bshah> :P
[20:08] <jensreu> I need some help there too but essentially trying wildly to figure out how to make shit into useful scenarios for you guys
[20:08] <Sho_> jensreu: do you need help there workload-wise or hand it off?
[20:09] <Sho_> (which is totally fine, priority is: we make progress)
[20:09] <jensreu> No I shouldn't but tomorrow if one of you have time I would love to sit down and do one or two together just so I get format right
[20:09] <Sho_> jensreu: ok, I am up for that
[20:09] <jensreu> Awesome! When are you up and about?
[20:09] <bshah> I will also be available tomorrow...
[20:09] <jensreu> bshah: <3
[20:09] <jensreu> when are you guys up?
[20:10] <bshah> I am mostly following generic CET work hours..... so... well.. anytime
[20:10] <Sho_> jensreu: I should be around from about 9-14 your time (most likely both earlier and later, but the above is fairly secure)
[20:10] <jensreu> ok so I have one meeting lemme check... brb
[20:10] <Sho_> (more likely earlier than later)
[20:11] <jensreu> Well lets do it earlier. You guys set the time since its two of you
[20:12] <Sho_> jensreu: what's the earliest you like?
[20:12] <jensreu> (and you are closer in time zone) meeting was set for thursday so I am fine
[20:12] <jensreu> errrr Sho_ 9:00 CET?
[20:12] <Sho_> sounds good
[20:12] <Sho_> basically i'll just ping you when i come in and you can pong at your leisure
[20:12] <bshah> 10:00 CET? 9 colides with lunch
[20:12] <jensreu> I mean I can go earlier but my brain is really slow to boot
[20:13] <Sho_> 10 is fine too
[20:13] <jensreu> 10 works for me too
[20:13] <Sho_> sweet
[20:13] <jensreu> Awesome so "10 CET make example scenarios" :)
[20:13] <Sho_> yep
[20:13] <Sho_> ok, so ... how about we do a poll on our announcement/communication/promo status-fu
[20:14] <bshah> or maybe... for instance.... how about we take one test case and hammer it out right now?
[20:14] <Sho_> let me check our notes for when we wanted to start communicating
[20:14] <Sho_> here it is: "promo launch deadline is first week of june"
[20:14] <Sho_> so we have about 20-30 days
[20:15] <Sho_> jens, do you think the vdg stuff etc will be ready by that (stickers and so on)?
[20:15] --> apol|desktop (~apol at 62.57.48.198.dyn.user.ono.com) has joined this channel.
[20:15] <Sho_> bshah: no objections, let's see after the promo question
[20:16] <bshah> cool
[20:16] <jensreu> Yes
[20:16] <jensreu> it will be no worry there
[20:16] <Sho_> sweet
[20:17] <Sho_> ok, do we want to do a testcase now?
[20:17] <Sho_> or tomorrow?
[20:17] <jensreu> (mostly because I've started fiddling on it right now the issue is printing it and most importantly... shipping/handling because that will be a full time gig for someone)
[20:17] <Sho_> wait, printing?
[20:17] <jensreu> tomorrow would be best for me...
[20:17] <Sho_> we're doing physical assets? :)
[20:17] <jensreu> stickers
[20:18] <jensreu> Well I thought with stickers that may make them more interesting :)
[20:18] <Sho_> when we said stickers i thought "a png to slap onto your blogpost" :)
[20:18] <bshah> ^ ;)
[20:18] <jensreu> OOooooh yeah well thats easy
[20:18] <Sho_> heh
[20:18] <Sho_> i'd actually prefer to do the testcase writing tomorrow too
[20:18] <soee> any plans to fix this activities switching issue (activity loosing wallpaper/widgets)?
[20:18] <bshah> I for instance wouldn't want bugs on my laptop back... :P
[20:18] <Sho_> mostly because i have a dinner date between two meetings
[20:18] <jensreu> I was going mad trying to figure out how to order stickers and handle shippiung from my apartment...
[20:19] <bshah> Sho_: okay sure....
[20:19] <Sho_> sweet thanks
[20:19] <jensreu> oooh dinner date?
[20:19] <Sho_> not a date-date
[20:19] <Sho_> :)
[20:19] <jensreu> :)
[20:20] <Sho_> ok, that leaves maybe talking about the wiki
[20:20] <Sho_> anyone got stuff to bring up re https://community.kde.org/Plasma/TestDays?
[20:20] <jensreu> Nothing beyond "Wednesday will be Jens doing things on the wiki after the meeting" (the ideal being banners etc there)
[20:21] <Sho_> kk
[20:21] <Sho_> ah, one more note about testcases:
[20:21] <Sho_> after we have done the writing tomorrow, i think the next step will be to actually try the samples out ourselves, to check whether they can be reasonably done
[20:21] <jensreu> YES! Good point!
[20:21] <bshah> I made sure that.. and well found bugs in it too... :p
[20:21] * bshah hides
[20:22] --> frinring_ (~frinring at x5ce54390.dyn.telefonica.de) has joined this channel.
[20:22] <bshah> I've some bugs to report already.. xD
[20:22] <jensreu> Well bshah you're a dev ;) https://www.youtube.com/watch?v=pFRFRwkDsog
[20:22] <Sho_> he
[20:22] <Sho_> *heh
[20:22] <Sho_> my image of the testcase will be the users finding bugs in the /wiki/
[20:22] <Sho_> *testday
[20:23] <Sho_> :)
[20:23] <bshah> :)
[20:23] <jensreu> Yeah that is one big thing: how do we handle incoming info and sorting it?
[20:23] <bshah> perhaps.. instead of allowing user to edit it themself.. we should use perhaps some kind of spreadsheet
[20:23] <bshah> or somehting
[20:23] <Sho_> imho: we're going to need handlers who talk to the users and translate into wiki notes
[20:24] <Sho_> so we will actually need to set start and end times and pen a rough schedule where people will have their "shift"
[20:24] <bshah> or maybe survey.kde.org ?
[20:24] <jensreu> Yeah...
[20:24] <Sho_> we've done stuff like that for booth duty on expos, so i don't see why not here too
[20:24] <jensreu> that will be one rough day for a lot of us :)
[20:25] <bshah> To me.. instead of wiki, we should setup something like survey...
[20:25] --> Clifspeare (~spencer at c-98-192-29-75.hsd1.ga.comcast.net) has joined this channel.
[20:25] <bshah> because.. that would give us parsable data..
[20:25] <Sho_> how do you see the survey working?
[20:25] <Sho_> i have a tough time imagining it
[20:25] <Sho_> i mean let me explain why
[20:25] <Sho_> the thing about bugs is that they are unexpected behavior
[20:25] <Sho_> it's hard to predesign a survey for unexpected behavior
[20:26] <Sho_> although
[20:26] <Sho_> hmm
[20:26] <Sho_> actually ...
[20:26] <jensreu> Well plus we would have to make a survey more or less per test
[20:26] <bshah> [Test if desktop is locked] [x] Yes  [ ] No
[20:27] <bshah> no?
[20:27] <Sho_> it would be cool if we had sort of a testday-specific bug entry form, where you pick which testcase you did, answer some questions like "did you understand the instructions?" and just get a textarea field for free form notes
[20:27] <Sho_> plus a success/failure thing
[20:27] <Sho_> and bshah has a point about parsable data then
[20:27] <Sho_> hmm i like it more and more the more i think about it
[20:28] <jensreu> If you guys are pro; I am pro.
[20:28] <Clifspeare> .info
[20:28] <Clifspeare> whoops
[20:28] <bshah> lets give it thinking for week and discuss it next week on how we want to implement test day..
[20:29] <Sho_> aye
[20:29] <jensreu> sure
[20:29] <bshah> also lets see what community thinks..
[20:29] <Sho_> but yeah i like the survey idea actually
[20:29] <bshah> (include points in notes)
[20:29] <Sho_> you'd pick the testcase, anwer multiple choice questions like "did you find the instructions easy to follow?" and "was the testcase a success?" as well as get fields for text explanations
[20:30] <Sho_> also some interesting info might be "how long did you need to complete the testcase?"
[20:30] <Sho_> then we could actually get some really interesting data, like
[20:30] <Sho_> how long do users actually need to play with widgets
[20:30] <Sho_> and what's the variance for that
[20:31] <jensreu> Yeah as well as random questions like "how did you feel the contact with the devs was?" or something?
[20:31] <Sho_> and a self-ratign questions "How experienced are you?
[20:31] <Sho_> i like it
[20:32] <bshah> :)
[20:32] <jensreu> Hey can someone send me a good screenshot of the "Plasma has crashed" window btw?
[20:32] <Sho_> unfortunately the easiest way to implement this will be something proprietary like google docs
[20:32] <Sho_> but
[20:32] <Sho_> we can talk about that next week
[20:32] <bshah> we may setup something @ survey.kde.org maybe
[20:32] <Sho_> aye
[20:32] * jensreu checking survey.kde.org
[20:33] --> JunkHunk (~ivan at 12.red-2-137-92.dynamicip.rima-tde.net) has joined this channel.
[20:33] <Sho_> one of my friends here works for Ipsos, the world's second largest market research firm, and she's an expert in survey design
[20:33] <Sho_> she can check the crap we cook up
[20:33] <jensreu> Huh... that would be awesome if she could give some pointers
[20:34] <Sho_> complex market research surveys are actually a lot like programming
[20:34] <Sho_> they have conditional branches, termination conditions and jumps to labels, all they lack is loops
[20:34] <Sho_> but i digress
[20:34] <Sho_> :)
[20:34] <Sho_> ok, do we have any more on the docket or are we all met out?
[20:35] <bshah> done i believe
[20:35] <jensreu> Nothing more for me except if someone had a screenshot of the crash window?
[20:35] <bshah> jensreu: kill -ABRT `pidof plasmashell`
[20:35] <Sho_> heh
[20:35] <bshah> this would crash plasmashell :P
[20:35] <Sho_> ok, /meeting
[20:36] <Sho_> keeping etherpad open until :40, then sending out notes
[20:36] <Sho_> thanks for attending!
[20:36] <bshah> \o/
[20:36] <jensreu> Awesome bshah :)
[20:36] <bshah> thanks for hosting
[20:36] <jensreu> Thanks for hosting Sho_