Meeting minutes
Agenda
<Ege> https://
* Continue the testing conversation wrt. profile
* Overall testing structure
* manual vs. automated testing
Ege: suggest to talk about testing structure
<ML setup the Agenda>
Minutes
Kaz: updated some off topics and speaker names
Lagally: any objections?
no
Overall testing structure
Ege: I can share my screen
<ML shows the current folder structure of the testing repo>
<Ege starts to share and explains the folder structure in the testing repo>
<kaz> wot-testing repo - data area
Ege: data folder is used to copy all test files in one place
<Ege shows the README of the July 2022 TestFest>
<kaz> wot-testing repo - 2022.07.Online area
Ege: for the September TestFest event we miss to provide a reated README
… we should provide all information in a single place (in testing repo)
Lagally: What is in the data/input_2022 folder?
Ege: all the data that is required to provide the implementation report
Lagally: Where the implementation report generated?
Ege: Locally on McCool's computer
Lagally: This is not transparent.
Ege: agree
Kaz: As I've been asking you all, we need to generate a clear guideline on how to submit test results. So I basically agree with Lagally's points. However, we should once clarify our issues and our existing resources first.
… And then we (including McCool) should discuss how to solve the issues based on the existing resources next.
… to improve the situation
<Ege creates an issue>
Sebastian: This is a known situation since a while, however, the script simple takes the input from the testing folder and creates the IR webpage
… we should continue this discussion when McCool is available again
Lagally: we should simplify the process befor the next TestFest
Kaz: We've been skimming the existing resources for 5 mins or so. So I'd suggest we record all those resources, and clarify what is done by whom (which repository and which tool on which folder based on which guideline)
Ege: we should avoid to ask developer to provide there inputs in different repos
<ML updates issue 478>
Issue 478 - Problems with Current Testing
Lagally: I would not see some issues here. However, I would prefer to see a simple description about the overall process
Kaz: We should create another list on the existing resources. We should handle 2 topics in parallel
Lagally: is the validation of TD inputs documented?
<Ege> /github.com/w3c/wot-testing/blob/main/.github/workflows/validation.yaml/
Ege: its based on the TD playground
Daniel: Last time the manual csv was changed during the testing week. We should avoid this for the next time
… should be also listed in the issue 478
<kaz> s/manual csv/manual.csv/
Kaz: we have only 10min left, we should clarify what should be clarified, and ask people to give inputs for the two GitHub Issues, one on the problems and another on the existing resources
<mlagally__> Issue 478 - Problems with Current Testing
<mlagally__> Issue 479 - Describe and consolidate existing process and tooling
Profile Testing
wot-profile PR 324 - December Testfest Preparation - implementation cross reference and xref tool
Sebastian: the content of the PR is fine, however, should go to testing repo
Lagally: Where should it go exactly?
Ege: this should go to the data folder
… data/input_2022/Profile
Kaz: are you ok, to move the content of the PR to the testing folder?
Lagally: I'm ok to provide the content to a different place, however, it should be clarified what the developer should do.
kaz: In that case, we can merge this PR itself today, and register this as an existing problem to Issue 478 and as an existing resource to Issue 479. Then we should continue the discussion on how to improve the current messy situation next week with McCool during the Testing call.
[ no objection to merge wot-profile PR 324 itself; However, this resource to be registered to the existing problem list (wot-testing Issue 478) and the existing resource list (wot-testing Issue 479). Also could be move to wot-testing repository after some more discussion during the next Testing call. ]
[adjourned]