PlugFest WebConf/2018

Agenda 10.12.2018 - Test Fest Day 1

Review Implementation Report tooling improvements
- Implementation report tooling
  - https://github.com/w3c/wot-thing-description/pull/314
  - https://github.com/mmccool/wot-thing-description/tree/updated-test-results
- Still need implementation.html files for some implementations
- TD Validation tooling
  - https://github.com/egekorkan/thingweb-playground/tree/assertionTest
- Result data
Testfest Logistics
- - Replace all the WoT calls week of Dec 10 (next week):
  - Monday: Script/Security slots (2h)
  - Wednesday: Editors/Main/Testing slots (3h)
  - Friday: TD slot (2h) for wrapping up
  - Details: https://github.com/w3c/wot/tree/master/testfest/2018-12-online/
CR Criteria, Requirements, and Resources
Assertion issues
TD Spec version
- How to integrate testing scripts with tagged version of TD spec (or branch?)
- Diff between today's Editor's Draft and last Working Draft:
Additional planning meetings?

Review Implementation Report tooling improvements
- Still need implementation.html files for some implementations
PlugFest reports
TestFest planning
- Schedule:
  - Week of Dec 10
  - During Mon (2h; 9pm-11pm JST) and Wed (3h; 9pm-12pm JST) call times
  - Fri (4pm-6pm JST) TD call to finalize results
- Data collection
  - Collect all TDs
  - Generate assertion results.csv manually or from TDs
  - Collect interop.csv data
  - Might still be useful to have preparation.md/results.md
- VPN
  - OpenVPN server and VLAN, at least for self-clients
- Wiki page
  - Kaz to set up template, add links from other places
  - McCool will fill in logistics data
  - Will re-use existing WebEx
Testing Subcommittee

Review Implementation Report tooling improvements
Review action items for collecting data
- Implementation descriptions
- Assertion test data results
- Proposed new assertions (codify preparation.md results)
PlugFest reports
- Panasonic, Fujitsu and Hitachi; others?
Testfest planning
Architecture draft testing and validation
Scripting API testing and validation

Update on TD Test plan and Implementation Report draft
- Implementation Report template
- Assertion extraction
- Merging implementation results via CSV files
- TO DO: Use headers in CSV files, create and merge PR with TD
- TO DO: template for implementation description(s)
- TO DO: Group assertions by category and section
- TO DO: Add file to capture "extra" assertions
PlugFest reports
- got messages from Panasonic, Fujitsu and Hitachi about their reports
- could be initial starting point for our implementation report
- need for description on directory (discovery/registration/lookup)
Tooling
- Test framework: Dave Raggett
- Should split the work from possible advanced effort for an automatic mechanism (e.g., Web Platform Test Framework)
- How to collect, organize and present results?
Testing Meeting in Dec Planning
Other

TPAC recap and planning
- https://github.com/w3c/wot/blob/master/testing/wot-test-plan-tpac-10-2018.pptx
Timeline
- Rough draft of "Implementation Report Plan" (CR entry doc):
  - Can we get a template?
  - Like plugfest report... but need to relate to assertions
  - Target: End of Nov (McCool takes action to do for next meeting)
- When a candidate release needs to be ready: end of Dec
- When things need to be frozen: end of Jan
- When tests need to be completed: before CR submission
- When CR is to be submitted, along with all tests: end of Feb
- Practical dropdead-deadline: end of March
Logistics
- When do we meet? Do we do a doodle?
- Separate repo for test suite?
- Do we need tests for the architecture document?
- Online "plugfest" with a focus on testing? When?
- How do we handle pen testing?
Test requirements for PR transition
- Want these done before CR submission, ideally...
- https://github.com/w3c/wot/blob/master/testing/requirements.md
- 1. generating assertions based on the specs (TD and WoT Architecture)
- 2. checking the assertions using the existing implementations
- 3. getting more than implementation for each feature/assertion.

Quick updates
Online PlugFest
- Pull requests
- Review any new documents
- Scheduling discussion
- Testing and validation update
Other

Quick updates
Online PlugFest
- Online PlugFest preparation document => https://github.com/w3c/wot/tree/master/plugfest/2018-sept-online
- Review of new preparation document submissions
- Review availability table and updates
- Please draft a preparation-YOU.md file in plugfest/2018-sept-online
Scenarios
Target Features
Testing

Quick updates
TPAC Logistics review (see Main Call Agenda)
Online PlugFest
- Online PlugFest preparation document => https://github.com/w3c/wot/tree/master/plugfest/2018-sept-online
- Please draft a preparation-YOU.md file in plugfest/2018-sept-online
Scenarios
Target Features
Testing

Quick updates
Online PlugFest
- Online PlugFest preparation document => https://github.com/w3c/wot/tree/master/plugfest/2018-sept-online
- Please draft a preparation-YOU.md file in plugfest/2018-sept-online
Scenarios
Target Features
Testing

Merged with main call.