PlugFest WebConf

From Web of Things Interest Group

The WoT Plugfest/Testing task force is responsible for organizing the Web of Things (WoT) Plugfests and also for organizing general testing procedures, data collection, and implementation report tooling for WoT deliverables.

Call Logistics

  • Time: Wednesdays at 9am US Eastern / 3pm Europe / 11pm Japan
    • NOTE: We may start at 15 past if the main call goes long.
  • Zoom: please refer to W3C Calendar
  • Note this is now a separate call from the Main call!

IRC

The IRC is used for the minutes, speaker queue, and sharing links etc.

Actions

  • Organize wot-test repo (McCool)
    • Copy over existing material from wot repo
  • Generate feature lists to abstract assertions, cover "informative" features, like use of MQTT, etc.
    • Copy over and improve implementation report generation scripts from wot-thing-description
  • Improve and generalize implementation report generation scripts (McCool)
  • Capture implementation descriptions in formal CSV tables
    • Include and generate report on interoperability testing
  • Plan Helsinki plugfest
    • Include formal data capture plans, including for conformance and interoperability
  • Finalize technical requirements
  • Set up remote access links, eg WebEx, Google hangouts, etc.
  • Capture more examples that can be shared
    • Link to wotify.org and other examples from wot-testing
    • Add links to formal implementation CSV files
    • Capture information on both producers and consumers, e.g. mashups in Node-RED
      • Even just screenshots
  • Improve test tools
    • E.g. scripts to "exercise" a device, use UI to confirm actions, etc.
    • See https://github.com/tum-esi/testbench - connect to impl/plugfest report
    • Standardize setup for implementations so they can be reproduced easily
    • Use semantics to guide testing; special vocab for testing?
  • Create procedure/template for Plugfest implementation preparation
    • Passwords, network setup, how to document implementations (incl authentication), behavior, etc.
    • Add link to documentation from CSV table, archive documentation in wot-testing

Action Items

Agendas

Agenda 31.05.2023

Chair: McCool

Scribe: Kaz

Agenda 24.05.2023

Chair: McCool

Scribe: Ege

Agenda 17.05.2023

Chair: McCool

Scribe: Kaz

Agenda 03.05.2023

Note: During Golden week, special case.

Chair: McCool

Scribe: Kaz

  • Minutes:
    • Defer until next testing call
  • Review new Test inputs
  • Planning
    • IR Report updates
    • Expected new results
    • Review priorities and deadlines
    • Define next steps
  • AOB

Agenda 27.04.2023

Testfest - Day 3 - Using Architecture call logistics.

Chair: McCool

Scribe: Ege

  • Minutes:
    • Defer until next testing call
  • Review new Test inputs
  • Planning
    • IR Report updates
    • Expected new results
    • Review priorities and deadlines
    • Define next steps
  • AOB

Agenda 26.04.2023

Testfest - Day 2 - Using Main (testing) call logistics.

Chair: McCool

Scribe: Ben

Agenda 25.04.2023

Testfest - Day 1 - Using Use Case call logistics.

Chair: McCool

Scribe: Ege

Agenda 19.04.2023

Chair: McCool

Scribe: Kaz

Agenda 12.04.2023

Chair: McCool

Scribe: Ege

Agenda 05.04.2023

Chair: McCool

Scribe: Cristiano

Agenda 29.03.2023

Chair: McCool

Scribe: Ege

Agenda 22.03.2023

Chair: Ege

Scribe: Cristiano

Agenda 15.03.2023

Chair: McCool

Scribe: Ege/Kaz

Agenda 22.02.2023

Chair: McCool

Scribe: Luca

Agenda 08.02.2023

Chair: McCool

Scribe: Cristiano

Agenda 01.02.2023

Chair: McCool

Scribe: Ege

Agenda 25.01.2023

Chair: McCool

Scribe: Ege

  • Minutes
  • Testimonials
    • Draft email explaining template, giving an example, for forwarding to AC rep
  • Implementation Reports
    • Tooling Issues
    • Gaps
    • Resolution plan
    • Another Testfest?
  • AOB

Agenda 21.12.2022

Chair: McCool

Scribe: Ege

Agenda 15.12.2022

Testfest - Day 4

Chair: McCool

Scribe: Kaz

Agenda 14.12.2022

Testfest - Day 3

Chair: McCool

Scribe: Lagally

Agenda 13.12.2022

Testfest - Day 2

Chair: McCool

Scribe: Ege

  • Minutes
  • Quick Updates
    • Already merged PRs - will review below
  • Active Devices
  • Implementation Report Updates
  • Review PRs
  • Next Steps
  • AOB

Agenda 12.12.2022

Testfest - Day 1

Chair: McCool

Scribe: Fady

Agenda 07.12.2022

Chair: McCool

Scribe: Fady

  • Review Minutes
  • Test fest preparation
    • Review instructions
    • CSV change log script
    • CSV format updates
  • AOB

Agenda 30.11.2022

Chair: McCool

Scribe: Sebastian

  • Review Minutes
  • test fest preparation
    • logistics
    • testing data organization (CSV files, etc)
    • scenarios

Agenda 16.11.2022

Chair: Lagally

Scribe: Sebastian

  • Overall testing structure
  • Continue the testing conversation wrt. profile
  • manual vs. automated testing
  • test fest preparation - scenarios


Agenda 09.11.2022

Chair: Lagally

Scribe: tbd.

  • Review Minutes
  • Implementation Reports
    • Check at risk items
  • Testfest/Plugfest planning
    • Schedule: Dec 12-16, times tbd.
    • Goals
    • Scenarios
  • AOB

Agenda 02.11.2022

Chair: McCool

Scribe: Kaz

  • Review Minutes
  • Implementation Reports
  • Testfest/Plugfest planning
    • Schedule: Dec 12-16, times
    • Goals
    • Scenarios
  • AOB

Agenda 12.10.2022

Chair: McCool

Scribe: Ege

Agenda 07.10.2022

Testfest

Chair: McCool

Scribe: Sebastian

Agenda 06.10.2022

Testfest

Chair: McCool

Scribe: Cristiano

Agenda 05.10.2022

Testfest

Chair: McCool

Scribe: Ege

Agenda 04.10.2022

Testfest

Chair: McCool

Scribe: Lagally

Agenda 03.10.2022

Testfest

Chair: McCool

Scribe: Cristiano

Agenda 28.09.2022

Chair: McCool

Scribe: Cristiano

Agenda 31.08.2022

Chair: McCool

Scribe: Kaz

  • Review Minutes
  • Architecture IR
  • Post-TPAC Testfest
    • Week of Sept 26
    • Focus on Profile *testing*
    • Possible "consumer" testings for all TDs (e.g. node-wot and Node-RED trying to read all TDs).
    • Architecture should not need *additional* testing (in theory)
  • AOB

Agenda 24.08.2022

Canceled.

Agenda 10.08.2022

Chair: McCool

Scribe: Ege

  • Review Minutes
  • Planning
    • Prep for CR Transition
      • Additional testing prior to CR/PR -> data/input_2022
      • Updated instructions (README, last testfest)/email (McCool)
      • Periodic merging and IR updates: daily
      • How to deal with changes to manual.csv e.g. when TD spec changes? Warnings from tooling for missing/obsolete/auto-manual/new changes?
      • batchValidation.sh had many missing -m options
      • still need many implementation descriptions (McCool drafts... then implementers can do fixes via PR suggestions)
    • Pre-TPAC Meeting
      • Sept 7 - resolutions for CR transition
      • Sept 5 should be deadline for CR transition IR and test results
    • Post-TPAC Testfest
      • Options: Sept 26-30; Oct 3-7; Oct 10-14; Oct 17-21 -> doodle
  • Review recent testing results
  • AOB

Agenda 03.08.2022

Chair: McCool

Scribe: Fady

Agenda 29.07.2022

Testfest - Day 5

Chair: McCool

Scribe: Cristiano

  • Review Minutes
    • deferred
  • PRs
  • Implementation Reports
  • Next Steps
    • Implementation Descriptions
    • Testimonials
    • Update todo.csv
    • Manual assertions
    • Assertion Review
      • Architecture
      • Security and Privacy
  • AOB

Agenda 28.07.2022

Testfest - Day 4

Chair: McCool

Scribe: Ege

  • Review Minutes
    • deferred
  • PRs
  • Implementation Reports
  • Assertion Review
    • Architecture
    • Security and Privacy
  • AOB

Agenda 27.07.2022

Testfest - Day 3

Chair: McCool

Scribe: Sebastian

  • Review Minutes
    • deferred
  • PRs
  • Implementation Reports
  • Assertion Review
    • Architecture
    • Security and Privacy
  • AOB

Agenda 26.07.2022

Testfest - Day 2

Chair: McCool

Scribe: Sebastian

  • Review Minutes
    • deferred
  • PRs
    • PRs with new data already merged
  • Implementation Reports
    • TD IR updated yesterday
    • Now running validation script on new inputs
  • Assertion Review
    • Architecture
    • Security and Privacy
  • AOB

Agenda 25.07.2022

Testfest - Day 1

Chair: McCool

Scribe: Kaz

  • Review minutes
    • Will defer all until next week
  • Logistics
  • PRs
  • Planning
    • Security and Privacy
    • Architecture
  • AOB

Agenda 20.07.2022

Chair: McCool

Scribe: Ege

Agenda 13.07.2022

Chair: McCool

Scribe: Philipp

Agenda 06.07.2022

Chair: McCool

Scribe: Cristiano

Agenda 29.06.2022

Chair: McCool

Scribe: Fady

  • Review minutes
  • PRs
  • Plugfest logistics
    • Move to Week of July 25
  • Implementation Reports
    • Manual test fixes in Discovery
    • TM testing issues still?
  • To Dos
    • Still missing implementation descriptions and testimonials

Agenda 22.06.2022

Chair: McCool

Scribe:

Agenda 15.06.2022

Chair: McCool

Scribe: Fady

  • Review Implementation Reports
  • To Dos

Agenda 10.06.2022

Testfest Day 5

Chair: McCool

Scribe: Fady

  • PRs
    • Last-minute updates
  • Implementation Report Drafts
    • TD
    • Discovery
  • At-Risk Items
    • Markup
    • Tables
    • Responsibilities
  • AOB

Agenda 09.06.2022

Testfest Day 4

Chair: McCool

Scribe: Ege/Kaz

  • Updates
    • Implementation Report Drafts
    • Gaps
  • TD PR on testing
  • Testing PRs
  • Issues
  • AOB

Agenda 08.06.2022

Testfest Day 3

Chair: McCool

Scribe: Fady

  • Updates
    • Implementation Report Drafts
    • Gaps
  • TD PR on testing
  • PRs
  • Issues
  • AOB

Agenda 07.06.2022

Testfest Day 2

Chair: McCool

Scribe: Ege

  • Updates
    • Implementation Report Drafts
    • Gaps
  • PRs
  • Issues
  • AOB

Agenda 06.06.2022

Testfest Day 1

Chair: McCool

Scribe: Jan

  • Organization
    • TD is ready, see README
    • Discovery still needs updates to template.csv; will be done by tomorrow
  • Instructions
    • Links to example PRs and issues (for work on addressing gaps)
  • PRs
    • Intel examples in TD
  • Issues
    • Intel example for gap-issue
  • AOB

Agenda 01.06.2022

Chair: McCool

Scribe: Jan

  • Review PF minutes
  • TestFest org
    • Calendar entries, WebExes, etc
    • Goals:
      • CR-Ready implementation report for TD
      • CR-Ready implementation report for Discovery
    • To Do (see README for event)
      • Testing inputs
      • Manual assertions
      • Implementation Descriptions
    • Documentation
  • PRs
  • Issues
  • Summer Testfest?
  • AOB

Agenda 25.05.2022

Chair: McCool

Scribe: Ege

Agenda 18.05.2022

Chair: McCool

Scribe: Fady

Agenda 11.05.2022

Chair: McCool

Scribe: Fady

Agenda 04.05.2022

Chair: McCool

Scribe: Fady

Agenda 27.04.2022

Chair: McCool

Scribe: Philipp

Agenda 20.04.2022

Chair: McCool

Scribe: Ege

Agenda 13.04.2022

Chair: McCool

Scribe: Philipp

Agenda 06.04.2022

Chair: Fady

Scribe: TBD

Agenda 30.03.2022

Chair: Fady

Scribe: Kaz

Agenda 23.03.2022

Chair: Fady

Scribe: Cristiano

Agenda 09.03.2022

Chair: Fady

Scribe: TBD

Agenda 02.03.2022

Chair: Fady

Scribe: Ege

Agenda 23.02.2022

Chair: Fady

Scribe: McCool

Agenda 16.02.2022

Chair: Fady

Scribe: Ege

Agenda 09.02.2022

Chair: Fady

Scribe: Cristiano

Agenda 02.02.2022

Chair: Fady

Scribe: McCool

Agenda 26.01.2022

Chair: Fady

Scribe: Kaz

Agenda 19.01.2022

Chair: Fady

Scribe: Cristiano

Past Content