<McCool> https://www.w3.org/WoT/IG/wiki/PlugFest_WebConf#Agenda_21.11.2018
<scribe> scribenick: mkovatsc
<kaz> PR 290
McCool: new section on what counts as
implementation
... added implementation section for Intel
... mk added implementation section for Siemens
... need to figure out how implementation is counted when a sw
component is shared
<McCool> https://github.com/mmccool/wot-thing-description/tree/updated-test-results/testing
McCool: now using terms "TD Consumer"
and "TD Producer"
... added several CSV files
... changes to the document need to go into template.html
... it imports other files
... implementations folder contains HTML blocks for each
Member, listing and describing their implementations and a
leading testimonial
... please give each implementation a unique id
("impl-<member>-<impl>")
<Zakim> kaz, you wanted to mention we should identify if each implementation is based on different code-base
Kaz: two points:
... 1. implementation should relate to specific code-base
... 2. should we really use this repo for this?
McCool: code-base aspect is
known
... my repo is only used to speed up things (merge into master
requires review)
... testing focuses on TD, thus it is in TD repo
Kaz: you could become moderator for TD repo to speed up
McCool: it is more about
authorization by the Editors, as spec was touched by PR
... best we wait until PR merge before people open PRs for impl
information
... best start by retrieving the template from my repo and make
PR to master
Toru: "TD Producer" has the notion of an automatic generator. What about manual TDs?
<kaz> Matthias: we're learning more and more about the actual procedure now
<kaz> ... the current assertion focused on whether it's correct or not
<kaz> ... need to see proper terms
<inserted> kaz: During the WoT
Chairs call today, I also mentioned "consumedThing" and
"exposedThing" as possible terms, but we can revisit the concrete
terminology later.
... The point at the moment is the terminology for the
Implementation Report should be consistent with the definition
within the WoT Architecture.
McCool: I understand my implementation with a manual TD still a unique code base and an implementation that produces a TD
Lagally: how stable is the tooling?
McCool: changes are planned to be
backward-compatible
... overall the testing plan is still "in flux", as we need
more information
... plan is to stabalize in two weeks for the online
TestFest
... data files will not change much, assertions will need to
change
Lagally: How can I play with this to check the resulting document for correctness?
McCool: adapt data files. script is supposed to pull them in automatically at some point.
Lagally: it is confusing that there are multiple testing directories
Matthias: note that w3c/wot" repo is IG!
McCool: IG material is general planning. Deliverable repos have concrete test documents for the REC process.
Lagally: is it a valid assumption that w3c/wot is not required for REC process?
McCool: yes
... using CSV for test results as they are easy to edit and
render nicely on GitHub
... CSVs can be broken down in multiple files, as each entry
uses an ID
... tool will colorcode the results
... red is critical, e.g., only one implementation for
security
... prefix your test results with your member identifier (e.g.,
"intel-")
Ege: having reports per assertions
will require to change the TD Playground, so that each
assertion is in its own JSON Schema
... Playground can only check syntax
Matthias: statements during today's
chairs call said that tests must focus on behavior, e.g., if a
parser must ignore unknown terms or must throw an error
... need to clarify what kind of assertions are actually
required from us
Kaz: I don't think we confirmed
that during the chairs call today.
... the W3C process document requires us to show evidence for implementability
McCool: there is a file
"extra-asserts.html" to file new/changes assertions
... @all, please have a look at your implementations in the
context of testing and report to us what you have
... need to think about interop and penetration testing and how
it fits into the report
... (something about two-stage process for the test plan
impl)
Kaz: As I (repeatedly) mentioned, CR
exit criteria defined by the W3C Process is showing that a
specification is sufficiently clear, complete, and relevant.
... On the other hand, I can understand people are interested in
testing interoperability between various implementations itself
and describing concrete behavior of each implementation (e.g., TD
consumer/TD producer).
... That's why I'm planning to talk with W3M to confirm what is
required from us
McCool: charter states security
testing
... please clarify what we need to provide
Ege: isn't interoperability shown when both client and server fulfills the assertions?
McCool: interoperability would go as
report into appendix
... AoB?
Toru: Could you please check my PR?
<inserted> wot PR 596
McCool: merging
... EoM