<kaz> scribenick: dape
<kaz> Agenda: https://www.w3.org/WoT/IG/wiki/Test_WebConf#Agenda
McCool: Any additions to the
agenda?
... we have plan
... 1. step, markup spec with normative assertions
<kaz> plan
McCool: RFC 2119 keywords
... anyone started with normative assertions ?
Kaz: think Sebastian and Zoltan started with it...
McCool: Section "Conformance" needs
to be added
... particular words MAY; MUST, ... etc need to be used
<kaz> scripting api draft
Kaz: Zoltan added conformance section as Section 9
<kaz> section 9
McCool: will be protocol binding separate document till to the end?
Koster: plan to provide normative statements for TD
McCool: for security it is the
same
... assume normative statements are a big chunk
McCool: 1. specification validation
(required by W3C)
... spec implementable and consistent
... 2. Implementation testing
... check implementation satisfy requirements
... a) scripting API
... b) TD validation
... general useful tool
... c) network interface described by TD
... would like to have an automated thing that tests this
... wonder which is actually necessary
Koster: do they need to be reference implementation
kaz: usually we don't have
"reference" implementations
... just need 2 implementations for each feature
McCool: 3. category is interop
testing
... difficult problem
Koster: we can have reference
implementation
... reference implementation helps to get started
McCool: Could be online tool
Koster: PlugFest could be venue
Kaz: usual W3C specifications are related to Web browsers
... so the W3C Testing Framework mainly handles Web browser testing
... on the other hand, we need server side also for WoT
... that's why, reference impl for server side is useful
Daniel: Wonder whether w3c testing only means spec validation ?
Kaz: the main purpose of "testing" for Candidate Recommendation transition is spec validation
McCool: we can add levels
... and we missed security testing
Koster: there is our spec we test against but there is more (e.g., in security)
McCool: we can just have
recommendation
... provide framework how to test
Kaz: maybe we might want to rename "security testing" to "advanced testing" which include security testing
McCool: I believe we have statement
in charter that requires us to test security
... in our tooling we can point to how to improve security
Kaz: ah, that's true
<kaz> WG Charter, fyi
[[
In order to enhance the security of WoT systems, we will also generate and implement a security testing plan which will include both functional and adversarial testing of the proposed standards and their implementations. We will only recommend an implementation of the proposed standards for use in production once it has passed such testing.
]]
McCool: I think we need validation
tool
... "functional testing" could be basic authentication
...
... hope we can reuse things that already exists
Matthias: w.r.t. CoAP there is some
penetration testing
... for CoAP we broke it down by feature
... test it pair-wise
... also test it with security enabled (difficult and maybe
also impossible with all combinations)
McCool: test common
combination
... w.r.t. reference implementation
... we need 2 separate implementations
... for example in the case of scripting we might just have a
limited number
... need to focus breaking things down by feature
... after that we can create table with common
combinations
... maybe PlugFest could be used to implement test cases
Koster: in physical realization:
we need system under test and test tool
... can have scripts for exposed thing and consumed thing using
node-wot
McCool: for PlugFest (or after) I
would like to have a skeleton testing framework in place
... implies that before the PlugFest we need test description
(before that test assertions)
Koster: need to talk about how to
implement
... what new tools we need
McCool: Next, will create section on
wiki for "Actions"
... normative assertions
... test descriptions
... each group should think about this
... we might want to have MD template
McCool: Q: Is there any test expert who could help us/me?
Matthias: mainly time issue
... talked to Ege Korkan
... he could volunteer
... will ask him
Koster: should also re-consider time slot
McCool: Kaz, can you do a doodle?
Kaz: will do
... BTW, we might want to think about
collaboration between PlugFest and Testing
... PlugFest call should start again
... possibly done within this testing call, but maybe need a
separate call
<kaz> Koster: right. plugfest has a bit different logistics cycle
<kaz> [adjourned]