Meeting minutes
Minutes
<sebastiankaebisch> Mar-17
any objection
no
Policy for backward compatibility check
<McCool> pproposal: we will use previous TD 1.0 submissions for TD1.1 (compatibility) testing if the only change needed is to add the context URL to the TD 1.1 version, in addition to the TD 1.0 context URL..
we need to add TD 1.0 namespace at the first place
<McCool> proposal: we will use previous TD 1.0 submissions for TD1.1 (compatibility) testing if the only change needed is to add the context URL to the TD 1.1 version, in addition to the TD 1.0 context URL.
Kaz: I'm ok with changing the namespace. For ECHONET is also ok. So I'd suggest we check their TDs with both 1.0 context and 1.1 context for backward compatibility check.
McCool: yeah
McCool: import all TDs from 1.0 TestFest, add v1.1 namespace
<McCool> would also be possible to keep them separate in the implementation report, under a "compatibility testing" column
any objection to the resolution?
no
RESOLUTION: we will use previous TD 1.0 submissions for TD1.1 (compatibility) testing if the only change needed is to add the context URL to the TD 1.1 version, in addition to the TD 1.0 context URL.
Testfest
https://
PR 270
McCool: let's start with the PR that passed the test
https://
McCool: going to merge
PR 271
https://
McCool: TDs already aligned the namespace for the v1.0 and v1.1 case
McCool: going to merge
PR 272
https://
Lagally: its about pumps that are online
… it also have event implementation in there
Lagally: should be compatible to v1.0 and v1.1
just see there is dataResponse which is available in v1.1
Ege: I don't see any problem here. Validator just follows the TD 1.1 spec
McCool: lets merge it . Maybe is a test case for the validator
any objection?
no
PR 274
https://
Ege: there are some problems with the version field
PR 273
https://
Ege: strange that the validator complaints about the context. It seems be correct
PR 276
https://
Implementation reports
Ege: test looks good. no fails in there so far
<Ege> my proposal: if you see the following "data['@context'][0] should be equal to one of the allowed values, data['@context'] should be string, data['@context'] should be equal to one of the allowed values, data['@context'] should match exactly one schema in oneOf" please merge
Lagally: I'm wondering if it is ok when we manipulate the TD of the submitter without asking before.
Expected proposal for PRs in general
Kaz: It's not clear what the "proposal" above is about, one of the PRs so far or all the PRs in general.
McCool: Ege will go over PRs 273, 274, 275 that have problems and try to solve the problem (if it is a validator problem), or make a review (if it is an input data problem)
<kaz> (We got clarification of Ege's "proposal" above, and so copy it here again
<Ege> my proposal: if you see the following "data['@context'][0] should be equal to one of the allowed values, data['@context'] should be string, data['@context'] should be equal to one of the allowed values, data['@context'] should match exactly one schema in oneOf" please merge
Implementation reports (revisited)
<MM shows rendered version of PR https://
McCool: we have good results from discovery
<MM shows the result of the discovery>
McCool: the validation tool for discovery is live testing
if understand correctly so far there are many non implemented features listed, however, this is not correct since the validation tool does not work correct
McCool: yes, we will check what is the issue here
Lagally: is there a base line from the TD validation tool which we can reuse?
<MM created a csv report and will submit it via a PR>
<McCool> the legal values for test results are "pass", "fail", "not-impl", and "null". Other values, like "passed" will be ignored (treated as null)
this template is for the architecture test report
McCool: please take a copy of the template file and modify your result
<MM showed an example for intel>
<McCool> https://
PR 277
Profile validation results 18.3.
McCool: (goes through the changes)
Lagally: (adds clarification)
… used the official JSON Schema for validation with a couple of additional constraints
McCool: merge this?
… useful to look at the results
(merged)
PR 278
PR 278 - Fix validation errors
Ege: tested in my fork
McCool: good to have it
… can you re-validate everything?
… will merge this PR itself
Ege: if there is any problem, will be shown as failed
McCool: ok
(merged)
PR 274 (revisited)
(problem fixed, and merged)
[Testfest part adjourned; 7min break]
Plugfest
agenda for the Plugfest part
McCool: I want talk about vlan, is there something else you want to discuss?
Lagally: do you have any feedback regarding Oracle devices?
… we can even discuss this offline
McCool: there are no issue added, this means no additional projects
VLAN for Plugfest
McCool: has anybody tried vlan?
Lagally: not me
<kaz> Setup instruction for SoftEther VPN client
McCool: I can't set the dhcp client
… try to debug it but with no success
… maybe is something about this pf2020 id
toumura: not sure
McCool: is there another place were I struggled the bridge
… I needed to know the server ip address
but I can't find it
… plus some steps are actually only needed for windows
… i fix also a couple of things
… some instructions for the AP would be useful
… just the name of the tool would be sufficient
… I haven't tried to do the no-bridge version
McCool: I want also to test mdns stuff
… I hope that we can keep things up for a month or so
Kaz: I suggest to file issues regarding this kind of problems
McCool: well, it is more a tool problem that test fest
… even if we can add an issue about VLAN set up process
Kaz: I mean not only this specific problem with VPN setting but all the problems from this whole week in general. Given today is the last day of our Testfest/Plugfest this time, probably we should use the last 10-15 minutes to wrap up all the problems and think about how to resolve those problems. Probably we need another follow-up meeting to deal with them.
… and derive some results
profile testing
<kaz> Oracle Profile results
Lagally: as you know we are working on the profile specification
… I want to experiment more with what we have right now.
McCool: you could document the process that you took to evaluate TDs
… in the Profiles directory
<michael lagally presents his work on Profile testing>
Lagally: dittos TDs have wrong time format, I filed a issue about it
… I have a WIP profile json schema
McCool: so you don't have all the profile constraints in there?
Lagally: not yet
Ege: I would recommend to not copy and paste the TD schema manually. It will be probably updated
… instead I would create another schema
… with just restrictions about the profile
<cris> +1
Lagally: thanks, I agree
… I hoping the experts of JSONSchema validation would help Profile taskforce to define the proper schema
Lagally: I agree
… we can do a double pass
… TD schema first than profile later
Kaz: this discussion implies that we might be going to need a dated version of schema similar to w3c publication process
McCool: I have a date version already
… tecnically is the process that calls two schemas. You don't need an include
Lagally: but the TD scheme include nice defintions
… that we can re-use
Cristiano: the problem is pretty similar to what we are dealing within the protocol binding templates
… we are expirimenting with validation with additional schemas
wip schema for modbus template https://
Ege: is there any thing that implements the new actions
Lagally: yeah blue pump has the new model
Ege: the validation schema is not an implementation report, it is only validation
Lagally: in the next pf calls we will probably talk about validation
McCool: I want to continue this discussion in a more generic way
wrap up
McCool: It would be nice to have a profile testing issue
… I'm also creating a plug fest TODO issue
… with a list of things that we couldn't test
… ege can you get some behavioral testing ?
Ege: I'm overbooked with other testing tasks
Lagally: behavioral testing can be done manually
McCool: some assertions might be done automatically
Kaz: I would suggest we split the todos into TestFest and PlugFest
… any other thing to do as follow up?
Kaz: we should clarify expectations of collaborators
Lagally: could we continue on Wednesday ?
McCool: good idea
Issue 280 - Testfest 2022-03 Followup
[adjourned]