4490 2007-04-25 02:56:06 +0000 Round4 negative tests 2007-04-25 03:00:27 +0000 1 1 1 Unclassified WS-Policy Test Suite FPWD PC Windows XP RESOLVED FIXED http://lists.w3.org/Archives/Public/public-ws-policy/2007Apr/0034.html P2 normal --- 1 asirveda fsasaki public-ws-policy-qa oldest_to_newest 14877 0 asirveda 2007-04-25 02:56:06 +0000 The round4 doc [1] mentions negative tests but doesn't actually says what's involved. We can assume that we're supposed to try to Normalize the xml files and that the engine MUST (should?) fail in some way - but it would be nice if the doc stated this so that everyone knows what "success" means. [1] http://dev.w3.org/cvsweb/~checkout~/2006/ws/policy/interop/WS-Policy-Scenarios-Round4.doc?only_with_tag=Release-1-20070322 thanks -Doug ______________________________________________________ STSM | Web Services Architect | IBM Software Group (919) 254-6905 | IBM T/L 444-6905 | dug@us.ibm.com 14878 1 asirveda 2007-04-25 02:57:13 +0000 I decided to draft up a possible solution for this issue - add a new section 7 as follows: - - - - - - - - - - - - - 7. Negative Testcases As part of this round of testing a set of WS-Policy expressions will be provided. Each one will be invalid in some way. Implementations will be expected to attempt to Normalize each expression and detect that there is an error. It is beyond the scope of this document to define what action each implementation needs to take, however, a successful result will be considered to be achieved if some kind of corrective action is taken. Some possible actions might include: generation of a fault or attempting to fix the expression so it is no longer invalid. - - - - - - - - - - - - - thanks -Doug ______________________________________________________ STSM | Web Services Architect | IBM Software Group (919) 254-6905 | IBM T/L 444-6905 | dug@us.ibm.com Reference - http://lists.w3.org/Archives/Public/public-ws-policy/2007Apr/0037.html 14879 2 asirveda 2007-04-25 02:58:10 +0000 >ACTION-276 Asir to respond to Doug and propose scenarios >paragraph to clarify negative test scenarios. Thank you for a concrete proposal. We are wondering if the test description could be simpler by dropping phrases or words such as error, action, corrective action, generation of fault, attempting to fix, invalid, etc. Here is a suggested editorial revision (no changes to content): "7. Negative Testcases The Round 4 Negative Test directory contains a set of non-conformant policy expressions. These are WS-Policy unit test cases. Implementations are expected to attempt to normalize these expressions and detect non-conformance. A detected non-conformance is considered to be a successful outcome." Regards, Asir S Vedamuthu Microsoft Corporation Reference - http://lists.w3.org/Archives/Public/public-ws-policy/2007Apr/0060.html 14880 3 asirveda 2007-04-25 03:00:27 +0000 Reference: http://www.w3.org/2007/04/18-ws-policy-minutes.html#item05 cferris: relates to the negative cases; how they are detected ... no objections to closing this issue with the following resolution <cferris> ACTION: Asir to open and close new issue on test scenarios related to negative testing and the proposal here http://lists.w3.org/Archives/Public/public-ws-policy/2007Apr/0060.html [recorded in http://www.w3.org/2007/04/18-ws-policy-minutes.html#action02] <trackbot> Created ACTION-282 - Open and close new issue on test scenarios related to negative testing and the proposal here http://lists.w3.org/Archives/Public/public-ws-policy/2007Apr/0060.html [on Asir Vedamuthu - due 2007-04-25]. <cferris> RESOLUTION: Issue XXXX closed with resolution in http://lists.w3.org/Archives/Public/public-ws-policy/2007Apr/0060.html