Workshop on Quality
Assurance at W3C
ETSI PTCC
Protocol and Testing Competence Centre
anthony.wiles@etsi.fr
About ETSI
- European Telecommunications
Standards Institute
- Develops a wide range of
specifications as Europe’s contribution to world-wide
standardisation of telecommunications and associated
domains
- Over 600 Members, 4000
experts
- Successful ETSI technologies
include
- GSM, 3G UMTS, HiperLan, TIPHON
(VoIP), DECT, TETRA, IN, ISDN (and B-ISDN) etc.
- Visit our website
- http://www.etsi.org
Producing Better
Standards
- The ETSI watchword is
INTEROPERABILITY
- A well-defined, accurate and
unambiguous standard is key to the development of products based on
that standard
- The ETSI membership believes that
the pragmatic validation of standards and testing of products based
on those standards greatly enhances the chances of
interoperability
- Pragmatic approach
- should not delay the standardisation
process
- and give real
added-value
- ETSI has two unique resources for
validation and testing
- PTCC (Protocol and Testing
Competence Centre)
- Bake-off Service
Verification
- Formal (automated) checking of the
internal consistency and correctness of the protocol
- Difficult (impossible) to apply on
informal specifications
- SDL tools provide powerful
simulation environments
- discover protocol deadlocks,
livelocks, unreachable states, implicit consumption of signals,
unused data etc.
Validation
- Checking that the protocol really
provides the (logical) functionality expected of it
- Only limited validation can be
applied to informal specifications
- SDL tools provide powerful
simulation environments
- visualization as MSCs
… and
Testing
- Conformance Testing checks that the
implementation provides the functionality defined by the (standard)
requirements
The Development
Process
Conformance
Testing
- ETSI produces Conformance Test
Suites
- does not perform the
testing
- does not perform
certification
- Have streamlined the process to
concentrate on areas related to interopability
- critical behaviour, basic
interconnection
- options, combinations of
options
- error recovery behaviour
- Use STFs (Specialist Task
Forces)
- Feedback to TBs an integral part of
the process
Testing
Specifications
ISO/IEC-9646
Typical Test
Suites
- 3G (3GPP)
- GSM
- GPRS
- TETRA
- TIPHON
- DECT
- IN, INAP
- ISDN, B-ISDN
- VB5
- Etc.
Use of
TTCN
- Tree and Tabular Combined
Notation:
- Versions 1 and 2 developed by ISO
SC21 WG3 (1984 - 1997) as part of the widely-used ISO/IEC 9646
conformance testing standard
- ISO/IEC 9646-3 (edition 2) and ITU-T
X.292
- Version 3 developed by ETSI TC MTS
(1998 - 2000)
Specialist Task Forces STF 133 &
STF 156
Advantages of Using a
Standardised Test Language
- One test notation for many testing
applications
- Cheaper education and training
costs
- Facilitates the application of a
common methodology
- Easier maintenance of test
suites
- Off-the-shelf tools
- Universally understood syntax and
operational semantics
- Tests concentrate on the meaning of
the test
- Constant maintenance of the
language
Motivation for
Producing TTCN-3
- Modernization
- technology has changed since TTCN
was first developed
- Wider scope of
application
- should be applicable to many kinds
of test applications not just conformance (development, system,
integration, iop …)
- Harmonization
- should be the first choice for test
specifiers, implementors and users both for standardized test
suites ...
- … and as a generic solution in
industrial product development
Scope of
TTCN-3
- Kinds of testing to which TTCN-3
could be applied
- Conformance -
Interoperability
- Configuration -
Compatibility
- Performance - Stress
- Robustness -
Integration
- Functional - Load
- Reliability - Fault
tolerance
- Scalability - Degraded
mode
- Unit - Product
- Development - Design
- Interface - System
Main Capabilities of
TTCN-3
- Core format is a text based
language
- Core can be viewed as text or in
various presentation formats
Use With Other
Languages
- TTCN can be integrated with other
'type and value' systems
Major Elements of
TTCN-3
- Built-in and user-defined generic
data types (e.g., to define messages, service primitives,
information elements, PDUs)
Example Core (Text)
Format
Example Tabular
Format
Example Graphical
Format
Use of TTCN-3 for XML
Tests
- Open Settlement Protocol
(OSP)
- EP TIPHON XML-based
protocol
- work on test specs. will start
mid-April (STF 176)
- will use TTCN-3 (Core
Language)
- 3GPP
- currently extensively using
TTCN-2
- initially will use TTCN-3 for
testing XML-based protocols
- work on test specs. will start
2002
Use of TTCN-3 for SIP
Tests
- IETF Session Initiation Protocol
(SIP)
- EP TIPHON SIP Profile
- mainly telephony aspects
(SIP-T)
- Work has started (STF 166 and STF
176 plus voluntary effort from Nokia, Ericsson, GMD Fokus (Testing
Tech), ACACIA, FSCOM)
- initial set Test Purposes
exist
- have a number of executable tests
running
- will continue until Dec.
2001
Contact
details
- Phone
- +33 (0)4 92 94 43 18
- Fax
- +33 (0)4 93 65 38 51
- e-mail
- pex@etsi.fr
- Web
- http://www.etsi.org/ptcc