W3C Distributed Tracing Working Group Workshop in Seattle, November 2019

This week, the W3C Distributed Tracing Working Group workshop was hosted in Seattle - the famous cloud city. We discussed the current status of Trace Context specification and future improvements. The main focus was made on passing identifiers via response headers, correlation context and advancing trace context into other protocols.

We had great attendance - over 20 people attended from Amazon, AppDynamics, Dynatrace, Google, Instana, Mailchimp, Microsoft, New Relic, LightStep, Uber Technologies, University of Washington, and W3C.

View from the workshop conference room @Microsoft. Photo credit to Christoph Neumüller.

Status of the specification

We discussed the current status of migrating to W3C Trace Context headers. The specification is transitioning into Proposed Recommendation status which opens the door for the implementers to finalize implementation. OpenTelemetry has already released support with more telemetry vendors planning to support and transition to W3C Trace Context headers in the coming months. There were two big developments in the last year. Microsoft .NET apps now have W3C Trace Context built-in. Dynatrace shared the exciting news of full support of W3C Trace Context in the Dynatrace OneAgent.

Migration to W3C Trace Context was a hot topic. Though it appears simple on the surface, it is a complicated problem because of backward compatibility concerns. We discussed motivation and benefits for end-users of the migration, migration plans for large distributed applications, and how sharing success stories may help accelerate this migration.

We will soon publish a blog post explaining the benefits of moving to W3C Trace Context headers. Morgan McLean (Google), Sergey Kanzhelev (Microsoft), and Sonja Chevre (Dynatrace) will be working on a migration plan whitepaper. Victor Soares (New Relic) and Lionel Godolphin (Microsoft) will be on the lookout for success stories that we can share.

Response headers

Morgan McLean (Google) lead a discussion where we brainstormed user scenarios, semantics, and the format of the response header. This topic generated a lot of interest in the context of Load Balancers, about which Anssi Alaranta (Amazon) shared his experience, as well as in the context of including browser interactions as part of a distributed trace. Also, Bernhard Lackner (Dynatrace) and Dave Raffensperger (Google) presented a proposal for tracking resource requests in the browser via response headers.

As an action item, we will be investigating the possibility of reusing the Server-Timing header defined by W3C Web Performance group. Also, we will be working on writing an initial specification that addresses scenario of enabling browser user interactions tracking. Daniel Dyla and Daniel Khan (Dynatrace) will be working on an initial proposal.

Correlation Context

Yuri Shkuro (Uber) gave a great presentation of use cases and issues with Correlation Context.

Alex Fedotyev (AppDynamics), Bogdan Drutu (Google), Daniel Khan (Dynatrace), Sergey Kanzhelev (Microsoft), Yuri Shkuro (Uber Technologies), Matthew Wear (Lightstep), and Justin Foote (New Relic) plan to work on this specification.

Group picture

Business Process Monitoring and distributed tracing

Lionel Godolphin (Microsoft) presented his experience building and operating Business Process Monitoring (BPM) systems and how it relates to distributed tracing. We discussed the procedural and technical aspects of implementing BPM. Interesting topics were Correlation Context applicability for BPM, a relationship of transaction monitoring identifiers with BPM workflows.

Other Protocols

Sergey Kanzhelev (Microsoft) presented options for propagating trace context in text and binary format and we discussed tradeoffs for using Unicode format since it’s available in text and binary format, or keeping the ASCII format required by the HTTP protocol. Our preference is to be consistent and use ASCII format in other protocols. We also discussed trace context propagation through AMQP and MQTT and options for where and how to inject context. The group will propose options for each protocol and link to the current specification.

Other topics

You can find the full agenda and meeting notes in this Google doc: W3C Distributed Tracing Working Group workshop (Nov 2019). Recordings will be uploaded later.

(post created by the Distributed Tracing Working Group)