Propagation format for distributed context: Baggage

W3C Working Draft

More details about this document
This version:
https://www.w3.org/TR/2022/WD-baggage-20220928/
Latest published version:
https://www.w3.org/TR/baggage/
Latest editor's draft:
https://w3c.github.io/baggage/
History:
https://www.w3.org/standards/history/baggage
Commit history
Editors:
Sergey Kanzhelev (Google)
Yuri Shkuro (Facebook)
Former editors:
Alois Reitbauer (Dynatrace)
Morgan McLean (Google)
Daniel Khan (Dynatrace)
Feedback:
GitHub w3c/baggage (pull requests, new issue, open issues)
public-trace-context@w3.org with subject line baggage (archives)
Discussions
We are on Slack.

Abstract

This specification defines a standard for representing and propagating a set of application-defined properties associated with a distributed request or workflow execution.

This is independent of the Trace Context specification. Baggage can be used regardless of whether Distributed Tracing is used. This specification standardizes representation and propagation of application-defined properties. In contrast, Trace Context specification standardizes representation and propagation of the metadata needed to enable Distributed Tracing scenarios.

Status of This Document

This section describes the status of this document at the time of its publication. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at https://www.w3.org/TR/.

This is the First Public Working Draft (FPWD). There are a few implementations of this protocol available. Experimental interoperability scenarios were run and have demonstrated promising results. The specification will be progressed into Candidate Recommendation stage. After that, the drafts for binary, AMQP, and MQTT protocols will be written to make sure the concepts and structure defined in this specification can be ported to other protocols.

This document was published by the Distributed Tracing Working Group as a Working Draft using the Recommendation track.

Publication as a Working Draft does not imply endorsement by W3C and its Members.

This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document was produced by a group operating under the 1 August 2017 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

This document is governed by the 2 November 2021 W3C Process Document.

1. Conformance

As well as sections marked as non-normative, all authoring guidelines, diagrams, examples, and notes in this specification are non-normative. Everything else in this specification is normative.

The key words MAY, MUST, MUST NOT, and SHOULD in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

2. Overview

The baggage header represents a set of user-defined properties associated with a distributed request. Libraries and platforms SHOULD propagate this header.

3. Baggage HTTP Header Format

The baggage header is used to propagate user-supplied key-value pairs through a distributed request. A received header MAY be altered to change or add information and it SHOULD be passed on to all downstream requests.

Multiple baggage headers are allowed. Values can be combined in a single header according to RFC 7230.

3.1 Header Name

Header name: baggage

In order to increase interoperability across multiple protocols and encourage successful integration, implementations SHOULD keep the header name lowercase.

3.2 Header Content

This section uses the Augmented Backus-Naur Form (ABNF) notation of [RFC5234].

3.2.1 Definition

baggage-string         =  list-member 0*179( OWS "," OWS list-member )
list-member            =  key OWS "=" OWS value *( OWS ";" OWS property )
property               =  key OWS "=" OWS value
property               =/ key OWS
key                    =  token ; as defined in RFC 7230, Section 3.2.6
value                  =  *baggage-octet
baggage-octet          =  %x21 / %x23-2B / %x2D-3A / %x3C-5B / %x5D-7E
                          ; US-ASCII characters excluding CTLs,
                          ; whitespace, DQUOTE, comma, semicolon,
                          ; and backslash
OWS                    =  *( SP / HTAB ) ; optional white space, as defined in RFC 7230, Section 3.2.3

token is defined in [RFC7230], Section 3.2.6: https://tools.ietf.org/html/rfc7230#section-3.2.6

The definition of OWS is taken from [RFC7230], Section 3.2.3: https://tools.ietf.org/html/rfc7230#section-3.2.3

3.2.1.1 baggage-string

List of list-members with optional properties attached. Uniqueness of keys between multiple list-members in a baggage-string is not guaranteed. The order of duplicate entries SHOULD be preserved when mutating the list. Producers SHOULD try to produce a baggage-string without any list-members which duplicate the key of another list member.

3.2.1.2 key

A token which identifies a value in the baggage. token is defined in RFC7230, Section 3.2.6. Leading and trailing whitespaces (OWS) are allowed and are not considered to be a part of the key.

3.2.1.3 value

A value contains a string whose character encoding MUST be UTF-8 [Encoding]. Any characters outside of the baggage-octet range of characters MUST be percent-encoded. Characters which are not required to be percent-encoded MAY be percent-encoded. Percent-encoding is defined in [RFC3986], Section 2.1: https://datatracker.ietf.org/doc/html/rfc3986#section-2.1.

When decoding the value, percent-encoded octet sequences that do not match the UTF-8 encoding scheme MUST be replaced with the replacement character (U+FFFD).

Leading and trailing whitespaces (OWS) are allowed and are not considered to be a part of the value.

Note, value MAY contain any number of the equal sign (=) characters. Parsers MUST NOT assume that the equal sign is only used to separate key and value.

3.2.1.4 property

Additional metadata MAY be appended to values in the form of property set, represented as semi-colon ; delimited list of keys and/or key-value pairs, e.g. ;k1=v1;k2;k3=v3. Property keys and values are given no specific meaning by this specification. Leading and trailing OWS is allowed and is not considered to be a part of the property key or value.

3.2.2 Limits

Note that the following limits are minimum requirements to comply with the specification. An implementor or platform MAY define higher limits and SHOULD propagate as much baggage information as is reasonable within their requirements. If a platform cannot propagate all baggage, it MUST NOT propagate any partial list-members. If there are multiple baggage headers, all limits apply to the combination of all baggage headers and not each header individually.

  1. A platform MUST propagate all list-members up to at least 64 list-members including any list-members added by the platform.
  2. A platform MUST propagate all list-members including any list-members added by the platform if the resulting baggage-string would be 8192 bytes or less. If the resulting baggage-string would be greater than 8192 bytes, some list-members MAY be dropped until the resulting baggage-string is 8192 characters or less.

If a platform does not propagate all list-members, it is left to the implementer to decide which list-members to propagate.

3.2.3 Example

The following example header contains 3 list-members. The baggage-string contained in the header contains 86 bytes. 82 bytes come from the list-members and 4 bytes come from commas and optional whitespace.

baggage: key1=value1;property1;property2, key2 = value2, key3=value3; propertyKey=propertyValue
  • key1=value1;property1;property2
    • 31 bytes
  • key2 = value2
    • 13 bytes
  • key3=value3; propertyKey=propertyValue
    • 38 bytes

3.3 Examples of HTTP headers

Assume we want to propagate these entries: userId="alice", serverNode="DF 28", isProduction=false,

Single header:

baggage: userId=alice,serverNode=DF%2028,isProduction=false

Here is one more example where values with characters outside of the baggage-octet range of characters are percent-encoded. Consider the entry: userId="Amélie", serverNode="DF 28", isProduction=false:

baggage: userId=Am%C3%A9lie,serverNode=DF%2028,isProduction=false

Context might be split into multiple headers:

baggage: userId=alice
baggage: serverNode=DF%2028,isProduction=false

Values and names might begin and end with spaces:

baggage: userId =   alice
baggage: serverNode = DF%2028, isProduction = false

3.3.1 Example use case

For example, if all of your data needs to be sent to a single node, you could propagate a property indicating that.

baggage: serverNode=DF%2028

For example, if you need to annotate logs with some request-specific information, you could propagate a property using the baggage header.

baggage: userId=alice

For example, if you have non-production requests that flow through the same services as production requests.

baggage: isProduction=false

3.4 Mutating baggage

A system receiving a baggage request header SHOULD send it to outgoing requests. A system MAY mutate the value of this header before passing it on.

Because baggage entry keys, values, and metadata are not specified here, producers and consumers MAY agree on any set of mutation rules that don't violate the specification. For example, keys may be deduplicated by keeping the first entry, keeping the last entry, or concatenating values together.

The following mutations are allowed:

If a system receiving or updating a baggage request header determines that the number of baggage entries exceeds the limit defined in the limits section above, it MAY drop or truncate certain baggage entries in any order chosen by the implementation.

If a system determines that the value of a baggage entry is not in the format defined in this specification, it MAY remove that entry before propagating the baggage header as part of outgoing requests.

4. Security Considerations

Systems relying on the baggage headers should also follow all best practices for parsing potentially malicious data, including checking for header length and content of header values. These practices help to avoid buffer overflow, HTML injection, and other types of attacks.

4.1 Information Exposure

As mentioned in the privacy section, baggage may carry sensitive information. Application owners should either ensure that no proprietary or confidential information is stored in baggage, or they should ensure that baggage isn't present in requests that cross trust-boundaries.

4.2 Other Risks

Application owners need to make sure to test all code paths leading to the sending of the baggage header. For example, in single page browser applications, it is typical to make cross-origin requests. If one of these code paths leads to baggage headers being sent by cross-origin calls that are restricted using Access-Control-Allow-Headers [FETCH], it may fail.

5. Privacy Considerations

Requirements to propagate headers to downstream services, as well as storing values of these headers, open up potential privacy concerns. Using proprietary ways of context propagation, vendors and application developers could always encode information that contains user identifiable data. This standard makes it possible for systems to operate on a known, standardized header to restrict propagation of sensitive data in the baggage when crossing trust boundaries.

Systems MUST assess the risk of header abuse. This section provides some considerations and initial assessment of the risk associated with storing and propagating this header. Systems may choose to inspect and remove sensitive information from the fields before processing or propagating the received data. All mutations should, however, conform to the list of mutations defined in this specification.

5.1 Privacy of the baggage header

The main purpose of this header is to provide additional system-specific information to other systems within the same trust-boundary. The baggage header may contain any value in any of the keys. As such, the baggage header can contain user-identifiable data, however no key or its value or properties is given semantic meaning by this specification. Systems MUST ensure that the baggage header does not leak beyond defined trust boundaries and they MUST ensure that the channel that is used to transport potentially user-identifiable data is secured.

A. References

A.1 Normative references

[Encoding]
Encoding Standard. Anne van Kesteren. WHATWG. Living Standard. URL: https://encoding.spec.whatwg.org/
[FETCH]
Fetch Standard. Anne van Kesteren. WHATWG. Living Standard. URL: https://fetch.spec.whatwg.org/
[RFC2119]
Key words for use in RFCs to Indicate Requirement Levels. S. Bradner. IETF. March 1997. Best Current Practice. URL: https://www.rfc-editor.org/rfc/rfc2119
[RFC3986]
Uniform Resource Identifier (URI): Generic Syntax. T. Berners-Lee; R. Fielding; L. Masinter. IETF. January 2005. Internet Standard. URL: https://www.rfc-editor.org/rfc/rfc3986
[RFC5234]
Augmented BNF for Syntax Specifications: ABNF. D. Crocker, Ed.; P. Overell. IETF. January 2008. Internet Standard. URL: https://www.rfc-editor.org/rfc/rfc5234
[RFC7230]
Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing. R. Fielding, Ed.; J. Reschke, Ed.. IETF. June 2014. Proposed Standard. URL: https://httpwg.org/specs/rfc7230.html
[RFC8174]
Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words. B. Leiba. IETF. May 2017. Best Current Practice. URL: https://www.rfc-editor.org/rfc/rfc8174