Meeting minutes
Intro
Sudeep: today, we have guests from media & entertainment, Web Performance, Browser Testing & Tools
Sudeep: main agenda today is a presentation on network emulation tools for browsers, by Jonas
… he had given a presentation & demo in TPAC19
… he'll be sharing news on their project
Web & Networks IG charter
Sudeep: an updated charter for the IG is under review by the W3C Advisory Committee
… until April 2nd
… please do bring this to the attention of your AC Rep
Dom: make sure to get your organization to chime in, important to show support for the work
Song: ongoing discussions in Multicast / Broadcast in M&E; network emulation can help on join work in W&N and M&E
Network Emulation Trace Tool For Browser
Sudeep: introducing Jonas, who presented Link performance prediction before
… today, he'll present some of their work that enables emulating network conditions from within a browser developer environment
… including a proof of concept demo
… after the presentation, we'll have discussion on what collaboration on this topic might look like
[Jonas showing slides https://
Jonas: this is a follow up to some of the topics we presented at TPAC19
… on link performance prediction
… we have a number of internal tools to capture network quality and play it back
… discussions in TPAC showed there is possible interest in that tooling in a wider community
… we've reworked them and made them available as open source
… 3 type of tools: trace emulator, trace capture and a trace format (allowing a library of existing network traces)
… these tools were built initially in the context of testing link performance prediction and its usage in apps
… but the tools we're presenting here today are more widely applicable, independently of prediction
… e.g. to show the impact of network quality variation on the behavior of your app
… with a library of trace files, this lets you test your app in very different network situations (e.g. NYC subway, driving in bangalore, etc)
… On top of that, we have our performance prediction capabilities
… We'll also touch on next steps
Link Performance Prediction presentation at TPAC 19
Jonas: [slide 4] Our tools are collection of tools for chromium browsers that gives you a visualization of network trace
… tool released on GPL2 - we're hoping to get contributions in addition to re-use
… [slide 6] In terms of emulation, the idea is that you should be able to get the experience as if you where in the circumstances where the traces were collected
… A second aspect is to help apps leverage predictions to adapt their behavior; useful e.g. in context of media streaming
… [slide 7] illustrates the network trace when testing connection speed on fast.com
… as throttled by Chrome itself
… [slide 9] the capture tool is also available on github, can be run from https://
… you configure it with various parameters on the capture session you want, e.g. frequency, bandwidth cap
… At this point, it's URL based; there are a bunch of speed connection test available out there that we're thinking of integrating directly in
… both end points of the connection matter in collecting the traces
… you can also collect GPS data as you collect info, and keep your screen on (important on mobile phone)
… mobile usage tends to provide the most interesting traces
… Right now, it's limited to chromium browsers
… [slide 10] once you start capturing, it generates JSON data
… that can be saved to the clipboard, which has worked well for us
… [slide 12] The trace format is pretty-simple, JSON-based, inspired by other W3C formats
… it starts with a header with general information, followed by a sequence of entries that describe the network characteristics and the capture circumstances
… the format is described in the github repo
… [slide 13: screenshot of a trace]
… format is flexible, e.g. no requirement to have both up- and down-link
… [slide 15]: We've been collected traces into a library, despite the limits imposed by the pandemic
… we're trying to collect traces from multiple locations, multiple operators
… we welcome contributions on traces too
… [slide 17] if you're interested not just in emulation, but also in prediction
… we have developed a set of algorithms to improve the UX based on predictions of how network conditions might evolve
… with a particular focus on cloud gaming, media streaming
… e.g. prefetching more data when you expect the enter a lower-bandwidth area
… these algorithms can be tuned e.g. on how much bandwidth change should trigger a behavior change
… [slide 18] the github repo has an hello-world demo to show how to pick up a prediction
… both pull & push approaches are available
… [slide 19] you get back a prediction with probabilities of expected changes
… [slide 21] What is our roadmap?
… we're quite open - we'll be continue working on this, with a plan to expand the emulator based on feedback we'll receive from different usage scenarios
… in addition to downlink bandwidth, uplink, and latency are good candidates
… other metrics are possible too
… more trace files is also in the plan
… we've extended the MPEG-DASH library to include LPP and drive-tested it with various operators
… to demonstrate how it can improve streaming
… we're very interested in seeing if others are interesting in driving this, e.g. as an MPEG-DASH extension
… our patches are still to be released, but we'd be happy to start sharing them early before we complete our internal processes for the full release
… we're also open to change how predictions get surfaced
… the remaining slides give background info on how to set the tools up, and gotchas that you may face in that process
Sudeep: thank you Jonas
Ed: (from AT&T) capturing upload & latency would be great - is it something already in progress?
Jonas: it won't be hard to do - we've been looking for feedback & interest
DavidBurns: Browser Testing Tools WG chair - we're extending how WebDriver work and are looking at network emulation
… e.g. to block requests to a CDN (who are bad when doing automated testing)
… is this something that you'd be open for BTT to extend
Jonas: sure! we're not trying to drive this into a specific direction
… we're happy to try and get these tools as useful as possible
… I'm open to any suggestion; open to figure out the right way to collaborate
Piers: I'm assuming you're using the Web browser throttling API to implement the emulation
Jonas: yes
Piers: have you calibrated that it gives the expected performance?
Jonas: we've done a fair bit of testing
… on the emulator side, it has worked quite well
… the capture side has been more challenging
Piers: in terms of capture, you're using XHR - have you considered using Fetch? WebRTC?
… for media delivery, you could look into capturing download segments
Jonas: that's part of the feedback we're looking into for the dash patches
… we haven't looked into that particular question
… quite a few aspects of network performance & metrics is app dependent
… we're open to other ways to do this
… there are different ways of calculating bandwidth metrics which may be of interest
… we'll invest more based on interest
Piers: what other techniques are using to measure?
Jonas: we have more complex tools that aren't part of the open source release at this time, but may become so based on interest; but the current tools are already pretty good
Song: The tool is meant to be used for mobile environments; I had a quick test with the we-chat built-in browser - it could be a very useful tool in the context of we-chat too!
… for the functionalities, I'm interested in seeing if that tool can help in the discussion around broadcast / multicast - I'll bring some of my colleagues in to look into this
Jonas: do please let us know of interest - that's going to be key in further work on this
Barbara: have you done any testing with live streaming?
Jonas: are you referring to the dash extension or the chromium browser?
Barbara: either - testing in the live streaming usage
Jonas: the trace collection is just raw data; when you go to the emulator side, the actual usage doesn't really matter
… we have used the emulation while watching live streams
… but it's probably the dash.js aspects that makes the live streaming use case more interesting
… we've spent a good amount of time on this; the patches haven't been released yet
Barbara: broadcasters, online gaming are likely good use cases for this
… In addition to WebRTC, WebTransport is another network usage that is relevant
… I wonder if these traces could help in that space too
Yoav: If I understand correctly, the application of emulation is based on the dev tools level emulation - is that correct?
… i.e. the same network emulation that Chromium dev tools would trigger
Jonas: correct
Yoav: my personal is experience is that this type of emulation is not ideal - it's done above the socket layer, which can trigger various effects that are different from what would happen from emulation happening at the network level
… it's a design choice Chromium folks did to avoid pollution across tabs
… but it can have significant different on the results when it comes to server behavior and prioritization
… it may be worth integrating with some network emulation tools to better emulate what would actually happen on the network
Jonas: our internal tool operates at the network level
… but it's a lot more challenging to set up and deploy
… this is the trade-off between ease-of-use and accuracy
… I think I would be cautious with relying only with this browser extension for validating considerations
… but it's probably already a significant improvement to what developers would get exposed to by default
yoav: I agree that it's good context for development; for testing, integrating with webpagetest.org a very popular testing utility
… I don't know how easy it would be to integrate with that - might be an interesting angle
… this would help maintain the ease of use while having more realistic network conditions
… for testing that goes beeond the development
Dom: the trace format is independent of how the traces are captured or at what levels the emulation runs, correct?
Jonas: correct
Next steps
Sudeep: good to see good suggestions and feedback
… there seems to be value in bringing that kind of capabilities in browser dev tools
… also intersection with network predictions, usage in the context of MPEG DASH
… I'm calling for people to express their interest, in terms of possible improvements, additional traces
… is there a need for trace standard format standardization? relation to the HAR format?
… with regard to adoption by web developers, Yoav's point on webpagetest seems very relevant
… who would be interested in contributing to this? intersections with other IGs/WGs?
… we could set up a dedicated CG to follow up on this if there is interest
… a TF in this IG
… please send us an email if you're interested