Meeting minutes
Sudeep: Welcome to the 11th Web & Networks IG teleconference
… Shachar Zohar is with us today, from Peer5, as a guest speaker, co-founder and CTO of the company, working on reliable and scalable content delivery
… P2P is one of the topics of interest for our IG, and we had a couple of presentation on similar subjects
… Shachar will talk about some of the basics of enterprise CDNs and how P2P can be an answer to some of the challenges in that space
Shachar: thanks for the introduction - I'm Shachar, CTO and co-founder of Peer5 which we founded some years ago
… to leverage WebRTC to provide solutions in verticals around video streaming
… WRT this talk, we've looking at solving problems with congestion of live event in enterprise network
… [slide 2]
… real-time protocols are usually used for video conferences, with very low latency (typically <1s)
… this contrats with HTTP video stream (like HLS, DASH) used in other environments have higher latencies
… this works by splitting videos in segments in different resolutions and exposing these in timestamped manifests
… that architecture creates latencies, but it comes with pros - no jitter thanks to buffer, no packet drops
… more efficient video encoding given that there is no latency constraint
… that means also better video quality at a given bitrate
… it's simpler: stateless servers with cache that can be scaled easily
… it's standards, interoperable and can be played on any device, can rely on industry standard toolchains and libraries
… it can have network congestion baked into it - which we don't see happening for now in real-time protocols
… in this talk, I'm focusing on video streaming using HLS and Dash, not videoconferencign
… [slide 3]
… the problem statement is when a live event within a corporate environment creates congestion in a corporate network
… This is because HTTP streaming requires all the users to download their own version of the video segment given the unicast nature of HTTP
… the congestion happens because of the load of the # of users watching the stream multipled by the bitrate
… when it reaches the capacity of each of the network offices, it creates congestion
… [slide 4]
… the solution for this problem is eCDN - enterprise content delivery network
… not to be confused with a CDN (from which the term is inspired) - the technology & the problems are quite different
… there are 3 technologies to help solve this:
… * caching - the caching proxy server will cache the segments from which users of each office location will fetch their segments
… only the cache server will hit the cloud-based central server
… so most of the connections avoid hitting the internet link
… the problem is that it requires installing, running and maintaining such a server, which is also difficult to dimension
… Another solution is to use multi-cast - each site would need to be configured to do multicast routing
… That too requires lot of maintenance and configuration
… Each stream needs to be set up in advance, be routed in advance for live
… For large on-demand content, it becomes pretty much unfeasible
… The last solution is to use a P2P architecture
… in which all the users are connected between each other
… and instead of hitting the cloud server, users get their segments from others in the office
… it is sometimes considered less secure when a client is involved
… [slide 5]
… in terms of P2P technologies, there are 2 main approaches, with and without an agent
… Agent based P2P technologies require installation (often a barrier in a corporate network) and it raises security questions given the reliance on a widespread single software
… The agent less solution leverages WebRTC in order to create the P2P connectivity
… it doesn't require any installation - it's just JS in the video page
… it means significant less risks
… But it means it is limited to browser APIs which makes it harder to develop
… [slide 6] shows the architecture of the Peer5 P2P agentless architecture
… in blue, the streaming platform
… in orange, the cloud infractracture
… at the bottom, the corporate network with 2 sites (possibly geographically distant)
… these sites can get data from the blue part, or from other users in the side
… the zoomed-in of a computer in the bottom part shows more details on how each client retrieves data in this architecture
… the peer5 backend helps with discovery of other users and bootstraps the P2P connections
… [slide 8] WebRTC P2P Challenges
… the first issue is with segmenting offices - we don't want P2P connection among users that are not in the same office
… since that would still require using the internet link (which we want to avoid congesting)
… The problem has been solved acamedically using traceroutes and see which one converge
… but traceroute is not exposed in the browser - that might be worth bringing in for standardization
… the problems with traceroute: there are several ICMP attacks that can be leveraged
… and more concerning would be the fingerprinting/privacy aspects as traceroute exposes lots of unique data
… which is something we want to avoid
… The mitigation would be not to enable that API by default, but only enable it when the user chooses it, or when the corporate IT department decides to expose it by policy
… for users that are not aware, this would be disabled by default
… [slide 9]
… Another challenge is P2P over virtual path
… detecting clients that are tunneled VPN is critical to avoid congesting the internet link
… that information is available on the device
… but the problem of exposing it in the browser is similar (fingerprinting), with similar mitigations
… Thanks you, and happy to get questions
Dom: re traceroute, do you know the full traceroute output, or could you use a smaller surface of data?
Shachar: exposing the IP, hostnames, latency and number of hops would be required
Dom: how would you scope this to make palatable to IT policies?
Shachar: limiting it from only a single origin and to a limited target
Dom: have you looked into building a browser extension that would demonstrate this?
Shachar: not as a browser extension, but we have an electron PoC
Sudeep: re traceroute, is this about multi-sites distribution, or is this about picking the right peer in the P2P architecture?
Shachar: these perspectives are intertwined
… we need P2P to avoid congestion over the expensive internet links
… but doing P2P without taking into consideration the LAN topology, we might still overload the congested links
… we need traceroute to determine which peers are in the same cluster / LAN
Sudeep: is there any sort of JS agent which can run in each of these peers accessing the video stream and would multicast the information with regard to their availability on the local network?
Shachar: something similar to what you suggested is mDNS (multicast DNS)
… the multicast TTL is set to 1, which means only users in the same network segment (one hop away from the multicasting user) will receive it
… but if you have multiple segments in a single office, you want them to discover each other
… in any case, I don't think there is a multicast discovery mechanism within JS
Sudeep: we have a similar conversation in the context of discovery edge servers in the context of offloading compute
… I wonder if the discovery aspect could help address this in a different way
Shachar: You're right that this has likely applicability here
Zoltan: re caching vs multicasting vs P2P ("eCDN technologies" slide)
… you mentioned that having an application was an issue due to security and management perspectives
… whereas in the browser, you're limited by what browsers are willing as powerful APIs
Shachar: from a business perspective, installing software within large customers with slow processes from security is extremely challenging
… more generally, the trend is to go to the cloud, with very little appetite for installing proprietary software
… so using the browser is a much more practical approach
… we've been able to manage some of these challenges within the current constraints
Zoltan: I agree that a browser front-end is preferable for cloud-based services
… I'm asking because other powerful APIs have been facing these questions
… these are challenging questions in W3C
… but you seem to confirm it's worth it
Shachar: yes, I think it's a MUST
Zoltan: do you have any specific proposal in terms of what API to standardize?
Shachar: not at this point - this is something that could be started within this IG
… not sure if WebRTC would be a natural fit for ICMP/traceroute
Dom: re VPN, this used to be exposed in WebRTC but was removed due to safety concerns
… how much of the topology can be determined from existing info (e.g. RTT, or IP name schemes)
Shachar: we're already some of this with pretty good level of success
… in some cases, our customers provide us with a mapping of IP address but that is brittle
Dom: it's interesting that you can already do most of it - that may be the basis of building the case that exposing just a bit more would not create very different privacy/security surfaces
… can't you get the info you need via STUN?
Shachar: different corporate networks may not make this practical (e.g. exposing random IP addresses, or same IP address across different offices)
Song: there are challenges with regard to privacy - can you explain more specifically the benefits to the enterprise customer? is it lower latency? saving costs?
Shachar: it's not so much about saving costs - today, corporate offices have a limited amount of bandwidth
… e.g. an office may a 1Gb/s to the internet; a stream might be 2M/s - if you have a 1000 users watching that stream (e.g. the CEO talking about their networks)
… this creates a 2G/s requirement, which will suffocate the internet link
… this means some users won't be able to watch the stream, and other applications will also be blocked or slowed down
… this is what the eCDN technologies are scoped to solve
Song: we are planning the Web conference for my company China mobile, with 100K people across the country
… right now, some people will group in meeting rooms, others will watch from home or on their mobile phone
… Would eCDN still cover this scenario, with users in these different setups?
… if this would work, this would be good motivation to consider the privacy/policy mitigations
Shachar: P2P can be used in a hybrid environments with some people at office and others at home
… e.g. by improving the situation for people on the same ISP
Song: as a 5G operator, we are also planning a 5G-based broadcasting or multicasting on the top of the 5G radio network
… if this might be interesting in this context, let me know
Dom: re enterprise policy for browsers - what exists today? any convergence across browsers?
Shachar: Harald shared with me that there doesn't seem to be any effort on this at this time
… lots of possible interesting topics for W3C here
… Chrome, Edge and Firefox have their enterprise mode with centralized configuration, but they're not standards
… they're not using the same registry keys
Dom: so I'm hearing several possible standardization topics:
… * exposing traceroute/ICMP or network topology one way or another
… * standardizing enterprise policies or at least a shared way to enable super-privileged APIs in specific environments
… * expose NIC interfaces
Sudeep: thank you for the presentation!
… we'll look at these ideas, also try and find a commonality with some of our other discussions