Meeting minutes
ChrisN: Background is TV app performance discussion from TPAC 2021
Lightning
ChrisL: I'm at Comcast, our goal was to use a different rendering technology on devices for pure performance reasons
… Rendering is very slow, as you navigate it switches from one thing to another, no animations, not a great user experience
… Due to low-end performance on devices. TV devices have great GPUs for video playback, but the CPU isn't high end
<ChrisLorenzo> https://
ChrisL: What Lightning does is to still use a browser, but a WebGL based canvas
… Video shows a comparison between a Lightning app vs an HTML app, it's twice as fast in FPS
… Gentle animations between one image and the next. If you want to build an app, we're at the point where you can't use HTML and CSS for performance
… HTML creates a DOM tree and CSS applies styling. WebGL talks to the GPU, draw at particular coordinates
ChrisN: Concur from a BBC perspective with the problem statement.
ChrisN: Agree, we're seeing similar issues
John: Impact of multiple applications running on TVs?
ChrisL: Devices have limited memory, but the cost is you can't load multiple applications at the same time
… Each app might require 350MB memory, so having 3 apps running at the same time you can hit a 2GB threshold quickly
… If you don't write the app well, you get memory leaks
… Performance is relatively stable once the app is started
John: So slow to make channel changes
ChrisL: Yes. There are processes running in the background that uses say 10% CPU
… I'll share an article
<ChrisLorenzo> https://
<ChrisLorenzo> Performance - https://
Kaz: You mentioned some benchmark tools, we can see potential issues like CPU power. We used to tackle a cloud browser approach some years ago
… Should we look at issues first in more detail, then look at various approaches to address performance at different layers
ChrisL: Comcast did cloud based rendering for many years. The drawback is you need a big cloud
… So ideally you move to computation to the device, ideally a browser. But the downside is it's slow to render
… You should be doing more server side pre-processing to minimise the computation on the client
Kaz: Load sharing approach is being looked at in Web & Networks IG
John: We've discussed the variety of approaches being taken in the media industry, so bespoke solutions
ChrisL: One approach is native, larger companies do that, primarily C++, then they have to get it running on each device, OS, and firmware
… Becomes a large task to work with manufacturers to integrate
… Different flavours of native, such as Android UI, some companies use WASM with a runtime that creates another app on the device, then web browsers
… Use Swift on Apple
… There becomes a lot of ways to build applications, each is different per manufacturer, so hard to make it work across devices
… Lightning is a way of developing web based applications. But we still have the issue of how to get it onto each manufacturer's devices. No standard deployment model for applications
… For TV apps, you can give a URL, host yourself. With Tizen you have to ***
… For developers would be beneficial to have some kind of progressive web app store to install easily on TVs
Andreas: Question about standardisation. How would your approach affect accessibility, if you can't use ARIA tags
… How would you compare the expressiveness of the Lightning approach compared to HTML? And how easy is it for implementers to use the accessibility features?
ChrisL: It's actually relatively easily. Learning all the ARIA tags requires a specialist. With Lightning, you navigate with arrow keys, and speak out
… We use text to speech and the browser speech synthesis API
<ChrisLorenzo> https://
ChrisL: It's a simple approach, but developers have to write a lot of strings for what you want to say. Could get to a point where we have something similar to ARIA tags, but need a component based approach
Andreas: Did you test with standard screen readers?
ChrisL: No, there's no inspection into a canvas tag. We experimented with a library that converts to HTML markup
… With HTML you use the standard elements, but with canvas they're just objects. No standard for that across multiple applications
… We created a Lighning UI component library. The Button component has ARIA-like labels
Kaz: Regarding the potential interoperability problem between the existing HTML approach and this new approach, I've been working on a possible speech / smart agent workshop
… Interop of speech processing platforms is a topic there.
ChrisL: Some of the compatibility is: use the web browser, but how to make the browser the foundation for how we make applications
… Bring your own runtime means more memory and resource usage. A browser with a standard WASM runtime allows a shared environment
… Load from disk or URL, different deployment model. Questions around offline mode with these applications. Solved in some way with PWA in the web world.
… TV manufacturers need to ramp up to adopt these patterns, create a unified standard
Kaz: Relates to MiniApps approach?
ChrisL: Yes. HTML and CSS is slow rendering. Way forward could be a) like Cobalt, simplified HTML and CSS, b) with WASM, c) MiniApps, with something like HTML that renders UI
… Up in the cloud ideas, how to get people onboard?
John: If the goal is to come up with a good solution, but have it be adopted, we'd need companies to buy in. Engage people early in discussion on requirements for problem solving and on the spec choices to be made
… In WAVE we didn't want to move forward until we have one significant TV service and one manufacturer bought into the process, to ensure there's at least some path to adoption in the industry
… That part, the front-loading of getting likely candidates to say they're interested is critical to its success
ChrisL: In questions we get, we see confusion about developing TV applications, how do i deploy on a Comcast, Samsung, etc device?
… Education is needed
John: Also business models, getting things into a store
… The PWA model has a discovery model, find the app that can play this content. Becomes so much simpler for a developer to test and verify their app
… It's a game changer. But it runs counter to some business models, hence why it's important to get major players bought in
… Things that may appeal to us as having a good design may not work for business reasons
… Important for W3C to be more relevant in this space
… What's the process, the path forward? Looking back at how MSE and EME were created, the industry got deeply involved. What's the thinking?
ChrisN: We use HbbTV, it's an HTML based environment but with broadcast delivery integrations. Can we get them on board?
ChrisL: What we do is to extend the Window environment with a TV object.
John: Two concerns, accessibility from Andreas, and hybrid solutions as Chris mentioned
… There's a need to capture the requirements. We have a problem statement, and use that before we start to propose solutions
… I'm encouraged by how Lightning had to solve these problems, shows there's a path
… Three concerns: we bring the existing HTML infrastructure to bridge the gap between broadcast and web world, have an off-ramp for all that. And all the work done on accessibility, including speech as Kaz mentioned
… All the tools that people created around accessibility. Thirdly, and critically, the streaming services and device manufacturers need to be bought into the problem statement and path foward
… We managed to do that for MSE, as we had a number of companies that made it possible. We need to get that same kind of momentum for creating a framework
ChrisL: Getting all manufacturers to adopt a standard browser, I'll spend some time looking at Tizen. Painful to have to set up a virtual machine. I'm used to developing in Chrome with standard JavaScript practices
… I want to write my app for your TV in Chrome, should have to do minimal debugging, and be easy to get started
… The sooner we can get to the point where there's a standard browser, not a unique development environment
John: CTA WAVE, when they adopted HTML as the app development framework, and been frustrated that it hasn't become the platform of choice, they created automated test infrastructure for media playback
… It's automated with QR codes and a camera for running tests. Game changing for manufacturers, makes verification easier. Having a standard browser, then all the tests are free to use,
… verify all the media profiles, CMAF, test content, tools. It would be a compelling reason to adopt an HTML5 based framework as an entry point
… Need a TV manufacturer who'll be willing to engage and be a partner in the development
ChrisL: Comcast has been doing this, they have RDK, X-Class TV, working with HiSense. Ideally you'd want them, Samsung, LG
… I think there's complexity, RDK creates the OS that has the browser, stripped down, device drivers. Also video payback is difficult
… We often dont' use the video tag, and use the AMP player. Tests can show where there are browser or device driver issues
cpn: other input that I get from our TV developers is inconsistency of implementations, e.g. some combination of properties for CSS animations. That's even within the regular HTML/CSS rendering model.
… It makes me wonder whether there's something there that could be looked at. Even if we introduce a new rendering approach, we'll have the legacy problem.
… Devices that support the old model, and devices that will support the new model.
… I think that the plan that John put forward is the right one. Pulling the right people.
JohnS: Maybe if someone could write things done, that could help get at least one TV manufacturer engaged.
… Document that details the problem statement, a kind of roadmap for the project. Looking for a ready-made proposal or call for proposals?
… Also, you're right that it should be a collaboration with other organizations. I think there needs to be some outreach to them.
ChrisL: One direction is to push for adoption of a solid web browser across devices. That won't solve the HTML/CSS performance issue but will simplify application deployment process.
… Once you have a standard browser in there, how do I inject a TV API in there?
John: I'd like your proposed specs
… seems to be correct to me
… having another meeting to break out the points would make sense
… let's include Jon Piesing and somebody from IPTV Forum as well
… ATSC as well
ChrisN: can we capture the steps?
ChrisL: I have a summary of the steps
… simpler way to develop apps like app store is a step
… and then improve the performance using WebGL, etc.
ChrisL: Step 1: Standard browsers of TVs, a simple way of getting apps on to TV. Step 2: How to increase performance on TVs, WebGL or WASM. Standard approach allows accessibility
… Then broadcast TV integration and talk to the TV hardware. Add APIs to the Window object
ChrisN: Write something, capture all the aspects we've talked about. How far to go before sending liaisons?
John: Work small first, talk to relevant people, then can broaden out later
Kaz: I'd like to talk with the NHK guys about this. This discussion could be part of a potential Task Force, can see the possibility
ChrisN: Want to run this as a joint WAVE and W3C thing, so everyone can be included and work is complementary
John: I agree, perhaps we'll talk offline on how best to coordinate, but it's smart to do that from the beginning
<kaz> [adjourned]