W3C

– DRAFT –
MEIG monthly meeting

01 November 2022

Attendees

Present
Andreas_Tai, Chris_Lorenzo, Chris_Needham, Ewan_Roycroft, Francois_Daoust, FW, Hiroshi_Kajihata, Hisayuki_Ohmata, igarashi, John_Riviello, Jordan, Kaz_Ashimura, Kinji_Matsumura, Nassar, Pat_O'Neill, Ryo_Yasuoka, Tatsuya_Igarashi
Regrets
-
Chair
Chris_Lorenzo, Chris_Needham, Tatsuya_Igarashi
Scribe
cpn, kaz, tidoust

Meeting minutes

Introduction

Chris: Today we'll hear about TV application development at the BBC. Anything else for today?

TV application development

Slideset: https://w3c.github.io/media-and-entertainment/meetings/2022/MEIG-2022-11-01-BBC-TV-Development.pdf

[Slide 1]

Jordan: I work on TV iPlayer at the BBC

[Slide 3]

[Slide 4]

Jordan: Firstly, will talk about how it's built
… We have one codebase across all devices, with configurations to perform optimally across devices
… We had a TV abstraction layer library (TAL), an opinionated approach. We've now extracted out different parts into more componentised libraries

BBC TAL library

[Slide 5]

Jordan: A device specific configuration, CSS transforms, etc. For our UI we have JSX and CSS components, a mix of server side and client side rendered
… Logic is vanilla JS
… For playback we use our open source player. Some UI libraries, for our design system, which will be open sourced
… LRUD Spatial for spatial navigation. Arnold, handles back journeys in the application

[Slide 6]

Jordan: Any code targeting the TV environment is transpiled to ES5.1. We can write in TS or ES6. A linter runs on the built output to detect if any problematic or unimplemented APIs are used
… The bundle uses Webpack or Rollup, for tree shaking and code splitting
… We have a certification process for browsers against our own spec

[Slide 7]

Jordan: We have a sandbox environment for local development. Our UI components are SASS and JSX, which allow us to common web development tools such as Storybook, to share with our UX team

[Slide 8]

Jordan: In our design system, the compiled CSS adds appropriate vendor prefixes, depending on the device
… The system contains a normalised stylesheets which overrides some of the agent-specific weirdness
… Disparity with SVG feature support, we arrived at through trial and error

[Slide 9]

Jordan: Testing, we have a specification that guarantees a minimal set of support accross devices

[Slide 10]

[Slide 11]

[Slide 12]

Jordan: For newer features, we have device compatibility experiments. We can run arbitrary code during application load, which we use for compatibilty experiment tests, and we populate an MDN style compatibility matrix for TVs, tells us which APIs or CSS features works or which devices are problematic
… We can use that to hold back certain features on devices that don't support
… We have a test harness for each webpage, runs in an isolated docker container, with pupeteer integration tests, take screenshots to verify

[Slide 13]

Jordan: We also have manual testing. Running on 400+ devices is a huge test burden, we can't do that. Instead we have canary devices in the office, which are indicative of whether a feature will work across a wider range of devices
… For example a specific historic WebKit version
… Anything that we test that isn't well supported, we hold back through config. Avoids device-specific logic tests. It's capability test driven

[Slide 14]

[Slide 15]

Jordan: Deployment. App may be pre-installed, or installable from a device specific store

[Slide 16]

[Slide 17]

[Slide 18]

Jordan: We use a lightweight wrapper app that points to a well known URL. Points to a config that delivers a configured application. We use the user agent string to identify the device

[Slide 19]

Jordan: Opportunities for standardisation

[Slide 20]

Jordan: First one, already explored is spatial navigation. LRUD spatial is our library, a generic solution, to be open sourced
… Our implementation is HTML attribute based, if you want specific behaviours you need to write your markup in a certain way
… CSS SpatNav spec from 2017, it would be good to revisit that
… Non functional requirements, hard to define, hard to measure reliably

[Slide 21]

Jordan: With CSS transitions, a device can be spec compliant even if it only runs at 1fps
… Fallback, if a device doesn't support, elements will just snap to space
… If a device performs poorly, better not to enable the feature
… With canvas animation, with our boot-up start application, we found some devices crash when rendering. If there's a way to query maximum size, developers could use it across a wider range of devices
… Currently rely on user agent to pair the app with device-specific config. Can't rely on feature detection in the app. Privacy concerns with UA string, and UA CH, but we do need a way to recognise the device

[Slide 22]

Jordan: Remote debugging. Each page is isolated, test with puppeteer. Better if we can run against a set of representative engines
… Running automated scripting tasks requires differnt integrations. Want a standardised approach to remote debugging

[Slide 23]

Jordan: Application manifesets. There's Web App Manifest. Extending to add rich metadata to populate into app stores would help developers

[Slide 24]

Jordan: System messaging. Allowing the host to notify of state changes. Voice events, I believe HbbTV is looking at that. Also Page Lifecycle API for more fine grained control of lifecycle events. Benefit from having a richer interface with the host, network conditions, etc

Page Lifecycle API

[Slide 25]

ChrisN: You mentioned quite a number of things, where to start?
… What's your thought on the manifest suggestion?

ChrisL: I'm at Comcast, and we mostly work with Comcast-specific devices
… How to standardize manifest files for different devices is important
… for easier tests
… I'm interested to see the iPlayer app

ChrisL: Can you talk more about the code. Is it all HTML and CSS?
… Is it open source?

Jordan: Some building blocks are open source, such as TAL. We haven't open sourced the newer module components, but we plan to
… For UI components, spatial navigation will be open sourced soon. The application itself isn't open source

Jordan: [Demos iPlayer development]

ChrisL: Does it use standard ARIA tags for accessibility?

Jordan: Spatial navigation relies on semantic HTML. For screenreader support, we use ARIA labels
… Some screen readers don't work so well, so we only enable support on certain devices, but the ARIA tags are always there

ChrisL: You mentioned remote debugging not being standardised. Which devices don't support the current standard?

Jordan: Chromium and Gecko, and previously Opera devices with Presto each have their own solution. So our tooling has to allow for 5 different strategies for remote debugging
… Sometimes the debuggers aren't available, manufacturers are hesitant to open up ports on the device

ChrisL: Is the app running in a specific browser or the pre-installed browser?

Jordan: Our app is just a webview that points to a URL, so it uses the pre-installed browser

ChrisL: Wondering about web performance, on what proportion of devices where you can use things like CSS transitions?

Jordan: On performance, we do an annual assessment if we can use a particular library or framework. We've trialed React and Preact, and others. Our UI is server-side rendered, not much dynamism when it hits the client
… An exception is the playback UI, which is dynamic, so we use Snabbdom, the underlying Vue.js DOM library. It's just the virtual DOM part, we have got decent performance from it
… Problems typically are memory related, holding a large representation of the DOM in memory. Or when the libraries generate target code that don't transpile to ES5.1. To work around that we'd need to write our own vendorised version of the library

Snabbdom library

Jordan: For device support, offloading to the GPU like ??, we're up to 60% of users on devices that support CSS transforms
… Even there, hinting that older browsers like, such as will-change on a carousel, for example

CSS Will Change spec

Jordan: Even within CSS transforms there are certain capabilities we turn on and off
… We test manually. We haven't found an effective way to define an acceptable level of performance in our spec, and how much burden does it put on manufacturers?
… We'd want to have a way of testing that isn't burdensome for manufacturers. A big area for investigation

ChrisL: WAVE group was looking at a test app, and to be certified you need to hit a benchmark

ChrisL: How do designers work with TVs? In Comcast we get designs with drop shadows, etc, things we can't do

Jordan: We have marketing, design, engineering. With the current version of the UI marketing suggest things that may not be possible. When designers look at it, they're more pragmatic
… In enginerring, we push them to be more ambitious, try not to be too limited. Start by aiming high, then compromise down

ChrisL: Have you looked at LightingJS framework, a canvas based renderer? Interested in your feedback

Jordan: Not so far.

LightningJS

ChrisN: I think other people in BBC are looking at it, could be a topic for a future meeting?

ChrisL: What is the development experience like building HTML and CSS?

Jordan: We spend a lot of time getting it performant, the initial development time relative to that is quite small
… Figuring out the different levers to have in place, to get optimisation, then getting to a usable experience takes longer
… Performance issues mean that we've had to choose between getting rich impression stats, page heat maps, or better animation performance, you can't have both
… So getting optimised on devices takes a lot longer than initial development
… Is Lightning whole page or components?

ChrisL: It's components. Happy to talk further and give a demo and walk through

ChrisN: You've covered quite a lot of ground, we want to capture everything. Re spatial navigation, you mentioned open-sourcing your library
… The spec was being developed by LG, but not sure what happened that means the spec hasn't progressed
… But if it can be done through script, to what extent would it be beneficial to standardize?
… If it's because it's so common? Or does the library need to have lots of strategies to support all devices?

Jordan: I assume it would be more optimized if natively supported.
… Our library's only dependency is on Element.getBoundingClientRect which is well-supported across browsers.

ChrisN: On remote debugging, you mentioned things being device-specific. Is there a standard that isn't widely supported. Or are there capabilities missing from the standard?

Jordan: Probably the de facto standard is Chrome's WebDriver implementation.
… There is a Chromium spec, which moves quite rapidly, hard to tell what we can rely on.

Kaz: Also interested in Spatial Navigation. These days, Metaverse is getting more popular, maybe we might want to think about navigation within 3D video content, and including collaboration with other types of devices such as smartphones.

Jordan: We don't currently run on any other devices that support more than left/right/top/down
… but it definitely sounds like an opportunity to explore

Kaz: Thanks! I'm involved in IoT standardization as well as Media Entertainment, so interested in cross-group scenarios.

ChrisN: What could be done upstream to make sure that the standards are more interoperable before they get to you and trigger these interoperability issues. A question for everybody.
… E.g. whether the set of test cases in test suites such as that developed by CTA WAVE are representative to identify functionality issues such as the SVG one you mentioned.
… One of the things that I've heard in a previous presentation on that topic is that tests can take a long time to run. So question on how you identify relevant tests.
… The System messaging is also interesting. You talked about memory usage and I don't know whether that is covered in any W3C spec.
… If in practice, constraints mean that some features cannot be used, it makes me wonder whether some kind of introspection mechanism would be useful.
… Right now, you need to track user agents and configure features accordingly.
… May be difficult to agree at the standards level.

ChrisLorenzo: TV manufacturers try to bring performance at low cost, application developers need to struggle to get the max out of the device, etc.
… It's hard to get all of the people involved going in the same direction.
… Small moves will probably be needed.

ChrisN: Makes me wonder about some kind of baseline specification that could be targeted at TVs.
… Some content producers benefit from having more end-to-end control over the experience.
… The notion of shared runtime is an idea that we've been discussing but haven't reached a critical mass of supporters so far to push things forward.

JohnRiv: Confirming that there's interest is useful. That's what we're willing to do in the next couple of months. Feel free to reach out to ChrisN, ChrisL or myself.

ChrisN: I think that what we'd like to do is capture the points you've raised into a document. We're collecting good feedback, today, and during our previous call from NHK, and from ChrisL's previous presentations.
… That would allow us to look at next stages.
… Two things: 1) common runtime, 2) what can we do about performance issues.
… They're both worth considering at this stage.
… Perhaps ChrisL and I could look at capturing this feedback in a document.

Jordan: Yes, there's future standards, and then there are standards already there that we should push to get supported.

ChrisN: For each of the capabilities that you've been raising, being able to say whether it's a standard problem, an implementation or adoption problem, a testing problem, etc.
… And documenting gaps, e.g. introspection for capabilities.
… If anybody is interested in helping collect something like that, help would be welcome.

ChrisN: Thank you Jordan and everybody for joining! Look forward to seeing you all in the next meeting in December!

[adjourned]

Minutes manually created (not a transcript), formatted by scribe.perl version 196 (Thu Oct 27 17:06:44 2022 UTC).