Meeting minutes
Welcome
ChrisN: Welcome, this is the first meeting for a while, hope to maintain regular meeting schedule in future.
Slideset:
https://
... Agenda for today, we have a lot
topics
… Launching web apps on TV devices,
… Media Content Metadata CG proposal
… Media WG updates
… TPAC 2024 plans
… If you think of any potential agenda items, very
welcome. Anything else for today?
Song: There is a media
workshop coming up in a few weeks
… Francois supposed to be there to give a
talk
… I just wanted to remind everyone.
… Hopefully it will be open for attendees from all
areas
ChrisN: If it's open, we can join it. I'd be happy to share the details with MEIG members. Thank you!
Launching Web applications on TV devices
ChrisN: ChrisL generated a demo video, showing the approaches for TV app development across different devices.
(shows ChrisL's video)
ChrisL: what happens is testing on one device
ChrisL: I'm here to
talk about what it's like to do TV app development.
… I want to start off simple with here is a demo
app that I've built as a web developer.
… It's pretty easy to get started I have VSCode, I
have my demo app up and then I can hit npm start and it just pops
up in Chrome.
… So now I'm in Chrome I have everything working. I
can use my arrow keys to move around, etc.
… I can do development just like I were a web
developer (which I am).
… I have access to Chrome, all the Chrome Dev
tools, and this is great.
… Then what happens is we need to start testing it
on devices and there's many different TV devices, and the process
for all those different TV devices is different, so this video is
really trying to go through what are those differences, and how can
we make things simpler?
… So again, starting off we have just Chrome, a
normal development flow, this works really well. It's pretty
straightforward.
Raspberry Pi 3
ChrisL: Next I usually
move to my Raspberry Pi 3. So I have a Raspberry Pi 3 on my desk
and it comes with this RDK. It has RDKimage loaded on it, which has
a development URL so the device actually has an IP address.
… I connect to that IP address and it gives me this
great interface that I can go in here and I can just set the URL
that I actually want to load to test on, so I can click "set" and
then I have the Raspberry Pi hooked up through my laptop through a
USB-C capture card so I actually can see this is the device
here.
… This is the Raspberry Pi output at 1080P and
inside this so if I have focus on this I can actually press the
arrow keys and it will actually move the focus it will send those
keys to the Raspberry Pi device.
… This is probably the easiest way to do testing on
an actual device, and a Raspberry Pi 3 is probably as performant as
some of the TV devices out there, but this gives me a full test
environment on a simulated TV device that I can actually play with
and see what the experience is like.
… I think this is the pinnacle of what I would like
to see for other TV manufacturers.
… I should be able to enable developer mode and
have a port that opens up that I can connect to the device and then
have just an input box for saying this is the URL I want you to
load.
LG Web OS Simulator
ChrisL: For our next
device we're going to talk about LG, which is Web OS. They offer a
simulator, which they have a whole bunch of different versions of
simulators.
… I was able to download the Web OS TV 6.0
simulator for for my Mac so you can download this, unzip it, and
then it will actually just give you an executable so you can double
click to open the Web OS TV simulator from here.
… You'll get this little screen here and you have
to add the web app so in order to add the web app you need to go
into the file launch app and then you have to give it a directory,
so this will open up a directory for you, which they give you an
example hosted app, which is this code here.
… Essentially the hosted web app is just a little
folder that you have to tell it there's an index.html file in there
and then you can set the index.html file this window.location.href
to redirect the screen to your demo app URL.
… So that's what I'm doing here, so now if we go
back into the simulator I can actually double click this, it will
open up the app here and now I can use my keys and work with
this.
… This is a pretty good experience, it's a little
more difficult, you have to download this 80 megabyte
simulator.
… To get it up and going you have to download this
hosted web app, you have to figure out putting a URL in there, and
then loading it up.
… The other thing is this will give you a TV app
experience as far as calling the APIs and simulating what it's
actually like running on the devices, but it won't allow you to do
performance testing because you're running at pretty much native
speeds on your developer box, so you're going to get 60 frames per
second no matter how fast you go so you can't really test the
performance on the device but otherwise I think this simulator is a
pretty good experience.
LG Web OS on a TV
ChrisL: Next we'll
actually try loading on TV, which is a much different
experience.
… So now we move on to trying to get our
application going on the Web OS TV.
… I have a physical TV here and I've spent the last
two hours, I've time boxed it, to try to get my web app loaded on
there, and I want to go through that process.
… And really again this is about the process, so we
did the simulator that works pretty well, it's pretty easy to get
up and running.
… It worked on my Mac, I couldn't get an older
version of Web OS 4 running on my Mac, I mean once you get that you
have to figure out the Java runtime, you have to make sure it
works, there's a VirtualBox that you might have to
install.
… I tried doing it on my Windows machine too.
That's again additional work which I again time boxed and I didn't
want to try to finish that up. LG provides this web OS tools, this
was actually very easy to install, this is just npm install Web OS
tool CLI.
… Once you do this you get these command line tools
this Ares CLI to actually deploy your app to a TV device and then
this is the steps using the developer guide to go through so I
originally got a USB cable and tried connecting that to TV and I
realized I didn't have to do that.
… The TV is connected with an IP address, I had to
go through this Ares generate, I bundled my hosted app using the
Ares generate command.
… No, I already had an app created so I had to
bundle my app, deploy the web app to a TV, so I had to package the
app.
… I did that and then in order to install the web
app, this is where it gets a little bit tricky, you have to first
go create an LG account, then on the TV you have to enable
developer mode, you have to download a developer mode app so you
have to go in the LG store, install the developer mode app then you
have to log in with your email address and password which is always
fun to do with a remote control, entering your email address and
password.
… So after a few minutes of that and updates on the
TV, the developer mode app is actually pretty good. You just enable
SSH on there, it opens up the ports and then using this CLI you can
find the TV, set up the device.
… I went through these instructions it was very
smooth, and eventually I was able to connect. I was able to package
up my web app and then do this Ares install.
… All of this worked really well, I got to the
point where I went to run the app and then I think there's some
stuff that I have to debug because it didn't fully run and that's
again on my end.
… But then I tried to debug it, which they do have
an inspector for the LG TV, unfortunately it's tied to the web OS
version since this is a Web OS 4 TV that I have it won't work with
the latest version of Chrome, it works with Chromium 68 which then
I had to download. I can't download that old version on my Mac so
then I couldn't get it working on my Mac.
… I tried doing it on my Windows machine. I got it
running there but then I realized on my Windows machine I then have
to install the Ares tools because it creates a proxy for you to use
the inspector so I can't connect through my Mac, I have to connect
through my PC, redo all the steps to be able to debug through
Chrome.
… Again I spent probably an hour or two hours going
through all these steps, tying it together, getting it up, still
not fully able to debug the app yet, but this was quite a process,
and ideally it would be a lot simpler with like not needing an
account to log into the TV, maybe there's a developer mode already
on the TV that I can just toggle on that gives me access, but
overall I think the instructions and everything are clear, it just
takes a little while to run through them all and actually get on
the TV.
Samsung Tizen
ChrisL: So now I'm
moving on to getting Tizen working.
… I have a monitor here that is Samsung, it's an
excellent monitor, USB-C it's connected to my desktop right now and
that's what you're seeing on the screen this is actually the
Samsung monitor.
… I think the first struggle that you run into
trying to deploy to a Samsung Tizen TV is there are two separate
URLs with very similar docs, so I'm going to be spending my time
figuring out how to actually deploy an application to this monitor
and figure out the bundling process and what I need to
install.
ChrisL: Once I get it
up and running I'll report back with the steps.
… OK, so after a few hours of trying to get my web
app going on a Samsung device, I've basically given
up.
… I tried, I have the Tizen CLI installed, you have
to get the Tizen Studio installed, then there's VS Code extensions
which I got installed and then you need to install the .NET
Framework.
… It's really difficult, I've basically given up
because I have only a few hours and in that few hours I was still
not able to get it going.
… I think one of the big issues is there's
conflicting documentation all over the place, with this Developer
Samsung versus tizen.org. Some of the documentation is outdated,
there's there's no clear path forward.
… A lot of the Samsung tools are made for the
mobile phone plus also TV plus the watch and they try to make it
one way of doing it for all their devices which then makes it more
complicated to get it going on a TV.
… I did get my monitor to have developer mode on it
but in the long run I was able to bundle it up.
… I just don't know how to connect to the TV device
yet. I'll probably try continuing a little later.
Wrap up
ChrisL: I guess that
kind of wraps up. The purpose of this video is to demonstrate how
simple it is to do the development, to run it on a dev box that is
made for running, and just you give it a URL.
… And then as you move into LG TV that was pretty
decent, it still took an hour especially you don't know what you're
doing.
… And then we go on to the Samsung TV, I I still
haven't been able to get it going and I have years of experience
working with many different tools.
… And then I have a HiSense TV and that's like
another different development process, and ideally we should get to
the point where you can just turn on developer mode on your TV give
it a URL and it launches your website. That's the
ideal.
… So hopefully this was informative, people can
learn something from this video. Thank you!
ChrisN: Thanks! Any questions?
Piers: I'm curious about the image on the Raspberry pi. Is that something that's available?
ChrisL: It's a Raspberry Pi 3 with with RDK and WPE browser
ChrisN: So there's a
lot of variations for TV devices, depending on the TV
manufacturers.
… Ideally you want to be able to point to web
server on your local network for testing purposes.
… We would need to engage the manufacturers, to
progess towards a common approach.
… It would be interesting to see it
happen.
Song: LG WebOS is quite
new to me.
… It looks interesting for interactive app
devleopment.
<ChrisLorenzo6>
https://
Song: If developer
wants to use the simulator, it's dedicated to to particular
manufacturers, or does it work with any Android TV platform? If
WebOS is available other platforms? Any special requirements for
remote controls?
… Any special requirements for remote
control?
ChrisL: The remote
control is pretty standard, but keys can vary across devices, but
we can map them to left, right, up, down controls.
… LG has a magic remote with mouse pointer and
click events.
… Primary goal here is for getting on TV devices
for testing.
… All have a browser, so the dev environment should
be standard, but the thing that differs is performance, for testing
if the app is fast enough, because the TV is much slower than your
development box.
… TV device might have only 1GB memory, also CPU is
much slower
ChrisN: OK, let's follow up this. Thanks. Anything else?
(none)
Media Content Metadata CG proposal
Endo: I'll share my screen
https://
… Media Content Metadata CG Proposal
Endo: We had a breakout
session at TPAC 2023
… We confirmed that more discussion is needed in
W3C. So we decided to set up a CG, an open community to exchange
ideas between a wide range of industries
… The group will share best practices for media
content metadata, in Japanese language, focusing on existing
business issues
… We plan to propose the group in May.
… The group has two scopes: media industry
collaboration and outside media industry
collaboration.
… We have member candidates from industries,
including 5 non-W3C members. I'll share progress with MEIG, and
want to discuss with MEIG members.
Chris: MEIG is happy to support, look forward to seeing your progress.
Kaz: Endo-san, do you have a timeline for launching the group, at some point in May?
Endo: We plan to have
an event at the NHK open-house on 13 May.
… We hope to propose and establish the group before
that event.
Spatial media requirements
ChrisN: In the March 19 Media WG meeting, Apple presented gaps identified for spatial media on Vision OS devices
ChrisN: I'll just run
through the topics briefly.
… First, presentation of stereo video, a single
video stream with different content for the left and right
eye.
… Pointing to the need for rendering capability
detection.
… It's not really a decoding problem, useful for
different devices with different rendering capability.
… Next thing is motion safety when presenting
immersive video on immersive device.
… If there is too much motion, people may feel
unwell
… so should have some kind of metadata to detect
it, then the device can respond by adjusting how the content is
rendered, maybe in a less immersive way.
… Another topic is display capability detection,
how to detect the capabilities of full screen mode?
… The context for this: on Vision OS devices, the
browser is presented as a 2D plane in the 3D
environment.
… So you can essentially see the browser in a 3D
environment. And the browser can show 3D content within the web
page.
… For example, the proposed HTML model
element.
… So this is about applying a similar kind of
approach to video media presentation.
… Next, an accessibility related requirement was
identified for caption depth data
… I expect Apple would think of this as an
extension to WebVTT,
… to associate each caption with depth information
to avoid overlapping with 3D objects in the video.
nigel: Z order sounds
like an HTML thing, which is already supported by TTML and
CSS.
… You would need to use the correct units, identify
the actual distance,
… so there's a need for some better coordinate
system.
ChrisN: Yes, that's
interesting.
… It's not really Z order for video, it's more
about the depth of the scene.
… I think that was more how Jer described the
requirement than how they might see a solution.
… But is this something that is being worked on
elsewhere?
Nigel: The Immersive
Captions CG published a report,
… it's probably worth maybe asking them to join a
meeting to discuss.
… I'll circulate the document here.
ChrisN: Thanks
… We had some pretty productive discussions with
the Immersive Web community some years ago, but not followed up
since then.
Kaz: I also think this
is interesting and exciting. Thing about the synchronisation among
objects, data, and captions would be important for 3D video and
metaverse applications
… As a basic platform for next-gen video services,
I think it would be important for smart city purposes as well. They
think about metaverse and city-verse as a framework
… and digital twin platform for handling devices
and virtual presentation. The direction should become more
important for this spatial media discussion
… Existing standards like TTML and WebVMT should be
good candidates.
… The smart city IG should start next month, we're
wrapping up the wide review at the moment
Andreas: Chris, you
mentioned MEIG's previous discussions. We talked about it at TPAC
Fukuoka.
… In TTWG we were thinking of starting activity on
immersive captions, but we didn't see enough industry
interest.
… Also the immersive web standards were missing,
some baseline to work with to provide a solution for immersive
captions.
… At the time, we thought if we started something,
TTWG would try to align efforts on TTML and WebVTT so they follow
the same rules.
… There have been some discussions, meeting notes,
an incubator issues, may be worth collecting them again.
ChrisN: Good idea to collect the resource. Is there growing interest in this at the moment?
Atai: This came up at NAB just recently, there seems to be some interest now.
Song: Regarding
interest from developers in the CG, one reason could be there may
not be enough members from TV developers
… The features are relevant for people for
different industries, but the spatial media and TV OS topics are
interesting for me.
ChrisN: We have an Overview of Media Technologies for the Web document. Could be a useful place to document the current status and needs.
https://
… Your help welcome to do pull together those
resources and update the report.
Kaz: As Media WG and MEIG chair, do you want to think about joint discussion with Spatial Data on the Web WG?
<kaz> Spatial Data on the Web Working Group
ChrisN: I think MEIG should do that, let's follow up on possible topics.
ChrisN: To complete the
spatial media requirements, next is 360/180 viewport controls, so
controls on the video element to let users change the
viewpoint.
… Spatial soundscpes, need clarifcation on what's
needed here, it's possible Web Audio is enough.
… Environment dimming support, so controlling
brightness of the outside environment.
TPAC 2024 plans
ChrisN: Should we
organize a F2F meeting during TPAC 2024 on 23-27 Sep in Anaheim,
California?
… what topics to discuss? One proposal already is
next generation audio, covered at previous TPACs.
… Anything else? I would probably request 2 meeting
slots, so a morning or afternoon.
… Also do we want joint meetings, e.g., with TTWG.
Nigel, let's follow up on that.
Kaz: We need to respond to W3T by 20 May, so want feedback and ideas by end of next week.
ChrisN: Agree, that would be ideal, or the following week at the very latest.
[adjourned]