Meeting minutes
<Hew> Reminder that the meeting is on Zoom today...
This meeting
Nigel: Today we have DAPT issues, IMSC-HRM (one pull request/issue)
… and any update on the charter FO.
… Is there any other business, or anything to make sure we cover?
Hew: I can give some feedback on DAPT-REQs
Nigel: That'd be great, thank you
group: no other business
IMSC-HRM - Coalesce empty ISDs into non-empty ISDs w3c/imsc-hrm#50
<Github> https://
github: https://
Nigel: This PR is primarily to deal with the clears and short gaps between ISDs - is that right?
Pierre: We're trying to address the industry practice of 2 frame gaps between successive
… subtitles or captions.
… Today the HRM imposes a very high cost to those short gaps.
… This PR addresses that.
… I think we're down to just one question:
… Should there be any cost to displaying an empty ISD?
… i.e. painting it and presenting it
… I am arguing that because the model assumes that subtitles are painted and presented on
… a graphics plane that is then overlaid on to the related video object, then displaying nothing
… costs nothing because you don't display that pane altogether.
Andreas: Makes sense to me.
Pierre: Another goal of this PR is to make as little change as possible.
… That's where my mind is, why I'm arguing that there is no cost with presenting and painting empty ISDs.
Andreas: As I understand there's no algorithm that would calculate a clearing action, right?
… What it means to have an empty ISD that follows an ISD with content, that would then be a clearing action.
Pierre: My suggestion is that clearing happens only in the buffers where things are going to be painted.
… Before drawing a new non-empty ISD you have to clear the back buffer, and that has a cost.
… You first clear that, then draw the non-empty ISD, then when the presentation time comes in you
… flip the buffer and display it.
… If you clear it before then there's no cost because you just don't display the buffer.
… Clearly the "clear" is related to the cost of clearing the buffer onto which an ISD is painted.
… What I'm suggesting is that in fact there is no need to paint empty ISDs so there's no cost to not displaying
… the graphics plane, period.
Nigel: That's an argument I hadn't understood before, so good to know.
Pierre: By the way I'm just trying to come up with something that requires as little change as possible
… to the model. I hadn't thought exactly this 2 weeks ago for instance.
Nigel: Good to clarify that you're not saying that the cost of CLEAR is zero.
Pierre: Yes, the cost of CLEAR is not zero, but there's nothing to clear for empty ISD.
Nigel: Then the question is if we always need at least one buffer to be composited, and
… therefore that we need to count the cost of clearing a buffer at least once for each clear.
… Or can we just say "stop compositing" at no cost.
… I think the conservative approach is to say that we assume there is always exactly one buffer
… being composited, and therefore we need to count the cost of preparing an empty buffer if there is
… to be an empty ISD after the current one.
Pierre: Within the model that's not possible with only 2 buffers.
… If you clear the back buffer for the empty ISD and then present it then you have to wait
… for the empty ISD to be presented which defeats the purpose. You would need a third buffer.
… What I have in mind is that the screen is being refreshed by the video.
… There's no cost to not painting on the next refresh. Not drawing an overlay is a no-op.
… When the next frame comes in you draw neither of the two buffers.
Andreas: The HRM is a theoretical construct to calculate presentation cost of ISDs.
… That doesn't mean that is how it is implemented in practice.
… You could view it like you do but of course in practice it could be like Nigel says when empty ISDs come in.
… Follow a logic that is not yet implemented in the HRM.
Pierre: Sure, exactly, I agree.
… I think what we're trying to do here is to tease out where the main complexity is, how it scales.
… I think I'm arguing, I've convinced myself, in general, you could have a cost to not displaying pixels,
… you could have a third buffer, maybe your implementation is on still images and clearing has a cost,
… but in the case of video the cost of not displaying an ISD is not substantial and can therefore be safely
… ignored.
Nigel: Sudden moment of clarity for me that the current pre-PR HRM is what you have to end up with
… if you have a 2 buffer model.
… I think what we're discussing here is that the idea is the implementation flips from compositing
… buffer A to B to A to B etc
… but in my mind I'd imagined that there was a compositing plane C
… and the task is to blit A to C, B to C, A to C etc which would have a very different set of constraints.
… I'm not sure which it is right now.
Pierre: The presentation buffer Pn-1 is directly connected to the display in figure 2.
… This was designed with a simple TV architecture in mind, where one or the other is displayed on screen,
… or read directly by the display circuitry.
Nigel: I see what you mean about fig 2.
… But again, fig 2 has no switch about whether the presentation buffers are connected to the display or not.
Pierre: We could add that to the model, that with empty ISDs nothing is displayed.
… The PR has text that goes beneath that figure, but we could modify the figure to make it clearer.
Nigel: Yes. I'm just wondering which is preferable, to insert a switch to the display
… or to posit a 3rd buffer.
Andreas: A new buffer would add more complexity to the model, right?
Nigel: I don't really think so.
Andreas: For me the question is if for some implementations there would be some cost when an
… empty ISD is going through the change, and if this cost is high enough to be taken into account
… for the calculation of complexity.
Pierre: Exactly.
Andreas: The cost is negligible and not go into the calculation.
Nigel: I think that introducing the third buffer and counting the cost of a CLEAR would not
… fail anything that passes today, but there are some documents that could potentially fail it,
… and we should catch those.
Pierre: Concerned about the editing complexity that would introduce.
Nigel: I haven't done the exercise, it seems like it _should_ be simple, but maybe not.
Pierre: If you want to try making the edit, go ahead. I'm not sure what it will look like.
… You need to introduce the cost of drawing into the back buffer.
Nigel: I would have to make the assumption of a cost-free blit into the third, composition buffer.
Pierre: If you want to give it a shot, as we pointed out, adding the cost of a clear for an empty ISD
… will land up with results that are closer to what we have today. It's not going to
… invalidate documents that are valid today. That's not my concern.
… It's updating the text in a way that does not cause more trouble than it solves.
… I'm happy to modify figures - I have the master files, though they may be checked in.
Andreas: I want to echo a bit Pierre's concern more about the complexity of the specification and
… unexpected consequences of adding a new concept.
… The issue is that the whole model is already not so easy to get, from non experts
… or people who really need to implement it.
… Adding a new component just increases the complexity of the model.
… From an understanding point of view.
… When we consider adding it, we should compare the benefit of adding it to the
… benefit we get from getting closer to reality for this particular case.
… We heard that even if we add a component it would not really change the results very much.
Nigel: Either proposal is a change to the model.
… I'm concerned about clarity of that change.
… The risk is that it's so subtle that it is not actually noticed properly.
… I'm not sure if the mathematical results would be identical.
Pierre: They would not be because you'd count one extra CLEAR per sequence of ISDs.
… Neither proposal would invalidate a set of valid documents, which we don't want to do.
Nigel: I'm interested to know what is the edge case between the two ideas where
… one would validate a document and the other would say it is invalid.
Pierre: The cost of that extra clear would be included in the cost of painting the following non-empty ISD, right?
Nigel: I think so.
Pierre: Let me try something.
… Scenario: non-empty ISD, empty ISD, non-empty ISD
… The model limits the complexity by saying painting the second non-empty ISD takes some time
… and the system has some time to do that, and what we're saying now is that the
… time available to paint the second non-empty ISD is the time between the presentation time of
… difficult to say without graphics on screen - [shares screen]
… [discussion of 2 buffer model plus 3 buffer model by looking and pointing at the rendering time figure]
Pierre: We could schedule some time for a call next week?
Nigel: That's a good shout - I should be able to do that.
Pierre: I'd really like to get to the point where we ask for feedback, and
… we need to complete this issue before we do so.
Nigel: Yes
Pierre: I am available at the same time next week
Nigel: Me too
SUMMARY: Discussions to continue offline and possibly in additional call
DAPT-REQs
Hew: Our AD operations and product owner reviewed DAPT-REQs and felt
… that it captured their current workflows and that there would be no problem
… capturing what they do, within it.
… They raised possible future questions.
… For example richer pan options if you want to get creative in e.g. 5.1 audio or other distribution approaches.
… They wondered about extensible synth voice options to include tone.
Nigel: Emotional tone maybe? Pitch is already available.
Hew: Yes, that could well be what they meant.
… If you think about dramatic AD particularly they try to carry the tension or emotion of the programme.
Hew: That was it.
Nigel: Great, thank you, always very helpful to get reviews.
Charter FO Status update.
Nigel: As discussed last week I submitted the Chair's report, and I got an acknowledgement back.
… I expect the Chair's report to be appended to the document.
… Atsushi, anything to add?
Atsushi: The report will be made available to the FO Council in the next couple of days.
… Stepping down from Council is ongoing - I am not sure when they will actually start to discuss it.
… I believe they have 2.5 weeks for stepping down prior to Team report so they should start shortly.
Nigel: Thanks for the update. Any questions?
… I see nobody on the queue.
… Q from me: did the Objectors respond to the draft Team report?
Atsushi: I have not seen a response. WG participants can see the updated report at the same URL.
Nigel: Thank you.
Meeting close
Nigel: Thanks everyone! [adjourns meeting]