13:00:50 <RRSAgent> RRSAgent has joined #gesture
13:00:50 <RRSAgent> logging to https://www.w3.org/2018/05/30-gesture-irc
13:01:00 <Zakim> Zakim has joined #gesture
13:01:24 <chaals> Present+ Fuqiao, Navid, Chaals, Max_Dapeng
13:01:29 <chaals> Meeting: Gesture events
13:01:59 <chaals> -> https://github.com/JuntaoPeng/GestureEvents/blob/master/GestureEvents.md basic proposal
13:02:16 <chaals> i/Present/scribe: chaals
13:03:47 <chaals> present+ Xiaozhen
13:04:19 <chaals> present+ AnQing, YanDong
13:06:02 <chaals> chair: Max
13:06:36 <chaals> ML: This is the second meeting of the CG - the first meeting happened last year, before launching the CG.
13:06:49 <patrick_h_lauke> patrick_h_lauke has joined #gesture
13:06:51 <chaals> ... Today we have a proposal from Alibaba.
13:07:06 <patrick_h_lauke> present+ patrick_h_lauke
13:07:07 <chaals> agenda+ AnQing introduce gesture events proposal
13:07:25 <chaals> agenda+ issues for further discussion
13:07:37 <chaals> agenda+ further work
13:07:41 <chaals> agenda+ Anything else
13:07:47 <chaals> zakim, take up item 1
13:07:47 <Zakim> agendum 1. "AnQing introduce gesture events proposal" taken up [from chaals]
13:08:15 <chaals> [slides being shown on Webex]
13:08:49 <chaals> AQ: Let's look at the motivation, and what to do. Basically we want standards for gesture events on mobile phones, coming from the smartphone OS.
13:09:09 <Max> Max has joined #gesture
13:09:09 <chaals> ... purpose is to specify common interactions like pinch, swipe, or double-tap.
13:09:45 <chaals> ... we propose to define gesture events. The motivation is simple, W3C defines touch events and pointer events but these describe limited operations. In daily life we have to use more events.
13:10:10 <chaals> ... So Web developers write JS to implement gesture detection from a sequence of touch/pointer events. This is very inefficient.
13:10:37 <chaals> ... There is also a lot of learning to make this work.
13:11:11 <xfq> RRSAgent, make minutes v2
13:11:11 <RRSAgent> I have made the request to generate https://www.w3.org/2018/05/30-gesture-minutes.html xfq
13:11:25 <chaals> ... enabling gesture events will improve performance, making user interaction work more smoothly.
13:11:25 <xfq> RRSAgent, make log public
13:12:22 <chaals> Navid: You want to do gesture detection, so you expect the OS to find the gesture and pass it to the browser by name?
13:12:42 <xfq> RRSAgent, make minutes v2
13:12:42 <RRSAgent> I have made the request to generate https://www.w3.org/2018/05/30-gesture-minutes.html xfq
13:12:58 <patrick_h_lauke> everybody's muted
13:13:01 <NavidZ_> I lost audio too
13:14:41 <patrick_h_lauke> i think the question was: do you expect these events to be generated like touch/pointer events (by the UA), or by the OS
13:14:50 <patrick_h_lauke> referring to the "originate from smartphone OS" line
13:15:13 <patrick_h_lauke> but i believe that line is in the slide more as a "gestures are a thing that started gathering momentum/be a thing on smartphones"
13:15:27 <patrick_h_lauke> rather than "it's generated by the OS"
13:16:06 <NavidZ_> Thanks Patrick :)
13:16:12 <patrick_h_lauke> anytime :)
13:16:28 <chaals> AQ: We want to handle these on the browser, but the functionality originates in the OS.
13:16:41 <patrick_h_lauke> (i think the answer to the question is: it'll be a JS API similar to touch/pointer events, generated by the browser)
13:17:05 <patrick_h_lauke> (unless the browser can HOOK into the OS' own gesture recognizer)
13:17:09 <NavidZ_> But does the detection happen in the browser? do you expect the gesture event to come from the OS?
13:17:16 <NavidZ_> like a tap event from the Android?
13:17:28 <NavidZ_> I don't think Android has such a thing
13:17:35 <NavidZ_> it has some helper classes for that
13:17:41 <NavidZ_> which detect the gesture based on the raw input
13:17:55 <patrick_h_lauke> for platform-agnosticity you probably want the browser to have its own recognizer
13:18:11 <chaals> AQ: We want to standardise some common gesture events [see slide]
13:18:14 <patrick_h_lauke> but to leverage any helper available at platform/OS level perhaps?
13:18:34 <chaals> q+ to note that some events are handled differently e.g. with screenreaders running.
13:18:37 <xfq> yeah, that's what I think too
13:19:16 <patrick_h_lauke> (random: https://twitter.com/patrick_h_lauke/status/778558022381600768) ;)
13:19:23 <chaals> Topic: Discussion points - usage of gestures?
13:19:59 <chaals> AQ: [shows some demos from the Web]
13:21:17 <Max> q?
13:21:46 <patrick_h_lauke> i think in general, both the use cases and the fact that user and authors want gestures is undisputed. the crux of the problem is getting agreement, AND getting all the players to agree that this can be standardised in light of IPR concerns
13:21:51 <chaals> AQ: There is maybe 20% touch-based scrolling, 15% of sites using touchstart, from chrome telemetry metrics.
13:23:09 <patrick_h_lauke> chaals: if i have an iphone, and turn voiceover on, double-tap and swipe and possibly others are taken up by voiceover. if a site relies on gestures, it won't work in this scenario
13:23:20 <patrick_h_lauke> q+
13:24:01 <xfq> ack ch
13:24:01 <Zakim> chaals, you wanted to note that some events are handled differently e.g. with screenreaders running.
13:24:22 <patrick_h_lauke> chaals: it would be good to have an abstraction - of what the user WANTS to do, rather than what gesture they are executing
13:25:43 <chaals> PHL: Chaals' point was that it would be good to avoid having event names that are tied to specific user movement on the platform - e.g. double-tap, or swipe'left.
13:26:06 <NavidZ_> patrick_h_lauke: Do you refer to say like click to indicate activation and we do fire that as well on tap?
13:26:13 <chaals> ... instead it would be good to have events that describe the user intent - zoom-in, pan-up, etc.
13:26:38 <chaals> ...  The proposal has fairly high level things already, buut we could go a bit further.
13:26:54 <chaals> ... Chaals' other point. Voiceover swallows e.g. double-tap, swipe, ...
13:26:57 <Max> ok, thanks Chaals.
13:27:42 <chaals> ... so if the site relies on that, the user will not be able to generate them. Or if the site overrides the input then the user will not be able to use their own interface.
13:28:44 <patrick_h_lauke> (i hope i encapsulated chaals' concern plus my take on it)
13:28:48 <chaals> ... Having standardised events, even at the level of "double-tap" or "swipe" means the platform can intervene, and enable users of e.g. VoiceOver to do *something* that generates a named double-tap, even if their actual interaction movement is different. So this would be better than the current situation.
13:28:50 <patrick_h_lauke> (and that i didn't crack up)
13:29:00 <chaals> [+1 to what Patrick said]
13:29:23 <patrick_h_lauke> and yes, taking the abstraction even further, would be Indie UI Events
13:29:28 <patrick_h_lauke> but that was abandoned/died
13:30:24 <chaals> [Note that if the browser is generating these events, it needs to make sure there is a way for the platform to intermediate and generate events with a different gesture. This will fail unless there is real implementation of the requirement.]
13:30:36 <chaals> [And the Web has a history of getting this wrong ;( ]
13:30:45 <chaals> AQ: We will return to this question...
13:30:51 <patrick_h_lauke> i personally think the proposed events (tapevent, panevent, pinchevent, longpressevent [which would be context click?], doubletapevent, rotateevent) are already fairly high level
13:31:12 <patrick_h_lauke> they're at least one step up from processing was touch/pointer coordinates to divine a gesture
13:31:37 <patrick_h_lauke> and Indie UI Events style events would be even more abstract. but those fell down a rabbit hole of trying to capture all possible user intents
13:31:39 <chaals> AQ: We see other usage data with lower numbers, but it is clearly something that developers are trying to achieve.
13:32:27 <chaals> ...  having these events will reduce page size, increase efficiency.
13:32:47 <patrick_h_lauke> in short: i personally think these gesture events as proposed here are a good stepping stone, as it's something authors are doing right now/are wanting to do, and it avoids potential for needing to constantly invent new abstract events that encapsulate intents
13:33:08 <chaals> YD: The point is to bring a mobile need to the W3C standard.
13:33:49 <chaals> [like Patrick, I agree that we need something in this space, but we need to make sure that it doesn't break because of assuming that a user gesture always means the same thing...]
13:34:08 <chaals> YD: We are trying to bring Web technology to enable the same power as mobile applications.
13:34:23 <patrick_h_lauke> but the meaning of a gesture is handled by the developer, so it's not our problem?
13:34:42 <patrick_h_lauke> for reference, apple's proprietary gesture events https://developer.apple.com/library/content/documentation/AppleApplications/Reference/SafariWebContent/HandlingEvents/HandlingEvents.html#//apple_ref/doc/uid/TP40006511-SW23
13:34:54 <patrick_h_lauke> and microsoft's gesture events https://docs.microsoft.com/en-us/previous-versions/windows/internet-explorer/ie-developer/dev-guides/hh673557(v=vs.85)#Gesture_object_events
13:35:09 <patrick_h_lauke> https://docs.microsoft.com/en-us/previous-versions/windows/internet-explorer/ie-developer/dev-guides/hh673557(v=vs.85)#gesture-object-and-events
13:35:20 <chaals> ... For the browser implementation, we built on a Wix? runtime. The events are needed by developers.
13:35:41 <patrick_h_lauke> and note, both apple and microsoft have not expressed any interest in standardising this
13:35:47 <patrick_h_lauke> as there are IPR issues
13:36:29 <patrick_h_lauke> for reference, Indie UI Events https://dvcs.w3.org/hg/IndieUI/raw-file/default/src/indie-ui-events.html
13:36:34 <xfq> s/Wix/Weex/
13:36:37 <chaals> YD: We think the abstraction like pinch and double-tap are reflecting the user intent, and this is helpful.
13:37:44 <chaals> FQ: double-tap is a user movement. That is one way. Another is that we standardise a set of user events - ZoomIn, Zoomout. The user interaction might be double-tap, or pinch...
13:38:14 <chaals> YD: OK, understood. From the developer, we think the meaning of a gesture depends on the application.
13:38:57 <chaals> ... double-tap can sometimes mean zoom in, sometimes mean zoom out...
13:39:04 <patrick_h_lauke> q+
13:39:19 <chaals> ... the meaning of the intent will be best left to the application developer
13:40:04 <chaals> PHL: In an ideal world we would have something like IndieUI tried. It would be good to have intent events instead of events that describe specific interaction.
13:41:15 <chaals> ... But it tried to capture a large number of user intents, and ended up with a very small set that could work, and nobody actually implemented the idea in the 3 years since it has been released.
13:42:29 <chaals> ... So there is some problem making that happen. As new inputs and new devices appear, there is a risk that we need to add new event types to deal with new intents user can have, and that seems like it would be very slow. So I don't think the high-level abstract definition that we would really like will actually occur.
13:43:10 <chaals> ... The proposal here is not reflecting user intent, but it does seem like a step forward from what we have now.
13:43:21 <Max> q+
13:43:26 <chaals> ... I will emphasise that the main problem in this space has been IPR issues.
13:43:31 <chaals> ack pa
13:43:53 <patrick_h_lauke> q-
13:44:16 <chaals> Max: This is not a new requirement, and we know it will bring benefits. The IPR issue is also important.
13:44:25 <Max> q-
13:45:01 <chaals> AQ: As PHL noted, Microsoft and Apple are unwilling to participate for IPR reasons. Is there a path forward for dealing with this issue?
13:45:19 <patrick_h_lauke> (as a side note, mentioning that even for standardisation of Touch Events apple raised a concern / initated a PAG)
13:45:28 <patrick_h_lauke> (if that's the right terminology...)
13:45:38 <chaals> Max: If we just define the API, does that get around the IPR issue?
13:45:40 <chaals> q+
13:46:16 <patrick_h_lauke> https://www.w3.org/2012/te-pag/
13:46:24 <patrick_h_lauke> :)
13:46:39 <chaals> [An API definition is not likely to infringe, but implementing it probably will...
13:47:06 <chaals> ... so if you have a proposal that nobody can implement, we didn't achieve anything useful....]
13:48:16 <chaals> [So we need to look at the actual IPR. If there are expired patents in this area, that means there are things you can implement....]
13:48:18 <patrick_h_lauke> and beyond being ABLE to implement, there also needs to be the DESIRE to implement
13:48:22 <NavidZ_> q+
13:48:28 <chaals> ack me
13:48:57 <patrick_h_lauke> noting how Apple has refused so far to do anything in the Pointer Events space
13:49:53 <chaals> Max: maybe there are multiple ways to implement an API. Whether that infringes something depends what specific patent covers
13:50:06 <chaals> ... Does W3C have resources to use for studying patents in this area?
13:50:25 <chaals> NZ: I think it would be useful to talk to the implementors, since they are the ones who are most concerned.
13:50:39 <patrick_h_lauke> q+
13:51:00 <chaals> Max: Problem is that the implementors are not IPR experts. E.g. in W3C Working Groups, if there are IPR issues a PAG will be formed with IPR experts, who are different people.
13:51:11 <NavidZ_> q-
13:51:17 <chaals> ... how can we get these resources in the CG.
13:51:53 <chaals> ... We need to understand what the IPR is related to this proposal, and the status of that IPR.
13:51:56 <chaals> ack pa
13:51:58 <Max> q
13:52:07 <Max> q?
13:52:49 <chaals> PHL: As well as IPR, this won't work unless browsers *want* to implement this. Working with pointer events, there was a time where Google were not keen to implement them even though they were  involved in developing it.
13:53:19 <chaals> ... because Apple refused to even talk about this. Google's thought was that this would not be interoperable so the implementation time would be wasted
13:54:10 <chaals> [Yandex felt the same way, from the developer side, wondering why they should spend all the time to make their sites work using pointer events when they would still have to make them work with plain touch events because browsers would not accept pointer events].
13:55:26 <chaals> PHL: If there isn't interoperable implementation, we either have to accept what e.g. Apple does which mmay have IPR risks and other technical issues, or we will force developers to do double work to deal with interoperability problems.
13:55:56 <chaals> Max: Important point. From developer perspective, interoperability is important. Browser vendors may have their own thoughts...
13:56:18 <chaals> ... This seems like a complex problem, unless browser vendors reach a common agreement. Otherwise this will be difficult to move forward.
13:57:14 <patrick_h_lauke> i would say, as Rick Byers and other mentioned in the email discussion leading up to this, a more productive approach would be to see if a polyfill/library could be developed/maintained collaboratively
13:57:25 <xfq> https://lists.w3.org/Archives/Public/public-wicg/2018May/0005.html
13:57:37 <patrick_h_lauke> as both a proof of concept, and as possible incentive for standardisation talk
13:57:47 <chaals> [I note that e.g. Alibaba can work with UCBrowser and find a few more high-profile people prepared to make the investment as a proof of concept - if it is good enough, developers might start to create pressure for interoperable implementation, which is sort of what happened for Pointer Events]
13:58:50 <patrick_h_lauke> AQ: [shows examples of some technical problems, like gestures that cross iframe boundaries and being able to detect these]
13:58:52 <Max> q?
13:58:58 <chaals> AQ: Handling e.g. pinch across iframes is difficult.
13:59:20 <chaals> [Yes this is difficult, do you have any concrete proposals to handle it?]
14:00:13 <patrick_h_lauke> (i think that question came from Rick or Navid, not me)
14:00:31 <chaals> [this applies to any multi-touch gesture - rotate, even swipe]
14:01:33 <chaals> [It also happens where there isn't a clear indication of what is the iframe boundary - e.g. a user starts a rotate with two finers each near the border between an iframe and the parent. Should the iframe get the rotate, or the parent?]
14:01:37 <patrick_h_lauke> q+
14:01:43 <chaals> ack patrick_h_lauke
14:02:17 <chaals> PHL: Probably the next steps would be looking at some of the polyfill libraries, maybe making a proof of concept to show how these things could look.
14:02:57 <chaals> ... maybe as chaals said you can get enough momentum to ship a proof of concept implementation, that might motivate further discussion and development.
14:03:35 <Max> q?
14:04:33 <chaals> ... Hammer seems to be abandoned, but developing on something like that would be helpful to show why this work would be useful
14:04:44 <xfq> -> https://hammerjs.github.io/ Hammer.js
14:05:00 <chaals> ... (or at least triggering the lawyers who might think they hold patents over this)
14:05:15 <chaals> ... Max: So next steps might be proof-of-concept implementations...
14:05:21 <patrick_h_lauke> "flying a kite for the lawyers ;) "
14:05:36 <Max> https://weex.incubator.apache.org/
14:05:57 <chaals> YD: We tried to implement this in weex runtime. I think next time we will have some demos based on this.
14:06:04 <chaals> s/Wix?/Weex/
14:07:26 <chaals> ... Maybe the blocking issue is IPR. We cann raise the question and try to involve people who know more about this - or developers who might have good use cases to move this work forward.
14:07:52 <Max> q?
14:08:31 <chaals> [thanks to all, adjourned]
14:08:33 <patrick_h_lauke> good stuff, thank you
14:08:37 <chaals> rrsagent, draft minutes
14:08:37 <RRSAgent> I have made the request to generate https://www.w3.org/2018/05/30-gesture-minutes.html chaals
14:09:11 <Max> thanks all!
14:09:43 <patrick_h_lauke> patrick_h_lauke has left #gesture
14:16:50 <xfq> s/mmay/may/
14:16:55 <xfq> RRSAgent, make minutes v2
14:16:55 <RRSAgent> I have made the request to generate https://www.w3.org/2018/05/30-gesture-minutes.html xfq
14:22:24 <xfq> s/Weex?/Weex/
14:22:27 <xfq> RRSAgent, make minutes v2
14:22:27 <RRSAgent> I have made the request to generate https://www.w3.org/2018/05/30-gesture-minutes.html xfq
14:23:58 <xfq> RRSAgent, bye
14:23:58 <RRSAgent> I see no action items