Editing TF face to face

23 Aug 2015

See also: IRC log


Chaals, Ryosuke, SamW, Enrica, SimonS, Grisha, Ojan, Piotr, Florian, Johannes, Johan


<johanneswilm> https://www.w3.org/wiki/Fall_2015_Editing_Taskforce_F2F_meeting

<scribe> scribe: chaals

Ojan: Interested in all three specs - input, selection and ce=events

RN: Not clear whether ce=events and input events are different enough to need differnt specs.

JW: One is the host for the other. Inputs has events that should fire on the old contenteditable, whereas new one isn't going to work backwards.

RN: Not convinced. If we want to make it so JS sets selection the browser doesn't normalse it, that should be always, not just sometimes.

Ojan: Have some detailed thoughts on minor things…

… high level thoughts: Like providing lower-level primitives people can build on. Ideally we implement a subset of cE=true.

… so cE would be like sandbox, with a list of things it enables. E.g. true, or, "events something other thing"

… would make sense to split events, one part enabling events aother enabling text input.

JW: it doesn't currently enable text input.

Johan: maybe movements should be a different thing from text input, layering features.

JW: That has been discussed the last week - should we have caret movement?

Caret movement

JW: There are things we don't know about that happen in the browser, and some of what we are hearing is contradictory - can we change contentEditable or will it break everything?

… so for example @@ has been sayign we need something new if we are going to change the normalisation of selection, but that can't be changed or we break the web.

… And I don't know if that's the case. Unclear to me with caret movement - if you reove normalisaiton of selection, can you have caret movement and will that break the selection?

RN: You mean if the caret is rendered elsewhere?

JW: If it is actually moved into eg a bit of SVG.

RN: Think that is something that we could fix as bugs.

… if the element is not visible we cannot render the caret, but that's obvious

JW: WIthout normalisation of selection the browser handles the caret movement but the caret can disappear sometimes so you need to implement it in JS already.

EC: If you let JS handle movement there could be a place where the caret goes and the browser cannot render one e.g. where there isn't a height computable.

JW: You normalise the position now to make sure this doesn't happen

RN: That isn't why we do normalisation. So even if we lost that you should be able to render the caret where you want it.

… it is normalised for editing, to reduce the edge cases we need to handle.

OV: Where browser isn't handling editing you could imagine not normalising.

RN: If that's what we are doing we dhould not normalise at all, unless we are running execcommand and normalise it there.

OV: there are some thing in common across blink and webkit codebases. We would also have to normalise right before copy. I think that is a reasonable change to make, but it will be some serious work.

RN: Not happening overnight but does need to happen.

JW: copy is the only exec command left that we need. But that's only relevant if you have a selection. Problem for editors is just the caret. If you only have a caret and people copy, you cannot copy?

OV: The idea is that you would not normalise the selection, but take the copy and then normalise what you have copied.

JW: That sounds even better.

Piotr: Copying doesn't have to put something to clipboard, we can do this in javascript directly.

JW: But we initialise that through an execcommand

PK: But ckeditor puts the content into the clipboard manually so we get the right stuff in there.

FR: My impression is that this area hasn't been a high priority area - how mch can we reasonably expect will happen?

Ojan: Think this is the hardest part and most "at risk".

… we will do selection eventually, but cannot give a timeline.

FR: You don't have anyone obviously ready to do it?

Ojan: Not sure…

SW: Browsers operate based on what others do… so may copy, especially once two browsers start doing something.

RN: There is also a question of benefit to cost…

Ojan: Want to convince our team soon to implement beforeinput and kill textinput. ContentEditable events? Should do, it is simple. Selection normalisation is hard...

Piotr: There was a difference between webkit and blink - when I added ranges, blink started returning the same ranges.

RN: You (ck) now normalise at execcommand, right?

PK: Yep.

RN: That part isn't that hard…

Ojan: So the thing I said is hard, we already did, right?

RN: Well, the hard part is to respect what you already did when you insert...

JW: SO two positions. We don't do caret movement in the browser. I think that is almost possible except maybe line endings and bidi boundaries. Other position is caret movement but we don't specify that. People making editors have been asking for standard caret movement, because moving in the block direction would be different.

… Don't see harm in having caret movement, but we may build code for something that wasn't specified and be stuck supporting that.

RN: We could have caret movement supported by ce=events, or we could have a caret that would allow you to move between elements.

Ojan: The question is should we specify caret movement? You can imagine having it as an optional piece of ce=

JW: Should we have caret movement prior to having a spec, or should it just not move?

RN: We should spec both at the same time

FR: Can you turn them on and off seperately.

Ojan: Proposing sepeartely.

SW: Seems odd to have a spec that says if you touch the keyboard text doesn't happen but arrow keys do, so makes sense.

RN: And in collaborative editing you already have to render carets, so you don't want browser caret movement.

FR: Think spearate modes make sense, and we say it happens and we will need to specify it.

RN: Not sure if it makes sense to have caret but not events. As soon as you have typing, you need caret, makes no sense seperately?

Ojan: Not so sure. E.g. you have special widgets but don't want to deal with typing yourself?

RN: But if you have IME, you need caret there
... So should events and caret be disjoint, or caret be a superset of events?

SW: So what do we mean by caret movement? Arrow keys / mouse, or implicit carets when you paste, or what?

PK: Drag and drop is affected too.

FR: We can say caret movement is implementation dependent, but we need to define early in what triggers movements. E.g. drag and drop, ...

RN: HTML5 says selection shouldn't move on drag/drop, but webkit/blink doesn't do that

FR: Worth trying to scope what triggers there are for caret movement.

RN: Yep.

… There are some platform-dependent things, e.g. keyboard combinations may or may not trigger a selection.

… but e.g. ordering of events should be specified

… on Mac we don't have touch events, but on iOS you have touch events moving the selection.

Enrica: Selections won't be drawn without user interaction, but they exist in the DOM and can be acted on.

RN: So if you interecept touch events, and do things to the selection when the user can't see it, you confuse people.

Ojan: Sam's point is interesting. If you paste, when you only have events and selection but not caret, what happens?

… so maybe makes sense to have superset relation

RN: We can define it like that for now, and see if we need to change it later.

Ojan: Don't see anything that will break, for the moment, so let's go that way.

RESOLUTION: We think we want strict subsets - events is a subset of caret is a subset of typing which is a subset of true.

Ojan: Think it makes sense to have one spec for editing, one for selection...

JW: There is also execcommand...

Ojan: Don't integrate that.

JW: Right but e.g. clipboard want things to be updated in that spec.

RN: Might need to specify caret movement in selection API...

JW: Yeah, we'll just say there will be a spec at some point.

RN: If it is going to be specced, it will happen in selection spec.

[spec development synchronisation …]

RN: So say a browser moves selections, where do they move to? Right now they are browser-specific… makes sense to specify that.

Ojan: Yeah, spec should describe what is going to happen there.

RN: In most applications what you want is the selection to be in the text node, not in an element boundary…

Ojan: Need to discuss this with Mozilla.

FR: If we can arrange call in, Ehsan from Mozilla can do that.

Ojan: There are a couple of points that would be good. Selection normalisation.

EC: Is it worth trying to standardise where selection will be on caret movement, or leave it to application?

… if we define strict rules we don't leave freedom to app developer about what is going to happen next…

CMN: We should standardise, but need to include the possibilities which makes things complicated...

Ojan: We should define primitives so you can hook into all the places, even if you don't implement those in your high-level.

FR: Doing reasonable things for people especially in regards to i18n makes sense because it saves a lot of work for app developers.

RN: If people are implementing at a low level like ce=events you have to do more work than if you're doing ce=true...

FR: Yeah, but i18n in particular is an area that people really don't know what they needed to know, so doing it for them is useful.

RN: Websites can always get things wrong. make things mostly work unless they specifically ask to do things otherwise.

… what if you want to show a widget on top of the caret… in multiple writing modes?

… it's a crazy idea, except people want to do that.

FR: new CSS?

… there are user stylesheets...

SW: Yeah, although they don't seem to work out well.

… What are the real problems?

JW: By making most basic tasks easy and not break, we can try to ensure editors can get the hard things like vertical correct…

RN: Example, at a boundary of vertical, the caret is shown differently depending on which direction you came from.

EC: Are we talking about the caret, or selection?

RN: Think selection…

EC: In that case eg vertical becomes more important, because selection is a pretty important thing beyond just editing.

SS: To the point about boundaries, you're inside the element or outside it, and those are different positions in the DOM.

RN: Meant bidi boundaries

Ojan: We would give primitives so you can put the caret where you want it, but we wouldn't make all the controls for where the browser moves the caret if it is handling it.

RN: So do we expose that the caret moved from left to right to get where it is now?

FR: The question is how do you find out where the caret gets to? Do we need to specify the superset?

JW: There are also end of soft line breaks.

RN: We have split caret in webkit…

… do we expose that as well?

Ojan: Yes. Don't see an alternative if you want stuff to work.

RN: And then we have to add more for each system…

CMN: Yeah, but this is likely to be a reasonable sized set to try and track.

Ojan: When we write down what editors want to do, it becomes clear what we need to expose. So we should do that exercise.

JW: We have been talking about that and haven't come up with a whole lot of new requirements...

RN: SUre, but there are very few people who even know all the things.

Ojan: So we need to look at the primitive, as an extension to selection - which end of line or bidi are you at?

SW: Do we need to expose things that match our internals?

Ojan: Where a single DOM position can be two different screen positions, yes we do.

RN: So e.g. before or after a line-break.

JW: getclientrects gives two rectangles in those cases - in which is the caret - that is the question…

RN: Right, because the answer now is "depends".

JW: So are there any other cases than "in one, in the other, or in both?"

RN: For bidi you need one more bit of info - the bidi level the caret is on.

JW: So having 2 carets, of which one is preferred, is that semantically different from having a single visible caret?

CMN: SO the need for bidi-level is when you extend the selection at a bidi boundary.

FR: Are we talking about enabling all things people do today, or determining what they can do?

JW: So you need a get and set of the bidi level...

SW: What do editors need here?

… if the browser says "when you modify selection one character forward this is what we will do" is that enough, or do you need to be able to know more?

PK: Bidi is really unclear, having "what we will do" is a good start. I don't think many of us editors want to mess with that on our own.

… understanding why the browser chose the location, that's not critical

FR: But if you are controlling teh caret, do you want to control how it is shown?

PK: Yes.

Ojan: You want to be able to save and restore character position with full fidelity, so you need to have that information.

SW: So if you have a shadow cursor of where something would go, you don't need the nitty gritty.

Ojan: Yes, when you remove the DOM and then need to put the selection back. Or collaborative editors...

RN: Don't think we want to have browsers implement other browsers' selection behaviour

JW: Most common use case is end of line - you reimplement in JS moving up and down…

… RN you already started on a proposal right?

RN: For line breaks yes. For bidi, I think we need another attribute for bidi-level…

Ojan: Can we add those to range? So you can get/set on a range?

RN: Yes, in the context of multi-range.

Ojan: should not be just set-position.

SW: We need some new objects that represent the internals of selection...


proposed resolution: we want information in selection that says where the caret is - before/after linebreak, where in a bidi boundary it is, and a way to say how to put it in a given place (before, after, or at both ends). Primitives for a superset of all browser behaviour, but browsers will choose for themselves where to put it and leave editor implementors to normalise that if they want it

<rniwa> line break issues: https://github.com/w3c/selection-api/issues/32 and https://github.com/w3c/selection-api/issues/51

RESOLUTION: we want information in selection that says where the caret is - before/after linebreak, where in a bidi boundary it is, and a way to say how to put it in a given place (before, after, or at both ends). Primitives for a superset of all browser behaviour, but browsers will choose for themselves where to put it and leave editor implementors to normalise that if they want it

RN: So we are probably looking at a trinary state…

… need to decide whether this will be on a range or selection.

Ojan: Use cases: Save and restore with selection in place, and being able to take the caret and do things to it. I think the latter should be a different API.

FR: You have enough information from the browser to expose the state, but not necessarily enough power allowing overriding of platform conventions

JW: If you want to move the caret in block direction by yourself and it jumps from start to end of the line, what do you do?

[How browser internals really work for this matters]

RN: hesitant to expose implementation-dependent rectangles, because people will rely on it.

JW: If you look at aloha2, they just disallow one caret location.

FR: When you move down from a non-Bidi line into one, where does the caret go?

[moving up and down ends of line vs offset that is at the end is tricky… but not horribly so?]

RN: The issue is restoring the history of that movement.

Ojan: A react style virtual DOM, where you are constantly reloading, is a stronger use case.

… feels like this is just another bit of information...

RN: we want to add more and more info...

Ojan: Yeah, but it isn't an infinite list.

FR: Dropping down from a straight line to a bidi boundary, you want to be able to set a physical caret location.

RN: How does the app do this?

JW: Point to caret position thingo helps do that.

RN: If we had a clone selection, and modify the clone, you don't modify the actual user selection. If you move the selection down, you are done...

… why figure out the position manually?

JW: Many times you want to put the character eg into a floated block…

FR: You don't want to rely on the browser to move your caret, you want to be able to do it yourself.

CMN: editors want to override the browser placement.

RN: Not sure we want to allow that

Ojan: You can just try all the bidi positions to get the one you want.

JW: We can and do put carets everywhere, except for linebreaks and bidi boundaries. Not having those two possiblities is problematic.

Ojan: Think we need some object that has all the state the browser uses internally - offset node, line point, bidi level, now I think inline-position…

… not worried which one it should be. seems reasonable that the API returns this object, which doesn't seem hard to implement, and would meet all the use cases

Johan: If you want to get to a point that is not visual, you cannot. You need to send the point inside the rectangle.

SW: Think the solution is that this is on document, probably need to put it on elements as well.

FR: And you should be able to do it in the viewport of the scroller.

Ojan: We don't hit-test things that are clipped. If we need to do that it is new...

JW: Aloha has this problem. Why not just scroll and then move there?

PK: What if you have slow scrolling, how do you get there?

Ojan: We should leave this for now...

SW: Is main use case to extend selection into something that is not visible?

… if so wouldn't API to extend selection be better?

FR: Extend forward and backward? Assumes browser caret movement is adequate, which isn't always true.

SW: Just API that says "if you move the caret, here is where we will put it" so you can check, then decide what to do.

EC: What Ryosuke said about cloning the selection without affecting user selection.

Ojan: That's backwards…

… we don't hit test for this. There is a separete code path for that.

EC: caretrange from point does hit test in webkit

RN: If you have selection modify on a clone you I can't think of a use case that means you also need to convert x/y to putting caret.

FR: Because e.g. the browser cannot get to the place where you want to go.

JW: You want to implement caret behaviour in your system that is consistent across different platforms.

FR: If you have DOM position plus line-end you know where it goes, but for position plus bidi level you don't.

RN: You can predict - it is beginning, end, or both, reliably but dependent on platform.

Ojan: If you get the rect where the caret goes, is that enough?

… you get caret position. Caret range from point returns that. There is a method to get the rect(s if there are two). Is that enough?

RN: In visual selection you want to be able to say "select from this coord to that one"...

JW: So in effect you can specify line start/end and an equivalent for bidi?

[Ryosuke draws a complex case to look where the offsets are in RTL text with mixed bidi levels]

RN: You will not be able to do restoration across different browsers, because they use different offset numbering in the line.

proposed: For now we can live with not being able to solve the edge case of knowing which end of a bidi block you get to when you move the DOM across to a different browser

RESOLUTION: For now we can live with not being able to solve the edge case of knowing which end of a bidi block you get to when you move the DOM across to a different browser

Ojan: CSSOM has offsetnode, offset and getclientRect - we will have to reconcile with that…

SW: Only Mozilla does that.

Ojan: We can convince them to change that.

<rniwa> Filed https://github.com/w3c/selection-api/issues/58 Need an API to get the current caret rects


<rniwa> Filed https://github.com/w3c/selection-api/issues/59 We need to expose line direction coordinate as a property


JW: Benjamin Peters looked at this and his first conclusion was that if we do inline IME we need to allow typing because it changes the DOM along the way…

… which is why we did ce=typing. Then Koji pointed out that this could be handled somehow in shadow DOM, so we could have inline IME input and when the content is constructed you create the beforeinput events and don't change the DOM in the meantime.

[We are talking about how inline IMEs work. Some are a popup, others show a composing character and then a final…]

SW: Do we count e.g. a composing character and then the result?

RN: Think this is more of a ligature case.

OV: IMEs are arbitrary programs that do their own internal thing. There is a pipeline of start composition, do stuff, end composition, and then you commit the resulting text… sort of…

… so for inline IMEs browsers modify the DOM as you type mid-composition, so if you autosave it might grab the half-composed thing. Escape will undo the composition.

RN: This varies depending on which language…

OV: dependent on language, OS, …

JW: supposedly this can be done without changing the DOM. Benjamin talked to the office team, ...

FR: is autocorrect like an IME?

CMN: Yes, OV: no...

RN: Doesn't work exactly the same, but you are trying to replace the range of text with something else.

OV: But there is no middle point, you replace something with something else.

FR: swipe keyboards?

RN: Yes.

JW: Mac-style composing accented character?

EC: That's not composition - the combining character is there, it is just that you might change it subsequently.

OV: On android everythig is IME.

JW: The issue is if you have a collaborative editing environment, and you send a tilde ready for composition, can someone else make a mess there?

… if we can get it down to this character is being committed, that would simplify many things when you coordinate editing

OV: The option-based combination on Mac is an IME...

CMN: The yandex translate input will let you type some letters in Latin, and offer an auto-complete as if it were a russian keyboard.

OV: What if you start composing an email, and hit send mid-composition.

RN: Real users don't - but anyway, the send confirms the current suggestion.

[This is highly variable according to platforms]

RN: Let's say you have <p><b>H</b>ello</p> and your caret is before the H, and the IME wants to insert…

… we attach the shadow DOM, conceptually speaking - shadowroot -> <b> -> H

… with the cursor at the H

OV: With shadow DOM you replace what is rendered with something else. So in HTML you won't get the composition. I don't like that because the lowest level element can be aribitrarily big.

… I think you want the thing in the DOM, but control over whether it synchs to collaboration, e.g. to demonstrate how to do the composition mid-typing.

… what if we put a composition element where you are composing?

RN: That was the original proposal.

JW: Main problem is with split text nodes.

… say we put the composition in the middle of two text nodes, start composing in the middle, and cancel, do you merge the two nodes which used to be one?

OV: Text nodes get re-normalised all the time…

RN: Problem modifying the real DOM you enter ce=typing, and you end up modifying a bunch of DOMs

OV: I'm a bit confused :S

RN: In ce=events we don't want to modify the DOM at all.

OV: I was assuming that in ce=events you have to get the composition events as they come.

RN: Different IMEs use different ways to e.g. identify what bit of the current composition is being worked on.

OV: How does browser do it?

RN: It's broken in blink. But Firefox gets it.

OV: We should expose what the browser does… draw an underline

FR: That varies enormously. Do you want to expose that as a superset of all platform behaviours?

OV: Today we have compositonStart, …Update and …End.

… if browsers tell you what the user typed and what they think the current compositoin is, that's enough…

RN: You need seperate ranges for each part of a composition. There is some privacy concern, because it can expose user-defined dictionaries.

OV: Seems far-fetched concern...

[not so much…]

[Rysouke gives example of how splitting japanese text in different ways exposes things in your dictionary, such as names.]

OV: Don't think this is a big deal...

RN: This is as bad as spellchecking, which people have said they don't want exposed.

[FR explains how to use this as a fingerprinting technique]

JW: collaborative editors are breaking because they get the composition events as they come.

OV: This is where I am going with the composition. A good editor wouldn't let a collaborator mess with what is being modified. Today you can't stop that because you don't know what is being composed.

… my concern with shadow approach is performance.

RN: We could do that with caret but don't want to put it in the real DOM.

JW: Koji said we don't need that...

OV: But I think he is wrong.

JW: If you save in the middle of composition, what do you do when you load again? You don't want to arbitrarily merge sibling text modes.

OV: Makes no sense to me to have event-only thing where you are modifying the content typing in an IME.

FR: Agree. But for visual aspect, who should display?

OV: Depends on IME…

…inline part is done today by the browser.

RN: Not necessarily. Depends on teh browser.

FR: So you want to make it the javascript's job choose what to display?

OV: yes.

JW: Can we get hold of Koji to get input on this?

OV: Koji's approach is possible but not performant...

JW: Is having stuff in shadow DOM less performant than normal DOM?

OV: No, it is that you have to re-render...

CMN: worst case is <pre><code> …[4000 lines of code] … </code></pre>

RN: You could have a mode where a caret element is disabled. EMpty in most cases

JW: You split text nodes…

RN: Yes but you never remove the caret. You see them all, you just don't show the characters

JW: When you split text nodes to put a composition element in the middle you have to send to everyone that you split the nodes.

… this is complicated.

RN: But you can do that easily enough.

JW: There may be other reasons why you have separate text nodes…

… e.g. so people are overwriting different parts of the same text node.

RN: You don't want to multiply split text nodes - that would be super inefficient.

JW: Avoiding splitting text nodes seem preferable.

RN: Don't think you have large text nodes

OV: when you have a code element you do.

RN: you can put <br>s in to split the nodes

OV: But that doesn't help in the shadow because you still take the element.

CMN: what if ce=events doesn't let you mess with composiotion and if you want to do that you have a new ce=composition

RN: Can't think of a use case where you want to handle composition on your own?

CMN: I can but they are a uncommon.

OV: Google Docs doesn't use the system IME…

CMN: Think use cases for messing with composition are obscure enough to justify a seperate layer…

RN: only use case I can think of is when you want to custom style the text being edited, e.g. spacing.

FR: How does Firefox get this right?

RN: It gets the info from IMEs and renders itself.

… we don't want web authors to do this

OV: We should be able to expose all this to the script

CMN: Not as a high priority. There is low value for most people, it is very difficult, and unlikely to be done right.

… Not that it should be impossible, but it isn't a priority to expose first.

FR: And it must not be the only way to handle this - developers should be able to just rely on system IME

RN: We want the editing to make it easier to develop editing software, this is a hard and often unnecessary piece.

OV: OK with an incremental path. Don't like that we have IMEs that are modifying the DOM as they go. If we had composition element you throw the composition into that, and that gives you control of what happens in the composition.

RN: That is the design. You could set where the shadow DOM shenanigans happen.

OV: Don't see that.

RN: We have "user about to type". You can get that to set insert point. We don't have a real shadow DOM. When text is being composed in a real DOM, script can interfere with that and it causes trouble.

OV: Are you going to change ce=true to do this?

RN: Yes, we could.

OV: Top level needs to do the same as the bottom level.

RN: THink it makes sense to do this in ce=true.

OV: I don't.
... Suggesting that in compositionStart you tell the browser where to put the composition

… difficulty, you need to deal with where the characters go...

FR: What can happen to caret while you are composing?

RN: It is somewhere in the composition.
... WOuldn't be rendered while composition is happening.

JW: As long as solution *permits* having composition in shadow DOM I don't have a problem.

OV: Think my proposal works fine…

RN: Proposal is to set the insertion point?

OV: Yes.

RN: That is the selection model, right?

JW: We have all the pieces necessary?

[we think so]

OV: So it is going to modify the actual DOM...

PK: If I create a blank text node between two others, and tell the browser to insert there, will that work?

OV: Hmm. Should be possible, not just on an element.

PK: Text node will be modified as you work?

OV: Only if you ask for that. You can do it elsewhere if you like.

… composition point is text node plus offset into it?

RN: need to spec what happens when the author modifies the text-node mid-composition

PK: Think it should break the composition.

FR: Seems reasonable. You shouldn't do that anyway…

RN: So we are coming back to ce=typing...

OV: Just for active composition. How is this different?

JW: Because you don't change the DOM until you have the composition complete.

OV: Why is that better? Why do we care about modifying the real DOM?

RN: You may have some states on server

OV: But in this case you have control over that yourself.

… if you don't want to mess with text nodes you don't have to.

RN: I guess so.

… for consistency we want to support regular text input as well or it will be weird.

OV: I was envisaging that you handle composition yourself.

SW: So does your vision include the dropdowns?

OV: No, you say where the dropdown goes, and you show teh text.

… I hear editors saying they are not about to start writing their own IMEs.

JW: If the basic things work you can do the high level stuff. If the basics don't work, then you do less.

… your proposal that you have to create things doesn't seem terribly hard.

RN: So maybe we should just go back to ce=typing. That would make more sense

JW: Why did we go away from that in the first place?

OV: I'd like ce=events someday. But don't think this stuff belongs to the browser.

RN: Don't think we want apps to be doing this - even huge important apps get it wrong.

OV: I want IMEs to be included with standard libraries.

… imagine we have ce=events, and you imported an IME library.

… That would be fine. But if you want to change it, you can.

FR: Doubt people will do that. MS, Google and Apple ahve all failed to get it right.

<inserted> [chaals and Ojan agree that we want something like ce=typing for the easy case, and to be able to rebuild it all in js using ce=events]

OV: So what was the pushback on ce=typing?

RN: When we started exploring spellchecking

[security/privacy issues from spellchecking, spellchecking crossing element boundaries, so editors disable spellchecking, use a 3rd party spellchecker…]

JW: Gmail uses spellcheck from a 3rd party but also uses built-in. So you have both cases. If we can turn off spellcheck, that's good enough.
... Weren't there issues with carets in typing?

RN: Where you are IME composing in typing you cannot change styles.

… in plain typing you can do that…

… so eg you can have beforeinput say "user wants bold…"

FR: Because we do not want to specify events relating to IMEs now, we want to skip that and do ce=typing

RN: Think we should focus on typing today. Don't need to put the future things in...

OV: Think we should note in the spec our expectations of what we will do in future

JW: Right, and focus on typing for now.

RN: Sure.

OV: For chrome on mac with chinese, if you move selection during compositionstart into an element you can already make that work… (for this one test case)

… if that doesn't work it is the sort of thing we could get into one of these specs.

FR: If you are in typing or higher, you get composiiton.

OV: If you are using events, you implement the whole shebang, you don't get input.

FR: If you are in typing or higher and canceling all caret events, what happens to caret movements during composition?

RN: You can cancel them

RESOLUTION: We are going to focus on describing ce=typing as a priority.

FR: Caret movements during composition are a bit magic and you don't want them to be overridden.

RN: Right.
... It's a bad idea to change carets/selection when a user is typing in general… don't do that.

… only tricky thing about IME is that sometimes the caret will disappear, e.g. japanese.

CMN: Don't think that is weird to actual users because that happens already.

OV: Think it is fine.

RN: When you have composition text script doesn't know where it starts and ends in the DOM.

OV: You can if you move the thing on compositionStart and create a place.

… and caret weirdness is something you can handle if you go into the lower level and do your own IME.

RN: Think when you are composing we want to say "please replace this bit of text with that bit".

OV: When you first start you get compositionStart, then updates, and then end with a beforeinput. And you say that has a request to replace text.

RN: Should we fire an event for this, or just repace the text?

OV: In my mental model, we just replace. That's part of typing. There is a beforeInput you can cancel…

RN: So we fire that for each one of the replacements.

OV: it's IME specific. if it is inline with multiple changes, you fire multiple events for them.

RN: What if the user deletes?

… we need to ask the app when we delete it because browser cannot. That ends up weird.

… if we allow replacement, or deletion, then we are allowing the browser to delete text niside a text node.

… Once you deleted past the text node, the browser won't handle it, but inside the text node teh composition handles it.

OV: It is weird to have typing without deletion, and to have that you get execcommand.

RN: So you do this in two steps, asking the app to handle the actual replacement

OV: Think that is OK.

RN: The pushback was that people would make the assumption deletion never happens.

OV: App authors will get IME wrong if they don't think about it.

SW: Seems like we don't need to make it trivial to get this right, but that it is feasible to manage...

OV: There are not that many people who want to write an editor...

RN: For spellcheck we do need a way to say please replace a range, already.

… so we want to use that in the IME case too.

… so I would prefer to always use beforeInput requesting replacement, even for IME replacing text nodes.

OV: Who does underlines?

RN: Browser. But we know the text node where we insert.

JW: Where is content during insertion?

[in the text node]

OV: Backtrack - what if we had ce=events but instead of magic shadow we give you beforeInput that says replace this with that, and browsers do the underlines for now, until we later explain what happens for real?

RN: If you don't use the same text node to insert, we have no idea what happened…

OV: Yeah. just break.

SW: In some way that we will define.

RN: We wanted to know that what we are trying to insert is what the IME is seeing.

SW: Why do we need to ensure that?

FR: So what will happen if you do that...

… cancel composition and do a text diff?

SW: You let them insert that...

OV: And the underlines become unpredictable and we cancel the composition

FR: But then you need to do a diff...

SW: You just have it go funky.

OV: Think that is fine. Lets you do what you want, it's a lower level version of the same thing.

FR: Think it is fine but should specify the error case.

OV: Sure, but doesn't amtter what we say.

FR: most dev-friendly is if you put something else then cancel composition, but that requires diffing. Putting funky underlines isn't that great.

OV: Don't.

FR: You'll have a problem where authors don't consider people who use IMEs

OV: Don't think this is a big problem in practice.

PK: So you ask for stuff to go into a text node, but where.

RN: what if you have a react thing? text node is gone...

OV: React has a model for stateful elements, that you would have to use. But that is a reasonable thing in react.

PK: What about big selections?

<johanneswilm> JS IME, chinese, 3 years old: https://github.com/hermanschaaf/chinese-ime

FR: Browser says "delete selection", app deletes it, and says "now insert this".

RN: Text node will be super hard to figure out in practice. Thing we want to underline might not exist.

OV: What is you get beforeInput and tell browser "here is where I put the composition"

JW: Just composition, or character input in general?

OV: Could be on compStart / update. So react can just reset it.

RN: Think we should do it on beforeInput so auhors don't have to listen to comp events.

OV: Default would be where the selection is.

RN: Just use the selection...

OV: Sure, seems fine.
... Note that this is for ce=events.

RN: Oh…

JW: So you get requests for changes as beforeInput events, during composition phase.

OV: As long as you don't move around in the middle of composition, you are OK.
... You do composition start, and at some point there the selection is at a point. Then you get your first character, "replace text(null, "я") …"

… wherever the thing started, you have to tell the browser where that is.

FR: You do that with selection.

RN: When you insert a character you need to move the selection forward.

… feels weird.

FR: During composition, selection must enclose the entire composition.

RN: The text node where the caret is contains the composed text.

… seems weird.

OV: So we add API to say "here is where composition goes"

… all to work around us not telling how we do underlining.

CMN: Don't think it is an issue if a caret disappears in an IME where it always has done that.

JW: You have to manually insert all the characters. How does the browser know that it is part of the same composition.

FR: You replace, then tell the browser where that is.

JW: You have replace content, and then say where the content you replaced is, and then you get a line.

RN: This API puts the app in control of the text node, and the app tells the browser where the underlining should be.
... In our original proposal we have a hidden text node where the browser does stuff, and when finished we tell teh app to insert the text node. Ojan's proposal is to request replacements character by character, and the app replies with a pointer to where the change happened.

PK: Maybe instead of a text node we specify a range, that must be anchored within one.

EC: Don't we already have the target range for any input event?

RN: Yes, but once you mess with that, you don't know where it went.
... Feels like few people will get this right

FR: Yeah, but the alternatives are worse.

OV: We have ce=Events with shadow DOM thing, or with "put this in a text node", or ce=typing which fixes that but has the delete issues to deal with.

EC: So we decided not to focus on events, lets try and focus on typing for now.

… all the app developer needs to know is that this is happening. They get all the events, but also notified if this is happening inside an IME session. So they can just leave it alone, to avoid creting problems.

… we have an API that notifies everything consitently, but you can tell them "this is in IME, so do nothing".

RN: If you delete an entire text node and there is an element boundary, the deletion needs to be done by the app.

EC: Yeah but you are not in composition anymore.

RN: That leads us to where we have two delete events

FR: composeDelete and delete. first you ignore, second you implement yourself.

OV: Seems fine

RN: Seems like the simplest thing we could do today.

… least insane of our current set of options.

FR: Are these cancelable?

<ojan> https://w3c.github.io/editing/input-events.html

OV: [segues to] input events are in composition. most IMEs don't support canceling updates… (bottom of link above)

… basically most IME don't let you cancel stuff they did.

JW: All makes me think when compstart happens, move it to some sensible point…

OV: Think this is reasonable. Cancelable should be false for events that cannot be canceled so authors know if they can.

FR: Set it per IME, or per event type?

OV: Don't feel strongly either way. Few IMEs allow cancelling so it seems mostly academic.

RN: japanese lets you cancel teh whole thing...

OV: But not a particular keypress in teh middle, right?

RN: Right.

… just say those things cannot be canceled.

EC: Not sure we can finalise all these. Start with something like no cancelable, and then do the homework to see if there is anything else we can say or negative implications.

OV: Fine to say these are not cancelable.

FR: So if we do this, does it leave door open to do =events the way we are starting to explore?

RN: Think we could do that…

FR: Seems compatible to me, but I am not confident I know...

… hope we can still do this one.

JW: There is a chinese IME in JS on the web - is there anything wrong with that?

OV: Just talking about the way the IME interfaces, not the entire IME from scratch.

FR: It's not a trivial task - it is a *lot* of work.

proposed resolution: We will have typing with compstart (cancelable) when you can set a selection point, then a series on non cancelable beforeinputs to replace text, and two kinds of delete events (one inside one across event boundaries)

RESOLUTION: : We will have typing with compositionstart (cancelable) when you can set a selection point, then a series on non cancelable beforeinputs to replace text, and two kinds of delete events - one inside composition not cancelable, and a "normal" one to be handled by the application

accessibility and IMEs

RESOLUTION: We will ask the accessibility people to look at IMEs and the problems we should have considered here but didn't.

<scribe> ACTION: chaals to ask accesibility people about IMEs [recorded in http://www.w3.org/2015/08/23-webapps-minutes.html#action01]

<trackbot> Created ACTION-758 - Ask accesibility people about imes [on Charles McCathie Nevile - due 2015-08-30].

Multiple ranges

SW: Apple wants to bring these back.

OV: Controversy is how to do it.

… @@'s suggestion was that unless the API looks teh same for single and multi range, people will get it wrong. His idea was to just expose a node list.

… don't think that works.

… nobody depends on multi-range today.

FR: How do people note there is more than one node in the list?

OV: That's the common case, so you have to handle it.

EC: How do you handle selecting part of a text node

OV: That was to be determined.

SW: 2 options...

… 1. Assume nobody pays attention because it was only in one browser.

OV: Anecdotally, it is alleged even developers could not get it right.

SW: Not sure I buy that. Appkit has it and it has worked fine - you get an array of ranges…

… there are fewer appkit apps than pages, but also pages don't all need to mess with ranges in interesting ways.

… there are needs for it, but the number of important cases might well be relatively small

… We could try to implement the way firefox does it, or we could leave window.getSelection alone, and add a new window.getMultipleRangeSelectionItemsThingMethod()

… and start a marketing campaign.

… Not sure the latter is worth doing. The existing API has some oddities, but it isn't that crazy.

RN: We could forbid people from modifying ranges directly.

… just return copies, can only add things from user selections...

FR: Cases where it is useful may include editable fields and noneditable, and trying to select one or the other…

… there are use cases beyond visually contiguous things

SW: Makes sense

FR: But don't need to be able to do any crazy thing...

RN: Having ce=false inside a selection makes me think having a list of nodes makes more sense because you can just remove nodes.

… you have a list of nodes that are inside a selection

SW: Seems more complex than existing model from FF.

OV: Yes

JW: What about pseudo elements that cannot be covered by selections.

FR: If we are inventing something new, we should think of that this time.

OV: One issue with ranges is that they are live. That makes me incline to a new API more than anything else.

SW: There are a bunch of other things that you would want in a new selection API - each subselection to have positions and affinities to know how they are selected, endline state, …

RN: affinity only matters in collapsed.

SW: Thought it mattered in extending a selection.

[some discussion of this on whiteboards]

OV: Authors can modify selections in various places, so you do need to be able to collect endlines etc in each range

RN: Affinity only matters for single DOM offset.

OV: Do you want a new range class with the endpoints having bidi level, line affinity, … ?

SW: Think your initial point is more pressing. A new selection algorithm we would probably not make live.

… if you want to change things you listen for a selection change and re-get the selection.

OV: If you hold on to live ranges, you get a lot of slowing down

… you could have deadSelection/Range …

FR: If we say we cannot drop ce=true, I think we should get around specifying it properly.

… if execcommand takes in we have a whole set of new dragons to specify. If we have a new API, we don't get a herd of dragons.

SW: Think it would work OK with execcommand.

… worried that we will not specify execcommand in the short future. It will take a monumental amount of work to get the spec complete and implemented.

… I'm not sure where it is going. So if we want to do something like this we have to pave our own way. Think that's fine, and might help improve execcommand, but don't think these should block each other.

SW: Anyone else get pressure to implement multirange selection?

OV: Not much. For flexbox a bit.

FR: Have heard some statements saying that it would be a nice thing to have for a better world

OV: There are people who believe that, but not sure about users.

SW: That's pretty much our case too.

Ilya: how does it benefit web authors?

SW: Don't think it helps authors, the drive is users.

OV: More about copy paste than anything else. Select something that has a position fixed in the middle that you don't want there (except when you do want it there)

EC: Shifting problem from cannot do it to "how do we make a UI" we learn a lot.

OV: There are not specific requests, but it does come out of more general requests.

EC: question for e.g. editor developers

Johan: We fake up table column selection…

PK: It isn't extremely critical, it would be nice.

… some of our algorithms wouldn't handle multi ranges

RN: Think a pain point is range can get into a messed up state. If you don't allow ranges to be modified, and they have to be sane, it would be good.

FR: Why not just get exception for doing dumb tings?

RN: Lots of edge cases to deal with

OV: It is slower, is one problem.

… but probably OK.

OV: giant tables can be arbitrarily slow

SW: But isn't something people would do with real selection...

SW: Only argument in favour of existing DOM selection is not duplicating API surface area.

OV: That is a big pro…

… the big con is live ranges.

SW: Maybe we start down this and then decide dead ranges are more efficient.

OV: What appkit does makes total sense.

… difference between getting a list, and querying into a list is important. Getting a list is clearer.

SW: There is a cognitive burden. If we did it today we would return a JS iterator.

… so you have to iterate it each time.

FR: So arguments for new API: dead vs live, cognitive load, collision detection

RN: If only browsers can make the selection...

… we could create multirange selections limiting what users can do to select

OV: That limits code selection.

FR: the thing with a floated section in the middle of somewhere won't let you include whether you pick the thing in the middle or not.

SW: Being able to construct a selection is important.

EC: Also becuse you need to be able to restore it.

SW: And you would be limiting what browser can do in the future.

OV: I am much more comfortable with multirange that returns iterator, and even more so if the object weren't live.

SW: SHould think about how we can get away with that. I guess we have don that before.

FR: So what happens to the current one? Do we develop it as well?

OV: Other problem with firefox' is that start and end don't make sense in a multi-range world.

… ranges should have anchor and focus. First range is anchor range, last is focus.

FR: Do we want to support "out-of-order" ranges?

OV: To match visual you have to do that.

EC: How do we want to manipulate the selection? In its entirety, or each range? Then you run into collisions…

RN: What if we only allow setting an array of ranges?

SW: This isn't going to be expensive.

OV: It will if you disallow overlapping.

RN: You could just normalise when you make overlapping ranges.

OV: Normalise we do is in a copy of the selection.

SW: Think the big thing we need is more implementation experience.

OV: Are you going to do that?

SW: Yes, we're planning on doing it...

OV: Would love to see dead ranges, iterator, @@@@

… would be nice to pare it down, too.

… setbase and extend doesn't match the rest of the API

RN: Think that came from web compat requirement.

SW: Should add new names for setanchor/focus.

OV: right.

[noodling on ideas for a new potential selection object, ranges…]

EC: Next order of business would be to define reasonable behaviour for copy.

OV: and paste.

… don't think we should spec execcommand in a hurry, but cut/copy/paste would be good.

JW: Clipboard API wanted updates to that. I preferred they put methods for security etc in clipboard API, and in execcommand we could just point to their spec, so they don't rely on each other.

OV: mostly care about the behaviour of what copy does.

JW: We asked for a copy method, they said it will come. Until then they update execcommand on those points.

FR: Do we think ce=true and execcommand should be done in a decade or so?


Summary of Action Items

[NEW] ACTION: chaals to ask accesibility people about IMEs [recorded in http://www.w3.org/2015/08/23-webapps-minutes.html#action01]
[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.140 (CVS log)
$Date: 2015/08/25 09:05:35 $

Scribe.perl diagnostic output

[Delete this section before finalizing the minutes.]
This is scribe.perl Revision: 1.140  of Date: 2014-11-06 18:16:30  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: RRSAgent_Text_Format (score 1.00)

Succeeded: s/typing/ce=events/
Succeeded: s/@@@/Ehsan/
Succeeded: s/with/… with/
Succeeded: i/OV/[chaals and Ojan agree that we want something like ce=typing for the easy case, and to be able to rebuild it all in js using ce=events]
Succeeded: s/run away/move it to some sensible point/
Succeeded: s/>/?/
Found Scribe: chaals
Inferring ScribeNick: chaals
Present: Chaals Ryosuke SamW Enrica SimonS Grisha Ojan Piotr Florian Johannes Johan
Got date from IRC log name: 23 Aug 2015
Guessing minutes URL: http://www.w3.org/2015/08/23-webapps-minutes.html
People with action items: chaals

WARNING: Input appears to use implicit continuation lines.
You may need the "-implicitContinuations" option.

[End of scribe.perl diagnostic output]