Technical Architecture Group Teleconference -- 30 May 2013

<trackbot> Date: 30 May 2013

<scribe> Scribe: Jeni Tennison

<scribe> ScribeNick: JeniT

Noah's thoughts as outgoing chair

noah: I hope to keep this short
... I've been on the TAG for 9 years
... chair for 4 years
... the role of the chair is to help the group think about what success is, and to help them deliver
... there are lots of new people in the group, and it's reinventing itself, and that's great
... none of this should be seen as an intention to stop you doing the things you're doing
... these are reflections for longer term
... this is an unusual group
... other groups have clear deliverables, and customers waiting for your work, who want to build on it
... there's a route for feedback in working groups
... we don't have that in the TAG
... one failure mode is thrashing on stuff that isn't focused
... worth reflecting on what the scope of the TAG is, and having 70-80% of the work fitting within that scope
... you are the senior technical steering committee for the web
... the only other steering committee does finances and stuff
... and the web is a really important piece of infrastructure
... evolving much faster than the phone system etc
... you have responsibility for a system that's really important and touches a lot of things
... from time to time think about it that way
... stretch out beyond your comfort zone sometimes
... you all have expertise in lots of important stuff
... for example, when Henry brought up the use case of librarians
... and whether they could use URIs given the persistence they need
... from time to time ask where the opportunities are
... you have to ask what makes the web different and valuable
... my answer comes back to Metcalf's Law, or network effects
... a virtuous positive feedback cycle where adding things to the network adds value to the whole web
... that gives you an imperative to think about things like how international the web is
... there was a time when putting a web server in a printer would have seemed weird
... now we take it for granted
... the TAG needs to think crazy like that, with some guiding principles
... ask yourself about whether it's worth being able to link data into the rest of the web
... you can get very busy working on stuff that you know is important
... web app developers are not a reliable proxy for everyone you have to worry about
... they're not a proxy for librarians for example
... this is a plea to think about the potential and the risks for the web
... this forces you to go outside your comfort zone
... don't just keep in touch with the communities that you know about
... the other thing I'd say is that we're the architectural group, and it's not entirely clear what that means
... this group should make sure the web works in 30/50/100 years
... phone numbers were built to scale, they scale to mobile
... think about building to scale
... for example, I think the surface area should be kept small
... you can decide what matters, I'd encourage you to think about it, to try
... it's easy to make decisions in an agile way when you have contact with your users, in an agile way
... but it's an art to make design decisions without that, as Tim did
... I'd encourage you to look at Tim's design notes
... a lot of what's in WebArch is a clarification and cleanup of those thoughts
... that doesn't make them all right
... but if you're on the TAG, I'd encourage you to look at the decisions that Tim made
... sometimes you should stand up to people, making longer term decisions
... when you do these things you can be wrong
... for example, we could see long term benefits with XHTML, and eventually the community came back and said it was wrong
... the three pillars of web architecture are URIs, HTTP and HTML
... Tim has said in order of importance they are URIs, HTTP and then content stuff (HTML, CSS, JS)
... a lot of stuff we're looking at at the moment is in the third category
... you should look for things in the first two categories
... in particular linking
... because that's what makes the network
... the reason people don't stay in iPhone Apps is because they want to visit other things on the network
... that's why it's important to think about unique names
... for example, what about internationalised names?
... one last thing
... it took me a while to realise how many ways Tim's presence adds to the group
... Tim works on a much broader range of this stuff than most of us do
... I'd encourage you to find ways to take advantage of not only his smarts and expertise, but also the scope of his knowledge and work
... finally
... because there's no one asking for things
... you have to be very motivated
... there's a huge amount of stuff to go out and proactively get into
... work hard, make it easy for your chairs
... and thank you
... thank you for Tim for making me part of this

timbl: thank you for chairing for such a long time, and for your contributions

ht: I'd like to follow up on one thing that noah said
... W3C has not done as well as we wish we had at having user representatives highly visible
... there are a handful of people who are clients
... users are underrepresented in the W3C
... they're a constituency that the TAG needs to keep in view, because no one else is going to do that
... that's a corollary of noah's point that the dev community are not the only customers
... not the only people we're responsible to
... trying to balance the requirements of different classes of user agents and different types of users are really important

dka_: the point about linking is really important, one of the things I want to bring up in the packaging discussion
... how do we have a packaged app and still use linking
... if it's possible, should we be influencing the work in that way

Futures / Promises

slightlyoff: we often create APIs which take the form dothing(opt1, ..., callback)
... where callback is a function that takes a single value

timbl: where the value passed in is the context

noah: you get context from the closure

slightlyoff: the simple synchronous version is contents = read(opt1, ...)
... in the synchronous version you'd wrap it in a try/catch
... that doesn't work in an async version
... so you end up another parameter for the error, or you have a separate error handler callback
... this shows up in a lot of APIs
... often when you're doing IO

timbl: the contract is that you'll get only the main callback or the error callback called?

slightlyoff: it happens on a WG by WG and API by API basis
... there are all kinds of patterns
... you might get an operation handle back
... op = read(...);
... op.onsuccess(...).ondone(...).onreadystatechange(...).onprogress(...)
... there are many different names for success
... they all reify down to a single event handler
... function(evt) { var controls = evt.value, evt.contents etc }
... there's a temporal dead zone between reading and binding event handlers

marcosc: this is freaking awful

slightlyoff: what people end up with is that they do the callbacks at some time in the future
... this can be linked into the HTML event loop
... some APIs define this well, some don't
... it's weird, it's verbose, there's inconsistencies
... it makes it hard for developers to work with

noah: almost every OS eventually gets an eventing system

slightlyoff: we should be driving towards a standard way of doing this
... this shouldn't be hard

timbl: what you're trying to do is express something in a synchronous kind of way in a asynchronous world

slightlyoff: promise = read(...);

noah: there's a type that is returned by read() which is consistent across APIs

slightlyoff: that's right, you can have p.then(func(v) {}, func(e) {}).then(...)
... then is called immediately but the functions aren't

noah: the read() delivers the value to the promise, the promise delivers the values to the functions

slightlyoff: you can call then multiple times

timbl: the thens occur in any order?

slightlyoff: they're dispatched in the order they're registered

ht: how does f2 know the name of the promise?
... how does it chain?

slightlyoff: the then() creates a new future
... so you can chain another then() on that future
... this is an important standard way of doing this historically
... many languages have this, for example Python has futures
... Javascript has no syntactic support for it, so it's just through these functional contracts
... C++ has Futures for example
... JS has gotten by with libraries that are similar to this
... they're really common, and people are using this pattern at scale
... so we want to get our house in order from the DOM perspective

ht: if there's no language-level support for this, how do the libraries which implement these solve the race condition problem?
... if they have to just work with callbacks

slightlyoff: in the web platform we have several APIs that give us the ability to delay work for the next turn
... they give you the ability to run code soon but not synchronously

timbl: timeout(0) for example

slightlyoff: JS and the DOM get along but don't have a contract
... JS has no idea of an event loop
... we're hoping to add something in ES7
... the DOM has been responsible for everything to do with time, except getting the time from the system clock
... the libraries use a call-me-back-soon hook

[discussion of GC]

noah: for medium/high performance this'll give you what you need

slightlyoff: this gives you standard ways for error handling too

timbl: I've found things get lost currently, do we get a guarantee that one of these things get called?

slightlyoff: I will talk about that later
... there are lots of libraries, want to make them compatible, have made Promises A+
... a proto-standard scrum, to standardise just the essential pieces of this contract
... eg number of arguments, the timing
... not the rest of the API design
... DOM needs this badly
... JS needs it too

timbl: which bits?

slightlyoff: generators and iterators which are nearly pythonic in their style
... and which aren't explained, they're just syntax
... it would be nice if we could describe them as promises
... in most cases we're on the UI thread

wycats_: we need them for modules

slightlyoff: it helps us reason about IO
... it would be good if both DOM and JS had a standard contract for their users
... since January we've been working with the Promises A+ community and the library authors

wycats_: all the pure JS libraries have coalesced around this

slightlyoff: annevk's draft locks down the ambiguities
... this is good for DOM
... we can make backwards-compatible fixes to existing APIs by returning a promise rather than void
... we want to make sure that APIs standardise around this same pattern

timbl: so this has consensus?

slightlyoff: there's no active hostility

annevk: there's some debate about specifics

slightlyoff: this was a hot topic at TC39 last week
... the big change is that we've renamed 'Future' to 'Promise'

timbl: was there any change to semantics?

slightlyoff: no
... there are two camps about how to use these things in the wild
... generally coming from perspectives of producer vs consumer
... the Promise can be thought of as the value
... or you can think of the value being received at another time

wycats_: abashed monadic promises

slightlyoff: we've won this battle in TC39 and DOM

annevk: you should say 'platform APIs' rather than DOM

slightlyoff: yes, we have agreement to use the same design, and to coordinate

<wycats_> you should consider this TAG work

slightlyoff: and it will eventually end up in the language

dka_: what can we as the TAG do, what's the output of our work on this?
... and what external bodies should the TAG be influencing to get consensus & buy-in?

marcosc: from a WG perspective, where we've been trying to use these things is guidance on how to hook into the spec
... like examples

annevk: the spec doesn't have hooks yet

marcosc: if the TAG made a document that gave examples for spec authors
... to make sure we're using the right language

wycats_: an explainer

timbl: is there JS for implementing it?

slightlyoff: yes, I've written one for example

wycats_: (to DKA) you should consider this TAG work

dka_: making that assumption, what is the output of the TAG?

annevk: note on promises

wycats_: a 'how to make APIs' guide that fast-tracks Futures

dka_: it's useful to be able to point to a document in TAG space that has the links to the relevant work

marcosc: and bug annevk to make sure we can hook into the spec

slightlyoff: we're not finished on the spec
... we've been successful at preventing design needs from leaking into the superclass
... eg notification of progress
... you need a subclass of a promise to give that
... need to get agreement on common subclasses
... the TAG could help spec authors understand how useful it is to have a common contract
... we've backdoored this because libraries have agreed, just have to standardise it
... the TAG should weigh in to say it's a good thing to do

timbl: have every jQuery implementation have a complete Promise API?

wycats_: it's an evolving story

slightlyoff: there are many opportunities here
... if we have a standard platform Promise type it's possible to add tools
... eg to show a list of outstanding promises that haven't been resolved
... if we can't standardise on a type we can't do that

wycats_: all existing libraries can wrap jQuery promises to create ones that meet their APIs

timbl: what's nice about try/catch is you can wrap it around a whole bunch of stuff

annevk: you can combine promises into a new promise

wycats_: the design is to make it easy to express things as if they were synchronous
... xhr.then(success).then(success2).then(null, err)
... and that desugars into try { xhr(); success(); success2(); } catch (e) { err() }

<annevk> Yves, ETA is a week or two, not sure if you asked that

<annevk> Yves, ETA for renaming that is and fixing a few things

<Yves> yeah, I was asking for ETA on the new version of the spec

timbl: what you want to do is write an asynchronous case in as concise a way as sync ways

slightlyoff: we can't get there without this

wycats_: in unix you have file descriptors for example
... but you need the abstraction first, something that represents the eventual thing from IO
... you need the primitive first

slightlyoff: this is the primitive
... and this is why I think the TAG needs to help drive this through the platform as quickly as possible
... it would be best if all the platform APIs use this pattern
... in the same way as we're relatively uniform over events
... there will be a similar issue over Streams
... there's no core idea of Streams
... then ES7 will probably bite off language-native events
... and it would be good to make sure that was unified with DOM events
... from an architectural perspective

timbl: using a promise has an overhead but is the idea that in the future will the language optimise it?

slightlyoff: anything that's common enough will get optimised
... especially if it's targeted by benchmarks
... having it standard opens up possibilities for optimisation
... it's never going to be actually free

wycats_: V8 happily optimises DOM constructs

slightlyoff: the more that V8 knows about the internal representation, the more it can do

plinss: what are the concrete next steps for the TAG?

slightlyoff: I need help convincing spec authors that this is a good thing to do

timbl: write what you think and we call it a Finding
... then you get a TPAC slot to point people at it

dka_: I think that fits nicely in how we described our work
... we have Note on Promises on the list
... this gets into the question of how we operate
... I'd like to see placeholder document for that before the end of the meeting
... which we use as a reference point
... if you have a commitment to write the content of the document, we can start shopping it around
... we can use it as a basis for the group engagement that we talked about yesterday
... even starting with discussion on web audio

plinss: we have to get consensus on whether this is something the TAG wants to throw its weight behind
... TAG Finding + TPAC slot might be appropriate, but do we need something quicker than that?

marcosc: it's hard to move people off the events model, so it needs to be clear how and when to do that

wycats_: the explainer needs to go into benefits and limitations

timbl: common open source code too?

marcosc: Alex has done some work on IndexedDB & LocalStorage for example, refactored

timbl: is it horribly cumbersome to do that?

marcosc: it's nice. The XHR example is very easy to understand
... integrating with events too

timbl: you mean the developer is using events as well?

marcosc: you can remodel the same thing to hook into the events, and use the promises you create to give an API that's very simple to use
... showing that is 20 lines of code and very useful

ht: that's a very high leverage way of engaging the web community

plinss: do we have consensus?

[yeses]

.RESOLUTION: The TAG supports Promises and will help drive their adoption throughout the platform

JeniT: what are the arguments against?

wycats_: one argument is the monadic promises thing
... which I think we can leave TC39 to argue
... the argument that is more relevant is from Node.js
... they coordinated around error-first callbacks
... where errors are called as the first parameter

slightlyoff: those are nicer in some ways
... by presenting the error as the first argument, the fact there might be an error is emphasised

timbl: I standardised on that because I was losing too many errors

wycats_: if you choose to use that pattern, you'll handle the error

slightlyoff: we can evolve Promises to do error-first callback

wycats_: error-first callbacks don't separate the call from the eventual value
... so it's hard to build abstractions around it
... you have to use all() around a bunch of functions
... it's hard to do combinations

annevk: the concern I've heard the most is completely different, that they're not stable yet

<wycats_> I wouldn't say it's hard, but it's hard to package up a bunch of eventual values and do abstractions around them

slightlyoff: yes, there's disbelief and unfamiliarity

wycats_: many people using Node.js use libraries that use Promises

timbl: the TAG can show the adoption path
... to show that new libraries can support old code
... demonstrate there's only benefit in supporting promises
... push it to libraries, then to developers

noah: are they actively pushing back?

wycats_: they can't avoid adopting, because they use V8

annevk: there are two conversations, one around the platform APIs and one around error-first callback

noah: we have to coordinate across W3C and others

<wycats_> https://npmjs.org/package/q

<wycats_> 322 dependents

noah: Node.js has expertise in dealing with async stuff, and they're objecting
... are they worried about object creation overhead?

wycats_: I haven't heard them bringing up performance

slightlyoff: you're already creating objects, in either model
... the Node.js community doesn't have the same latency issue as a game developer

<dka_> A refinement of the proposed .RESOLUTION: The TAG supports the continued development of the work on Promises, including helping to drive their adoption, socialisation of the work with more communities, and the continued refinement of the concept…

noah: if the TAG is coming close to making a resolution, we should reach out to that community

wycats_: even if it doesn't solve Node.js's problems, it solves platform API's problems

slightlyoff: the Node.js community isn't our community
... it's divorced itself from standard web APIs in several cases
... for example inventing a module system diverting from ES6

timbl: can I propose we wrap-up Futures/Promises, and queue as an agenda item the relationship between Node.js and the platform

wycats_: the Node

s/wycats_: the Node//

dka_: I suggested a clarification to the resolution wording

<wycats_> the node community believes that constant API churn is bad for their ecosystem, and therefore do not want to adopt ecosystem-wide paradigm changes if their solutions are working

.RESOLUTION: The TAG supports the continued development of the work on Promises, including helping to drive their adoption, socialisation of the work with more communities, and the deployment and development of the concept

<wycats_> they have developed node during a period of extraordinary language stability, and they are going to have to grapple with more language change eventually, beyond just promises

timbl: the message is that we have consensus between Ecmascript and the platform, and that's big enough to say we're not going anywhere else

Yves: is it too early to write a document?

wycats_: no

.RESOLUTION: The TAG supports the work on Promises, including helping to drive their adoption, socialisation with more communities, and the deployment and development of the concept

RESOLUTION: The TAG supports the work on Promises, including helping to drive their adoption, socialisation with more communities, and the deployment and development of the concept

ht: I wonder if we should try for a W3C workshop on this

[BREAK for 20 mins]

<Zakim> ht, you wanted to suggest a workshop

[BACK]

ht: we should have a workshop on the larger topic of platform API stakeholder consistency

marcosc: primitives

Yves: something at TPAC?

ht: possibly
... W3C workshops are another tool that we could employ

wycats_: the process of workshops aren't familiar or helpful for web developers
... particularly requiring position papers

ht: it's possible to have workshops without requiring position papers

noah: was the meeting a waste of time?

wycats_: it ended up meaning the workshop didn't contain developers, which made it less useful

slightlyoff: there were meetings last year that I participated in
... an invited group of people
... that and W3C workshop have similarities and impact
... the difference in style gives different outcomes

wycats_: you're too charitable
... at the workshop and developers at the Offline workshop
... the developers & annevk & darobin removed ourselves from the workshop
... the predefined structure of the workshop didn't work

dka_: in defence of workshops
... I understand the issues, but part of the point is to try to be an open forum
... what Alex described is a bunch of people who know each other
... the whole idea of the W3C workshop process is to announce publicly that it's going to happen
... to try to get people in the room who aren't talking to each other already
... and the point of the position papers is to eliminate kooks
... you have to have something coherent to say

wycats_: the people who wrote position papers were kooks

timbl: philosophically, if you want to get people together you have to find a way of doing it
... we found too many people coming to workshops
... we found that asking for a small paragraph in email as a position paper set a bar
... later as things got bigger we found that we needed a programme committee
... workshops shouldn't turn into conferences
... think of a way of getting the right people in the room

wycats_: I was invited but I wasn't able to speak

marcosc: it was an unconference!

wycats_: most of the people there weren't familiar with the actual problem

dka_: let's discuss this over lunch

wycats_: we have to discuss this before I'm happy with a workshop

Yves: this is why I suggested TPAC because you can present to a larger amount of people

noah: you're over thinking this

<darobin> [more precisely, most of the people there were familiar with the *use case* problems — i.e. that they couldn't deliver — but weren't familiar with the actual technical problem]

<Yves> and more generally WG that would need to hear about such topic

noah: if a workshop is potentially promising, make it someone's responsibility to organise it
... asking people to write stuff in advance is a good thing because it helps people think about what they want to say

<annevk> Basically, we should have a barcamp around the web platform. Workshops / conferences are a pain.

noah: put it together in an appropriate way for getting the people you want in the room

dka_: I was one of the chairs of the Offline workshop, so I want to know about what was wrong about it
... for this topic, we can move on

Streams

annevk: we don't know enough about this to talk now

TAG operations

dka_: we've talked about enough content to be able to ground this discussion about how we operate

plinss: the one thing on Promises to tidy up is a next action

dka_: marcosc, you gave a briefing to the SysApps WG about how to operate, I was wondering how much of that we could adopt

marcosc: there's what I did with SysApps, and what slightlyoff, wycats_ & annevk have done with private repos that they've then opened up
... we have a github organisation & repo that JeniT's already used for her slides
... it's easier in some respects than CVS
... on the other hand there's lots of infrastructure currently in place for the TAG
... like generating minutes etc it's super useful

<annevk> https://github.com/w3ctag/ exists

dka_: how we use github and moving things over to github is one key topic

annevk: we have a github organisation already

noah: the main things that we've gotten from CVS from the past decade is instantaneous path to publication in W3C space, with ability to maintain long term persistence
... sometimes that's useful to get coverage with IP rules

<annevk> Btw, if people can give me their usernames I will add them

noah: I've been told that some WGs have cloned repos into W3C space, but I don't know about the control over URIs

<annevk> Yves, plinss, dka_, ht, timbl?

timbl: github does not naturally allow you to host a website, it doesn't give you control over mime types

wycats_: you can manage those within github

ht: what's wrong with CVS? it's not broken

wycats_: other people who don't have CVS access can't collaborate with you

ht: if we have projects that need collaboration, fine, but we don't always need collaboration
... can't we use CVS for most of our work, and use github when we want to collaborate

timbl: Henry's concerned about public write access

wycats_: github doesn't give public write access

dka_: this is why I asked marcosc to introduce the topic from the perspective of other WGs
... then we can talk about how we mix this approach with the existing W3C approach to get the best of both worlds
... this isn't an either/or

noah: my question was about what control you get over URIs

marcosc: currently, W3C has a htaccess file that redirects from W3C space to github space

noah: is it easy to make multiple URIs and include dates in URIs?

marcosc: yes

wycats_: there's a huge gap in understanding about how github works here

dka_: noah, you're conflating how we are doing it and how we should do it

noah: we've needed dated versions so that we can discuss drafts in meetings and having a dated version referenced from the minutes

marcosc: when you check something in, you can see diffs for changes

wycats_: you can do everything you want to do with tooling on top of gh-pages

noah: when the URIs are mirrored from W3C space, do you get stability?
... what happens if/when github blows up?

wycats_: we can move the network storage and get the URI persistence

timbl: I like github in lots of ways, pull requests, a lot of stuff on top of git
... what W3C gives you is persistence for history, so you can go back and check
... we have the CVS record
... we can see what happened, who changed things etc
... at the moment all that is within w3.org
... we have comma tools and so on

wycats_: why don't we have that with github?

timbl: W3C has a persistence guarantee

annevk: you can clone the git

timbl: we could every night sync onto w3c space

dka_: including the commit history

timbl: the problem is that if you use the github tools like issues etc then it doesn't include that history
... anything that's decentralised, I'm happy to use whatever tools we like
... I'm worried about tracker, the messaging

wycats_: the pull request feature is the most problematic part

plinss: the CSS WG mirrors our repositories onto github
... we don't use issue trackers
... our test suites and our drafts are on github
... the canonical source is Mercurial on W3C space
... we have git and hg versions of both repos
... I'd propose we do the same thing
... using the github collaboration for pull requests and so on
... github have APIs for those

<annevk> Everyone should be added to the owners of the w3ctag thing now except for ht. Could not find ht on GitHub.

plinss: we can use those to pull out that data

<ht> I should be there. . .

<ht> Hang on

plinss: I'm syncing all the issues etc for 'Shepherd' from github

timbl: I'd love to see the issues etc be written out into a standard format

plinss: we can capture all that data and keep our own archive of it
... I like the social part of github, I'm not comfortable to rely on them for our storage

dka_: the question I asked was about what happens when HP buys github
... we need to have a story for that

plinss: we can't rely on that

- DRAFT -

Technical Architecture Group Teleconference

30 May 2013

Attendees

Contents

Noah's thoughts as outgoing chair

Futures / Promises

Streams

TAG operations

Summary of Action Items

Scribe.perl diagnostic output