IRC log of xproc on 2010-11-05

Timestamps are in UTC.

08:10:57 [RRSAgent]
RRSAgent has joined #xproc
08:10:57 [RRSAgent]
logging to
08:11:33 [Norm]
Topic: Investigating methods for making pipelines easier to build
08:12:06 [Norm]
Vojtech: We're reimplementing the DITA toolkit in XProc (instead of Ant+Java extensions)
08:13:09 [Norm]
...It's about 9000-10000 lines of XProc
08:14:47 [Norm]
...Top-level pipeline consists of about 20 steps
08:14:59 [Norm]
...If you didn't want to do, for example, conref processing (one of the steps in the middle)
08:15:12 [Norm] would you turn that off or replace that with something else?
08:15:37 [Norm]
Vojtech: Suppose you want to do it just slightly differently
08:16:07 [Norm]
...Currently only option is to cut-and-paste the entire pipeline and change the one part I want to change.
08:17:00 [Norm]
...It would be nice if you could pass a step in dynamically or do some sort of replacement
08:17:06 [Norm]
Alex: Or pass in a subpipeline
08:18:22 [Norm]
Norm: The problem is that there's no obvious, single step that you want to override, you need to access them all
08:19:14 [Norm]
Vojtech: Even if they copy-and-paste to change the definition of a pipeline, the original pipeline may be called from somewhere else where they have no control.
08:19:28 [Norm]
...It would also be nice to have library-level variables or constants.
08:21:22 [Norm]
Norm: I can imagine we might do that...
08:21:56 [Norm]
Mohamed: But it's not really variables, you want to override them, so it's more like parameters or options.
08:21:58 [Norm]
Norm: That could get messy.
08:22:07 [Norm]
More discussion
08:22:18 [Norm]
Vojtech: I really want to replace all steps of a given type, not just a single instance.
08:23:35 [Norm]
Florent: It seems that there are two aspects here: the ability to override an entire step type (like XSLT import precedence) and the ability to modify the internals of one type.
08:25:23 [Norm]
Norm: Maybe you could make it work with just a mechanism to override whole step types. If you want to override only some, you can conditionalize the internals.
08:25:33 [Norm]
Mohamed: But how would you know what instance it was?
08:25:41 [Norm]
Norm: Yes, I think we'd have to provide a function to get the step name.
08:25:56 [MoZ]
MoZ has joined #xproc
08:27:36 [Liam]
Liam has joined #xproc
08:28:01 [Norm]
Alex: Sometimes the pipeline author knows where the extension points are, where the author is making choices, and that's maybe slightly different.
08:28:28 [Norm]
Norm: That's where p:eval would be useful. You could let users pass in a pipeline.
08:30:03 [Norm]
...You can kind of do that today by putting in calls to step types that are undefined, then the user has to import both your pipeline and declarations for those types.
08:30:12 [Norm]
...(I wonder if that's actually true)...
08:32:12 [Norm]
Alex: It might be nice if we could make that easier; some mechanism for declaring steps "abstract" and then some mechanism for defining them when you import the pipeline.
08:33:51 [Norm]
Alex: I wonder if users will be able to find ways to define where their extension points need to be or if we'll need a "redefine" mechanism.
08:37:57 [Norm]
Mohamed: I don't think I want to see the ability to change a single instance of a type, but being able to change a whole type seems like something I might want to see.
08:40:31 [alexmilowski]
alexmilowski has joined #xproc
08:43:28 [Norm]
Norm: Isn't this just like what we were talking about in XSLT WG about overriding templates?
08:43:30 [Norm]
Mohamed: Yes.
08:43:35 [Norm]
Norm: Ok. I'm not dreaming.
08:46:07 [Norm]
Topic: "Streaming" XProc
08:47:36 [Norm]
Mohamed: The idea is the same as it was at the beginning of XProc. Being able to sort out, whatever we call it, saying how you can make your pipeline more streamable.
08:47:55 [Norm]
...We already say that if you use last(), you've probably impacted streaming in the pipeline.
08:48:00 [Vojtech]
Vojtech has joined #xproc
08:48:19 [Norm]
...It would be nice if we had a document that described a profile of XProc that would improve streamability.
08:48:37 [Norm]
...Innovimax is working on an implementation of XProc using multi-threading and streaming.
08:49:13 [Norm]
...We are planning to issue the project at the end of March. It is based on XML Calabash. We already have some interesting results.
08:49:32 [Norm]
...We're working at the same time on a streamable subset of XPath with a different research agency.
08:50:59 [Norm]
Mohamed: We have customers with large volumes of data that can't use DOMs and they can't rewrite all their tools.
08:52:14 [Norm]
Norm: That could be interesting. If you had a defined streamable subset of XPath, you could have a switch to analyze a pipeline for streamability.
08:52:49 [Norm]
Mohamed: Like the work we're doing in XSLT 3.0 now for streaming, we should be looking at XProc with streaming in mind.
08:53:21 [Norm]
Alex: It always depends on what you mean by streaming. What's your bound?
08:59:58 [Norm]
Some discussion of the value (or not) in defining a streamable subset of XPath (whatever that means)
09:00:28 [Norm]
Florent recounts the example in XSLT of two XPath expressions interacting in ways that prevent streaming even though the expressions are simple.
09:02:12 [Norm]
Discussion leads to general agreement that a study of use cases is necessary and would be valuable.
09:05:15 [Norm]
Mohamed: Having a spec is only the first step. The second step is to have a document that describes usage patterns that will improve performance.
09:08:25 [Norm]
Mohamed suggests that we could have a workshop on pipeline performance.
09:11:19 [Norm]
Norm: So do we want to ask Liam to put something in the charter about investigating streaming or performance?
09:11:27 [Norm]
General nods of agreement.
09:11:32 [fgeorges]
fgeorges has joined #xproc
09:41:23 [Norm]
Norm has joined #xproc
10:10:45 [alexmilowski]
alexmilowski has joined #xproc
10:41:34 [fgeorge]
fgeorge has joined #xproc
11:25:19 [Norm]
Topic: More discussion of possible extension steps
11:26:43 [Norm]
ACTION: Alex to review common concurrency patterns to see what might work best for us (.e.g. countdown latches)
11:27:32 [fgeorge]
fgeorge has joined #xproc
11:56:21 [Norm]
rrsagent, set logs world-visible
11:56:25 [Norm]
rrsagent, draft minutes
11:56:25 [RRSAgent]
I have made the request to generate Norm