XProc Candidate Issues

Hi Geert,

Sorry for my late reply, I had to do other things than XProc... not so 
exciting things actually ;-)!
That's clear with p:load needed here.. I could actually have guess it !

The spec is clear about this  : viewport is just a scope on specific 
elements selected by xpath, it's not aiming at loading any external document
"Each matching node in the source document is wrapped in a document 
node, as necessary, and provided, one at a time, to the 
viewport's/<http://www.w3.org/TR/xproc/#dt-subpipeline> subpipeline/"

The goal of p:viewport seems to change the content of selected elements 
in the source document :
"The result of thep:viewportis a copy of the original document where the 
selected subtrees have been replaced by the results of applying the 
subpipeline to them."
which is not really what I want to do here, I might be able to use this 
for my purpose but I think an iteration element is more adapted.

The difference I see between p:viewport and p:for-each is about the way 
to select elements "catch" elements : p:viewport uses a match attribute 
and whereas p:for-each uses a select attribute. I suppose 
match="h:object" is more efficient than select="//h:object" (on big 
documents).

My documents are not so big, so for the moment I keep on using 
p:for-each, but I think it's good to be aware of this (If i'm true with 
my interpretation)

Keep on learning,
Thanks for your help,

Best Regards,
Matthieu

Le 14/10/2011 20:25, Geert Josten a écrit :
> Hi Matthieu,
>
> Actually, you would need a p:load in front of the p:xslt step in my example as well (at least, that would be most logical). I overlooked it, sorry.. :P
>
> Kind regards,
> Geert
>
> -----Oorspronkelijk bericht-----
> Van: xproc-dev-request@w3.org [mailto:xproc-dev-request@w3.org] Namens Matthieu Ricaud-Dussarget
> Verzonden: vrijdag 14 oktober 2011 16:33
> Aan: xproc-dev@w3.org
> Onderwerp: Re: sequence of filtered documents as input of p:xslt
>
> Hi Geert,
>
> I finally find the time to test p:viewport, but I don't manage to get it
> do what I want.
> As far as I understand p:viewport split my whole XHTML document into
> small ones with only<object>  element in it.
> That means my XSLT has to read the @data attribute and then switch to
> the SVG file itselft with document() function.
> I actually would like to keep the XSLT simple and independant from xproc
> pipeline (so I can run it directly on any SVG file on my system)
>
> This is the code I've tried :
> <p:viewport match="h:object">
> <p:output port="result" primary="true"/>
> <p:xslt name="PDFTronSVG2epubFixedSVG">
> <p:input port="stylesheet">
> <p:document href="../xslt/PDFTronSVG2epubFixedSVG.xsl"/>
> </p:input>
> <p:input port="parameters"><p:empty/></p:input>
> </p:xslt>
> <p:store encoding="UTF-8" omit-xml-declaration="false" indent="true">
> <p:with-option name="href" select="concat('PAGES_SVG_01/',
> tokenize(document-uri(/),'/')[last()])"></p:with-option>
> </p:store>
> <p:identity>
> <p:input port="source">
> <p:pipe step="PDFTronSVG2epubFixedSVG" port="result"/>
> </p:input>
> </p:identity>
> </p:viewport>
> The p:store doesn't work as expected cause document-uri(/) doesn't refer
> to the SVG file as I expected.. it seems document-uri(/) is a nul
> sequence when processing a "virtual" xproc document ?
> A document called "PAGES_SVG_01" has been created on my file system
> which is a copy of the whole XHTML doc I had as input.
>
> Hoping my interpretation are correct. Let's say it if not !
>
> I think I'll turn to p:load solution as Romain suggests.
> Anyway it's good to learn new step which might be usefull in another
> context.
>
> Thanks for your help,
>
> Regards,
> Matthieu
>
> Le 13/10/2011 14:09, Geert Josten a écrit :
>> Hi Matthieu,
>>
>> I did something similar a few times. I just used a p:viewport to do the trick. The p:viewport takes the XHTML with the objects as input. You give the viewport a match on object. That will cause the viewport to return the object elements as documents one by one. You can pass that straight-forward into p:xslt. Put a p:store behind p:xslt to store the output of the XSLT. Add an empty identity after p:store to make sure p:viewport has output.
>>
>> <p:viewport match="object">
>> 	<p:xslt .../>
>> 	<p:store .../>
>> 	<p:identity... />
>> </p:viewport>
>>
>> Note that this will remove the objects from the XHTML stream. You will need to reroute input here or there if you want to do more with them, or with the XHTML in whole..
>>
>> Kind regards,
>> Geert
>>
>> -----Oorspronkelijk bericht-----
>> Van: xproc-dev-request@w3.org [mailto:xproc-dev-request@w3.org] Namens Matthieu Ricaud-Dussarget
>> Verzonden: donderdag 13 oktober 2011 13:32
>> Aan: XProc Dev
>> Onderwerp: sequence of filtered documents as input of p:xslt
>>
>> Hi all,
>>
>> Continuing discovering xproc, I have a new problem today :
>>
>> One of the steps of my pipeline gives for result an XHTML document that
>> contains many
>> <object data="fileA.svg">,<object data="fileB.svg">   elements.
>>
>> Of course each referenced "file{?}.svg" exists in a specific directory,
>> let's call it "SVGdir".
>>
>> I'd like to iterate on those file{?}.svg and apply an XSLT
>> transformation to each one (and store the result in separate directoty).
>>
>> I don't want to iterate the "SVGdir" directly cause it can contains some
>> svg files which I don't want to transform cause they are not referenced
>> in the XHTML document.
>>
>> I think there are 2 options :
>> - filtering the directory files according to the XHTML
>> - starting from the XHTML to iterate on the good SVG files in the directory
>> I thought the 2nd option is better.
>>
>> (remind : I don't want the XSLT to be applied on the XHTML document
>> itself, cause I'd like it to be independant : it takes one SVG in input
>> and get one SVG in output.)
>>
>> The spec says that<p:xslt>   can have a sequence of documents as input.
>> But I don't find a way to make that work :
>> - should I give an xpath collection() for p:xslt/p:input/@select =>
>> xproc error
>> - should I iterate the<object>   elements within the XHTML doc and then
>> getting the svg document as input :
>> <p:for-each name="for-each-html-object">
>> <p:iteration-source select="//h:object">
>> <p:pipe step="generateECF" port="result"/>
>> </p:iteration-source>
>> <p:xslt name="PDFTronSVG2epubFixedSVG">
>> <p:input port="source" select="document(concat('PAGES_SVG_0',
>> tokenize(@data,'/')[last()] ))">
>> <p:pipe step="for-each-html-object" port="current"/>
>> </p:input>
>>              [...]
>> </p:for-each>
>> =>   xproc err:XD0023:Invalid XPath expression
>>
>> I will continue to investigate for other solutions, but if you have any
>> advices, they are welcome !
>>
>> Kind Regards,
>>
>> Matthieu.
>>
>


-- 
Matthieu Ricaud
IGS-CP
Service Livre numérique

Hi Romain,

And also sorry for my late reply.

I have tested with */@data and it works !

The context node is actually the document itself, that means I have to 
select the document root element with "*" just like I would do in xslt 
with :
<xsl:template match="/*">
which is not the same as <xsl:template match="/"> which matches the root 
node (cf. http://www.dpawson.co.uk/xsl/sect2/root.html)

I actually used a variable to store the @data attribute and make the 
xpath explicit  :
<p:variable name="svgSrc" select="/h:object/@data"/>

Everithing's fine !
Thanks for helping,

Best Regards,
Matthieu.

Le 14/10/2011 19:22, Romain Deltour a écrit :
>> With <p:with-option name="href" select="concat('PAGES_SVG_0/', 
>> @data)"/> I get an error : GRAVE: err:XC0011:Could not load PAGES_SVG_0/
>> It seems @data is empty at this point.
>
> Right. The XPath context at this point is a *document* (with 'object' 
> as its root element), @data returns the empty sequence.
>
> I suppose it would work with:
>
> <p:with-option name="href" select="concat('PAGES_SVG_0/', */@data)"/>
>
> Romain.
>
> Le 14 oct. 11 à 16:58, Matthieu Ricaud-Dussarget a écrit :
>
>> Hi Romain,
>>
>> Thanks for your suggest !
>>
>> It almost works but I still have problem to store the load the good 
>> svg file.
>>
>> This is the code i've tested :
>> <p:for-each name="for-each-html-object">
>> <p:iteration-source select="//h:object">
>> <p:pipe step="generateECF" port="result"/>
>> </p:iteration-source>
>> <!--<p:output port="load-svg"/>-->
>> <p:load name="load-svg">
>> <p:with-option name="href" select="concat('PAGES_SVG_0/', @data)"/>
>> <!--<p:with-option name="href" select="concat('PAGES_SVG_0/', 
>> 'epub0001.svg')"/>-->
>> </p:load>
>> <p:xslt name="PDFTronSVG2epubFixedSVG">
>> <p:input port="stylesheet">
>> <p:document href="../xslt/PDFTronSVG2epubFixedSVG.xsl"/>
>> </p:input>
>> <p:input port="parameters"><p:empty/></p:input>
>> </p:xslt>
>> <p:store encoding="UTF-8" omit-xml-declaration="false" indent="true">
>> <p:with-option name="href" select="concat('PAGES_SVG_01/', 
>> tokenize(document-uri(/),'/')[last()])"></p:with-option>
>> </p:store>
>> </p:for-each>
>>
>> With <p:with-option name="href" select="concat('PAGES_SVG_0/', 
>> 'epub0001.svg')"/>, it works, but it's always the same document that 
>> is processed.
>> But it is processed 3 times, which proof the iteration on object 
>> works fine (there a 3 object in my document)
>>
>> With <p:with-option name="href" select="concat('PAGES_SVG_0/', 
>> @data)"/> I get an error : GRAVE: err:XC0011:Could not load PAGES_SVG_0/
>> It seems @data is empty at this point.
>> I have tried @h:data (prefix "h" binded to xhtml namespace), but I 
>> get the same error.
>>
>> The spec indicate the xpath context inside an iteration is "the 
>> sequence of documents that will be processed by the p:for-each", that 
>> means each document is here <object data="..."/>, so I should be able 
>> to get the data attribute, no?
>>
>> Any ideas ?
>>
>> Regards,
>> Matthieu.
>>
>>
>>
>>
>> Le 13/10/2011 14:13, Romain Deltour a écrit :
>>>> - should I iterate the <object> elements within the XHTML doc and 
>>>> then getting the svg document as input
>>>
>>> That's what I would do.
>>>
>>>> (...)
>>>> <p:input port="source" select="document(concat('PAGES_SVG_0', 
>>>> tokenize(@data,'/')[last()] ))">
>>>> (...)
>>>
>>> The select expression is applied to the document connected on the 
>>> input port. What you rather want here is to load the SVG documents 
>>> using a dynamically computed URI, which is then connected to the 
>>> primary input port of the XSLT. You can do that with p:load, e.g.:
>>>
>>> <p:load>
>>> <p:with-option name="href" select="concat('PAGES_SVG_0', 
>>> tokenize(@data,'/')[last()])"/>
>>> </p:load>
>>>
>>>
>>> To summarize, the workflow would be:
>>>
>>> p:for-each (iterate the objects)
>>>  p:load (load the document after having computed the URI)
>>>  p:xslt
>>>
>>> Hope this helps,
>>> Romain.
>>>
>>> Le 13 oct. 11 à 13:31, Matthieu Ricaud-Dussarget a écrit :
>>>
>>>> Hi all,
>>>>
>>>> Continuing discovering xproc, I have a new problem today :
>>>>
>>>> One of the steps of my pipeline gives for result an XHTML document 
>>>> that contains many
>>>> <object data="fileA.svg">, <object data="fileB.svg"> elements.
>>>>
>>>> Of course each referenced "file{?}.svg" exists in a specific 
>>>> directory, let's call it "SVGdir".
>>>>
>>>> I'd like to iterate on those file{?}.svg and apply an XSLT 
>>>> transformation to each one (and store the result in separate 
>>>> directoty).
>>>>
>>>> I don't want to iterate the "SVGdir" directly cause it can contains 
>>>> some svg files which I don't want to transform cause they are not 
>>>> referenced in the XHTML document.
>>>>
>>>> I think there are 2 options :
>>>> - filtering the directory files according to the XHTML
>>>> - starting from the XHTML to iterate on the good SVG files in the 
>>>> directory
>>>> I thought the 2nd option is better.
>>>>
>>>> (remind : I don't want the XSLT to be applied on the XHTML document 
>>>> itself, cause I'd like it to be independant : it takes one SVG in 
>>>> input and get one SVG in output.)
>>>>
>>>> The spec says that <p:xslt> can have a sequence of documents as input.
>>>> But I don't find a way to make that work :
>>>> - should I give an xpath collection() for p:xslt/p:input/@select => 
>>>> xproc error
>>>> - should I iterate the <object> elements within the XHTML doc and 
>>>> then getting the svg document as input :
>>>> <p:for-each name="for-each-html-object">
>>>> <p:iteration-source select="//h:object">
>>>> <p:pipe step="generateECF" port="result"/>
>>>> </p:iteration-source>
>>>> <p:xslt name="PDFTronSVG2epubFixedSVG">
>>>> <p:input port="source" select="document(concat('PAGES_SVG_0', 
>>>> tokenize(@data,'/')[last()] ))">
>>>> <p:pipe step="for-each-html-object" port="current"/>
>>>> </p:input>
>>>>         [...]
>>>> </p:for-each>
>>>> => xproc err:XD0023:Invalid XPath expression
>>>>
>>>> I will continue to investigate for other solutions, but if you have 
>>>> any advices, they are welcome !
>>>>
>>>> Kind Regards,
>>>>
>>>> Matthieu.
>>>>
>>>> -- 
>>>> Matthieu Ricaud
>>>> IGS-CP
>>>> Service Livre numérique
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>> -- 
>> Matthieu Ricaud
>> IGS-CP
>> Service Livre numérique
>>
>>
>
>


-- 
Matthieu Ricaud
IGS-CP
Service Livre numérique

Hi Geert,

Once again, thanks a lot for your help.

> <p:xslt ... name="xslt"/>
> <p:store .../>
> ...
> <p:error ...>
> 	<p:input port="source">
> 		<p:pipe step="xslt" port="result"/>
> 	</p:input>
> </p:error>
I thought the input port of <p:error> shall contains the error message 
itself (within <p:inline> for example). I guess I have to get 
information on how to deal with error code and message to display.
Anyway the problem is just above :
<p:when 
test="count(distinct-values(/igs:aggregate/doc/pages/page/@viewportX))!=1">
<p:error ...>
</p:when>
My guess is that the xproc processor is not able to perform the xpath 
test above because it can't define a context node at this point of the 
pipeline... because just above is a <p:store> which doesn't through any 
output!

> Another option, which is most often easier, is to put a p:identity with such an input binding after p:store:
Yes you are right, it works !
And I now understand why my former xslt step direct after <p:store> 
works : I piped the input to the good step ... jumping back over the 
<p:store>.
Cool that's clear !

> But since this is a common case, it is worthwhile to declare a helper step to do that.
100% agree !
> I wrote the following, which combines a p:store with such input rerouting, together with a p:choose so the p:store is only done if a debug parameter was passed through..
I actually don't know how to use such a self define step.
My xpl file starts with <p:declared-step> element : It seems I can't add 
a <p:declared-step> within my root element <p:declared-step>.
I thought I should put it in another xpl document and then import it 
from my current xpl with <p:import> but... It's seems <p:import> is not 
allowed as child of <p:declared-step>.

Maybe I should use <p:pipeline> as root element but I actually thought 
it was better to start with <p:declared-step> so I can really understand 
binding concepts in Xproc.

But this consideration is a bit out of the subject : I will keep the 
<p:identity> solution and go further with my pipeline. Later I'll try to 
make it more simple and maybe modular.

Thanks again for your response.

Matthieu

Just jumping in and out in the middle, regarding compactness..

It would already help a lot if the specs could be tuned such that you can rely more on default behavior, or could write things with less characters:
- You would not need the p:with-option stuff if attribute value templates were supported from the start.
- Things like generating empty input taking up 5 lines (p:identity with p:input with p:empty inside), can't that be done shorter?
- Maybe something specific to XMLCalabash, but why can't I just do p:variable with a p:pipe connecting to parameters port? It complains about context, forcing me to pipe the parameters through a p:parameters first, and rerouting input around the p:parameters as well.
- Lot's of extensions don't produce output, but do take input without doing much with that (most notably the file io extensions for example). Why not just let them do identity transform as well, making it much easier to do a large sequence of such steps..

Just a few cents..

Kind regards,
Geert

> >     *deep breath*
> >
> >     Going back to my earlier comparison: if XProc is like XSD, then
> >     tools like XMLSH are like Relax-NG. Both XMLSH and RNG (or RNC)
> >     solve the same goals as W3C specs, and they both do it with a
> >     very significantly lower learning curve.
> 
> Would a non-XML syntax for XProc help?
> 
> I have one. I presented it as a "PechaKucha" lightning talk at XML
> Summer School.
> 
> XML Syntax:
> 
> <p:pipeline xmlns:p="http://www.w3.org/ns/xproc"
>             version='1.0'>
> <p:serialization port="result"
>                  method="xhtml" indent="true"/>
> 
> <p:xinclude/>
> 
> <p:xslt>
>   <p:input port="stylesheet">
>     <p:document href="dbslides.xsl"/>
>   </p:input>
> </p:xslt>
> 
> </p:pipeline>
> 
> "Compact" syntax:
> 
> pipeline {
>   serialization "result" with method="xhtml", indent="true"
>   xinclude
>   xslt {
>     input "stylesheet" {
>       document "dbslides.xsl"
>     }
>   }
> }
> 
> What struck me most after writing it is that it's not really much more
> compact than the XML syntax. Jeni Tennison took up the challenge to
> produce a more natural compact syntax. I'm having trouble getting
> JavaCC to swallow it, but I'll get there eventually, I expect.
>

> What struck me most after writing it is that it's not really much more
> compact than the XML syntax.

I had the same feeling, although it does save a few characters.

>  Jeni Tennison took up the challenge to
> produce a more natural compact syntax. I'm having trouble getting
> JavaCC to swallow it, but I'll get there eventually, I expect.

Interesting, please keep us posted!

BR,
Romain.

Le 12 oct. 11 à 16:29, Norman Walsh a écrit :

> Zearin <zearin@gonk.net> writes:
>> <opinion feedback="wanted" feedback-types="flames comments  
>> questions suggestions misc">
>>    The barrier of entry to XProc newcomers is needlessly high. I
>>    think of it sort of like W3C’s XSD spec: it’s capable of
>>    achieving powerful goals—but it bends over backwards so much to
>>    accommodate implementors that the end users spend all their time
>>    trying to learn all the “inconvenient corners” of XProc  
>> instead
>>    of making progress on their projects.
>
> Could you be more specific?
>
> In the case of the outputs-must-be-the-same rule, that's for logical
> consistency of the pipeline, not implementor convenience. We could
> have added a special rule for p:error, but that has costs too and it's
> not obvious to me that the benefits outweigh the costs.
>
>>    XProc was invented to fill a sorely-needed role for macroscopic
>>    XML processing—but just getting it to work is an obstacle that
>>    has often cost me more time than I would have spent using
>>    simpler tools. But I tried to force myself through it anyway,
>>    because I wanted to stick to as many official XML tools as
>>    possible.
>
> Simpler tools, or just different tools? Can you enumerate lessons
> learned along the way?
>
>>    *deep breath*
>>
>>    Going back to my earlier comparison: if XProc is like XSD, then
>>    tools like XMLSH are like Relax-NG. Both XMLSH and RNG (or RNC)
>>    solve the same goals as W3C specs, and they both do it with a
>>    very significantly lower learning curve.
>
> Would a non-XML syntax for XProc help?
>
> I have one. I presented it as a "PechaKucha" lightning talk at XML
> Summer School.
>
> XML Syntax:
>
> <p:pipeline xmlns:p="http://www.w3.org/ns/xproc"
>            version='1.0'>
> <p:serialization port="result"
>                 method="xhtml" indent="true"/>
>
> <p:xinclude/>
>
> <p:xslt>
>  <p:input port="stylesheet">
>    <p:document href="dbslides.xsl"/>
>  </p:input>
> </p:xslt>
>
> </p:pipeline>
>
> "Compact" syntax:
>
> pipeline {
>  serialization "result" with method="xhtml", indent="true"
>  xinclude
>  xslt {
>    input "stylesheet" {
>      document "dbslides.xsl"
>    }
>  }
> }
>
> What struck me most after writing it is that it's not really much more
> compact than the XML syntax. Jeni Tennison took up the challenge to
> produce a more natural compact syntax. I'm having trouble getting
> JavaCC to swallow it, but I'll get there eventually, I expect.
>
>>    <aside>
> [...]
>>        I don’t know what to make of this. …On the other hand, I
>>        have never written any script that generates XProc or XSL
>>        for me. (I wish I were that good! I envy devs that can do
>>        this.)
>
> I've certainly written stylesheets that produce stylesheets. And I'm
> one of the outliers who's unconvinced that moving from an XML
> pattern-based syntax to a string syntax for XPath (back way before
> XSLT 1.0 came out) was a good thing. Water under the bridge.
>
>>        What are the pros and cons of designing a language to work
>>        with XML using an XML syntax vs. a non-XML syntax?
>
> That's a good question. I'm toying with the idea of writing about it
> for XML Prague 2012. You know, in my copiuos free time.
>
>>    Of course, there is the XProcVNext page on the XProc wiki.
>>    There’s some very good ideas there. Has anyone else looked at it
>>    recently?
>>
>>    No?  Now’s a good time to start! ☺
>>
>>    Is there any official work (as in W3C) on improving XProc at the  
>> moment?
>
> No. The XProc WG is not chartered to do new work. We're going to talk
> informally about next steps at the upcoming f2f in CA. It's a bit of a
> chicken and egg problem. Trying to get the W3C Membership and our
> member companies to support work on V.next would be aided by greater
> adoption of V1.0.
>
>                                        Be seeing you,
>                                          norm
>
> --
> Norman Walsh
> Lead Engineer
> MarkLogic Corporation
> Phone: +1 413 624 6676
> www.marklogic.com

On Oct 12, 2011, at 10:29 AM, Norman Walsh wrote:
> Zearin <zearin@gonk.net> writes:
>> <opinion feedback="wanted" feedback-types="flames comments questions suggestions misc">
>>    The barrier of entry to XProc newcomers is needlessly high. I
>>    think of it sort of like W3C’s XSD spec: it’s capable of
>>    achieving powerful goals—but it bends over backwards so much to
>>    accommodate implementors that the end users spend all their time
>>    trying to learn all the “inconvenient corners” of XProc instead
>>    of making progress on their projects.
> 
> Could you be more specific?


Sure.  

Geert’s earlier reply was a good start.  I’ll start with that:


On Oct 12, 2011, at 10:48 AM, Geert Josten wrote:
> Just jumping in and out in the middle, regarding compactness..

> 

> It would already help a lot if the specs could be tuned such that you can rely more on default behavior, or could write things with less characters:

Yes.

> - You would not need the p:with-option stuff if attribute value templates were supported from the start.

YES!!!

Having to use p:with-option is so unbelievably annoying!  It’s annoying to write.  It’s annoying to read.  

How would you feel if you were forced to pass all non-static arguments through an extra function in your programming language?  For example, let me simulate an example in Java:

//  Example #1
// (Apologies for any syntax errors—I only know the very very basics of Java.)
String staticArgument = "non-dynamic string";
aFunctionBeingCalled( staticArgument, with-argument(aDymanicArgument()), with-argument(anotherDynamicArgument())  );

Now, compare that to this:

//  Example #2
String staticArgument = "non-dynamic string";
aFunctionBeingCalled( staticArgument, aDynamicArgument(), anotherDynamicArgument());

Tell me: Do you honestly prefer the first example to the second?

What if you had no choice but to type all that extra crap…EVERY…SINGLE…TIME…you called a function that had arguments evaluated at runtime?  

And the more dynamic I want my pipeline to be, the worse it gets.

What’s more—the first example is still significantly leaner than the XProc equivalent.  My example only added 1 extra level to the code (the with-argument() wrapper function).  In XProc, each dynamic argument adds so much more extra:

A new child element (<p:with-option />)
Specify which option (@name)
Pass the value (@select, OR yet ANOTHER child element—most of which also have their own attributes that need to be typed)

I mean, whenever I hear someone complain that XML is too verbose, I ask them what editor they code in.  I mean, it’s 2011—editors can do most of that automatic typing for you!

…But then, there’s <p:with-option />.  An extra child element, plus attributes, plus the potential for still more child elements of its own…just because a step’s argument is evaluated at runtime?  Come on!  

Sigh.  

I know there was a reason behind this.  I know it exists for the implementors.  

But ultimately, the implementors are not the end-users of XProc.  We are.  

It’s as though everyone agreed we needed a macroscopic XML tool, but the moment the W3C started working on it, they forgot all about the end users and only listened to implementors.


> - Things like generating empty input taking up 5 lines (p:identity with p:input with p:empty inside), can't that be done shorter?


YES!!!  

Seriously!

Would it have caused World War Ⅲ to do this?

<p:input port="source" empty="true" />


Or this?


<p:input port="source" select="empty()" />


I don’t care which.  It’s the equivalent of passing null to a function in other languages.  It shouldn’t be any more complicated than that.


> - Maybe something specific to XMLCalabash, but why can't I just do p:variable with a p:pipe connecting to parameters port? It complains about context, forcing me to pipe the parameters through a p:parameters first, and rerouting input around the p:parameters as well.


Still more of the same.  Designed for implementors.  

We end users pay the price.


> - Lot's of extensions don't produce output, but do take input without doing much with that (most notably the file io extensions for example). Why not just let them do identity transform as well, making it much easier to do a large sequence of such steps..


OMG, yes!  

I have never had so much extra work just to store something in the middle of a pipeline.  

…

I’m so sorry, Norm.  

It’s just…XProc makes me angry.  

(Please don’t hurt me!  ☹  I feel like I have just betrayed a childhood hero…I just never imagined something I looked forward to so much would turn out to be this frustrating to use.)

…

So far I’ve just agreed with the points from Geert’s earlier post, but there’s still more on the XProcVNext page on the wiki.  

I don’t understand 100% of the suggestions there, but the rest of them—most of them—make complete sense to me.  I agree with them completely.


> Simpler tools, or just different tools? Can you enumerate lessons
> learned along the way?


Good question.  Sometimes those two things are indeed confused.  

But, I do actually mean simpler:

XMLSH

some XProc + some other scripts (bash, Python, XMLSH, or 3rd party utilities I download)
I started to write an XProc pipeline, but some would-be steps proved too annoying to write.  Sometimes I can’t figure out what’s wrong, and sometimes I just don’t feel like being forced into the extreme excess verbosity (described above).

Sometimes I wanted to automate something with a script, but ran into an endless stream of trivial “nitpick” errors that I gave up, and resort to a combination of search-and-replace and some of Oxygen’s refactoring functionality instead.  Even if I have to type a keyboard shortcut 100 times to it—it’s preferable to staring at an XProc pipeline and “edit-run-error-edit” over and over again.  


Regarding “lessons learned along the way”—I don’t remember everything, because over time I’ve used XProc less and less, and used XMLSH more (though I still use both)…

But off the top of my head, some things are:


Processing <p:http-request /> results is hard
This isn’t XProc’s fault—just an unfortunate reality of dealing with tag-soup HTML.  (And Tidy is useless if you want X/HTML5. ☹)

Doing anything with the filesystem in XProc is an exercise in masochism.  Having to reconnect an input in the middle of a pipeline whenever I use a <p:store /> makes me want to break things.  
There have been several times I actually preferred run the pipeline, waiting for Oxygen’s “results” view, and copy-paste the results into a new document.  
The EXProc steps admittedly help a bit.  Why they weren’t included in the core spec is beyond me.  
On the other hand, I kinda wish the EXProc filesystem steps were also available as functions.  That way I could just use them as functions in p:inputs and p:outputs (or use them with p:options in my own steps).

XProc does not lend itself well to “learning by exploration”.  (This is XMLSH’s greatest strength, in my opinion.)  In XProc, I spend too much time trying to understand errors, figuring out how to work around stuff like <p:store />, looking up what steps and options do, and so on.  Before long I’ve forgotten what I started the pipeline for, and instead I’m trying to remember all these other things in order to get it to work.


> Would a non-XML syntax for XProc help?

[…]

> "Compact" syntax:
> 
> pipeline {
>  serialization "result" with method="xhtml", indent="true"
>  xinclude
>  xslt {
>    input "stylesheet" {
>      document "dbslides.xsl"
>    }
>  }
> }
> 
> What struck me most after writing it is that it's not really much more
> compact than the XML syntax. Jeni Tennison took up the challenge to
> produce a more natural compact syntax. I'm having trouble getting
> JavaCC to swallow it, but I'll get there eventually, I expect.


How interesting!  

Actually, the compact syntax makes some of the required-verbosity a bit less frustrating.  (But I do mean a bit less.  I’d rather be able to just pass in $variable wherever I needed it.)

Hmm.  You say the compact syntax isn’t really much more “compact”…I’m not sure I agree.  If you’re talking about total lines of code, I’d agree.  But your example is both less typing and much easier to read.  

On the other hand…I’m not sure how to express what I’m feeling.  I’ve designed XML grammars, but I’ve never designed a language syntax before.  My mind keeps flipping between different syntaxes—your example above, an equivalent script in XMLSH, and a mix of syntactic choices from other languages.  (By “other languages,” I’m specifically thinking of Python, Ruby, and XPath—all of which I think are powerful, easy to read, easy to write, and just “feel” elegant when I use them.)




> I've certainly written stylesheets that produce stylesheets.

I know! ☺ 

I looked at the DocBook XSL that comes with Oxygen.  I saw the comments stating “auto-generated—do not edit!” in stylesheet after stylesheet.  

I bow to your superior skill.  (With envy.  ☺)


> And I'm
> one of the outliers who's unconvinced that moving from an XML
> pattern-based syntax to a string syntax for XPath (back way before
> XSLT 1.0 came out) was a good thing. Water under the bridge.


Oh, wow!  The thought of that gives me shudders…

I’m curious—why do you feel this way?  

XPath is a very big part of what makes many flavors XML so attractive to me.  Definitely one of my favorite things in the XML universe.  It makes SQL seem clumsy and oafish by comparison. :)


>>        What are the pros and cons of designing a language to work
>>        with XML using an XML syntax vs. a non-XML syntax?
> That's a good question. I'm toying with the idea of writing about it
> for XML Prague 2012. You know, in my copiuos free time.


I don’t have 1337 programming skills, Norm.  But I’ve offered to help with some grunt-work on your websites/documentation before.  I figured you were too busy to respond, but if you reading this, the offer’s still open.  (E-mail me off-list if you’re interested.)

I enjoyed your talk from XML Prague 2011.  If you decide to discuss this for 2012, I’d be very interested.


>>    Is there any official work (as in W3C) on improving XProc at the moment?
> No. The XProc WG is not chartered to do new work. We're going to talk
> informally about next steps at the upcoming f2f in CA. It's a bit of a
> chicken and egg problem. Trying to get the W3C Membership and our
> member companies to support work on V.next would be aided by greater
> adoption of V1.0.


What a Catch-22!  If I want XProc to get better, I have to use the version that makes me want to cry.  

I feel like…

♫ Everything I know is wrong!  
Up is down,  Black is white, and Short is long… ♬


Sigh…What a day.


—Zearin (Tony)