– DRAFT –
RDF-star WG meeting

Approval of minutes from the last two meetings: 1 2

<ora> PROPOSAL: Accept last two week's minutes

<pfps> +1

<ora> +1

TallTed: minor typo on the 20th, it is minor, can be ignored

<Dominik_T> +0 (not present)

<niklasl> +1

<olaf> +1

<fsasaki> +1

<AndyS> +1

<pchampin> +1

<gtw> +1

<lisp> +1

<doerthe> +1

<ktk> +1

<Souri> +1

<tl> +1

<AZ> +0 (not present at either meeting)

RESOLUTION: Accept last two week's minutes

CR Drafts

<TallTed> +1

pchampin: recent version of w3c process extended notion of CR
… we now have CR snapshots
… and CR drafts
… we have not used the latter notion yet
… we are frozen in the three specs in the state they were
… we could have enabled automated publication syncing all changes into CR drafts
… question: do we want to do that, as long as we do not make substantive changes it does not preclude us to got to REC
… in case we make substantive changes we need a new snapshot
… so question to the group: do we want to re-enable automatic publication
… to give more visibility to changes we made since CR
… e.g. RDF concepts, the CR was published april 7th, we did many adjustments
… all these are only visible if you dig deeper
… it is mainly a question of visibility

<Zakim> AndyS, you wanted to ask what the difference is.

pchampin: we need to work with CR snapshots anyway for moving to REC

AndyS: a question, what is the difference between CR snapshot and CR draft

<Tpt> -1

AndyS: what goes to CR: snapshot or draft?

pchampin: many groups are now publishing on TR what they have as an editors draft
… they want to have the TR version as recent as possible
… CR draft is any version after a CR snapshot was published
… we are now in CR, calling for implementations
… and we continue to improve the document
… if we want to say "there have been substantive changes", we can move from a CR draft to a new CR snapshot
… if there are no substantive hcanges, we can go from the latest draft to RC

S/RC/REC/

AndyS: suggest to have the live version as recent as possible

ora: what is the downside? is there any?

pchampin: the same as having ecidna published
… some WG want to have publications more meaningful to express WG endorsement
… for this they do not sync ecidna with a separate branch
… they merge then only once things are ready to go to TR
… we have adopted the "release early, release often" model during draft state
… we may consider that CR is different
… or remain in the "release early, release often" mode

AndyS: so no difference towards the timing

pchampin: yes

<ora> PROPOSAL: Move to the new CR snapshot/draft model?

<ora> +1

<niklasl> +1

<fsasaki> +1

<pchampin> +1

<olaf> +1

<Dominik_T> +1

<AZ> +1

<lisp> +1

<AndyS> +1

<doerthe> +1

<gtw> +1

<tl> +1

<Souri> +1 to CR draft as long as it is indicated prominently that it is indeed a draft and has a pointer to the official CR-snapshot

pchampin: CR drafts are drafts, now checking if they point to the CR snapshot
… the history link shows the latest snapshot

<pchampin> https://www.w3.org/standards/history/vc-data-model-2.0/

pchampin: that is the histroy. You see always the link to the history in a draft

ora: so you want to have one click?

souri: if there is a search involved it would not be obvious

ora: other thoughts?
… should we should propose this to the people who work on the publishing tooling?

AndyS: we can use the "latest published version" link

pchampin: "latest published version" is most recent version on tr
… if that is the cr draft, you will end up with that
… some specs have a latest rec, that is it

pchampin: we can create custom headers and try to maintain one to the latest snapshot

souri: would be great

RESOLUTION: Move to the new CR snapshot/draft model?

ora: pchampin will set things up

Presentation of an LLM based 1.2 implementation (by lisp) 3

<lisp> https://dydra.com/data/protocol-process-1.pdf

<lisp> https://dydra.com/data/protocol-process-2.pdf

<lisp> https://dydra.com/data/1_2-process.pdf

lisp: above links are three diagrams about using an LLM with W3C docs to perform development tasks
… was an exchange in sparql task force regarding the state of the sparql test suite
… we saw that specs where not in sync, I wanted to check how an LLM can help
… that was first step, see the first diagram
… in all cases the prompts where very simple. I asked to make corrections, deletions
… the LLM did that and produced a version that gregory described as deficient in some ways
… so I looked into how this can be made better
… the 2nd diagram provides more control, more constraints on the LLM
… to provide controls
… so that links between assets could be checked better

ora: what is the difference between solid, dashed and dottet lines

lisp: solid is data flows from documents
… solid lines forward are references forward from the LLM
… the dashed lines were prompts from me
… then setting up a test runner to run LLM until agreement is done, with a semi automated loop
… the pattern got applied
… to the process I followed over 6 weeks to implement sparql 1.2
… from the start of doing turtle
… to doing n-quads, n-triples and sparql-protocol and graph protocol
… pointing an llm to the rec, concepts, and asking to write a requirements document for the implementation
… the requirements document was about what it had to do about the implicit aspects
… then I asked the llm to re-write the implementation, again in an automated loop
… the execution in testing also for triple terms was with very simple prompting

TallTed: did the requirements document have MUST and SHOULD etc.

lisp: the doc had also SHOULD etc., the implementation was only for MUST

ora: so an LLM wrote an implementation?

lisp: the LLM took an conformant implementation for 1.1 so that it passes all tests for 1.2.

lisp: Common Lisp is not creating an obstacle for the LLMs
… if one wants to look at loosing an LLM for providing support for implementation - this was six weeks of work
… it was able to extract the requirements from the implementation, modifying the existing implementation
… and some LLM tooling has detailed logs of what was done
… recorded in very explicit terms

pchampin: the idea that LLM tooling gives a detailed account of what was done
… the idea is to say: someone takes responsibiilty of what was done

pchampin: not sure if LLM tooling responds to that concern

lisp: the LLM has a certain ability to interpret an LLM

pfps: pchampin point is:
… it does not help to produce a large document that makes explicit from which comes from
… that can not be done by a WG
… just using a long document does not help as a justification in a WG

lisp: very likely one would not read that
… but have an LLM that can use it

TallTed: LLMs still concoct their own information
… I recently tried to process pronounciation rules
… the LLM came back with full assurance about some rules but was not correct

TallTed: I had to say "this is wrong"
… the meat of the W3C policy is: a human has to have reviewed what came forward

TallTed: I need a human reviewing it

lisp: that is why I went through the history to provide a perspective of the problem
… the first pass was unguided
… the 2nd pass had a requirements doc which I reviewed
… I reviewed the transcript as it worked
… the full process implementing 1.2. had the same order of control
… it came up with proposals
… where I had to go back to ask: where did you get this from?
… it was given a process which had controls on it
… it had at the end the tests
… it executed all tests except the ones I said: you can skip
… so if these controls are considered: the concerns are addressable

ora: now handing over chairing to pchampin, need to drop out

tl: the work seems to rely on test cases a lot
… not everything is covered in test cases
… how do you or does the LLM deal with that?

lisp: the success criteria was with regards to the W3C test suite

Review of open actions, available at 4

niklasl: worked on my action item

Review of open PRs, available at 5

pchampin: anything to discuss?

AndyS: w3c/rdf-concepts#283 is ready
… on sparql query, w3c/sparql-query#416 is ready
… and w3c/sparql-query#400 too

pchampin: there is a change request from olaf on 400

AndyS: on turtle, w3c/rdf-turtle#157 is ready
… on rdf tests, looking now

pchampin: so w3c/sparql-query#416 is ready on sparql

<TallTed> s/and 400 too|and w3c/sparql-query#400 too|

AndyS: will look and do that

<pchampin> w3c/rdf-n-quads#108

<gb> PR 108 Undefined errors (by lisp)

pchampin: james pr on n-quads was discussed previously
… the essence of my comment on that pr is:
… once we propagate change that have been made in n-triples and then this to n-triples then conflicts will arise
… so would suggest to make the pr against n-quads
… if we do that changes we have a conflict later down the road

lisp: I saw your mail, the group should decide on that
… you said that some aspects in n-triples should be retained
… I do not think that is true
… the group should decide on that

pchampin: we have a chicken and egg problem
… any PR is just a proposal
… the current PR does not show what will be removed

lisp: n-triples and n-quads should agree by including the n-quads content
… that is what I will do

pchampin: we talk past each other. I would like to keep the n-quads PR in a parking lot for the moment
… and make a PR on n-triples first
… the most up to date doc is n-triples
… so make an PR in n-triples that is equivalent, the group can decide on that

lisp: OK, will do

pchampin: any other business?

adjourned