<scribe> Scribe: Ted
<scribe> scribenick: ted
Ted: I think we are all tiring of
not having a name and resolved to decide this week
... I want to take a straw poll, try to reach consensus and
announce to public-automotive list (we have many inactive
observers - 130 subscribers)
[review of current proposed namees and straw poll]
Gen2 name discussion on Github
<oberstet> : ted hi there, unfort. the webex link I have wants me to signin .. cant do. so .. currently double checking my email if I missed something ..
Peter: keeping it short is better, so not CVSiS
<oberstet> ok, mmh. I tried chrome and FF ..
Ted repeats merits of VISS version 2
https://www.w3.org/TR/vehicle-information-service/
Vehicle Information Service Specification
MagnusF wonders about corresponding RPC protocol spec
Ted: problem is VISS/VSS relation and for RPC we are using Vehicle Service Catalog, overloading term Service
MagnusF: we are leaning toward using WAMP for RPC so that may factor into name
Peter: I see problems using WAMP
for RPC, we are committed to gRPC
... it won't go over well here
Tobias: what is your
benchmark/concern for language
... Swift is fairly popular but we have strong support for
Perl, Python, Java (Android)
Peter: I like the platform but will have problems introducing it
Tobias: give it a try with Swift and let me know of any issues, we can address it
Peter: Go support seems outdated
Tobias: gRPC with Google's support is why
MagnusF: what about Erlang? it is used extensively in telco
MagnusF: v2d[evice] suggests need
to be able to run clients on iOS and Android, eg for remote
unlock
... is this a use case we see as part of service catalog?
Peter: absolutely
Steven: I can see that
Daniel: should we discuss if that is an RPC use case versus just sending signal? :)
Tobias: the WAMP piece on library
support, the main library on Android is made from us and was
the first WebSocket implementation for Android
... it runs cloud side as well
... iOS is a third party project and not as familiar with it so
cannot speak to current status
MagnusF: gRPC has Object C and
proven use in Volvo, people seem to like it
... we are keeping focus on gRPC vs WAMP
Tobias: so iOS part of your requirements
MagnusF: yes although we do not have formal requirements
Peter: iOS and Adroid both very
important for us, especially the later
... we are using Go on server side. Volvo Signal Broker (open
source version renamed) uses gRPC for proof of concept
autonomous drive
MagnusF: what is the main drawback for gRPC?
Peter: the three party decoupling
is better in WAMP
... I have done the Go implementation. versioning is an issue
and initial problems with Objective C
... Swift5.0 a problem and concern with new devices using it
such as Apple watch
... WAMP could let us influence the market but problem with our
current use cases and existing solution. we already worked out
issues for PoC we created
MagnusF: interesting to hear of
gRPC success
... I no longer represent JLR and cannot speak to their intent,
could see Toyota who I can't speak yet for going gRPC
... is there a performance issue, it uses protobuf
underneath
Peter: yes, not a problem for us even on a raspberry pi test platform
Tobias: gRPC doesn't work for cloud to client support, that is a requirement as well?
MagnusF: yes, also a requirement
Tobias: how do you do that?
MagnusF: HTTP2
Tobias: so gRPC over HTTP2?
... so listening end point in the car?
MagnusF: no, vehicle always has to initiate
Tobias: so vehicle can't be reached?
MagnusF: SMS initiates vehicle to
react
... car always runs connect system call to reach cloud after
receiving that message
Tobias: gRPC over HTTP2 standardized?
MagnusF: believe that is what the
implementation is
... yes HTTP2, protobuf and IDL for control language
Tobias: does that open port on device?
MagnusF: no, well not on vehicles I am familiar with
Tobias: how does callee request procedure to call
MagnusF: bidirectional
communication established
... drawback with gRPC is its existing IDL and how that can
conflict with FrancaIDL being used in service catalog
... not sure we want to hardwire ourselves into a Google
solution...
... we can do adhoc translation during build
Steven: do we define a schema
that can be mapped or does the gRPC or other protocol provide
that for us
... we can have different implementations underneath with
former
MagnusF: not sure where we are
now
... issues that Volvo ran into with mobile support for WAMP
gives nod toward to gRPC
Ted recaps, WAMP suggested by Gunnar back before Gen2 started, revisited with RPC
scribe: want to be clear on scope
of this topic
... not proposing derailing Gen2 although open to seeing how
they may fit, interested in different architectural ideas
... what WAMP does that is on our Gen3 (or extensions) wishlist
since we would get them for 'free', related to second major
topic so perhaps keep some of that subtopic for when joined by
other colleagues
... further consideration for RPC which as proposed will be
separate from Gen2 although desire to have architecturally
similar
Ted: @@
... he is at a disadvantage since he will need to talk to
others more about iOS support
Tobias: understood and how you
also want to learn about our experiences regarding compression,
push/pull etc
... I am willing to prototype based on VSS. I can take your
schema file, run through our tool chain including explicit
targets (eg JavaScript and Objective C)
... in the end this is all about using the car
... end goal is to be able to create these interactions with
vehicle
... gRPC ok in NodeJS runtime for browser clients using Webkit
and worth testing as well
MagnusF: there tends to be local NodeJS, gateway does inspection before sending out via gRPC or whatever
Peter: there are issues with browser / gRPC interaction
Tobias: ah because of lack of
HTTP2
... you can't do HTTP2 stream in browser tab, not
possible
... you would have to tunnel over a WebSocket
MagnusF: I have two or three data
points from different OEM saying they will only do native apps
and not support browser apps
... anyone know otherwise?
Peter: I know of PoC but we are not doing so in our projects
MagnusF: JLR Defender uses
Visteon's Phoenix for web apps via WebSocket gateway
... to NodeJS
Tobias: aren't you guys creating Web API at W3C? :)
Ted/Peter summarize history on original approach with Vehicle object in browser runtime and how we pivoted long ago to support broader range of clients
Daniel: regarding RPC we are not there, with common definition and toolset. I want more a protocol discussion before delving into eg protobuf
Tobias: this is my first time looking at this, how to model a car. the domain knowledge is the interesting part
Daniel: I can see multiple protocols being able to support the common data definition
Tobias: agree, the YAML can support multiple uses
MagnusF: this is split into two,
GENIVI focused on the catalog and signal definitions with W3C
on the corresponding protocol pieces
... I was trying to think of how to generate WAMP stubs from
existing tooling that exists at JLR from service catalog
Daniel: we are working on both at
the same time
... we maybe should postpone the W3C side for RPC instead of
trying in parallel
MagnusF: depends on who steps up
Tobias: for WAMP at least I feel
confident a few hundred lines of script code can read in JSON
for car model and output flat buffer files I can use as stubs
for WAMP
... we can compare that against gRPC
... propose giving that a try
MagnusF: we need to get a more detailed catalog, looking at you Steven
Steven: bit of resource issue
right now but should be able to get energy soon
... we have some internal tooling we should be able to open
source, previously gave a simpler version for YAML
... I don't want to be bottleneck
Tobias: don't want to be the bottleneck
Daniel: I am still a bit surprised at the format of the service catalog, wondering how we want to move forward
https://github.com/GENIVI/vehicle_service_catalog
Gunnar: I don't believe the difference between this early version and what we end up with as being very different
Tobias: this looks good, like the abstraction/description
Gunnar: to be clear we are not
doing away with VSS
... final format may be adjusted to Daniel's comment
... we already have a Franca to WAMP mapping from years
ago
... that is based on Franca instead of YAML
... FrancaIDL is about splitting from deployment model, not
sure if the changes are too large
MagnusF: deployment as in what executes where?
Ted points out good conversation but we are kind of ambushing Tobias and didn't provide clear requirements earlier
Tobias: service catalog
Ted: automobiles are similar to
some IoT and mobile devices, have sporadic connectivity, desire
to 'off-board' sometimes substantial data while seeking to be
efficient due to connectivity, cost, timeliness, etc
... focus of this discussion is on efficiency 'every byte
counts'
... we are creating an in-vehicle service that OEM may permit
outside connections to. also see desire to be able to do
pre-packaging of sampled data to local storage for further
inspection and later transmission
sub-topics:
* push v pull
* compression
* serialization
* sampling methodologies
Ted: I'll provide brief
explanation of each since they are intertwined
... from security perspective I strongly prefer push, many OEM
have open port which is a proven attack vector despite carrier
level safeguards
... those carrier safeguards (firewalls, VPN etc) and opening
port on mobile device (atypical compared to eg phones) costly,
requires complete trust in competance, creates single point of
attack to all vehicles and carriers known to have to comply
with state actors
... unsure when vehicle is online, polling each to see if they
are inefficient. appreciate WAMP approach of client (vehicle)
initiating dialog with server in cloud (OEM only at present but
subject to regulations, anti-trust etc - politics)
... regarding compression, with the exception of private
branches in VSS we know our data structure and have schema. Ulf
used a dictionary idea to greatly reduce
... realtime (time sensitive) data will be smaller
transmissions, do see larger batch use cases as well for when
connectivity is better (including at home, office or other
wifi)
... regarding serialization JSON is verbose, have UUID (not
that much shorter) which is why we are open to alternate
formats
... ideal would be a binary or otherwise compact format that
allows for some inspection to be able to prune and discard no
longer pertinent data from local storage before
off-boarding
... regarding sampling methodologies, we are seeing a mix at
present of fixed time interval, event based and curve logic,
possibly others
... Geotab interested in opening up their patent and curve
algorithm. this enables being able to present reasonably full
picture per signal (data point)
... it would be advantageous to support multiple sampling
methodologies in Gen2 implementations for mutiple in-vehicle
sampling applications to take advantage
... Dave previously equated this to lossy compression,
discarding unneeded data
... given time lets focus on compression and serialization
Tobias: both topics involve a
trade off
... depends on wire size and cpu
... if I am doing advanced compression to minimize bandwidth I
will need more cpu and memory
... if I don't care about the wire size, I'll dump data as
is
... consequence is there is no one best way
... second thing to keep in mind is to limit expectations. the
best compression will always depend on the data
... data needs to be qualified, what are we sending, jpeg of
cats?
<caribou> https://www.w3.org/TR/xbc-use-cases/ pre-EXI work examined different kinds of data
MagnusF: we are going to do a
control plane protocol, sending signals and service calls
... not looking to send image/video transfers
Tobias: good point, not using
large media data
... different set of compression algorithms, control calls and
structured
... WebSocket deflate is a very simple option, depends on
memory level
MagnusF: think we agree on a binary format such as protobuf instead of JSON
Tobias: agree, there is one reason to talk JSON
Carine: if you have any sort of
data in EXI or CBOR you can run deflate
... you can go JSON to EXI directly, same for CBOR
... you get benefit with smaller payloads, especially repeating
information EXI or CBOR will be more efficient that text if you
have a schema
MagnusF: we do have schemas for signals and RPC service calls, could be made into protobuf as well
Tobias: coming back to
serialization there is CBOR which is dynamically typed
... protobuf and flat buffers another for statically typed. can
I use schema language to describe a meta schema
... I can write a schema in flatbuffer that represents another
schema, allows me to do reflection
... if you want to store service descriptions in db. what is
the type of column for schema table
MagnusF: we had a schema
validator to check dumps for different types of targets
... https://github.com/GENIVI/vehicle_service_catalog
Tobias: that would be interesting for me to read
MagnusF: Steven needs to sterilize additional JLR code before open sourcing
<caribou> Don't forget to compare processing efficiency as well....
<caribou> not just the compression
MagnusF: there are many things up in the air as well. now we are discussing how to optimize a protocol
caribou, on receiving end presumably?
<caribou> both ends
Daniel: one proposal tomorrow we can look at is aligning VSS and VSC, some collapsing
Carine: we had roughly the same
kind of problem when working on EXI and delayed choosing our
solution
... we had all kinds of use cases, narrower set here
... we listed all properties we wanted the encoding to have,
benchmarked the different proposals
... size of the resulting encoding not the only criteria. I
heard someone mention roundrip time being important
... is the schema agreed upon
MagnusF: yes
Ted: clarification, we allow for 'private branches' which may not be known on receiving end
Carine: another aspect is
measuring deviation in EXI, we had a requirement to allow for
schema extensions negotiated on the line
... before EXI we had different candidates
... I'll provide link to measurements document you might find
useful
... another was compression efficiency required for the
different encodings
<Yves> other aspect: asymetry of encoding and decoding, if messages are always 1-1 or 1 to many, latency of encoding/decoding (I guess this is RTT). Having a "native" binary format (like cbor) is definitely better than compressing a string-based format like json
MagnusF: did you look into storing forward if data link was unavailable
Carine: didn't have that use
case
... maybe the military use cases are closest
MagnusF: looks interesting, for
regulations compliance we may need to lock down things and more
rigid, spec backed
... not sure over the wire agreed extensions may be a nightmare
and hard to explain to eg the EU
... did you look into versioning?
Carine: we had long discussions
on sending the schema, obviously more efficient to not
repeatedly resend it
... EXI headers aren't even mandatory
... in the end we created a Best Practices document on how to
negotiate schema, when you want to use it
... this is an established specification, six years old
MagnusF: any developments since, prompting changes?
Carine: no but understand CBOR in
IETF is more tokenization. EXI is more information theory, very
different
... CBOR can also be efficient for smaller transmissions but
not as familiar with it
... I know it know CBOR and EXI used in other places, ISO
standards
Ted: can you tell us about its uses, understand military and aerospace, latter being very similar to automotive
Carine: Navy was a key
stakeholder but we don't know their usage, highly
confidential
... they were interested in EXI for JSON
... the Working Group is closed but still have the people
around
<caribou> https://www.w3.org/XML/EXI/implementation-report/EXI_1_0_2nd_Edition.html
Ted: trying to remember one of the commercial solutions, believe it is AgileDelta and how much more efficient it is than the open source ones
<caribou> https://www.w3.org/XML/EXI/#implementations
Ted: any comparison to protobuf since that has been brought up
Carine: not sure, protobuf more
optimized for wire transmission
... when we did a comparison it was not compressing as much
Tobias: why would we do that?
Ted processing on end?
Tobias: why serialize as XML?
Ted: not needed to go to XML if JSON with a schema
Carine: it is not really about
XML, it is grammar based. schema description not necessarily
XML schema
... schema describe grammar of what is being encoded. if you
know how your JSON is structured you can generate an encoded
document directly
Tobias: can I generate a validator for arbitrarily describing some JSON stuff, can I get a payload validator?
Carine: you can decode and get events
Tobias: ah while the parsing happens
Carine: encoded knows schema, chooses the right grammar and decode in reverse
Tobias: I see
Carine: depends on implementation
Tobias: so this is for domain specific grammars
Carine: as soon as you have a
schema you can find a way to encode/decode events
... compare it to token based approach
... eg instead of 'sensor' you can have bit 1, preference on
prioritizing what is transmitted more frequently
... CBOR is token based and no optimization based on actual
data
Tobias: got it, if schema is known on both ends
Carine: you can use multiple
schemas with simply sending the short string identified (or URI
if needed)
... you do not need to encode all the stuff all the time
Tobias: thanks, much more clear
Carine: despite the name not tied to XML
Tobias: this is a crossover between serialization and compression
Carine: you may need to measure
differences in processing efficiency, what types of devices are
sending, bandwidth
... CBOR slightly longer and perhaps simpler
... deflate can be used in addition to EXI, more useful on
larger transmissions
... Adobe was interested for compressing PDFs for example since
gzipped had to operate on full document before being able to
use it
s/Ted points out @@/Ted points out we have people waiting to join for next topic. while experiences and choices with different RPC solutions helpful we clearly need to flush out requirements more, use cases and represent them in the service catalog/
Linked related on WebEx chat:
from Tobias Oberstein to Everyone: https://gitlab.com/leapsight/bondy11:28 http://leapsight.io/11:29 ^ rgd erlang ..11:29 from Adnan Bekan to Everyone: https://grpc.io/img/grpc-core-stack.svg11:35 from Tobias Oberstein to Everyone: https://grpc.io/blog/state-of-grpc-web/11:48 from Adnan Bekan to Everyone: they implement proxy11:49 from Magnus Feuer to Everyone: https://docs.python-cerberus.org/en/stable/12:00 https://jinja.palletsprojects.com/en/2.11.x/12:00 from Tobias Oberstein to Everyone: https://github.com/GENIVI/vehicle_signal_specification12:00 from Magnus Feuer to Everyone: https://github.com/GENIVI/vehicle_service_catalog12:01 from Tobias Oberstein to Everyone: https://github.com/google/flatbuffers/blob/master/reflection/reflection.fbs12:09 from daniel wilms to Everyone: https://github.com/GENIVI/vehicle_service_catalog/issues/212:11 from Tobias Oberstein to Everyone: fwiw, websocket supports deflate compression officially, autobahn-python also supports bzip2 and snappy over websocket https://github.com/crossbario/autobahn-python/tree/master/autobahn/websocket12:41 fwiw, rgd serializers here is a bunch of results https://crossbario.com/docs/benchmarks/serialization/index.html (sorry, company hosted, dont have other one)12:43 from Edward C Guild to Everyone: https://www.w3.org/TR/xbc-use-cases/12:44