Bug 18764 - MIDI messages don't all have a channel, and status should be part of data.
Summary: MIDI messages don't all have a channel, and status should be part of data.
Status: CLOSED FIXED
Alias: None
Product: AudioWG - OBSOLETE - Moved to Github
Classification: Unclassified
Component: MIDI API - OBSOLETE - Now on Github (show other bugs)
Version: unspecified
Hardware: PC All
: P2 normal
Target Milestone: TBD
Assignee: Chris Wilson
QA Contact: public-audio
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-08-31 22:51 UTC by Chris Wilson
Modified: 2012-11-19 16:04 UTC (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Chris Wilson 2012-08-31 22:51:08 UTC
After looking at all the MIDI messages, including sysex, and examining the other MIDI APIs out there, I don't think we should break out the status and the channel separately in sendMessage(), and I also don't think we should break out status and channel from the rest of the data in a MIDIMessage.

For the first point - Windows API names the first byte "status", but packs them all into a dword, and doesn't break channel apart; CoreMIDI uses MIDIPackets, which contain pure data (i.e. they don't distinguish status bytes from any other bytes; in fact, it does not even appear to be a requirement that there only be one MIDI message per packet, just that the packets contain complete MIDI messages.)

For the second point, when forwarding the data on to the underlying APIs, you will need the status byte as part of the data; it would be considerably easier to have it all together, and not significantly harder to access data[0] instead of status in MIDIMessage.  At any rate, MIDIMessage should definitely not have a channel.
Comment 1 Jussi Kalliokoski 2012-09-05 09:48:22 UTC
(In reply to comment #0)
> After looking at all the MIDI messages, including sysex, and examining the
> other MIDI APIs out there, I don't think we should break out the status and the
> channel separately in sendMessage(), and I also don't think we should break out
> status and channel from the rest of the data in a MIDIMessage.
> 
> For the first point - Windows API names the first byte "status", but packs them
> all into a dword, and doesn't break channel apart; CoreMIDI uses MIDIPackets,
> which contain pure data (i.e. they don't distinguish status bytes from any
> other bytes; in fact, it does not even appear to be a requirement that there
> only be one MIDI message per packet, just that the packets contain complete
> MIDI messages.)
> 
> For the second point, when forwarding the data on to the underlying APIs, you
> will need the status byte as part of the data; it would be considerably easier
> to have it all together, and not significantly harder to access data[0] instead
> of status in MIDIMessage.  At any rate, MIDIMessage should definitely not have
> a channel.

This is something I've been thinking about as well. It should be simple enough to separate the status and channel manually:

var status = 0xF0 & firstByte
var channel = 0x0F & firstByte

And merging them is even less of a hassle.

I think it might be a good idea to just stuff this information into a single byte, as is intended in MIDI anyway. Unless there are objections, I will make the necessary changes to the spec soon.
Comment 2 Jussi Kalliokoski 2012-09-05 12:14:53 UTC
Proposed changeset: https://dvcs.w3.org/hg/audio/rev/bf0e920450e6
Comment 3 Chris Wilson 2012-09-05 17:20:15 UTC
LGTM.

I'd also noted that it makes it easier to forward messages along (e.g. implementing soft MIDI Thru, which is pretty common apparently).

(In reply to comment #2)
> Proposed changeset: https://dvcs.w3.org/hg/audio/rev/bf0e920450e6
Comment 4 Chris Wilson 2012-09-05 17:55:23 UTC
Oh, wait - you only have 2 data bytes in sendMessage():
    void sendMessage(short data0, short? dataX)

It should still have a status byte:
    void sendMessage(short status, short? data0, short? data1)


(It may seem confusing, since we previously had status and channel separately, but even the MIDI standard (http://www.midi.org/techspecs/midimessages.php) refers to this byte (status + channel) as the status byte.  It also frequently does not contain the channel, e.g. for any system or realtime common messages.)
Comment 5 Jussi Kalliokoski 2012-09-05 18:11:50 UTC
(In reply to comment #4)
> Oh, wait - you only have 2 data bytes in sendMessage():
>     void sendMessage(short data0, short? dataX)
> 
> It should still have a status byte:
>     void sendMessage(short status, short? data0, short? data1)
> 
> 
> (It may seem confusing, since we previously had status and channel separately,
> but even the MIDI standard (http://www.midi.org/techspecs/midimessages.php)
> refers to this byte (status + channel) as the status byte.  It also frequently
> does not contain the channel, e.g. for any system or realtime common messages.)

Yeah, it is essentially what I meant, as in the first byte is the status byte. I can change the name to `status`, but what I'd really like to know is how to make rest arguments in WebIDL. It's currently not all so obvious what the meaning behind data0 and dataX is, at least compared to how obvious I want it to be. Maybe it should just be a note or something? If we have a WebIDL guru around here, I'd appreciate the help.
Comment 6 Chris Wilson 2012-09-05 18:36:12 UTC
Ah, I see.  I think WebIDL gives you two options:

void sendMessage(short status, short... data)

This operation is "variadic" - http://www.w3.org/TR/WebIDL/#dfn-variadic - which means "required status param, any number (zero or more) of data args following".  Or, you can have

void sendMessage(short status, optional short data0, optional short data1)

which uses "optional arguments" (http://www.w3.org/TR/WebIDL/#dfn-optional-argument) and means "required status param, either one or two data args following."  This would be my preferred form, since MIDI short messages shouldn't (IMO) be allowed to have more than three bytes.  Nullable types (e.g. short?) are, AFAICT, essentially the same as Optional types here, so

void sendMessage(short status, short? data0, short? data1)

is okay too - although I think this may technically allow you to call the function with:

sendMessage( value, NULL, value );

which is a little bizarre - optional is probably preferable for this reason alone.

MIDI Short messages (i.e. non-sysex) always have a status byte followed by zero, one or two data bytes per complete message.  What's currently in the spec defines one data byte, followed by an optional second data byte - so even if you don't want to call the first byte "status" (and I think we should), it needs another data byte.
Comment 7 Florian Bomers 2012-09-05 19:24:21 UTC
I think that the "variadic" form would be very nice:

> void sendMessage(short status, short... data)

It allows you to send Sys Ex, too. For example, it can be used for a fixed initialization Sys Ex message to a device. 

And it also allows you to send multiple short messages directly one after another (possibly even using running status), for example for (N)RPN's, or bank changes, (MSB, LSB, Program Change), or drum triggers with a note on followed directly by note off.

It might be a great convenience for programmers. And it is sort of "natural" for the serial nature of MIDI.
Comment 8 Chris Wilson 2012-09-05 20:11:04 UTC
This is actually NOT a good idea, due to how it affects the underlying system APIs.  MIDI itself is categorized into short messages - three bytes or fewer - and sysex.  Most system APIs delineate these too - Windows definitely does (where the short messages are packed into DWORDs for efficient passing to message handlers), and OSX's CoreMIDI kind of does as well (there is a separate SendSysex call, although you could technically send sysex in the MIDISend as well, with some limitations.  CoreMIDI does not delineate receiving Sysex from short messages, however.)  In Windows, for example, a short message is sent via a simple MMRESULT midiOutShortMsg(HMIDIOUT hmo, DWORD dwMsg) call; a long message requires a call to midiOutLongMsg(), which needs a header to be prepared and populated.

But even more importantly, if you truly are sending variable-length data, you probably want to put it in a UInt8Array in the arg list for efficiency - except that's NOT the most efficient way to handle 3-byte (or less) sends, which is going to end up being 99+% of all MIDI send/receives.
Comment 9 Florian Bomers 2012-09-05 20:56:47 UTC
thanks for commenting, Chris. I do not follow, though: I see this as an opportunity to create a MIDI API that makes sense in terms of MIDI and not in terms of how the API will be implemented underneath. The Windows MME API was created about 20 years ago when memory management and software interrupts were the main issues for realtime MIDI handling.

It's easy for the underlying implementation to inspect the data passed to the sendMessage() method and use the appropriate OS API function. It should be done for the sendMIDIMessage() variant, too.

At the core, MIDI is a serial protocol. It just transmits one byte after another. I do not see a reason to force API users to separate short messages from long messages. It is very convenient to send multiple short messages at once. And for the given examples, one "message unit" really consists of multiple 3-byte MIDI messages.

Regarding your second point: do you say that the performance of a "variadic" method is worse than creating an object with an array and passing that to a non-variadic method? That could be a reason to not use "variadic". But isn't it just a compiler issue?

For dynamic messages, and for received messages, the MIDIMessage approach is nice. But for sending (mostly) hardcoded messages, using var args is very appealing.
Comment 10 Jussi Kalliokoski 2012-09-06 06:05:55 UTC
(In reply to comment #9)
> thanks for commenting, Chris. I do not follow, though: I see this as an
> opportunity to create a MIDI API that makes sense in terms of MIDI and not in
> terms of how the API will be implemented underneath. The Windows MME API was
> created about 20 years ago when memory management and software interrupts were
> the main issues for realtime MIDI handling.
> 
> It's easy for the underlying implementation to inspect the data passed to the
> sendMessage() method and use the appropriate OS API function. It should be done
> for the sendMIDIMessage() variant, too.
> 
> At the core, MIDI is a serial protocol. It just transmits one byte after
> another. I do not see a reason to force API users to separate short messages
> from long messages. It is very convenient to send multiple short messages at
> once. And for the given examples, one "message unit" really consists of
> multiple 3-byte MIDI messages.
> 
> Regarding your second point: do you say that the performance of a "variadic"
> method is worse than creating an object with an array and passing that to a
> non-variadic method? That could be a reason to not use "variadic". But isn't it
> just a compiler issue?
> 
> For dynamic messages, and for received messages, the MIDIMessage approach is
> nice. But for sending (mostly) hardcoded messages, using var args is very
> appealing.

I'd think that if the underlying implementation has an additional branching operation to check the amount of arguments sent (which it probably has to do anyway) and optimizes for three, it's a ridiculous performance loss in this kind of an API, and not really something we should worry about in this case.

Thanks Chris for pointing me to the variadic arguments, I'm going to change the signature for the next update.
Comment 11 Chris Wilson 2012-09-06 13:09:18 UTC
I don't think I've made the point correctly.

For a large number of arguments, variadic args aren't great, because each arg has to be pushed on the stack; it's better to push a single pointer to an array of data, particularly typed arrays (like UInt8) in Javascript.  This isn't "just a compiler issue" - it's how the code is structured, and it's the API designer's responsibility to make an informed choice between variadic args (which are necessary for some APIs, like printf() for example0) and passing array references and/or having optional arguments.

MIDI really does have two types of messages - sysex, which are variable length, and everything else, which are 1-3 byte long messages.  The latter are the vast majority of messages.  I believe we should optimize the API around those two cases.  The underlying APIs are optimized around them.  

Please do not use variadic arguments to make the simple send() able to send arbitrary bytes.  Using optional arguments for a total of three bytes of arguments is a more appropriate choice.  Otherwise, the data will need to be inspected not only for length, but for partial messages as well, and we will need a lot more error returns to explain to users how they've violated MIDI protocol.  We need to think in complete MIDI messages in the API; this isn't just an arbitrary serial port.

As for multiple MIDI messages batched together - yeah, we could do something similar to that.  CoreMIDI effectively does this, since send() takes a packetlist; however, their packetlists have a significant overhead to creating and managing.  If we want to take that on, we should simply overload send to have:

send( short status, optional short data1, optional short data2)
send( MIDIMessage msg )
send( <sequence> MIDIMessage msgs );  // I think this is legal; need to dig in a bit more.

Again - I think variadic is the wrong choice here.
Comment 12 Florian Bomers 2012-09-07 09:09:19 UTC
thanks for taking the time to explain. 

It's not a big deal to me how I can send "long" messages (syntax is for illustration purposes only, excuse my poor knowledge of WebIDL?):

A) variadic
   send(0xF0, 0x40, 0x00, 0x7F, 0x20, 0xF7);

B) array
   send([0xF0, 0x40, 0x00, 0x7F, 0x20, 0xF7]);

C) MIDIMessage
   send(new MIDIMessage(0, [0xF0, 0x40, 0x00, 0x7F, 0x20, 0xF7]));

I can live with all of the above, but A) does seem the most straight forward, and I don't think that pushing arguments on the stack is worse than creating an array and pushing the array pointer on the stack.

> Otherwise, the data will need to be inspected not only for length, 
> but for partial messages as well, and we will need a lot more error
> returns to explain to users how they've violated MIDI protocol.

For this argument, I don't see the difference of A) and C). And C) will be available in any case, I guess.

However, I really don't think that a MIDI API should verify or otherwise enforce that the byte stream sent via the API conforms to the MIDI protocol. It's nice to optimize the API for short messages, but there should be a way for the user to send what she wants, it IS a serial protocol! MIDI is used in so many areas that it wouldn't be good to unnecessarily restrict the type of messages that you can send. IMHO, the Windows MME and CoreMIDI do not set good examples there (although Windows, at least, allows you to send arbitrary data via midiOutLongMsg()). ALSA on Linux gives you full power with the RawMIDI interface.

Summary: the method signatures are not so important to me as long as the API does not enforce the MIDI protocol. Inspection of the data should be done only for optimization and for formatting the data for the underlying OS functions (e.g. CoreMIDI).

Sorry for the fuzz...
Comment 13 Jussi Kalliokoski 2012-09-07 09:33:16 UTC
(In reply to comment #12)
> thanks for taking the time to explain. 
> 
> It's not a big deal to me how I can send "long" messages (syntax is for
> illustration purposes only, excuse my poor knowledge of WebIDL?):
> 
> A) variadic
>    send(0xF0, 0x40, 0x00, 0x7F, 0x20, 0xF7);
> 
> B) array
>    send([0xF0, 0x40, 0x00, 0x7F, 0x20, 0xF7]);
> 
> C) MIDIMessage
>    send(new MIDIMessage(0, [0xF0, 0x40, 0x00, 0x7F, 0x20, 0xF7]));
> 
> I can live with all of the above, but A) does seem the most straight forward,
> and I don't think that pushing arguments on the stack is worse than creating an
> array and pushing the array pointer on the stack.
> 
> > Otherwise, the data will need to be inspected not only for length, 
> > but for partial messages as well, and we will need a lot more error
> > returns to explain to users how they've violated MIDI protocol.
> 
> For this argument, I don't see the difference of A) and C). And C) will be
> available in any case, I guess.
> 
> However, I really don't think that a MIDI API should verify or otherwise
> enforce that the byte stream sent via the API conforms to the MIDI protocol.
> It's nice to optimize the API for short messages, but there should be a way for
> the user to send what she wants, it IS a serial protocol! MIDI is used in so
> many areas that it wouldn't be good to unnecessarily restrict the type of
> messages that you can send. IMHO, the Windows MME and CoreMIDI do not set good
> examples there (although Windows, at least, allows you to send arbitrary data
> via midiOutLongMsg()). ALSA on Linux gives you full power with the RawMIDI
> interface.
> 
> Summary: the method signatures are not so important to me as long as the API
> does not enforce the MIDI protocol. Inspection of the data should be done only
> for optimization and for formatting the data for the underlying OS functions
> (e.g. CoreMIDI).
> 
> Sorry for the fuzz...

No need to be sorry, this is an open discussion.

I agree with you on this, the idea of my original API was to provide the simplest possible API for making it possible to do anything MIDI and to enable user libraries to abstract further when necessary, in a fashion suitable to each project (if there's one thing I've learned during my time programming, there is no one ultimate API that suits all).

I've actually even had to simplify further from that, as the channels/status separation didn't make sense and the MIDIMessage interface was far too complicated for what it represents.

I'm actually having second thoughts about the whole sendMessage short form as well. The added value is starting to be quite miniscule, especially now that the MIDIMessage interface is a dictionary.

The performance value is really a ridiculous discussion point as even at their most crowded, MIDI devices hardly ever transfer more than 100 messages in a second.

port.sendMessage(0x80, 0x70, 0x60)
port.sendMIDIMessage({
  data: Uint8Array([0x80, 0x70, 0x60])
})
Comment 14 Chris Wilson 2012-09-07 23:48:54 UTC
Florian: I have to disagree.  This is a MIDI API.  MIDI is a software protocol.  I don't think we should be explicitly trying to make this a generic serial send/receive API.  All the wealth of devices I'm interested in out there- synths and keyboard controllers, drum machines, mixers, guitar amp simulators, DJ controllers, lighting systems, control pads like the Launchpad, and more have all designed for MIDI.  I don't think we need to go out of our way - e.g. validating every message - but we do need to design for the protocol, as timing can be important to real-world devices.  And most importantly - this is the point I really, really care about - we should be optimizing for the most important and commonly-used scenarios in sending and receiving MIDI messages.  Far and away the most common scenario is a three-byte "short" message - sysex is useful and needed, but you end up doing a tremendous amount with just note on/off and CC messages.

This also is my reasoning for wanting the "send a short message" call to be as trivial as possible - send( status, data0, data1) - because it is incredibly common.

Jussi - I disagree that "MIDI devices hardly ever transfer more than 100 messages in a second."  Or, more to the point - I can easily see wanting to be able to have a higher-speed connection than that.  Not because I am a high-speed-Rachmaninoff player, but because I want to be twisting knobs with instantaneous response, and because I've seen first hand that those sexy Ableton and Traktor controllers have multicolor button displays, and pushing data back and forth to those could easily outstrip 100 messages/second.  I could hit that from computer->Launchpad in my Conway demo easily, by hitting a button twice in a second.

I understand 

port.sendMIDIMessage( { data: Uint8Array([0x80, 0x70, 0x60] ) })

is only a couple of object allocations/constructions and pushing 3 parameters into an array (and pulling them back out) over

port.sendMessage(0x80, 0x70, 0x60)

However, as an author, it's cargo-cult boilerplate.  I think we should need to make this ultra-common case simple.
Comment 15 Jussi Kalliokoski 2012-09-08 07:02:28 UTC
(In reply to comment #14)
> Florian: I have to disagree.  This is a MIDI API.  MIDI is a software protocol.
>  I don't think we should be explicitly trying to make this a generic serial
> send/receive API.  All the wealth of devices I'm interested in out there-
> synths and keyboard controllers, drum machines, mixers, guitar amp simulators,
> DJ controllers, lighting systems, control pads like the Launchpad, and more
> have all designed for MIDI.  I don't think we need to go out of our way - e.g.
> validating every message - but we do need to design for the protocol, as timing
> can be important to real-world devices.  And most importantly - this is the
> point I really, really care about - we should be optimizing for the most
> important and commonly-used scenarios in sending and receiving MIDI messages. 
> Far and away the most common scenario is a three-byte "short" message - sysex
> is useful and needed, but you end up doing a tremendous amount with just note
> on/off and CC messages.
> 
> This also is my reasoning for wanting the "send a short message" call to be as
> trivial as possible - send( status, data0, data1) - because it is incredibly
> common.
> 
> Jussi - I disagree that "MIDI devices hardly ever transfer more than 100
> messages in a second."  Or, more to the point - I can easily see wanting to be
> able to have a higher-speed connection than that.  Not because I am a
> high-speed-Rachmaninoff player, but because I want to be twisting knobs with
> instantaneous response, and because I've seen first hand that those sexy
> Ableton and Traktor controllers have multicolor button displays, and pushing
> data back and forth to those could easily outstrip 100 messages/second.  I
> could hit that from computer->Launchpad in my Conway demo easily, by hitting a
> button twice in a second.

Nevertheless, the bottleneck is never going to be in this part. Even smartphones should be able to send at least hundreds of thousands of MIDI messages this way. And really, if the underlying API makes it slower to send completely custom messages, it's pretty easy to see the length of the typed array in the MIDIMessage and if it's <= 3, you'll know it's a short message.

sendMessage is just sugar, not really a meaningful optimization.
 
> I understand 
> 
> port.sendMIDIMessage( { data: Uint8Array([0x80, 0x70, 0x60] ) })
> 
> is only a couple of object allocations/constructions and pushing 3 parameters
> into an array (and pulling them back out) over
> 
> port.sendMessage(0x80, 0x70, 0x60)
> 
> However, as an author, it's cargo-cult boilerplate.  I think we should need to
> make this ultra-common case simple.

The API users will probably make boilerplate that fits them anyway, it's likely that there will be libraries that have separate functions for common cases, such as

noteOn(key, velocity)
noteOff(key, velocity)
controlChange(cc, value)

and they can internally use even a simple function like this for sending a message:

function send (data, /* optional */ timestamp) {
  port.sendMIDIMessage({
    timestamp: timestamp,
    data: new Uint8Array(data)
  })
}

/* send the message ASAP */
send([0x80, 0x70, 0x60])
/* send the message one second later */
send([0x80, 0x70, 0x60], performance.now() + 1000.0)

In fact I intend to make a library like that myself. ^^
Comment 16 Jussi Kalliokoski 2012-09-08 07:07:04 UTC
(In reply to comment #15)
> (In reply to comment #14)
> > Florian: I have to disagree.  This is a MIDI API.  MIDI is a software protocol.
> >  I don't think we should be explicitly trying to make this a generic serial
> > send/receive API.  All the wealth of devices I'm interested in out there-
> > synths and keyboard controllers, drum machines, mixers, guitar amp simulators,
> > DJ controllers, lighting systems, control pads like the Launchpad, and more
> > have all designed for MIDI.  I don't think we need to go out of our way - e.g.
> > validating every message - but we do need to design for the protocol, as timing
> > can be important to real-world devices.  And most importantly - this is the
> > point I really, really care about - we should be optimizing for the most
> > important and commonly-used scenarios in sending and receiving MIDI messages. 
> > Far and away the most common scenario is a three-byte "short" message - sysex
> > is useful and needed, but you end up doing a tremendous amount with just note
> > on/off and CC messages.
> > 
> > This also is my reasoning for wanting the "send a short message" call to be as
> > trivial as possible - send( status, data0, data1) - because it is incredibly
> > common.
> > 
> > Jussi - I disagree that "MIDI devices hardly ever transfer more than 100
> > messages in a second."  Or, more to the point - I can easily see wanting to be
> > able to have a higher-speed connection than that.  Not because I am a
> > high-speed-Rachmaninoff player, but because I want to be twisting knobs with
> > instantaneous response, and because I've seen first hand that those sexy
> > Ableton and Traktor controllers have multicolor button displays, and pushing
> > data back and forth to those could easily outstrip 100 messages/second.  I
> > could hit that from computer->Launchpad in my Conway demo easily, by hitting a
> > button twice in a second.
> 
> Nevertheless, the bottleneck is never going to be in this part. Even
> smartphones should be able to send at least hundreds of thousands of MIDI
> messages this way. And really, if the underlying API makes it slower to send
> completely custom messages, it's pretty easy to see the length of the typed
> array in the MIDIMessage and if it's <= 3, you'll know it's a short message.
> 
> sendMessage is just sugar, not really a meaningful optimization.
> 
> > I understand 
> > 
> > port.sendMIDIMessage( { data: Uint8Array([0x80, 0x70, 0x60] ) })
> > 
> > is only a couple of object allocations/constructions and pushing 3 parameters
> > into an array (and pulling them back out) over
> > 
> > port.sendMessage(0x80, 0x70, 0x60)
> > 
> > However, as an author, it's cargo-cult boilerplate.  I think we should need to
> > make this ultra-common case simple.
> 
> The API users will probably make boilerplate that fits them anyway, it's likely
> that there will be libraries that have separate functions for common cases,
> such as
> 
> noteOn(key, velocity)
> noteOff(key, velocity)
> controlChange(cc, value)
> 
> and they can internally use even a simple function like this for sending a
> message:
> 
> function send (data, /* optional */ timestamp) {
>   port.sendMIDIMessage({
>     timestamp: timestamp,
>     data: new Uint8Array(data)
>   })
> }
> 
> /* send the message ASAP */
> send([0x80, 0x70, 0x60])
> /* send the message one second later */
> send([0x80, 0x70, 0x60], performance.now() + 1000.0)
> 
> In fact I intend to make a library like that myself. ^^

Another thing is that it's not necessarily even an optimization for the common case, as it doesn't have the timestamp; MIDI sequencers or anything time-sensitive shouldn't rely on non-timestamped messages. Coincidentally, that boilerplate function I just made allows you to have timestamps with just adding one argument to the function call.
Comment 17 Florian Bomers 2012-09-08 08:25:07 UTC
Chris, I see your points (except the "software protocol", but that's OT). But I still don't understand why all that excludes a variadic method signature. It's a superset of what you propose.

For purpose of illustration some of the points Jussi and I made, a simple Windows implementation could look like this (again, free syntax):

function sendMessage(short status, short... data) {
  va_init(data);
  if (data.length <= 3) {
    midiOutShortMsg(hmo, (data[0] << 16) | (data[1] << 8) | data[2]);
  } else {
    this.sendMIDIMessage({data: new Uint8Array(data)});
  }
}

(of course, special case handling for length = 2, 1, or 0 will always need to be added)

So for performance, it'll be OK -- there is only a tiny bit more overhead for va_args init (which is done by compiler magic), and for one conditional.

For programmer's convenience it's the same as the non-variadic form, plus allowing you to send long messages in a convenient form, too.

For API specification and implementation, it does not add any difficulty, because there is already a way to send "arbitrary" variable length messages. It's necessary to define how these "arbitrary" should look like anyway, e.g. "One single MIDI message".

And as Jussi points out in different words: just because short messages are usually sent much more often than sys ex messages, it does not mean that a programmer will type a command for sending a short message more often.

My conclusion is that nowadays there is no technical reason to use different methods for sending/receiving short and long MIDI messages. Because you can do both with the same method. There is a ton of Windows software that does not handle Sys Ex because the API makes it so difficult to use it (for historically valid reasons). Here we have the chance to make it better.

Maybe Jussi's send() function signature will suit us both? The underlying implementation can easily optimize it for short messages...

And all this is just a couple of cents from me. I want to assist, not interfere!
Comment 18 Jussi Kalliokoski 2012-09-08 09:54:12 UTC
(In reply to comment #17)
> Chris, I see your points (except the "software protocol", but that's OT). But I
> still don't understand why all that excludes a variadic method signature. It's
> a superset of what you propose.
> 
> For purpose of illustration some of the points Jussi and I made, a simple
> Windows implementation could look like this (again, free syntax):
> 
> function sendMessage(short status, short... data) {
>   va_init(data);
>   if (data.length <= 3) {
>     midiOutShortMsg(hmo, (data[0] << 16) | (data[1] << 8) | data[2]);
>   } else {
>     this.sendMIDIMessage({data: new Uint8Array(data)});
>   }
> }
> 
> (of course, special case handling for length = 2, 1, or 0 will always need to
> be added)
> 
> So for performance, it'll be OK -- there is only a tiny bit more overhead for
> va_args init (which is done by compiler magic), and for one conditional.
> 
> For programmer's convenience it's the same as the non-variadic form, plus
> allowing you to send long messages in a convenient form, too.
> 
> For API specification and implementation, it does not add any difficulty,
> because there is already a way to send "arbitrary" variable length messages.
> It's necessary to define how these "arbitrary" should look like anyway, e.g.
> "One single MIDI message".
> 
> And as Jussi points out in different words: just because short messages are
> usually sent much more often than sys ex messages, it does not mean that a
> programmer will type a command for sending a short message more often.
> 
> My conclusion is that nowadays there is no technical reason to use different
> methods for sending/receiving short and long MIDI messages. Because you can do
> both with the same method. There is a ton of Windows software that does not
> handle Sys Ex because the API makes it so difficult to use it (for historically
> valid reasons). Here we have the chance to make it better.
> 
> Maybe Jussi's send() function signature will suit us both? The underlying
> implementation can easily optimize it for short messages...

Just to clarify, I'm not suggesting that the function is added to the specification. I don't think it's a good idea to add something that can be achieved with six lines of code to an API that's likely to be wrapped by various libraries anyway.

If it's about how many characters you have to type, I'd be happy to change the name of sendMIDIMessage() to send(). And drop sendMessage().
 
> And all this is just a couple of cents from me. I want to assist, not
> interfere!

Please don't be apologetic like that; in the standards process, all input is welcome and quite necessary actually. ;)
Comment 19 Srikumar Subramanian (Kumar) 2012-09-09 01:18:26 UTC
(In reply to comment #15)

I find the short form sendMessage API to be no simpler than the array version since the burden of having to refer to the MIDI protocol spec to find out how to construct the data bytes remains. I agree with Jussi that programmers will therefore write sendNoteOn kind of wrappers to even the shorter sendMessage API .. due to this remaining conceptual burden.

So, if the byte protocol is going to be exposed to API users without support for constructing the bytes, then having a single array based sendMIDIMessage (or simply send()) is strictly simpler. It may also help set expectations that no message verification will be performed by the API. 

If easing a developer's conceptual burden in the API is important, then it will be conceptually simpler to have multiple functions that hide the byte protocol - like sendNoteOn(timestamp, channel, pitch, velocity), perhaps with velocity in [0,1] range even. Covering the most common cases of noteOn, noteOff and controller change and allNotesOff may be enough.

If developers are to write their own wrappers, performance isn't an issue (I think). I found that a sendNoteOn wrapper on top of a shared Uint8Array based implementation gave me more than 10 million overhead calls per second on my macbook air (https://gist.github.com/3652045). That's 50x the capacity needed to send 100Hz control messages to each of 128 controllers in each of 16 channels through individual function calls.

Regarding naming, the array in MIDIMessage can hold more than one message, but the name suggests that it holds only one. Maybe adopt the CoreMIDI name "MIDIPacket" instead? A shorter send() function name also seems more correct for this reason.

> (In reply to comment #14)
> > Florian: I have to disagree.  This is a MIDI API.  MIDI is a software protocol.
> >  I don't think we should be explicitly trying to make this a generic serial
> > send/receive API.  All the wealth of devices I'm interested in out there-
> > synths and keyboard controllers, drum machines, mixers, guitar amp simulators,
> > DJ controllers, lighting systems, control pads like the Launchpad, and more
> > have all designed for MIDI.  I don't think we need to go out of our way - e.g.
> > validating every message - but we do need to design for the protocol, as timing
> > can be important to real-world devices.  And most importantly - this is the
> > point I really, really care about - we should be optimizing for the most
> > important and commonly-used scenarios in sending and receiving MIDI messages. 
> > Far and away the most common scenario is a three-byte "short" message - sysex
> > is useful and needed, but you end up doing a tremendous amount with just note
> > on/off and CC messages.
> > 
> > This also is my reasoning for wanting the "send a short message" call to be as
> > trivial as possible - send( status, data0, data1) - because it is incredibly
> > common.
> > 
> > Jussi - I disagree that "MIDI devices hardly ever transfer more than 100
> > messages in a second."  Or, more to the point - I can easily see wanting to be
> > able to have a higher-speed connection than that.  Not because I am a
> > high-speed-Rachmaninoff player, but because I want to be twisting knobs with
> > instantaneous response, and because I've seen first hand that those sexy
> > Ableton and Traktor controllers have multicolor button displays, and pushing
> > data back and forth to those could easily outstrip 100 messages/second.  I
> > could hit that from computer->Launchpad in my Conway demo easily, by hitting a
> > button twice in a second.
> 
> Nevertheless, the bottleneck is never going to be in this part. Even
> smartphones should be able to send at least hundreds of thousands of MIDI
> messages this way. And really, if the underlying API makes it slower to send
> completely custom messages, it's pretty easy to see the length of the typed
> array in the MIDIMessage and if it's <= 3, you'll know it's a short message.
> 
> sendMessage is just sugar, not really a meaningful optimization.
> 
> > I understand 
> > 
> > port.sendMIDIMessage( { data: Uint8Array([0x80, 0x70, 0x60] ) })
> > 
> > is only a couple of object allocations/constructions and pushing 3 parameters
> > into an array (and pulling them back out) over
> > 
> > port.sendMessage(0x80, 0x70, 0x60)
> > 
> > However, as an author, it's cargo-cult boilerplate.  I think we should need to
> > make this ultra-common case simple.
> 
> The API users will probably make boilerplate that fits them anyway, it's likely
> that there will be libraries that have separate functions for common cases,
> such as
> 
> noteOn(key, velocity)
> noteOff(key, velocity)
> controlChange(cc, value)
> 
> and they can internally use even a simple function like this for sending a
> message:
> 
> function send (data, /* optional */ timestamp) {
>   port.sendMIDIMessage({
>     timestamp: timestamp,
>     data: new Uint8Array(data)
>   })
> }
> 
> /* send the message ASAP */
> send([0x80, 0x70, 0x60])
> /* send the message one second later */
> send([0x80, 0x70, 0x60], performance.now() + 1000.0)
> 
> In fact I intend to make a library like that myself. ^^
Comment 20 Jussi Kalliokoski 2012-09-09 08:12:24 UTC
(In reply to comment #19)
> (In reply to comment #15)
> 
> I find the short form sendMessage API to be no simpler than the array version
> since the burden of having to refer to the MIDI protocol spec to find out how
> to construct the data bytes remains. I agree with Jussi that programmers will
> therefore write sendNoteOn kind of wrappers to even the shorter sendMessage API
> .. due to this remaining conceptual burden.
> 
> So, if the byte protocol is going to be exposed to API users without support
> for constructing the bytes, then having a single array based sendMIDIMessage
> (or simply send()) is strictly simpler. It may also help set expectations that
> no message verification will be performed by the API.

I think we'll have to at least check for partial messages. We can allow multiple messages [in a single packet], but allowing partial messages will be dangerous as the developer is not completely in charge of the order the packets are sent in. And I can't see any value in partial messages anyway.

> If easing a developer's conceptual burden in the API is important, then it will
> be conceptually simpler to have multiple functions that hide the byte protocol
> - like sendNoteOn(timestamp, channel, pitch, velocity), perhaps with velocity
> in [0,1] range even. Covering the most common cases of noteOn, noteOff and
> controller change and allNotesOff may be enough.

Yes, but I'm not quite convinced that even that is necessary. The source of the message may well not be the part of the program that actually sends the message, for example in a sequencer where there's a predefined track of events. The problem with trying to cater ease of use to the common case with MIDI is that there is no such thing, different programs require different approaches.

> If developers are to write their own wrappers, performance isn't an issue (I
> think). I found that a sendNoteOn wrapper on top of a shared Uint8Array based
> implementation gave me more than 10 million overhead calls per second on my
> macbook air (https://gist.github.com/3652045). That's 50x the capacity needed
> to send 100Hz control messages to each of 128 controllers in each of 16
> channels through individual function calls.

My point exactly. Even mobile devices have to be seriously on the edge of their performance already if communicating with MIDI is going to be a problem.

> Regarding naming, the array in MIDIMessage can hold more than one message, but
> the name suggests that it holds only one. Maybe adopt the CoreMIDI name
> "MIDIPacket" instead? A shorter send() function name also seems more correct
> for this reason.

Good idea!
Comment 21 Florian Bomers 2012-09-09 08:41:30 UTC
(In reply to comment #20)
> I think we'll have to at least check for partial messages. We can allow
> multiple messages [in a single packet], but allowing partial messages will be
> dangerous as the developer is not completely in charge of the order the packets
> are sent in. And I can't see any value in partial messages anyway.

For short messages, I agree. For Sys Ex, it is important to allow partial messages: many MIDI devices use big Sys Ex dumps (e.g. one single Sys Ex of 100KB), but cannot handle them if sent to them at full speed. Ideally, the MIDI bytes would need to be throttled for such devices, but sending in timed chunks of, say, 100 bytes will achieve the same. Or we add API methods to set the send speed...

> > the name suggests that it holds only one. Maybe adopt the CoreMIDI name
> > "MIDIPacket" instead?

fine with me, but "MIDIMessage" works the same to me.

Regarding the "sugar" method (i.e. a method that takes MIDI bytes directly as parameters): IMHO, it is useful and convenient (especially with var args) and will save a few CPU cycles compared to sendMIDIMessage().
But not having the sugar method and only provide sendMIDIMessage() will be OK for me, too -- I feel only a tiny bit of pain when allocating an array and an object for transmitting 3 bytes :)

> > A shorter send() function name also seems more correct for this reason.

yes, as previously expressed, I like that, too.
Comment 22 Jussi Kalliokoski 2012-09-09 10:28:33 UTC
(In reply to comment #21)
> For short messages, I agree. For Sys Ex, it is important to allow partial
> messages: many MIDI devices use big Sys Ex dumps (e.g. one single Sys Ex of
> 100KB), but cannot handle them if sent to them at full speed. Ideally, the MIDI
> bytes would need to be throttled for such devices, but sending in timed chunks
> of, say, 100 bytes will achieve the same. Or we add API methods to set the send
> speed...

Yes, this is a problem that needs to be thought about, thanks for bringing it up. Ideally the underlying implementation would take care of this (so that you could just send the complete message), but if so we need to specify how. It also is a bit orthogonal with previous discussion about allowing the UA to dump overflowing messages.

Maybe we need to specify the buffer handling in a way that implies that the buffers need to be big enough to hold large SysEx message dumps like that. After all, even 1MB at worst case isn't terribly much for one device given that the benefit is that the developers generally don't need to worry about the buffers getting overflown.

Ideas?

> > > the name suggests that it holds only one. Maybe adopt the CoreMIDI name
> > > "MIDIPacket" instead?
> 
> fine with me, but "MIDIMessage" works the same to me.
> 
> Regarding the "sugar" method (i.e. a method that takes MIDI bytes directly as
> parameters): IMHO, it is useful and convenient (especially with var args) and
> will save a few CPU cycles compared to sendMIDIMessage().
> But not having the sugar method and only provide sendMIDIMessage() will be OK
> for me, too -- I feel only a tiny bit of pain when allocating an array and an
> object for transmitting 3 bytes :)

Given that JS is a dynamic language, for most cases it's possible to optimize that allocation away if it really turns out to be big enough a pain point to justify that. -.-

Or the short signature can be added at a later date. But so far I haven't seen any data supporting the case that this operation is so expensive that it justifies complicating the API. Even for orders of magnitude larger performance gains it's usually hard to justify adding a special method for speed-up. I managed to get Number.prototype.clz() accepted for ES6 and even that required me to prove valid real-world use cases that would very significantly benefit from that method being a pain-point. XHR is now slowly getting support for chunked processing of data (try responseType = "moz-chunked-arraybuffer"), although it's an obvious pain point for any binary processing.

> > > A shorter send() function name also seems more correct for this reason.
> 
> yes, as previously expressed, I like that, too.
Comment 23 Florian Bomers 2012-09-09 22:12:27 UTC
(In reply to comment #22)
> Maybe we need to specify the buffer handling in a way that implies that the
> buffers need to be big enough to hold large SysEx message dumps like that.
> After all, even 1MB at worst case isn't terribly much for one device given that
> the benefit is that the developers generally don't need to worry about the
> buffers getting overflown.
> 
> Ideas?

we should to distinguish 2 problems here:

1) buffer overflow on sending side. I propose to handle this with a "false" return value of the send() function. The buffer should not get too big (except for scheduled messages) to prevent accidental bursts.

2) buffer overflow on the receiving side. This is what I meant: many hardware devices are not capable to receive/process incoming MIDI data when it comes at full MIDI speed. For the sender, it is impossible to tell that the receiver cannot keep up. It's up to the specific application to throttle as needed for a specific hardware device, e.g. by a setting for the user. It can be implemented by having an API method for throttling, or by sending only small chunks of the big Sys Ex messages with delays in between.
Comment 24 Chris Wilson 2012-09-10 00:09:01 UTC
Florian: my point around this whole conversation has been that any API should be optimized around its most important and common cases.  You made a statement that "just because short messages are
usually sent much more often than sys ex messages, it does not mean that a programmer will type a command for sending a short message more often."  I don't agree with that;  or more to the point, I don't think that is the cause&effect, but I do think programmers will type the code to send a single short MIDI message far more than they will to send sysex messages.

>My conclusion is that nowadays there is no technical reason to use different
>methods for sending/receiving short and long MIDI messages. Because you can do
>both with the same method. 

And my crescent wrench makes a pretty good hammer, too.

>Maybe Jussi's send() function signature will suit us both? The underlying
>implementation can easily optimize it for short messages...

I've maintained all along that I believe we need to have a send method that takes up to three bytes worth of arguments, and sends it immediately, with as little mental and programming overhead as possible.  Particularly in the controllerist world, that is an immensely important use case.  (and yes, sending as well as receiving.)

>And all this is just a couple of cents from me. I want to assist, not interfere!

+1 to Jussi's comment; comments are not interference, just potentially disagreement.  :)  I'd like to explicitly ask for MORE feedback from developers using MIDI today on this issue.

Jussi:
>Just to clarify, I'm not suggesting that the function is added to the
>specification. I don't think it's a good idea to add something that can be
>achieved with six lines of code to an API that's likely to be wrapped by
>various libraries anyway.

1) I disagree that it's not worthwhile to add a simpler API to skip six lines of code in a very common use case,
2) I don't agree that the MIDI API is that likely to be wrapped by various libraries that frequently, either.  Or maybe I'm just considering the authors of those libraries.

As to the various "performance isn't important here" comments - performance is always important.  But my point was really intended to be around the mental overhead of having to create two wrapper objects around three bytes of data.

Florian: are there are devices that choke on their own sysex in a single sysex packet?  I know of many devices (like my ancient Akai S612 sampler) that have a handshake protocol in sysex in order to keep from clogging the pipe, but I didn't remember any that had to have MIDI itself slowed down.
Comment 25 Florian Bomers 2012-09-10 08:06:38 UTC
(In reply to comment #24)
Chris, I am afraid that our discussion becomes circular...

Regarding the controllerist world:
Check out the MIDI spec of 50 controllers: I predict at least 45 of them require Sys Ex for configuration and often for real time control. Your Launchpad is one of the few control surfaces which does not use Sys Ex at all.
For any GM synth, to send master volume, you need Sys Ex.

> And my crescent wrench makes a pretty good hammer, too.

:) but what's the point? For the user who only sends 3 byte messages, it's totally irrelevant, and functionally equivalent, if the method is declared with up to 3 args or with var args. If it looks like a hammer, and works like a hammer, everybody is happy!

> As to the various "performance isn't important here" comments - performance is
> always important.  But my point was really intended to be around the mental
> overhead of having to create two wrapper objects around three bytes of data.

I agree, too, but still I don't see the point of limiting it to 3 bytes. With the variadic form, you'll get the performance and the mental optimization...

> Florian: are there are devices that choke on their own sysex in a single sysex
> packet?  I know of many devices (like my ancient Akai S612 sampler) that have a
> handshake protocol in sysex in order to keep from clogging the pipe, but I
> didn't remember any that had to have MIDI itself slowed down.

yes, in various forms. I know a couple first hand (sorry, no names, some of them are customers). At the time, devices wouldn't keep up with original MIDI speed. But they'd send out their bulk dumps slowly, too, so you wouldn't notice if you just record it in a sequencer and play it back. Nowadays, the processors in the devices are faster, but using USB can get them more bandwidth, too. I publish a MIDI bulk dump program, and based on customer feedback and support requests, its "throttling" mechanism is quite important. It splits up big Sys Ex messages.
Comment 26 Jussi Kalliokoski 2012-09-10 10:10:34 UTC
(In reply to comment #24)
> Florian: my point around this whole conversation has been that any API should
> be optimized around its most important and common cases.  You made a statement
> that "just because short messages are
> usually sent much more often than sys ex messages, it does not mean that a
> programmer will type a command for sending a short message more often."  I
> don't agree with that;  or more to the point, I don't think that is the
> cause&effect, but I do think programmers will type the code to send a single
> short MIDI message far more than they will to send sysex messages.
> 
> >My conclusion is that nowadays there is no technical reason to use different
> >methods for sending/receiving short and long MIDI messages. Because you can do
> >both with the same method. 
> 
> And my crescent wrench makes a pretty good hammer, too.

I laughed out loud, but the analogy doesn't really fit. ^^

> >Maybe Jussi's send() function signature will suit us both? The underlying
> >implementation can easily optimize it for short messages...
> 
> I've maintained all along that I believe we need to have a send method that
> takes up to three bytes worth of arguments, and sends it immediately, with as
> little mental and programming overhead as possible.  Particularly in the
> controllerist world, that is an immensely important use case.  (and yes,
> sending as well as receiving.)
> 
> >And all this is just a couple of cents from me. I want to assist, not interfere!
> 
> +1 to Jussi's comment; comments are not interference, just potentially
> disagreement.  :)  I'd like to explicitly ask for MORE feedback from developers
> using MIDI today on this issue.
> 
> Jussi:
> >Just to clarify, I'm not suggesting that the function is added to the
> >specification. I don't think it's a good idea to add something that can be
> >achieved with six lines of code to an API that's likely to be wrapped by
> >various libraries anyway.
> 
> 1) I disagree that it's not worthwhile to add a simpler API to skip six lines
> of code in a very common use case,

I've yet to see a demonstration that this is a more common case than any other.

> 2) I don't agree that the MIDI API is that likely to be wrapped by various
> libraries that frequently, either.  Or maybe I'm just considering the authors
> of those libraries.
> 
> As to the various "performance isn't important here" comments - performance is
> always important.  But my point was really intended to be around the mental
> overhead of having to create two wrapper objects around three bytes of data.

Performance is always important, no disagreement there, but saving a few CPU cycles doesn't justify adding an extra method unless there are demonstrated real world scenarios that suffer significantly from these extra cycles.

> Florian: are there are devices that choke on their own sysex in a single sysex
> packet?  I know of many devices (like my ancient Akai S612 sampler) that have a
> handshake protocol in sysex in order to keep from clogging the pipe, but I
> didn't remember any that had to have MIDI itself slowed down.

Florian:
I have to admit, I haven't heard of this before either. Could you elaborate this a bit? Is this some sort a memory allocation limit (I'd imagine the device has to be pretty old to not be able to handle its own system dump)? CPU limit (if so, I'd think that the device would just become unresponsive for a while)?

And how would one really counteract this, how do you know what size of chunks to send and how often? After all, based on your explanation it would appear that different devices have different limits? I also wonder how the devices handle the dump when it's complete if they can't handle it if it's sent as a whole...

Thanks for your input!
Comment 27 Chris Wilson 2012-09-10 16:50:26 UTC
(In reply to comment #25)
> Regarding the controllerist world:
> Check out the MIDI spec of 50 controllers: I predict at least 45 of them
> require Sys Ex for configuration and often for real time control. Your
> Launchpad is one of the few control surfaces which does not use Sys Ex at all.
> For any GM synth, to send master volume, you need Sys Ex.

I fully understand the Launchpad is special in that it doesn't have configuration controlled through sysex - it has remapper software.  It's not my only controller, by a long shot - not even the only one that doesn't use sysex - but I'd also point out the whole point of those controllers (like my Livid CNTRLR) that use sysex for remapping is so that developers can write software that just deals with straight controller and key mappings, and not have to write the manufacturer-specific sysex.

I've 
> 
> > And my crescent wrench makes a pretty good hammer, too.
> 
> :) but what's the point? For the user who only sends 3 byte messages, it's
> totally irrelevant, and functionally equivalent, if the method is declared with
> up to 3 args or with var args. If it looks like a hammer, and works like a
> hammer, everybody is happy!

I think we are conflating two issues, and that's not my intent.

Issue 1) I feel very strongly that the three-byte simplistic send is an extremely common case, and should be as trivial as possible (i.e. having to wrap an object and an array around those bytes is unnecessary and confusing to developers).

Issue 2) Assuming some simplistic, non-object-based send call is there for sending common MIDI messages, there is the issue of whether the variadic or optional forms should be used to describe it.  (AKA: is the simple form for messages of up to three bytes, or of arbitrary lengths?)  Here, I feel that using variadic encourages developers to use the simple form in cases where it's not really ideal; however, I frankly don't care that much.  I think it's a bit silly to use here, when my point is simply to optimise for an obvious common case (all those note on/off and controller messages, e.g.), not to improve sending arbitrary sysex (which I think will likely be in arrays already, given how sysex is typically used).  I disagree with using variadic here; I don't disagree to the point of taking my tricycle and going home, however.

I was referring to issue 1 when I made the crescent wrench comment - NOT whether the variadic or optional form was used.

> > As to the various "performance isn't important here" comments - performance is
> > always important.  But my point was really intended to be around the mental
> > overhead of having to create two wrapper objects around three bytes of data.
> 
> I agree, too, but still I don't see the point of limiting it to 3 bytes. With
> the variadic form, you'll get the performance and the mental optimization...

See above - I think the cases when you're sending sysex are different in nature.  Sysex messages tend to have a lot more than a single byte identifying what type of message they are, and therefore will likely have to have a boilerplate header in the code somewhere, likely as an array, already.  Again, this is not my highest-priority concern.

> At the time, devices wouldn't keep up with original MIDI
> speed. But they'd send out their bulk dumps slowly, too, so you wouldn't notice
> if you just record it in a sequencer and play it back. Nowadays, the processors
> in the devices are faster, but using USB can get them more bandwidth, too. I
> publish a MIDI bulk dump program, and based on customer feedback and support
> requests, its "throttling" mechanism is quite important. It splits up big Sys
> Ex messages.

Ick.  I had not realized that inside a single sysex message, common devices could not keep up; I knew some devices broke up their bulk dumps into chunks and sent the chunks slowly, relying on sequencer playback timing to keep it slowed down.  That will be unfortunate, then, and we'll have to allow partial sysex messages in MIDIMessage, a la CoreMIDI.
Comment 28 Chris Wilson 2012-09-10 16:58:47 UTC
 (In reply to comment #26)
> > 1) I disagree that it's not worthwhile to add a simpler API to skip six lines
> > of code in a very common use case,
> 
> I've yet to see a demonstration that this is a more common case than any other.

How can I demonstrate evidence that sending MIDI note on/off/CC/realtime messages is more common than sending sysex?  Obviously, pointers to my Github projects would seem biased.  :)

I know that a large part of my motivation in sparking getting a Web MIDI API off the ground was wanting to tie real-world controllers to the Web Audio API; as I have a pile of MIDI synths (and mixers, effect units, etc) in my studio, obviously I'd like web sequencing to work well, too, but of course everyone has their biases.  I think we should be optimizing (not SOLELY optimizing, but optimizing) around the use case of developers wanting to enable controllers (not just controllerist, but keyboards, drum pads, mixers, lighting, guitar controllers, etc as well) - and I do think that case heavily uses short messages (as they construct the messages on the fly most of the time, rather than having pre-recorded sequences of data.)
Comment 29 Florian Bomers 2012-09-10 21:50:31 UTC
(In reply to comment #26)
> Florian:
> I have to admit, I haven't heard of this before either. Could you elaborate
> this a bit? Is this some sort a memory allocation limit (I'd imagine the device
> has to be pretty old to not be able to handle its own system dump)? CPU limit
> (if so, I'd think that the device would just become unresponsive for a while)?

Mostly this was a series of smaller Sys Ex messages sent at full speed. This applies to older MIDI-DIN devices as well as USB MIDI devices. One reason is that they don't have a buffer large enough to hold the entire dump, so they need to process it as it's coming in, which is too slow to keep up if the incoming messages come without pauses. The other reason I'm seeing in quite a few (and new) devices is that specific Sys Ex messages will make the device unresponsive for a couple of milliseconds so it'll miss some following MIDI bytes.

But every now and then I get a support request where one big sys ex causes problems, and throttling helped.
Could be that the MIDI driver or the OS is messing things up, but because of the problems with the shorter Sys Ex messages, I always assumed it to be a problem on the receiver side.

> And how would one really counteract this, how do you know what size of chunks
> to send and how often? After all, based on your explanation it would appear
> that different devices have different limits? I also wonder how the devices
> handle the dump when it's complete if they can't handle it if it's sent as a
> whole...

Unless you're writing a general-purpose dump app, I don't see a problem. The app will be targeted to a specific device, so the programmer can take care of proper throttling/waiting.

I don't think we need to add API methods for that. My main point is that the API should not discard MIDI data if it's incomplete. It's the user's responsibility to send data that will make sense to the receiving end.

Ah, one other thing I've heard of (but not seen myself): MIDI streaming, a Sys Ex is started but never finished with F7, for realtime streaming of arbitrary (7-bit) data. FWIW.
Comment 30 Florian Bomers 2012-09-10 22:14:55 UTC
(In reply to comment #27)
> - but I'd also point out the whole point of those controllers (like my Livid
> CNTRLR) that use sysex for remapping is so that developers can write software
> that just deals with straight controller and key mappings, and not have to
> write the manufacturer-specific sysex.
> [...]
> See above - I think the cases when you're sending sysex are different in
> nature.  Sysex messages tend to have a lot more than a single byte identifying
> what type of message they are, and therefore will likely have to have a
> boilerplate header in the code somewhere, likely as an array, already.  Again,
> this is not my highest-priority concern.

yes, for some applications this is true, and the object form (aka sendMIDIMessage()) makes most sense, also for MIDI Thru/filter type applications.

But many controllers also use Sys Ex for realtime control, e.g. high-res faders, text/graphics displays.
And MIDI Show Control! I assume those guys will love a variadic send() function!

I like to think of MIDI as a protocol with variable length messages: 1 or more bytes per message. Which messages you send is up to the user. Drawing a hard line at the 3 byte boundary will penalize those sending messages with more than 3 bytes.

> [sending partial messages]
> That will be unfortunate, then, and we'll have to allow partial
> sysex messages in MIDIMessage, a la CoreMIDI.

I don't understand, Windows allows sending partial MIDI messages too?

Anyway, another thing to consider is that the implementation needs to handle splitting up received MIDI messages, too, if they get too big.
Comment 31 Chris Wilson 2012-09-10 22:27:30 UTC
(In reply to comment #30)
> I like to think of MIDI as a protocol with variable length messages: 1 or more
> bytes per message. Which messages you send is up to the user. Drawing a hard
> line at the 3 byte boundary will penalize those sending messages with more than
> 3 bytes.

I can understand that view.  My argument was there is a serious inflection point at 3 bytes-or-less; but again, my bigger concern is that we not, as you put it, "penalize" those sending messages of 3 or fewer bytes (by making them learn to create MIDIMessages and organize into arrays), not that we enforce a hard boundary at 3 bytes.  (Although I think it's a good idea, but I think I've beaten that horse enough by now.)

> > [sending partial messages]
> > That will be unfortunate, then, and we'll have to allow partial
> > sysex messages in MIDIMessage, a la CoreMIDI.
> 
> I don't understand, Windows allows sending partial MIDI messages too?

Yes.  I'm just starting to think the "MIDIMessage" name might be misleading, if it's not necessarily a complete message; but then there's the oddity in CoreMIDI that short messages are not to be split across MIDIPackets (I don't know if that's enforced, but it is stated in the docs), so I don't think we can just say "arbitrary buffer of any kind of data" either.
Comment 32 Florian Bomers 2012-09-10 23:10:02 UTC
(In reply to comment #31)
> hard boundary at 3 bytes.  (Although I think it's a good idea, but I think I've
> beaten that horse enough by now.)

yeah, my horse got its share, too!

> Yes.  I'm just starting to think the "MIDIMessage" name might be misleading, if
> it's not necessarily a complete message; but then there's the oddity in
> CoreMIDI that short messages are not to be split across MIDIPackets (I don't
> know if that's enforced, but it is stated in the docs), so I don't think we can
> just say "arbitrary buffer of any kind of data" either.

I'd opt for something like "MIDIMessage: a complete MIDI message. Sys Ex messages can be sent/received partially".
Comment 33 Jussi Kalliokoski 2012-09-11 10:15:47 UTC
(In reply to comment #28)
>  (In reply to comment #26)
> > > 1) I disagree that it's not worthwhile to add a simpler API to skip six lines
> > > of code in a very common use case,
> > 
> > I've yet to see a demonstration that this is a more common case than any other.
> 
> How can I demonstrate evidence that sending MIDI note on/off/CC/realtime
> messages is more common than sending sysex?  Obviously, pointers to my Github
> projects would seem biased.  :)
> 
> I know that a large part of my motivation in sparking getting a Web MIDI API
> off the ground was wanting to tie real-world controllers to the Web Audio API;
> as I have a pile of MIDI synths (and mixers, effect units, etc) in my studio,
> obviously I'd like web sequencing to work well, too, but of course everyone has
> their biases.  I think we should be optimizing (not SOLELY optimizing, but
> optimizing) around the use case of developers wanting to enable controllers
> (not just controllerist, but keyboards, drum pads, mixers, lighting, guitar
> controllers, etc as well) - and I do think that case heavily uses short
> messages (as they construct the messages on the fly most of the time, rather
> than having pre-recorded sequences of data.)

Hmm, do you think these use cases, where - as you said - the developers have a hard time wrapping their head around the conceptual overhead of port.send({data: Uint8Array([...])}), are better served by the short message pattern rather than a small library that provides them with methods like noteOn(), controllerChange() etc. and abstracts the send() interface away for them?
Comment 34 Chris Wilson 2012-09-11 23:34:37 UTC
(In reply to comment #33)
> Hmm, do you think these use cases, where - as you said - the developers have a
> hard time wrapping their head around the conceptual overhead of
> port.send({data: Uint8Array([...])}), are better served by the short message
> pattern rather than a small library that provides them with methods like
> noteOn(), controllerChange() etc. and abstracts the send() interface away for
> them?

Yes, because most of those noteOn()/controller() calls have higher-level semantics.  I would wrap a scenario-specific library with "turnOnLights()" calls or something, and those would have implementations that translate into 3 bytes of data.  We shouldn't need to have app libraries just to hide the details of the funky boilerplate, and I don't think in general the note on message vs controller semantic is a necessary one; you use what you need at that point in your app.  In some cases, you'll be writing a learning/remapping layer anyway, that might need to change what calls you're making.

Plus you're suggesting wrapping ANOTHER layer around what I'm already saying I think is too many layers.  :)
Comment 35 Srikumar Subramanian (Kumar) 2012-09-12 06:03:22 UTC
(In reply to comment #34)

Taking a look at Chris Wilson's github repo, the 3 byte sendMessage call has been used quite a bit to turn on/off lights/buttons on a controller.  For these cases, clearly sendMessage is the right level of abstraction and names such as noteOn/noteOff/controllerChange are not. Indeed, even the MIDI spec for noteOn as [0x90 + channel, noteNumber, velocity] is useless, since the byte values are used for entirely different purposes such as setting the colour of a button.

For those interested in making noises via a softsynth available on standard computers however, noteOn()/noteOff()/etc. are the right level of abstraction since it saves developers who may not be intimate with MIDI the trouble of referring to the MIDI spec first.

For the controller display cases, if you consider someone fresh to a controller who needs to lookup its manual to find out what MIDI messages to send to it, will the manual entry for a light on message be found to be worded as, say, "send a noteOn message giving the button number in the pitch field and the brightness level as velocity" or as "send a three byte message [0x90, buttonNumber, brightness]". If the former, then a noteOn-like API still makes sense. If the latter, then a raw 3-byte form makes sense. If both are used, then it may benefit developers to have both kinds of APIs.

So it looks to me like this debate boils down to whether the WG is to implement a MIDI API for its sound making aspects or for the general uses its protocol can be put to. 

> (In reply to comment #33)
> > Hmm, do you think these use cases, where - as you said - the developers have a
> > hard time wrapping their head around the conceptual overhead of
> > port.send({data: Uint8Array([...])}), are better served by the short message
> > pattern rather than a small library that provides them with methods like
> > noteOn(), controllerChange() etc. and abstracts the send() interface away for
> > them?
> 
> Yes, because most of those noteOn()/controller() calls have higher-level
> semantics.  I would wrap a scenario-specific library with "turnOnLights()"
> calls or something, and those would have implementations that translate into 3
> bytes of data.  We shouldn't need to have app libraries just to hide the
> details of the funky boilerplate, and I don't think in general the note on
> message vs controller semantic is a necessary one; you use what you need at
> that point in your app.  In some cases, you'll be writing a learning/remapping
> layer anyway, that might need to change what calls you're making.
> 
> Plus you're suggesting wrapping ANOTHER layer around what I'm already saying I
> think is too many layers.  :)
Comment 36 Jussi Kalliokoski 2012-09-12 07:20:52 UTC
(In reply to comment #34)
> (In reply to comment #33)
> > Hmm, do you think these use cases, where - as you said - the developers have a
> > hard time wrapping their head around the conceptual overhead of
> > port.send({data: Uint8Array([...])}), are better served by the short message
> > pattern rather than a small library that provides them with methods like
> > noteOn(), controllerChange() etc. and abstracts the send() interface away for
> > them?
> 
> Yes, because most of those noteOn()/controller() calls have higher-level
> semantics.  I would wrap a scenario-specific library with "turnOnLights()"
> calls or something, and those would have implementations that translate into 3
> bytes of data.  We shouldn't need to have app libraries just to hide the
> details of the funky boilerplate, and I don't think in general the note on
> message vs controller semantic is a necessary one; you use what you need at
> that point in your app.  In some cases, you'll be writing a learning/remapping
> layer anyway, that might need to change what calls you're making.

Of course, use the right tool for the job. But programming is all about abstraction. You have something and change the abstraction into something else, but in the end it's all just data. Keeping programmers from having to do abstraction in a low-level API is counter-productive.

> Plus you're suggesting wrapping ANOTHER layer around what I'm already saying I
> think is too many layers.  :)

Yep, but I'm suggesting the extra layers stay on the user side rather than the API side.

document.createElement('div') might be the most used case of creating a DOM element, but even though it saves a few characters, it really doesn't make sense to add a window.createDiv() method. If we start adding methods for saving a few characters we're going to end up with a framework instead of an API, and web frameworks should stay in the user-world side.
Comment 37 Jussi Kalliokoski 2012-09-12 07:26:24 UTC
(In reply to comment #24)
> 2) I don't agree that the MIDI API is that likely to be wrapped by various
> libraries that frequently, either.  Or maybe I'm just considering the authors
> of those libraries.

BTW, I don't think the short form is going to be very useful for libraries, because it'll seriously restrict their use cases if the library doesn't support timestamps.
Comment 38 Chris Wilson 2012-09-12 18:24:24 UTC
(In reply to comment #36)
> Keeping programmers from having to do
> abstraction in a low-level API is counter-productive.

Huh?  I'm not preventing programmers from doing abstraction at a low level; I'm saying forcing them to have an extra layer of abstraction around the concept of a MIDI message (when the call is already doing that) is unnecessary.  They're free to wrap any layers of abstraction around those messages that they would find useful in a particular scenario; I've never argued they should have semantic calls at the Web MIDI level, just that sending an up-to-3-byte short message needs to be super-efficient.

> > Plus you're suggesting wrapping ANOTHER layer around what I'm already saying I
> > think is too many layers.  :)
> 
> Yep, but I'm suggesting the extra layers stay on the user side rather than the
> API side.

And I'd certainly agree with that; I don't want to see noteOn() in the MIDI API.  I would, of course, expect lots of libraries to have a function named noteOn() (in fact, a couple of my examples do - but several do not).

(In reply to comment #36)
> BTW, I don't think the short form is going to be very useful for libraries,
> because it'll seriously restrict their use cases if the library doesn't support
> timestamps.

I disagree quite strongly.  In the sequencer use case, that may well be true; but I also expect sequencers will be structured around keeping data in MIDIMessage-compatible form, and wouldn't use the short form.  For my controllers, though, on the other hand, I'll absolutely want a library call to turn on lights, set encoder display levels, etc., that is not timestamp-based.
Comment 39 Florian Bomers 2012-09-12 20:05:09 UTC
(In reply to comment #37)
> BTW, I don't think the short form is going to be very useful for libraries,
> because it'll seriously restrict their use cases if the library doesn't support
> timestamps.

I wouldn't say that, either. Timestamps for sending is only useful for a certain type of apps. For all interactive and realtime MIDI, it's mostly useless. Also, you could argue that timestamps for sending data is high level, because you can easily handle scheduling of MIDI messages in a user lib. 

Of course for receiving data, it is essential that it's submitted with a timestamp.
Comment 40 Chris Wilson 2012-11-01 23:58:52 UTC
With the consolidation of sendMessage and sendMIDIMessage, I believe we have finally resolved the issues here.

https://dvcs.w3.org/hg/audio/rev/f939eca58e74.
Comment 41 Olivier Thereaux 2012-11-19 16:04:07 UTC
(In reply to comment #40)
> With the consolidation of sendMessage and sendMIDIMessage, I believe we have
> finally resolved the issues here.
> 
> https://dvcs.w3.org/hg/audio/rev/f939eca58e74.

Phew. Closing.