How would you transfer these types to other applications on the OS?

Drag and drop to an application outside the browser:

If the datatransfer data is an object that belongs to the browser (like the Date object) then the browser should select the most appropriate type for that object based on the MIME spec and the most appropriate behavior for the thing being dragged.
Below there's a list with default objects and their corresponding MIME and output formats.
For the objects not mentioned below, it's correct, by this spec, if the contents sent to the destination is the result of calling toString() on the object and defaulting the MIME to text/plain.
Below there's examples of the output format and MIME for some of the javascript objects specified in the HTML5 and the DOM level 4 specification.
Unless stated, all objects that are not referenced but extend any of the mentioned ones must follow the same rules as the ones mentioned. E.g. Function extends Object, so if a Function is given, it is treated just like it is specified for an Object.

Below, "data" is the contents given in the 2nd parameter when calling setData()


Window object: Output data.toString();

Object: Output data.toString();

Array (not array-like): Has the MIME of multipart/mixed where each element goes into its own part (see below).

ArrayBuffer object: Output is its contents in binary and the default MIME is application/octet-stream

RegExp object: Output is data.toString() and the MIME is text/plain (or text/X-*).

Blob object: Output is the contents stored in the blob and the MIME is as specified when the Blob was created. If such is invalid, then MIME is application/octet-binary.


Special cases:
Date object: Output date according to ISO 8601 in the combined date and time format. If the time zone is provided, then that must be included too.

File object: The browser should communicate with the other program the same way as if the file had been dragged from the OS's file explorer to the neighbor application. All required data that the OS specifies that is supplied in the process should be supplied.
As a test, a drag and drop towards the OS's native file explorer must succeed as long as the file is valid. As a second test, dragging from the browser onto any application must give the same results as dragging from the OS's native file explorer into that same application.

Element object (or something that extends it) the contents is the data stored in the outerHTML parameter. The default type is text/html or application/xhtml+xml depending if the document is specified as being HTML or xHTML respectively. The MIME allowed is limited to text/html, text/xhtml, application/xml and application/xhtml+xml and the masks text/X-* and application/X-*.

If it is a textDocument, then it contains the contents of the text parameter. The default MIME is text/plain and the MIME must match text/*.

If it is a primitive, then just do the equivalent to:
> primitive + ""
the default type is text/plain.

Special case:
If the object is an Array of File (it's an array such that in all its positions it only contains a mix of File objects and undefined), then all the files are sent to the destination application in the same order as they are in the source Array.

For other browser native objects not specified above, it's up to the browser to decide the best way to represent the data to the other application. Even then it must always guarantee that the MIME corresponds to the actual data being sent. Custom ("X-*") MIME must be avoided whenever possible. If the user specifies a valid MIME for the output contents or the MIME matches the mask "*/X-*", that MIME overrides the browser's choice.

If it is an non-native javascript object, just print the toString() call to the returned object by toString()(same output as when the object is sent to the alert() function).
The default MIME is text/plain and the MIME must match text/* (or text/X-*).


If it is an Array-like DOM object (like a NodeList), then it should apply the above for every Node that it contains in the list or collection. The browser must select the correct MIME for this and it must consider using multipart/mixed if appropriate.

For Arrays, each element belongs to a part (of MIME multipart/mixed). The browser should iterate through the array and convert each element into its part according to what is specified above and in the same order as it is in the array.
For arrays, the type the user specifies belongs to the elements that are in the individual elements of the array walk, not the array itself (the array is always multipart/mixed). Self references must never lead to infinite loop or an error (a warning is OK) during the conversion, other than that, it's up for the browser to decide how to deal with self references.



If the final application explicitly rejects the drop for the data with a custom MIME (and a custom MIME only), the browser may try that again with the MIME calculated by himself according to the above rules. If, by chance, both match or the browser cannot calculate the MIME himself or, if by past experience, the browser knows the destination program will also reject the 2nd try, do not try again.
Also, if a drop is re-tried by the user, the browser must never assume that the destination program will reject, so it should try to re-process the drop regardless.
The browser must never make the automatic 2nd try if it knows (by past experience) that the target application will deny the drop.

Issues and how to deal with them

If the call to toString() fails then:
if toString is not a function, or an Error happens inside the toString() method, the user agent must fire a "ErrorEvent" event on the script's global object which is cancelable and whose default action is to report the error to the user (in an error console (or similar), if possible).
The ErrorEvent's filename, lineno and colno must reflect the file name, line number and column number of where the corresponding setData() method is called.


The strictly necessary information stating that this had happened should be send to the final program (where the drop happened)(which may/may not trigger an error in the final application but, still, it might be able to call the attention towards the user). The message must follow the MIME type used rules. (E.G. if application/JSON, then it must be valid JSON, such as a simple JSON string)

If the MIME given is unknown to the browser (it must also try to cross reference with the well known "*/X-" MIME types), then the MIME is text/plain.

Here's some examples for each MIME:
application/JSON: "Error: \"Error occurred while processing the drag&drop on the origin application\""

text/plain: Error: Error occurred while processing the drag&drop on the origin application

application/octet-stream: Error: Error occurred while processing the drag and drop on the origin application\0
(\0 is the null character)

Example of custom MIME that the browser must know:
      A              is the same as          B
application/x-binary                application/octet-stream
application/x-javascript            application/javascript
application/x-zip-compressed        application/zip
image/x-icon                        N/A
etc...


The reason why exceptions inside the setData() method are not javascript exceptions is because the browser is free to evaluate the input (by the above rules) at any time it wants from the drag to the actual releasing the mouse button at the drop location, for performance reasons.


It's the browser's responsibility to enforce that the correct MIME is sent to the destination based on the content that is being supplied as long as the MIME does not match "*/X-*".
Note: Some objects are more restrictive than just "*/X-" and enforce it to be "image/*". For those the MIME must additionally match that. For example, if it is a TextDocument, only "text/*" is valid, so any "application/X-*" or "image/X-*" are invalid although all that match "text/X-*" will be automatically considered valid regardless of the contents of the TextDocument.

If the MIME supplied is wrong, it should be considered invalid and one of the above MIME must selected instead based on the actual content being sent.

For example, if the user says it is sending an application/json but it sends a DOM object, the browser should change the MIME to text/html (or application/xhtml+xml, depending on how it is structured).
If the user is sending a String and the MIME it tells is not text/* or */X-* ("*" represents anything), then it should be considered invalid and it would be changed to text/plain.

Unless otherwise stated, for all cases above, if the user specifies a MIME that matches the mask
>*/X-*

Then that is the MIME that the browser sends to the drop destination in its first drop attempt.


Simplification table between Object, accepted MIME and default MIME

Date		text/plain		text/plain

Window		text/plain 		text/plain

Object		text/plain 		text/plain

Array 		multipart/mixed		multipart/mixed

(typed array)	*/*			application/octet-stream

ArrayBuffer	*/*			application/octet-stream

RegExp 		text/plain		text/plain

Blob		*/*			if valid, as specified in the Blob, 
					else, 
					application/octet-binary

String		*/*			text/plain

Element		text/html, text/xhtml,	text/html (if doc is HTML)
		application/xml,	application/xhtml+xml (if doc is xHTML)
		application/xhtml+xml
TextDocument	text/*			text/plain

(the above does not include the acceptance of a MIME with the mask */X-*)

Sorry about the delay but I wanted to have something complete enough without wasting your time with WIP.

Please review my proposal and point any mistakes and errors with my logic.

If you want to, we could even make a protocol for sending Nodes between browsers but I don't think that that is necessary as sending Nodes between documents on the same browser is already a very fuzzy thing in the documentation.

This I presented is not a completely final form bit I think it is good enough to be applied to real life scenarios.

Feedback is appreciated.

Interesting. So basically you're saying we should special-case interesting things that might be added, and for everything else, just provide some default text representation. I suppose that could work.

Do you have any specific use cases in mind? That might be useful in convincing browser vendors to support this.

Any browser vendors interested in supporting this?

Errata:

Note: Some objects are more restrictive than just "*/X-" and enforce it to be "image/*"

Should be:

Note: Some objects are more restrictive than just "*/X-" and, for example, enforce it to be "image/*"

> Do you have any specific use cases in mind? That might be useful in convincing browser vendors to support this.

I don't have them for some of the situations. E.g. I don't really know what can be done about the window object or an Object object.

I do have them for some of the others, though.

Passing an array of File can be used in an AJAX file downloader. You select the files you want to download by ctrl + click and then drag them all to a directory in the OS's file explorer completely bypassing the OS's file picker.
You could also drag to a WYSIWYG text processor in order to attach the set of files to the document.

For File, in a system such as Mega (files are encrypted in the server and they are decrypted using javascript in the client as they are downloaded and then they are stored in memory), I could drag the downloaded file to a directory on my computer bypassing the file picker.

For an array of Date, you could select multiple Dates (mouse selection or ctrl + click or shift + click) on a calendar and then drag to a text processor or an external calendar manager that would identify the dates and do something with them (possibly mark them as occupied or open a dialog that allows the user to select what to do with those days/hours). It could also take an array of 2 dates as being an interval.

For an Element, it could be used to copy and paste the actual source HTML from one browser to another. It can be done to avoid no behavior if the user drags from the browser.

The Blob can be used to transfer images. For example, a javascript colaborative image processing program. You'd build the image and then, when complete, you'd drag to an image processor or attach to a word processor directly or you'd encapsulate it into a file to store in the computer while completely bypassing the browser's file picker.


I'm more of a problem solver than problem maker but I think this can give some clues about what this enables. It does not mean that it isn't possible with the current API's but (I believe) drag & drop is much more intuitive for most people

Files are already supported in the current API.

Currently commented out in the spec is the ability specifically to also support:

 - text/html, e.g. dragged from selections
 - Blobs
 - anything that can be structured-cloned
 - promises for any of the above

Would that be sufficient for your use cases?

Exposing text/html and blobs to other applications is trivial; exposing structured-cloned data would probably be done via JSON, with some special logic for handling loops and types JSON doesn't support natively. I've no idea how promises would be exposed; I guess it could be transparent if the platform API works that way, otherwise it would likely just not be exposed.

Sorry I lost the e-mail.

Yes, that looks good except for the missing Node.
Firefox already implements sending DOM nodes under the MIME: application/x-moz-node 

Source: https://developer.mozilla.org/en-US/docs/DragDrop/Recommended_Drag_Types#Dragging_Nodes

On the same page, it also talks about an implementations where a multitude of elements can be sent using the drag&drop API.
For example, for files, 
I didn't confirm now, but, IIRC, the current spec does not allow drag & drop of multiple files, just single independent files.
I don't know if mozilla's implementation idea is the best way to go or if mine's better (or if there's even a better one) but I think it is a good idea to define an implementation for it.

Outside these two, by what you mention, I think the spec (with the commented part) should cover what I mean.

Any updates

The feature's on my list of feature requests that need studying. The next step is to get buy-in from browser vendors.

Created attachment 1660 [details]
Wsop

If still desired, please file an issue against https://github.com/whatwg/html/issues/new.

OK. Done
https://github.com/whatwg/html/issues/2859
Does it look correct?

Yeah, though there's not a lot of interest from implementers in improving drag & drop it seems like.