27421 2014-11-24 22:38:58 +0000 Investigate having a tool which files bugs on implementations whenever the API behavior has changed in the spec 2019-03-29 22:48:32 +0000 1 1 1 Unclassified WHATWG Unwelcome unspecified PC Linux RESOLVED FIXED P2 normal Unsorted 1 bugs mike annevk d hober0 ian james Ms2ger philipj travil w3bugs sideshowbarker+unwelcome oldest_to_newest 115431 0 bugs 2014-11-24 22:38:58 +0000 Automating 'spec change -> bugs filed on implementations' would probably speed up updating implementations to follow the spec. 115462 1 Ms2ger 2014-11-25 11:40:07 +0000 Thinking in a bit of a different direction: I've been wanting a kind of dashboard that shows which commits still need tests. Perhaps this could also include a convenient button to file a bug with some fields already filled in / mark as "no bug needed". (Not volunteering.) 115476 2 annevk 2014-11-25 20:11:14 +0000 It might also lead to getter feedback on a specification a lot quicker. 124367 3 w3bugs 2015-12-06 01:10:41 +0000 * Mozilla Bugzilla API * https://wiki.mozilla.org/Bugzilla:REST_API * http://bugzilla.readthedocs.org/en/latest/api/index.html * WebKit Bugzilla API (they're running an older version) * https://www.bugzilla.org/docs/4.4/en/html/api/Bugzilla/WebService.html * XML-RPC endpoint: https://bugs.webkit.org/xmlrpc.cgi * JSON-RPC endpoint: https://bugs.webkit.org/jsonrpc.cgi * Google Code doesn't seem to have an API anymore (https://code.google.com/p/support/wiki/IssueTrackerAPI#Shut_Down ) * https://connect.microsoft.com/IE/Feedback doesn't seem to offer an API 124368 4 philipj 2015-12-06 17:56:39 +0000 Since not all changes will affect all implementations, a naive implementation of this would likely be too spammy to be appreciated. Something similar would be to require all spec changes to have accompanying test updates for web-platform-tests, and then those engines that regularly import test from wpt would notice with no extra action. Currently I think Gecko does it best, but Blink imports at some of the tests. 124370 5 Ms2ger 2015-12-06 20:06:14 +0000 Even though we do regular wpt imports, we don't really track new failures as they come in. 124371 6 d 2015-12-06 20:13:12 +0000 Here are projects that seem doable and useful to me: - A one-click process for converting a spec commit into a GitHub issue on w3c/web-platform-tests saying "A spec change was recently made; we need tests for it. Here is the commit message." Maybe include an optional comment from the button-clicker. - An easy to fill out form that allows filing bugs on multiple implementations at once. - Finds the right components to file on - Requires manual testing from the filer to determine which bug trackers to file on; the filer can check boxes to determine which to send it to. - Optionally: a tool that runs code snippets in all bleeding-edge browsers and tells you which fail, to automate the selection of which bug trackers to file on. - Optionally: an automated process where a commit message to a spec can include something like "Requires changes to: Gecko, WebKit" and the bugs will be auto-filed. - Optionally: a link to relevant failing web platform tests, probably input manually 129718 7 d 2019-03-29 22:48:32 +0000 I'm happy with our current working mode here. If folks want to automate more, let's discuss on whatwg/meta.