This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 27421 - Investigate having a tool which files bugs on implementations whenever the API behavior has changed in the spec
Summary: Investigate having a tool which files bugs on implementations whenever the AP...
Status: RESOLVED FIXED
Alias: None
Product: WHATWG
Classification: Unclassified
Component: Unwelcome (show other bugs)
Version: unspecified
Hardware: PC Linux
: P2 normal
Target Milestone: Unsorted
Assignee: Michael[tm] Smith
QA Contact: sideshowbarker+unwelcome
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-11-24 22:38 UTC by Olli Pettay
Modified: 2019-03-29 22:48 UTC (History)
9 users (show)

See Also:


Attachments

Description Olli Pettay 2014-11-24 22:38:58 UTC
Automating 'spec change -> bugs filed on implementations' would probably speed
up updating implementations to follow the spec.
Comment 1 Ms2ger 2014-11-25 11:40:07 UTC
Thinking in a bit of a different direction: I've been wanting a kind of dashboard that shows which commits still need tests. Perhaps this could also include a convenient button to file a bug with some fields already filled in / mark as "no bug needed".

(Not volunteering.)
Comment 2 Anne 2014-11-25 20:11:14 UTC
It might also lead to getter feedback on a specification a lot quicker.
Comment 3 Chris Rebert 2015-12-06 01:10:41 UTC
* Mozilla Bugzilla API
  * https://wiki.mozilla.org/Bugzilla:REST_API
  * http://bugzilla.readthedocs.org/en/latest/api/index.html
* WebKit Bugzilla API (they're running an older version)
  * https://www.bugzilla.org/docs/4.4/en/html/api/Bugzilla/WebService.html
  * XML-RPC endpoint: https://bugs.webkit.org/xmlrpc.cgi
  * JSON-RPC endpoint: https://bugs.webkit.org/jsonrpc.cgi
* Google Code doesn't seem to have an API anymore (https://code.google.com/p/support/wiki/IssueTrackerAPI#Shut_Down )
* https://connect.microsoft.com/IE/Feedback doesn't seem to offer an API
Comment 4 Philip Jägenstedt 2015-12-06 17:56:39 UTC
Since not all changes will affect all implementations, a naive implementation of this would likely be too spammy to be appreciated. Something similar would be to require all spec changes to have accompanying test updates for web-platform-tests, and then those engines that regularly import test from wpt would notice with no extra action. Currently I think Gecko does it best, but Blink imports at some of the tests.
Comment 5 Ms2ger 2015-12-06 20:06:14 UTC
Even though we do regular wpt imports, we don't really track new failures as they come in.
Comment 6 Domenic Denicola 2015-12-06 20:13:12 UTC
Here are projects that seem doable and useful to me:

- A one-click process for converting a spec commit into a GitHub issue on w3c/web-platform-tests saying "A spec change was recently made; we need tests for it. Here is the commit message." Maybe include an optional comment from the button-clicker.

- An easy to fill out form that allows filing bugs on multiple implementations at once.
  - Finds the right components to file on
  - Requires manual testing from the filer to determine which bug trackers to file on; the filer can check boxes to determine which to send it to.
  - Optionally: a tool that runs code snippets in all bleeding-edge browsers and tells you which fail, to automate the selection of which bug trackers to file on.
  - Optionally: an automated process where a commit message to a spec can include something like "Requires changes to: Gecko, WebKit" and the bugs will be auto-filed.
  - Optionally: a link to relevant failing web platform tests, probably input manually
Comment 7 Domenic Denicola 2019-03-29 22:48:32 UTC
I'm happy with our current working mode here. If folks want to automate more, let's discuss on whatwg/meta.