Future of the Open Web

• Upcoming
• Tentative
• Breakout Sessions

The phrase “Open Web” has served as a rallying cry for those who champion the benefits of the Web as we know it to humanity. However, there are subtle (and not-so-subtle) distinctions in what it means depending on who you talk to: some believe it to be a Web where all content is available no matter the purpose; others focus on “Open Web” in relation to the advertising market; yet others distinguish between open to humans-using-browsers vs. other clients.

At the same time, large parts of the Web have always been behind logins and paywalls: the so-called “deep Web”. Furthermore, it’s important to acknowledge that the reason that many parts of the Web have been free to access because they are advertising-supported – along with all of the concurrent privacy harms.

While there may not be strong agreement on what the Open Web is, it’s widely held that it is under more threat than ever, thanks to the emergence of AI. Content authors, publishers, and hosters may have been accepting of the bargain they got from search crawlers: let us access your content and you will get traffic from search results. They are much more reticent about exposing their content when it has no such quid pro quo.

As a result, blocking unknown or unwanted crawlers has become more common, and the Web Bot Auth effort is in the process of being chartered to enable known bots to cryptographically authenticate themselves – with the implication that those clients that don’t may be blocked. There are also proposals emerging for payment protocols specifically targeted at not only crawlers, but also bots and agents – and maybe even for browser micropayments.

There are also emerging concerns that AI will put gatekeeper platforms in the position to drive further concentration: to pressure publishers to submit content to them directly rather than taking on the cost of hosting content themselves, thereby further foreclosing what was previously open content.

These changes have led to questions about the future of the Open Web – whether we’ll have one, and what it will look like. A less open Web is one where it is more difficult to introduce new non-browser agents or run services that don’t have immediate financial advantage for sites. One where more data is locked up indiscriminately.

This session will explore questions in this area, such as:

• Is there a useful definition for the “Open Web”? Or is it better to have more specific goals?
• Are there useful metrics for the Open Web that can be tracked?
• What purposes do sites have for not being open?
• Is there more than one economic model that promotes the Open Web?
• How is privileged access for search engine crawlers changing?
• What impacts on the Open Web do we see from AI and “agentic” clients?
• Are there best practices for sites and services that could help mitigate those impacts?
• Are there standards activities that could help mitigate those impacts?

Agenda

Chairs:
Mark Nottingham, David Schinazi

Description:
The phrase “Open Web” has served as a rallying cry for those who champion the benefits of the Web as we know it to humanity. However, there are subtle (and not-so-subtle) distinctions in what it means depending on who you talk to: some believe it to be a Web where all content is available no matter the purpose; others focus on “Open Web” in relation to the advertising market; yet others distinguish between open to humans-using-browsers vs. other clients.

At the same time, large parts of the Web have always been behind logins and paywalls: the so-called “deep Web”. Furthermore, it’s important to acknowledge that the reason that many parts of the Web have been free to access because they are advertising-supported – along with all of the concurrent privacy harms.

While there may not be strong agreement on what the Open Web is, it’s widely held that it is under more threat than ever, thanks to the emergence of AI. Content authors, publishers, and hosters may have been accepting of the bargain they got from search crawlers: let us access your content and you will get traffic from search results. They are much more reticent about exposing their content when it has no such quid pro quo.

As a result, blocking unknown or unwanted crawlers has become more common, and the Web Bot Auth effort is in the process of being chartered to enable known bots to cryptographically authenticate themselves – with the implication that those clients that don’t may be blocked. There are also proposals emerging for payment protocols specifically targeted at not only crawlers, but also bots and agents – and maybe even for browser micropayments.

There are also emerging concerns that AI will put gatekeeper platforms in the position to drive further concentration: to pressure publishers to submit content to them directly rather than taking on the cost of hosting content themselves, thereby further foreclosing what was previously open content.

These changes have led to questions about the future of the Open Web – whether we’ll have one, and what it will look like. A less open Web is one where it is more difficult to introduce new non-browser agents or run services that don’t have immediate financial advantage for sites. One where more data is locked up indiscriminately.

This session will explore questions in this area, such as:

• Is there a useful definition for the “Open Web”? Or is it better to have more specific goals?
• Are there useful metrics for the Open Web that can be tracked?
• What purposes do sites have for not being open?
• Is there more than one economic model that promotes the Open Web?
• How is privileged access for search engine crawlers changing?
• What impacts on the Open Web do we see from AI and “agentic” clients?
• Are there best practices for sites and services that could help mitigate those impacts?
• Are there standards activities that could help mitigate those impacts?

Goal(s):
Discussion and planning

Materials:

• Session proposal on GitHub

Joining Instructions

Instructions are restricted to W3C users . You need to log in to see them.

Export options

Personal Links

Please log in to export this event with all the information you have access to.

Public Links

The following links do not contain any sensitive information and can be shared publicly.

• Download as ics