This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 6853 - restore meta keywords, search engines use them
Summary: restore meta keywords, search engines use them
Status: CLOSED DUPLICATE of bug 7525
Alias: None
Product: HTML WG
Classification: Unclassified
Component: pre-LC1 HTML5 spec (editor: Ian Hickson) (show other bugs)
Version: unspecified
Hardware: All All
: P2 normal
Target Milestone: ---
Assignee: Ian 'Hixie' Hickson
QA Contact: HTML WG Bugzilla archive list
URL: http://www.w3.org/TR/html5/
Whiteboard:
Keywords: NE
Depends on:
Blocks:
 
Reported: 2009-04-29 06:21 UTC by Nick Levinson
Modified: 2010-10-04 14:47 UTC (History)
6 users (show)

See Also:


Attachments

Description Nick Levinson 2009-04-29 06:21:28 UTC
The meta element for name="keywords" has recently been demoted to being a failed proposal, presumably thus not to be allowed under HTML5. I disagree and suggest the status be restored to that of a proposal. Search engines use it and for good reason.

1. Advice is often to integrate keywords into the visible text, especially near the beginning, and into headlines, especially in h1 and h2 elements. This is sound but has a limitation. I'm writing a page against using drugs to stay awake. I'm  telling readers that I don't know of any I'd want to recommend. Therefore, I don't want to list any. But I know some people will search by a drug name because they want to know if it'll help them stay awake. Therefore, I'm listing some drugs in a meta keywords tag. Whether a search engine applies the tag is up to the search engine's management, but I'd like the option.

2. The argument that keywords should all be in visible text and headlines is sound for attracting readers through search engines, but it can be counterproductive for retaining readers, especially through longer or multi-page content. Keyword-heavy writing style can lead to bad writing in the minds of some audiences. Better writing should be permissible without contradicting search requirements.

3. It is the only technology for its purpose. Unlike description, keywords are not meant to be visible to a searcher but are meant to influence results behind the scenes.

4. Google or some forum posters at Google say Google doesn't use them. The reasons I don't believe this are given below. More likely, Google uses them only a little and maybe only part-time, but that's good enough. They serve a purpose nothing else serves as well.

5. It works. I designed a website. When I searched in Yahoo and Google using periodless abbreviations appearing only in my meta keywords to clarify an otherwise confusing search request (the site was for someone who has the same name as someone else with kindred interests), the website came up, or came up higher, via both search engines.

6. It's been much abused. But that's not reason to prevent or discourage its proper use. If Google or Yahoo choose to give them no weight, that's their choice. But they also have the choice of analyzing them and using them selectively, for example, not using them if they're repetitive, apparently irrelevant, misformatted, or used in sites with certain themes associated with unreliable use of keywords.

7. Google inserts them into some of their own pages. Here's one:

<meta name="keywords" content="browser, browsing, web browser, internet browser, free browser, web applications, web apps, bookmark, web applications, search box, navigate the web">

It appeared in the source code for <http://www.google.com/chrome/index.html?hl=en&brand=CHMH&utm_source=en-et-abt&utm_medium=abt&utm_campaign=en>, as accessed 4-27-09. (Perhaps the keyword "web applications" appearing twice is not keyword-stuffing.)

Here's another:

<meta name="keywords" content="Google Friend Connect, Friend Connect, FriendConnect, add social features, add social gadgets">

It appeared in the source code for <http://www.google.com/friendconnect/?utm_medium=et&utm_campaign=en&utm_source=en-et-abt>, as accessed 4-27-09.

Those were not search results pages.

8. Yahoo inserts them into some of their own pages. Here's one:

<meta name="keywords" content="New York Police Department,Sen. Charles Schumer,Andrews Air Force Base,White House military office,White House,U.S. National,AP">.

It appeared in the source code for <http://news.yahoo.com/s/ap/20090428/ap_on_re_us/us_low_flying_plane;_ylt=ApaxPTErLp3DhJ9wj6jH6lSb.HQA;_ylu=X3oDMTJtdDFqZTQ3BGFzc2V0A2FwLzIwMDkwNDI4L3VzX2xvd19mbHlpbmdfcGxhbmUEY3BvcwM3BHBvcwM3BHNlYwN5bl90b3Bfc3RvcmllcwRzbGsDamV0Zmx5b3Zlcmlu>, as accessed 4-27-09.

Here's another:

<meta name="keywords" content="401k,Business,Financial Information,Investing,Investor,Market News,Stock Research,Stock Valuation,business news,economy,finance,investment tools,mortgage,mutual funds,personal finance,quote,real estate,retirement,stock,stocks,suzeorman,tax,track portfolio">

It appeared in the source code for the home page at <http://finance.yahoo.com/>, as accessed 4-27-09.

Those were not search results pages.

9. Here's one from an MSN page, not search results, and I understand MSN is the third leading search engine:

<meta name="keywords" content="IE, IE8, Internet Explorer, Internet Explorer 8, browser, web browser,  Microsoft browser, MSN, MSN Internet Explorer." />

It appeared in the source code for <http://ie8.msn.com/microsoft/internet-explorer-8/en-us/ie8.aspx?ocid=B037MSN55C0403A>, as accessed 4-27-09.

10. Here's one from a page at the Ask search engine site:

<meta name="keywords" content="nascar, race schedule, nascar drivers, race results"/>

It appeared in the source code for <http://www.ask.com/nascar>, as accessed 4-27-09. That's not a search results page.

11. If Google thinks meta keywords are a bad idea, it shouldn't mind competitors using them.

12. If Google only sometimes thinks meta keywords are a bad idea, Google would want them available for when they think they're a good idea, which means they'll want websites populating the tags in anticipation of Google's reliance on them.

13. Yahoo recommends using meta keywords: "Use a 'keyword' meta-tag to list key words for the document. Use a distinct list of keywords that relate to the specific page on your site instead of using one broad set of keywords for every page." <http://help.yahoo.com/l/us/yahoo/search/ranking/ranking-02.html;_ylt=Aqo20RnglJfFGUGT0ou1UxObqCN4>, as accessed 4-28-09.

14. Microsoft recommends using meta keywords: "Use a descriptive title tag, meta keywords, meta description, and H1 tag on all pages to identify its intended content, even if the content on the page itself isn't immediately accessible." <http://blogs.msdn.com/webmaster/archive/2009/02/03/optimizing-your-very-large-site-for-search-part-3.aspx>, as accessed 4-27-09.

15. SEO promoters say to add meta keywords. SearchEngineWatch in 2007 said so even though their use was limited: <http://searchenginewatch.com/2167931>, as accessed 4-28-09.

16. SearchEngineLand said in 2007 that Yahoo and Ask used them, and described a test showing it. <http://searchengineland.com/meta-keywords-tag-101-how-to-legally-hide-words-on-your-pages-for-search-engines-12099#>, as accessed 4-28-09.

17. For a test with an ambivalent result by Webmaster World, see <http://www.webmasterworld.com/google/3138562.htm>, as accessed 4-28-09.

18. Yahoo and Google apparently distribute their indexes to many nations and try to serve searchers in many nations, suggesting a likelihood that ranking criteria may vary by nation where the index is locally supported and according to the desires of local searchers. Thus, meta keyword reliance may vary by nation.

19. Other search engines around the world, such as Baidu, include some that are popular within their own nations, may compile their own collections, and may weigh factors differently. The tag should be available for them to analyze.

Conclusion: Google, Yahoo, Microsoft, and Ask all use them, if in very limited ways, and therefore implicitly support meta keywords. Some SEO firms support them and I don't know of any who oppose them when they think engines use them. Other engines may also use them. Meta keywords are the only technology that fits the need. The abuse is soluble. And the need exists.

Thank you.

-- 
Nick
Comment 1 Ian 'Hixie' Hickson 2009-06-28 10:46:36 UTC
I've marked it as proposed again.

However, I assure you that search engines really don't do much if anything with the meta keywords these days, and using it is no more effective than using a placebo.
Comment 2 Olivier Gendrin 2009-06-29 10:00:45 UTC
Use case : Twine uses the meta keyword value to pre fill the tag array in it's bookmarklet.
Comment 3 Julian Reschke 2009-08-19 09:10:36 UTC
I do not believe this is adequately addressed.

- meta/@name=keywords is in HTML 4

- search engine documentation implies it is used

So it appears that the burden of proof is on those who want it to be removed.

If we have solid research that shows that this is harmful (in that's it's almost always ignored, and usually used for spam), then the spec actually should include it and explain the problem.

Comment 4 Julian Reschke 2009-08-20 08:04:24 UTC
See <http://krijnhoetmer.nl/irc-logs/html-wg/20090819#l-288>
Comment 5 Ian 'Hixie' Hickson 2009-09-07 10:07:20 UTC
(In reply to comment #3)
> - meta/@name=keywords is in HTML 4

That is irrelevant.


> - search engine documentation implies it is used

Google doesn't use it. Some search engines do, as described in the comments above, but then search engines also use all kinds of other metadata names like terms from Dublin Core, and we're not going to put all those in HTML5 either.


> So it appears that the burden of proof is on those who want it to be removed.

HTML5 started with a clean slate. The burden of proof is _always_ on the request to add a feature.


> If we have solid research that shows that this is harmful (in that's it's
> almost always ignored, and usually used for spam), then the spec actually
> should include it and explain the problem.

I disagree; if it is harmful, or useless, or even not especially useful, then it doesn't belong in the spec.

It is in the wiki; people can use it. We need a community to maintain the wiki page, find or write specs defining the terms, reject or condone proposals, etc, but that is needed independent of the HTML5 spec.
Comment 6 Julian Reschke 2009-09-07 10:26:49 UTC
(In reply to comment #5)
> > - meta/@name=keywords is in HTML 4
> 
> That is irrelevant.

No, it's not irrelevant.
  
> > - search engine documentation implies it is used
> 
> Google doesn't use it. Some search engines do, as described in the comments
> above, but then search engines also use all kinds of other metadata names like
> terms from Dublin Core, and we're not going to put all those in HTML5 either.

Adding terms from Dublin Core is defined in DC-HTML, which uses the @profile extension point.

> > So it appears that the burden of proof is on those who want it to be removed.
> 
> HTML5 started with a clean slate. The burden of proof is _always_ on the
> request to add a feature.

We disagree on whether this constitutes "adding a feature".

> ...

I think this remains a bug, even if you close it as "WONTFIX". At some point, the WG will need to decide.
Comment 7 Nick Levinson 2009-09-12 21:24:32 UTC
>> Google doesn't use
If Google doesn't ever use meta keywords, why did testing come up positive and why did they insert them into their own pages? They use them at least sometimes. Maybe they fear encouraging their misuse so they don't talk about them, but still use them. Keywords fill a necessary function and there's no equivalent. Engines can ignore abuse-prone patterns. Tags on their own pages suggest proper use.

> Adding terms from Dublin Core
I gave up on Dublin Core, and plan to not have it in revised pages, despite the potential value of Library of Congress Subject Headings. Apparently, at least last April-May, DC was for specialized search engines and expansion into Google's space was not in DCMI's plans. It's also complicated and time-consuming, especially for non-XML authors, and LCSH is high-maintenance due to revisions. I asked DCMI to support HTML5 by reserving DC keywords, but I haven't heard back. If we want the DC system even if DCMI isn't interested, an easier solution is to incorporate the whole DC system by a single reference in the WHAT WG wiki to the DC website (DublinCore.org), saving you a lot of entry and maintenance, only reserving the DC prefix so non-DC names are not proposed with the same prefix.

Thanks.

-- 
Nick
Comment 8 Julian Reschke 2009-09-22 08:05:25 UTC
This is now in the Issue Tracker, see http://www.w3.org/html/wg/tracker/issues/79.
Comment 9 Ian 'Hixie' Hickson 2009-09-22 08:36:09 UTC
Marking WONTFIX since it's been escalated.
Comment 10 Maciej Stachowiak 2010-03-14 14:48:06 UTC
This bug predates the HTML Working Group Decision Policy.

If you are satisfied with the resolution of this bug, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document:
  http://dev.w3.org/html5/decision-policy/decision-policy.html

This bug is now being moved to VERIFIED. Please respond within two weeks. If this bug is not closed, reopened or escalated within two weeks, it may be marked as NoReply and will no longer be considered a pending comment.
Comment 11 Nick Levinson 2010-03-28 19:59:17 UTC
Evidence is that search engines, including the biggest, use meta keywords, and they are necessary and useful, so they should be supported in HTML5. I'm requesting escalation, which I think is already underway.

Suggested title: restore meta keywords, search engines use them

Suggested text:

The meta element for name="keywords", once demoted to being a failed proposal and then restored to being a proposal again, should be supported by HTML5. Search engines use it and for good reason.

1. Advice is often to integrate keywords into the visible text, especially near the beginning, and into headlines, especially in h1 and h2 elements. This is sound but has a limitation. I'm writing a page against using drugs to stay awake. I'm  telling readers that I don't know of any I'd want to recommend. Therefore, I don't want to list any. But I know some people will search by a drug name because they want to know if it'll help them stay awake. Therefore, I'm listing some drugs in a meta keywords tag. Whether a search engine applies the tag is up to the search engine's management, but I'd like the option.

2. The argument that keywords should all be in visible text and headlines is sound for attracting readers through search engines, but it can be counterproductive for retaining readers, especially through longer or multi-page content. Keyword-heavy writing style can lead to bad writing in the minds of some audiences. Better writing should be permissible without contradicting search requirements.

3. It is the only technology for its purpose. Unlike description, keywords are not meant to be visible to a searcher but are meant to influence results behind the scenes.

4. Google or some forum posters at Google say Google doesn't use them. The reasons I don't believe this are given below. More likely, Google uses them only a little and maybe only part-time, but that's good enough. They serve a purpose nothing else serves as well.

5. It works. I designed a website. When I searched in Yahoo and Google using periodless abbreviations appearing only in my meta keywords to clarify an otherwise confusing search request (the site was for someone who has the same name as someone else with kindred interests), the website came up, or came up higher, via both search engines.

6. It's been much abused. But that's not reason to prevent or discourage its proper use. If Google or Yahoo choose to give them no weight, that's their choice. But they also have the choice of analyzing them and using them selectively, for example, not using them if they're repetitive, apparently irrelevant, misformatted, or used in sites with certain themes associated with unreliable use of keywords.

7. Google inserts them into some of their own pages. Here's one:

<meta name="keywords" content="browser, browsing, web browser, internet browser, free browser, web applications, web apps, bookmark, web applications, search box, navigate the web">

It appeared in the source code for <http://www.google.com/chrome/index.html?hl=en&brand=CHMH&utm_source=en-et-abt&utm_medium=abt&utm_campaign=en>, as accessed 4-27-09. (Perhaps the keyword "web applications" appearing twice is not keyword-stuffing.)

Here's another:

<meta name="keywords" content="Google Friend Connect, Friend Connect, FriendConnect, add social features, add social gadgets">

It appeared in the source code for <http://www.google.com/friendconnect/?utm_medium=et&utm_campaign=en&utm_source=en-et-abt>, as accessed 4-27-09.

Those were not search results pages.

8. Yahoo inserts them into some of their own pages. Here's one:

<meta name="keywords" content="New York Police Department,Sen. Charles Schumer,Andrews Air Force Base,White House military office,White House,U.S. National,AP">.

It appeared in the source code for <http://news.yahoo.com/s/ap/20090428/ap_on_re_us/us_low_flying_plane;_ylt=ApaxPTErLp3DhJ9wj6jH6lSb.HQA;_ylu=X3oDMTJtdDFqZTQ3BGFzc2V0A2FwLzIwMDkwNDI4L3VzX2xvd19mbHlpbmdfcGxhbmUEY3BvcwM3BHBvcwM3BHNlYwN5bl90b3Bfc3RvcmllcwRzbGsDamV0Zmx5b3Zlcmlu>, as accessed 4-27-09.

Here's another:

<meta name="keywords" content="401k,Business,Financial Information,Investing,Investor,Market News,Stock Research,Stock Valuation,business news,economy,finance,investment tools,mortgage,mutual funds,personal finance,quote,real estate,retirement,stock,stocks,suzeorman,tax,track portfolio">

It appeared in the source code for the home page at <http://finance.yahoo.com/>, as accessed 4-27-09.

Those were not search results pages.

9. Here's one from an MSN page, not search results, and I understand MSN is the third leading search engine:

<meta name="keywords" content="IE, IE8, Internet Explorer, Internet Explorer 8, browser, web browser,  Microsoft browser, MSN, MSN Internet Explorer." />

It appeared in the source code for <http://ie8.msn.com/microsoft/internet-explorer-8/en-us/ie8.aspx?ocid=B037MSN55C0403A>, as accessed 4-27-09.

10. Here's one from a page at the Ask search engine site:

<meta name="keywords" content="nascar, race schedule, nascar drivers, race results"/>

It appeared in the source code for <http://www.ask.com/nascar>, as accessed 4-27-09. That's not a search results page.

11. If Google thinks meta keywords are a bad idea, it shouldn't mind	competitors using them.

12. If Google only sometimes thinks meta keywords are a bad idea, Google would want them available for when they think they're a good idea, which means they'll want websites populating the tags in anticipation of Google's reliance on them.

13. Yahoo recommends using meta keywords: "Use a 'keyword' meta-tag to list key words for the document. Use a distinct list of keywords that relate to the specific page on your site instead of using one broad set of keywords for every page." <http://help.yahoo.com/l/us/yahoo/search/ranking/ranking-02.html;_ylt=Aqo20RnglJfFGUGT0ou1UxObqCN4>, as accessed 4-28-09.

14. Microsoft recommends using meta keywords: "Use a descriptive title tag, meta keywords, meta description, and H1 tag on all pages to identify its intended content, even if the content on the page itself isn't immediately accessible." <http://blogs.msdn.com/webmaster/archive/2009/02/03/optimizing-your-very-large-site-for-search-part-3.aspx>, as accessed 4-27-09.

15. SEO promoters say to add meta keywords. SearchEngineWatch in 2007 said so even though their use was limited: <http://searchenginewatch.com/2167931>, as accessed 4-28-09.

16. SearchEngineLand said in 2007 that Yahoo and Ask used them, and described a test showing it. <http://searchengineland.com/meta-keywords-tag-101-how-to-legally-hide-words-on-your-pages-for-search-engines-12099#>, as accessed 4-28-09.

17. For a test with an ambivalent result by Webmaster World, see <http://www.webmasterworld.com/google/3138562.htm>, as accessed 4-28-09.

18. Yahoo and Google apparently distribute their indexes to many nations and try to serve searchers in many nations, suggesting a likelihood that ranking criteria may vary by nation where the index is locally supported and according to the desires of local searchers. Thus, meta keyword reliance may vary by nation.

19. Other search engines around the world, such as Baidu, include some that are popular within their own nations, may compile their own collections, and may weigh factors differently. The tag should be available for them to analyze.

20. According to commentator Olivier Gendrin, "Twine uses the  meta keyword value to pre fill the tag array in it's bookmarklet."

Conclusion: Google, Yahoo, Microsoft, and Ask all use them, if in very limited ways, and therefore implicitly support meta keywords. Some SEO firms support them and I don't know of any who oppose them when they think engines use them. Other engines may also use them. Meta keywords are the only technology that fits the need. The abuse is soluble. And the need exists.
Comment 12 Maciej Stachowiak 2010-03-28 21:49:15 UTC
Actually, meta keywords has already been restored, per the Working Group Decision recorded in bug 7525. Marking this one as a duplicate.

*** This bug has been marked as a duplicate of bug 7525 ***
Comment 13 Maciej Stachowiak 2010-03-28 21:49:39 UTC
Please change state to CLOSED if you agree that this is a duplicate.
Comment 14 Maciej Stachowiak 2010-03-30 03:51:04 UTC
Removing TrackerIssue keyword on the assumption that this is indeed a duplicate, so this doesn't throw off our checks for TrackerISsue bugs that are not listed on the issue status page.
Comment 15 Nick Levinson 2010-04-11 19:42:05 UTC
This is a duplicate, so I'm closing this.

I just emailed WHATWG re Web Apps. 1.0, Draft Std., Apr. 9, 2010, that in section 4.2.5.1, in the subsection "keywords", the UA should ignore spaces adjacent to a token-separating comma. It is common for authors to type a space after a comma and there's no harm in expecting UA designers to accept this as a norm. Technically, the UA should loop until no token-separating comma has an adjacent space remaining, on either side and regardless of quantity of consecutive spaces.

Thanks.