Why aren’t all my xml sitemap pages indexed in Webmaster Tools?

In the last few days I’ve encountered a surprising number of clients and even SEOs who don’t fully understand XML sitemaps, so I’m here to clear up some things.

Let’s say you half read a blog post somewhere that said “if your site doesn’t have an XML sitemap, your site will never be indexed and you will be poor, miserable and die lonely”.  So, you got your developer or SEO to make an XML sitemap for your website, or maybe you did it yourself with a free tool (because you’re cheap).  All giddy and excited, you submit your sitemap through Google Webmaster Tools and wait for the magical day for Google to crawl it.  Like Xmas morning, you creep down the stairs, log into GWT and start to cry because you see a report that looks like this:

Google Webmaster Tools Sitemap Report

“Only 262 pages indexed!” you scream.  “Why does Googlez hate me?  Imma fire my SEO and kick a baby!”

In a fevered response, you (or your SEO) goes line by line through your sitemap.xml file to make sure there are no broken links, or malformed URLs (good for you!), but you can’t find anything.  So instead, you resign yourself to being poor, miserable and dying lonely.

Well..  here’s something you may not have considered..

All URLs in a sitemap.xml file must return a 200 OK response

I find myself constantly amused by the number of XML Sitemaps I come across that have URLs that either 404 or redirect with a 301 or 302.  What’s even more amusing, is when I find URLs that have been disallowed via robots.txt.

So, to help you all understand why the URLs in your XML sitemap may not be indexing fully, I’ve made some easy-to-follow pictures!  Why?  Because I know how much you hate reading.

URLs in XML Sitemap returning 404 Not Found responses

sitemap.xml url returns 404 response

URLs in XML Sitemap returning 301 or 302 redirect responses

sitemap.xml url returns 301 or 302 response

URLs in XML Sitemap disallowed via Robots.txt

sitemap.xml url is disallowed via robots.txt

URLs in XML Sitemap returning 200 OK responses

sitemap.xml url returns 200 response

Now, before you start looking… no, this site doesn’t have an XML sitemap file.  Why?  Because they’re not necessary!  An XML sitemap is only a tool to help crawlers discover pages they might not normally find, usually because you have a crappy, unspiderable javascript menu that plays a Megadeth song every time you hover over it with your mouse, because your usability expert told you that was the future of the web.

This entry was posted in I Win At Internet. Bookmark the permalink.

100 Responses to Why aren’t all my xml sitemap pages indexed in Webmaster Tools?

  1. nairamania says:

    Good post with good illustration, thank you.

  2. Keith says:

    No problem… and as I can tell from your Nigerian IP address (197.255.175.138), XML Sitemap posts with illustrations are clearly the tonic for crippling poverty and governmental corruption! Happy to oblige!

    Sorry I removed your link, but you still haven’t responded to my email about requiring your bank account info so I can transfer some funds the government wants to seize from my family dynasty in Winnipeg…

  3. Andy follin says:

    so are you saying that if a page returns a 200 response it will definitely be indexed?

    or does the ‘*well, maybe’ imply something else?!

    • Keith says:

      A 200 response is required for Google to index a page, but it doesn’t mean it will definitely be indexed. That’s up to Google to determine if its relevant, useful, blah blah…

  4. Heather Wood says:

    That is an awesome post. I understand now what the heck is going on with page indexing. You never told me what I can do to fix it! No problem, At least I know what I should be doing to get it fixed ^_^

  5. How can you test whether your URLs get the 200 Response? Where do you find what response you get? Thanks for the info!!

    Kateryna

  6. Keith says:

    Clearly you are no l33t hax0r… however, if you are a web developer who’s actually getting work, you’re already using Firebug. You’ll want to learn how to use the “net” tab which will show you server responses for assets.

    If all else fails, use the SEO Book status code checker – http://tools.seobook.com/server-header-checker/

  7. Mark says:

    Great blog! My pain: Google Webmaster indicates everyting OK: sitemap, content, etc. Still, for over 2 months only 9 pages are indexed – I have more than 15,000. Even the first level/click urls are not indexed. Could you help? Best, Mark

    • Keith says:

      Ya sitemaps aren’t your issue. You’re hiding text (good ol’ 90s style cloaking… way to keep it old school!).

      Visit one of your artists pages and disable external CSS using the “Webdeveloper Toolbar” plugin for Firefox, then smack yourself in the face for using inline CSS to color your text white, but making the background black in your external CSS.

      Remember kids.. Google can parse inline CSS, so don’t use it unless you’re really lazy and hate the internet.. then it’s ok.

  8. Graeme says:

    Fun, Fun & More fun. This post literally pulled me from a depressed state and back to mirth & merriment.

  9. Rutger says:

    You’re funny. Thanks for this great post!

  10. Thanks for the advice! I submitted my sitemaps yesterday evening. Some have been indexed others are still pending. should i be nervous not all of them are indexed yet or does it take a couple of days?

    • Keith says:

      Unless your site is regularly crawled by Google on an hourly basis, you might as well hurry up and wait. It’s like anything with Google.. if your not famous or popular, the crawlers don’t care.

      Is your website followed around by gawking tourists or people with cameras?

  11. praveen says:

    the site has a search query in a page and it was disallowed for indexing by the robots code. now only 3 pages have been indexed, the main problem is when you search
    keywords it show inside pages and not my home page.

    • Keith says:

      Ok. Here’s my advice.. Go buy yourself a bag of chunky chocolate chip cookies and spend the evening googling “link canonical tag”, delete all wildcard disallows in your robots.txt, and then punch yourself in the face for wanting your homepage to show up for every keyword, as opposed to the internal pages that users would actually be searching for.

  12. Tony says:

    AHA! Finally I understand what’s going on with my sitemaps. Was driving me insane. Thanks for making it clear.

  13. Jeff says:

    Great post, I have a pretty basic question so sorry for boring you. with my sitemap it indexed 3/4 of my pages…I can fix that on my own, but under the “total indexed” in the health tab it says 0 indexed….sucks, whats the difference and how can i fix that?

    Thanks for your time, jeff

    • Keith says:

      Most questions bore me, particularly stupid ones… however, yours isn’t so stupid. There is a difference between the “indexed” sitemap pages and the “indexed status” under the health tab.

      The “indexed status” found under health is pointing out pages that haven’t been excluded from search results due to the meta noindex tag, duplicate content pages, or all those other factors that Google uses to keep pages out of search results. They’re not quite related to the xml sitemap. What I would do in your situation is compare your xml sitemap and/or your page inventory to the “site:” search parameter in Google. Might give you an idea of what isn’t there that should be there.

  14. Tan says:

    Now I know why half of my links doesn’t get indexed. I checked “Error’s” page and there was ~40 broken links. I repaired it, w’ll see how google responds to this. 🙂

  15. matteo says:

    hi, great post about sitemaps and great answers about common issues…
    i’m facing a problem with my sitemaps indexing: i saw it was falling down in the last month and found there was problem fetching the page with lynx… this resulted in MANY “not followed” errors…
    last week i fixed the problem, lynx can fetch pages again and “not followed” errors are decreasing, but the number of indexed pages is still going down…
    any hint?

  16. Meg says:

    I submitted the sitemap to Google Webmaster about a month ago. The sitemap shows as being processed today with 5787 urls submitted and the Indexed status pending. Why would the processing date show today and why do I have ZERO urls indexed?
    *Yes, I know that I have TONS of redirects which are not helping my situation and the site itself is archaic. It is a temporary solution until I finish with the development of the new site.

    • Keith says:

      Well, good on ya for finding a temporary solution. I’m guessing by now you’ve realized that Google webmaster tools updates REALLY slowly and I don’t actually need to answer this.

  17. Howdy! I could have sworn I’ve been to this site before but after browsing through a few of the posts I realized it’s
    new to me. Anyways, I’m definitely delighted I stumbled upon it and I’ll be book-marking it and checking back often!

    my page :: spam is my name, crap is my game

    • Keith says:

      I could have sword I’d seen this comment before, but I would’ve figured ASKIMET took care of it. I’m definitely delighted that you found a way past my filtering software!

  18. kostas says:

    Hallo Keith,
    have found your blog really useful. and really uplifting :O)
    i have two questions the following three:
    1.i come from greece where we use intonation to the words so does goole really make any difference between κωστας and κώστας?
    2.i have submitted my sitemap but not all my pages are indexed. on the other hand when i do a site:www.domain.gr search i get the pages listed as results. which info should i trust?
    3. does google make a distinction between capital and no capital letters?

    thanks for your time and good work!

    • Keith says:

      I have three answers the following two:

      1. The soldier desserted his desert in the desert, but no one will object to the object. Context and semantics beat spelling in all respects, even with search engines. I would guess that intonation is also taken into account with Google, but its all greek to me…
      2. I would trust the “site:” delimiter search over webmaster tools. Just keep in mind that just because you’re indexed, don’t mean you’re actually ranking. Those are different beasts.
      3. Your answers lie here and here

  19. sivakumar says:

    i have 125 pages website added sitemap successfully to google web master tools but not indexed all my pages only 30 pages where indexed

    i have checked my server response all my pages sERVER RESPONSE: HTTP/1.1 200 OK

    how to do getting all pages indexed

    Thank you

    • Keith says:

      Perhaps you can use some of your Hyderabad detective skills to investigate the duplicated pages linked from the footer of your homepage, Mr Mars.

  20. Sandeep says:

    Yes, well written post. I know xml sitemap is important for website faster crawling. I wonder why my sitemap is not indexed by google? How ever this is a very good tips. Thank you.

  21. Jo says:

    Keith, please please give me your thoughts and I will love you forever (if they help).

    Google has only indexed 34 of 61 submitted pages on my site, and I have no idea why. I’ve read your article, and all the replies above, and I can’t find anything that seems relevant.

    Can you see something I should be taking notice of?

    Thanks!

  22. Spanish Property says:

    Nice post! Is it possible to see what google has indexed from the sitemap? i.e. if I submit a sitemap with 500 urls but only 200 are shown as indexed in webmaster tools, how can I see which 200 have been indexed?

  23. David says:

    If i have pages that are in my sitemap but don’t have links through from the main site will they still be indexed? If so with the same ranking?

  24. Jon says:

    This is probably the funniest post about xml sitemaps i have seen, thank you keith.

    ps- use screaming frog tools and crawl html/text files to get a true picture of the actual size of your site. the “site:” command may bring back session id’s, paigination and other crap url’s- you can exclude all those in your screaming frog crawl.

    • Keith says:

      you’re right about Screaming Frog for getting site size, but software can’t tell you the number of indexed pages without getting your IP blocked.

      But ya, screaming frog. Big ups.

  25. Jonathon says:

    Hi there,

    I have 19000 images submitted to Google, only 4800 indexed. I’m using Nextgen Gallery plugin, this seems to be a common problem for users of this plugin. In spite of the number of users experiencing problems, there’s not much useful info from people about how they actually resolved the problem. Can you help at all please? Thanks.

  26. Dave M. says:

    Keith, you are wicked! thanks for the fun times dude! keepen it real!

  27. ram says:

    Hey… i got around 700+ links in my sitemap but Only 13 are indexed can you take a look and help me how to fix this?
    http://allcrapguide.com

  28. Matt Brown says:

    I’m having a hard time getting pages other than the homepage to show up in Google Search. Even the pages optimized for keywords won’t show up…just the homepage. Any suggestions? Most of the pages are indexed but seem to all have a 200 on them. even the ones that aren’t indexed.

  29. Jacques Levy says:

    I enjoyed your humor and some very good references. I tried recently to generate a sitemap using online tools for Mac OSX but the sitemap.xml files contained only the home page. Using list mode and loading the first 20 urls of my site into Screaming Frog reveals that my files are evidently in good order. So I have two questions: 1) Why do the generated sitemap.xml files not contain my whole site? 2) Why does Google Webmaster Tools report that line 9 of my html is missing the term url? Thanks for any insight you can provide.

    • Keith says:

      Screaming Frog is a web crawler just like any other bot. It won’t contain the full site if it can’t reach the URLs by following links. I think the real problem with your site what you used to build it (iWeb? isn’t that the old Apple web builder that was notoriously un-SEO friendly).

      Not sure what the “line 9” thing you’re getting from GWT, but I looked at the source of your site and it made me throw up in my mouth a little.. which is too bad because I actually like the design and the content.

  30. This is the best blog post about Xml sitemaps ever. Makes a load of sense but I’m curious to see how Google responds to a load of historic posts I’ve uploaded to my blog. Will it skip indexing because of the age of posts?

    • Keith says:

      I’d be curious to know that as well, actually. My thinking would be no, they’ll index just fine, but they likely wouldn’t be considered as “current” and be relegated to archival-style results.. depends on the content, I suppose.

  31. Akhil says:

    my site is about 2 weeks old, in the starting all my posts were indexed soon and i got regular organic traffic from google, but after 5-6 posts my urls are not indexed in webmaster tools, nor my main urls are showing in google anywhere. even if i copy the whole url and paste it in google search, it doesnt show any sign of it. It only shows the category and tags of that post url. Im worried if google banned me or im hit with any google update. I have another doubt taht any of my competitors did something wrong of my url. Please help me out….Im worried.

    • Keith says:

      I talked to Google yesterday about your site and he told me that he wanted to index all your pages but he doesn’t have enough Kleenex to deal with the mess.

  32. Mihir says:

    Hello,

    I have submitted sitemap, my pages are 1621 submitted but pages are not indexing since a week Just 23 pages are indexed by google. i have also done 301 redirects also . i am regularly updating content also. i am doing off page SEO but i don’t know that whay my pages are not indexing?

  33. allebookdownloads.communism.is.great! says:

    Hi Keith!

    I thought I was on a wrong page when I searched for sitemap errors in google. Was thinking that this was an x-men magazine website or something and hey this is actually what I needed.

    Here’s the scenario, I have changed hosting about 4 months ago and my rankings (all) sank to the bottom. When I checked my sitemap indrx at google webmasters, it only stated 6 url indexed out of 24 submitted. Now I understand that it’s all google that decides to index my submitted url or not. Is there a way to get indexed faster in bing than google other than submitted a url at bing toolbox?

    Great blog layout btw! reminds me of my primary school days!

    cheers!

    drey

    • Keith says:

      I thought I was on the wrong blog when I read your comment filled with backhanded compliments! As for your site, I’d sooner point fingers at the legally murky world of “ebooks” before worrying about site indexing.. but that’s just my primary school knowledge of digital distribution.

  34. lee says:

    hey Keith

    Graittude for an Original Site! Please can you advise me why only 10 of my 121 pages are indexed ? and any other help you may suggest to get my site to show up on google …. when I type in BodyTalk Cape Town…. which is the URL it dosent show up ….. give me a CLUE please !

    • Keith says:

      First, take a moment to digest this wisdom:

      Indexing and ranking are not the same!

      I know, I know.. pick your brains up off the floor because I JUST BLEW YOUR MIND!!! Holy crap, eh? Ice cream no longer tastes like it used to!

  35. Carmen says:

    Hi, I am wondering if we have a dynamic site where a page is categorized under several categories, will that count as duplicate content? For example, site.com/this-page, site.com/books/this-page, site.com/hard-back-books/this-page, site.com/books-to-subscribe/this-page. Thank you in advance.

    • Keith says:

      Ask yourself this question… is the content on one page the duplicate of another? Mull over this mystery and you’ll find your answer.

  36. RechargeTips says:

    Nice Article ! Well explained .. However I’m one of those pity people whose only 20-30 % URL got indexed 🙁

  37. puzzled by design says:

    This is helpful (and funny), but there are some other mysteries maybe you can address. And my js menu plays Wolves in the Throne Room, not Megadeth.

    Image Search shows many of my images indexed. But the Site Map bar graph in GWMT shows only a handful, The GWMT Site Map red bar graph _used_ to show most of them, for many years, but for the last six weeks, it shows less than 10% as indexed.

    So, images vanished from Image Search and GWMT Site Map index at the same time. The images have largely come back in Image Search over the last six weeks, but still no show in Site Map index. And the images that have come back in Image Search have a fraction of their former ranking. I’m not so worried about the poor showing in Site Maps index, since the images are in Image Search. I am concerned about the plummet in ranking that occurred at the same time. Indexing and ranking are not the same, but the de-indexing and de-ranking occurred at the roughly the same time, so it looks like there is some connection.

    No relevant errors shown in GWMT:
    Fetch and Render…check
    robots.txt….check
    Site Maps test…check
    200 a-ok all around as far as I can tell.

    I’m spinning my wheels in wordpress and GWMT forums, getting nowhere with this.

    http://bit.ly/1lNdLut
    Any advice appreciated, if you have the time.

    cheers.

  38. puzzled by design says:

    Thanks for replying, KG. Bitchin’ +1 back at you.

    I have a separate site map: /sitemap-image.xml. Both are wp plugins, both are zipped. I didn’t think about adding sitemap-image.xml.gz to robots, but lately I’ve been manually tossing my site map at google’s door like an obsessive paperboy (anyone remember newspapers?).

    I do understand that indexing and ranking are different aminals (sp). Let me tell you a lil’ story, full of horror and despair: For many years, until late August, GWMT Site Map showed about 60 images indexed. Most of these images ranked very well in Image Search for relevant terms. I inadvoitantly blocked my wp-content folder for several days in late August. I discovered my felonious ways through the GWMT Site Map red index bar graph. I changed the robots, and pages slowly began to reappear in the GWMT Site Maps index. But my former rankings were kaput.

    Then, overnight, in mid Sept, as witches convened around a cauldron boiling in a woods, the number of images in the Site Map index bar plummeted again, to 16 total, of about 170 submitted. Mind you, many images could still be found through site:mysite in Image Search. Now, another few weeks later, the number indexed in Site Map is still at 16. And most of my images are now in Image Search, but they still aren’t ranking for squat-diddly.

    I know I’m grasping at straws, and folks in the GWMT forum are stepping carefully around my battered corpse, but it seems that — even though the images are indexed in Image Search — there might be a connection between Site Maps only acknowledging 16 images indexed, and the overall skanky ranking of the images. I know indexing and ranking aren’t the same, but they both went kablooey at the same time, so is it all just a coincidence, and everything is really a-ok? Either there is a connection between the poor Site Map indexing and the poor rankings, or google is just punishing me for blocking the bot for a few days. Talk about holding a grudge,

    Somewhere, I’ve screwed the pooch….cdn, caching, robots, htaccess, I dunno. But google is treating me like a NKOTB instead of a creaky ten-year-old site with consistent if unspectacular rankings in an obscure niche. Why wouldn’t my former rankings come back after only several days of robots mischief over a month ago?

    I read that indexes are recommended for sites with thousands of pages, or many site maps. I have a measly cupla hunnerd pages, and two site maps, so I don’t know..

    Heh, “cheating.” *sigh* I feel like the more I dig into this SEO swamp, the less I know…You should take the brick wall on the road. It would rock.

    cheers.

    • Keith says:

      Yeah, I get that, its kind of a pain when things have gone mysteriously awry for no good reason. The price one pays for having their business model rely on a fickle multinational corporation.

      The real question would be the conversion rate for those image results. Traditionally, image results don’t convert for shit, were they converting well on your site?

      • puzzled by design says:

        Yes, images actually converted for me. My products are heavily visual. A lot of people looking for them don’t know what they’re called, but if they got one or two keywords close, I used to get my pictures in the top row, and a lot of click-throughs. Image Search was important for me.

        Now, over two months after my snafu, top-level images that used to be in the top row, are maybe in the fourth row or lower. Not great, but better than nothing.

        But the real problem are the second-level images in my gallery. They’re nowhere in image search. They’re indexed, but still not shown in the Image Sitemap index.

        GWMT forum folks tell me it can take over a year to ever regain rank.

        Pretty messed up for a five day self-inflicted robots goof.

        GWMT forum tells me its content, as if I’m running ads and pushing viagra. Scraper sites rank better for some of my images than I do.

        Google is effing evil.

  39. Har Smiht says:

    Hi all,

    I have read the post and check all think but still have a same problem last few months.

    My web site total page is 4000 and only 370 pages in google index. the problem start when i have change my hosting server. but working all ok after chnage server + google fatch my web site also.

    Can you please help me to find out problem. Same think in bing and yahoo also.

    Thank you!

    Har smith

  40. Maniche says:

    Hey keith,

    Can you please help as i have recently redirected my whole website from http to https last week and it show server response 200k, and if i check for sitemap indexing its not yet done.

    What could be the reason for it or is there anything else you can help me with. my site is localdiariesdotcom

    • Keith says:

      First ask yourself why you redirected everything to https and then maybe you’ll understand what happened to your indexed results.

  41. Tadas says:

    Hi Keith,
    Thank you for the interesting blog!

    We have launched a website with almost 1 million pages. Sitemaps were uploaded (two months ago) and very quickly Google indexed 147,000 pages. Now We are going nuts trying to understand why google doesn’t want to index everything.

    There are no errors,redirects or problems with robot file. Anything that we could do besides waiting and eating cookies?

    Thank you for your comment (I hiope it will include some sarcasm)

    • Keith says:

      If its a brand new site with a million pages, then yes, sit back and enjoy your cookies because unless you’ve got 1 million+ visitors a day, you’re gonna hurry up and wait.

  42. immortal says:

    Dear Admin and Other users,

    I added the sitemap to the google site. It was a success. hOWEVER, WHEN i SEE UNDER “CRAWL” THEN IN “SITEMAP” SECTION, i GET TO SEE THE FOLLOWING:

    SUBMITTED AS 33 iNDEX AS Pending

    DOES GOOGLE TAKE SOME TIME TO GET THE PAGES INDEXED. WHEN I SEE IN THE “ROBOT TESTER”, I GET TO SEE THE FOLLOWING CONTENT:

    User-agent: *
    Disallow: /feeds
    Disallow: /*/_/
    Sitemap: http://OMGITYPEEVERYTHINGINCAPS:80/system/feeds/sitemap

    PLEASE HELP ME TO GET THE PAGES INDEXED. I WONDER, AM I DOING SOMETHING WRONG?

    THANK YOU.

    • Keith says:

      Its a well known fact in the SEO and webmaster community that people who use cap locks when they type automatically get their pages kicked out of Google. The only way to fix this issue is to email Matt Cuts directly, however he won’t respond to your email unless you compose the email in Comic Sans, using all caps, and end every sentence with fourteen exclamation marks.

      Oh yes, also make sure the subject line is:
      MAATT CUTZ, WHY U NO INDEX MY SITE!!!!!!!!!!!!!! IZ DEFINIETY NOT SPAMS!!!!!!!!!!!!!!

  43. puzzled by design says:

    Keith, do you have an opinion on image sitemap format?

    I’ve been told in GWMT forums that it’s best as just raw urls, with no priority or frequency metadata.

    Meanwhile, support for the third most popular WP plugin tells me that search engines will reject sitemaps that don’t have priority and frequency metadata.

    • Keith says:

      I have no opinion of Image sitemaps, if they’re still a thing. Image ranking historically has been a magic sauce of three things:
      1. Image Alt Tags
      2. Image naming convention
      3. Contextual Text Surrounding the image

      So, if your image is named ‘stupid-ugly-dog.jpg’ with the alt tag “stupid ugly dog” surrounded by the text “This is a picture of a stupid ugly dog”, then the odds of your picture of a stupid ugly dog for the term “stupid ugly dog” will likely increase…

  44. Wilson says:

    Hi Keith,

    I’m stuck for more than a month already, I’ve submitted my website and sitemap to google webmaster tools, when i did a google search, it shows http://www.mywebsite.com/sitemap.xml instead of http://www.mywebsite.com? (not my actual domain)

    oNLY MY SITEMAP.XML GOT INDEXED, ALL OTHER SITE URL ARE NOT INDEXED. TRY SUBMITTING USING FETCH AS GOOGLE FOR INDIVIDUAL URL, BUT HAVE REDIRECTED ERROR 301

    may i know what is wrong? i’m very new as this is my first website.

  45. Surya says:

    Keith,, i had read about sitemap which dont indexed because the url have same content (duplicate internal content) with different url. and these url will not counted because it was crawled before. sorry for my bad english.

    my question is : i make some mistake when do post, and send it back to my draft. i change the url but not the content. does it will back indexed and counted in webmaster (in my sitemap list) ? or it will counted again for more few days.
    it seem my sitemap stuck, for example, it only show 54/60, i wait for 2 days. whats about canonical?

    thanks and sorry again for my english language.

    • Keith says:

      canonical url tags typically exist so that duplicate pages don’t get indexed, however I’ve been out of the SEO game for years now so I couldn’t even tell you if they’re a thing anymore. Want some real advice, forget about sitemaps and work on semantic tagging your site.. its THE FUTURE!!! (at least it was when I was pushing it back in 2007)

  46. I guess it takes time for all of your links to get indexed. A lot of my pages do have 200 OK responses, but it takes time for Google to crawl them all. This happens a lot with my portfolio items, but I’m sure they’ll be indexed once Google figures that my business exists. Thanks for the informative post!

  47. Google hates me and I don’t know why! I have submitted my sitemap multiple times and it only indexes 1 of 10. All the pages are 200 OK. I’ve looked for all the things you suggested but I must be missing something. Can you lend a hand to a fellow Canadian? If you can help, I will cheer for the Winnipeg Blue bombers and the Jets for 1 whole month!

  48. Abdullah says:

    Sitemap just got crawled and only 38 submitted out of 114..
    Btw using megadeth songs when mouse hover over some page is actully cool.. \NN/

  49. Alex says:

    Simply brilliant.
    The sarcasim truly does get you to think.

  50. Sneha says:

    Keith, should i have an image sitemap or not ?

  51. OJ says:

    @Keith Someone hails your work and you respond with an insult! What has being in Nigeria got to do with the rubbish you typed. Pathetic thinking

    • Keith says:

      Here’s the funny thing about WordPress, OJ. All comments have their IP address attached, so way I knew that spammer was from Nigeria is the same way I know that you posted from Washington DC.. or at least routed your IP through a server there. Its not, however, the same way I know you killed Nicole Brown Simpson.

  52. Gin says:

    wow! thanks a lot! Great post! very helpful

  53. Robbie says:

    Oh, if only this were all true!
    I have plenty of URL’s not being indexed, none of what you state is relevant to these URL’s. All are resolving with a 200, none are 404, none are redirecting.
    These are url’s in the sitemap submitted to Google more than 2 months ago, and they’re just not appearing and never have.

    • Keith says:

      Yeah, sitemaps aren’t magical, and they won’t help you get your shitty pages indexed if they’re shitty.

  54. aakeyy says:

    yeah, thanks for the smile on my face, well explained. like this article. 😀

  55. thegreatj says:

    Great post, thanks for your generous knowledge sharing! However, I am confused why nearly 3/4 of our pages/posts are not indexed despite all returning 200’s… To make things even more interesting, we rank great in Google for some very competitive terms and get a lot of organic traffic. Seems odd that we are clearly indexed heavily but WMT is showing that most of our pages are not indexed…

    Example page on our site linked for reference…

  56. usoro says:

    Thanks a lot. I also had the same problem but i think i now understand how indexing works

  57. Ugochi says:

    Hi Keith, I pray you find time to read this and respond. Please what is the difference between sitemap.xml and site.xml?

Leave a Reply

Your email address will not be published. Required fields are marked *