Browse by Tags
Sorry for any confusion there.
My goal was to get Google to crawl my sitemap default.aspx?feed=googlesitemap (or default.aspx?feed=sitemap both work) but inform Google to not index my rss and atom feeds default.aspx?feed=rss20 and default.aspx?feed=atom10.
When I put this line into my robots.txt file
Google did not index my ...
Search engine bots are quite thorough when indexing sites. If you're using Vine Type for a blog, as I do, you'll discover that the search engines may be indexing more than you would expect and perhaps more than you want. Google was indexing all the RSS and ATOM feeds on my site. Not just the main "all ...
Vine Type PRO users have the ability to generate search engine sitemaps. There are two types of search engine sitemaps
Google sitemaps actually came first and the industry reacted by declaring it, with Google's permission, as a de facto open industry standard. The two formats are ...