Onsite maintenance completed this morning with additional testing performed on the main index site, http://www.resumeadvantagepro.com/ We use Xenu Link checker to look for any onsite or external broken links or invalid URL issues. In running the report we do always find that there is an odd issue with a forbidden request 403 Error, specifically generated from the comment block. Not an issue but makes the report look a bit odd but nothing unusual for us. Per this reporting, no orphan files were reported or internal page errors. Currently we’re showing 417 total internal URL's by MIME type.
Additionally, we're updating the uploaded Google Webmaster sitemap. There is always the question of how many times or how often should you update a sitemap. In our case, we often generate a new .xml site map once or twice a month depending on the level of new content and blog activity we have. This round we submitted 108 pages to be indexed, up 13 sense our last sitemap update.
If you use the 'Crawl stats' report in Google Web Master Tools, you could also use that to determine how often to update a sitemap. For example the site www.interfacebus.com gets over 700 of it's pages crawled per day [average]. As the site only has 2,000 pages, the entire site gets crawled every month. Keep in mind that a bot may crawl a page more than once.
ReplyDeleteAnother bit of advice is to change the name of your site map every few up-loads. Some of these auto-generated sites that just grab data, might just find your sitemap and down load it, or link to it. Those computer generated pages do you no good, and just use server bandwidth reading your sitemap.
If your site-map does get linked to, it may start showing up in Google searches, which you may not want. The 'real' page is a better search return than a sitemap with hundreds of links.
Of course always add your sitemap to your robots.txt file so all the search engines can find it.