What are Meta Robot tags?
What are Meta Robot tags and how can I use them?
Meta robot tags are tags that you can use to tell robots(search engine crawlers) to, or not to, index the content of a page, and to follow or not follow any links that are on the page. It can easily be set on a page by page, post per post basis, especially with the use of Yoast’s WordPress SEO plugin.
It’s important to note that robots *can* ignore these meta tags though, so they are not foolproof.
There are many settings available, the first is for index, the second is if links should be followed. Here’s a look at how you configure these in the event that you should choose to do it manually.
<META NAME="ROBOTS" CONTENT="NOINDEX, FOLLOW"> <META NAME="ROBOTS" CONTENT="INDEX, NOFOLLOW"> <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
It’s quite painful to set these on a per page basis manually, we recommend using Yoast’s WordPress SEO Plugin to manage this