This guide to using noindex, nofollow, and disallow will come in handy if your web pages need to be invisible to search engines, indexing robots, and web page crawlers.

Noindex or Nofollow Noindex and nofollow are both meta tags that can be applied in the page’s HTML code, or by returning a noindex header in the HTTP request. The noindex attribute indicates which pages shouldn’t be indexed and the nofollow attribute indicates which links shouldn’t be followed. This guide to using noindex, nofollow, and disallow will come in handy if your web pages need to be invisible to search engines, indexing robots, and web page crawlers. Using noindex is useful if you don't have root access to your server, as it allows you to control access to your site on a page-by-page basis. Implementing noindex There are two ways to implement noindex: as a meta tag and as an HTTP response header. The NoFollow tag was created as a way to disclose to search engines that although your mom blogging site is adding a link to a certain destination, it is not endorsing the destination site. Initially used for blog comments to combat comment spam, it protected blogs from inadvertently linking to so-called bad neighborhoods on the […]

NoIndex and NoFollow may sound similar but they are quite different. NoIndex is a meta tag that is added into the header code of a web page to tell search 

You can use a special HTML tag to tell robots not to index the content of a page, and/or not scan it for links to follow. For example: