The X-Robots-Tag response header defines how crawlers should index URLs.
While not part of any specification, it is a de-facto standard method for communicating with search bots, web crawlers, and similar user agents.
Search-related crawlers use the rules from the X-Robots-Tag header to adjust how to present web Pages or other resources in search results....to adjust how to present web pages or other resources in search...listed. noindex Do not show this page, media, or resource in search...