To prevent unwanted content within the search indexes, website owners can instruct spiders to not crawl sure documents or directories in the regular robots.txt file in the foundation directory from the area. Additionally, a web page may be explicitly excluded from the search engine's database by making use of a meta tag unique to robots (usually ).