I’ve been having more problems with the Microsoft “Bing” search engine that DuckDuckGo uses. DuckDuckGo should use something other than Bing. Why don’t they just perform that functionality themselves?
I don’t want the content on the homepage of NOITV to be indexed by the search engines because the search results become bloated with links to the homepage.
However, when I add the tag <meta name="robots" content="noindex"> to the homepage it entirely shuts down Bing’s search results for the site, despite it being suggested in their docs to use that to exclude the homepage from the results. I also tried “noindex, follow” and “noindex, nofollow” but that didn’t help either. And I’m sure the robots.txt file is configured properly, etc..
Using the meta tag works fine for instructing Google to disallow the homepages from its searches, however.
The homepage URL is not even specified in the sitemaps to be indexed, yet it shows up in the results anyway on Google and Bing unless I shut it off with the meta tag.
So my options seem to be to either have “clean”-looking search results in Google only, or to have bloated-looking results in both. So I guess I’ll just go with the bloated ones for the time being until I can figure out what the problem is.
It is also possible that the site is being discriminated against because of its content, although other sites don’t seem to be blocked that way. Perhaps they don’t like that I communicate its information in such a palatable way.