The Discourage Search Engines Checkbox After Launch

One tick in WordPress, under "Search Engine Visibility", tells search engines not to index the whole site. It helps while a site is being built and does damage when it survives into launch. If you ship sites for clients, run a shop or blog of your own, or wonder why a new WordPress site is missing from Google, the details of what the box does decide how you test before and after go-live.

What happens when you tick it

The WordPress documentation page Settings Reading screen (Opens in a new tab) describes the "Discourage search engines from indexing this site" checkbox as a request. When it is checked, the page lists these effects: since version 5.3 a meta robots tag with noindex,nofollow is generated into the head section if wp_head is used; pings to update services are stopped; the "Update Services" option on the Writing screen is hidden; and normal visitors are still allowed in.

The closing note on that page matters most. Neither of these options blocks access to your site, and it is up to search engines to honor your request. Think of the box as a polite sign, not a lock. If a staging site must stay private, protect it with access control. Interface labels are quoted in English and may differ in a localized dashboard.

Mechanism by version

Version numbers come from the documentation and can change, so consult the current page before relying on them. On a modern install the effect comes from the meta tag inside each page, not from robots.txt.

Reading the tag Google sees

Google's robots meta tag specification (Opens in a new tab) defines noindex as not showing the page, media or resource in search results, nofollow as not following the links on the page, and none as equivalent to noindex, nofollow. It also says that when rules conflict, the more restrictive one applies.

Why a friendly plugin does not rescue you

Suppose an SEO plugin outputs an index directive while the checkbox adds noindex. With conflicting rules the restrictive one wins, so the page stays out. Practical suggestion: on launch day read the page source and confirm there is one robots tag with the value you expect.

Where the discouragement does not truly block

The robots.txt and noindex trap

Google's noindex documentation (Opens in a new tab) states that for noindex to work the page must not be blocked by robots.txt. If it is blocked, the crawler never sees the rule, and the page can still appear in results when other pages link to it. The robots meta tag page repeats the idea: these rules are discovered when a URL is crawled, and a robots.txt block means they are ignored.

Practical suggestion: look for a leftover "Disallow: /" line from a staging setup. With both layers closed, clearing the checkbox is not enough, because Google still cannot crawl the pages to learn that noindex is gone.

Hiding only part of a site

The checkbox is site-wide. Anyone who wants to hide a section needs page-level control, through the per-page field of an SEO plugin or the wp_robots filter. That decision lies outside this topic; what matters here is that if both the checkbox and a page-level rule apply to one page, the more restrictive rule decides.

A launch-day gate

The checkbox is forgotten because it was set on purpose months earlier and nobody lists it at go-live. Handling of whole staging environments, including password protection, is covered in fixing a staging site indexed by Google; the launch-day gate table concentrates on the WordPress checkbox.

The same Google documentation says an X-Robots-Tag header with noindex or none works for non-HTML files such as PDFs. The WordPress checkbox produces a meta tag, so headers are the job of the server or a plugin. For the difference between the two methods read the noindex tag and X-Robots-Tag guide.

When clearing the box is not enough

• An SEO plugin applies its own site-wide noindex.
• A caching layer still serves the old output.
• The server or CDN adds an X-Robots-Tag header.
• A robots.txt block survived from staging.
• Individual pages were set to noindex one by one.

These are possibilities drawn from site work, not statements in the documentation. Fixing one does not prove the others are absent, so run all six checks.

Recrawl expectations after you clear it

The noindex documentation says Googlebot has to recrawl a page before it sees the rule, and that revisiting a page can take months depending on the page's importance. It adds that you can ask for a recrawl to speed things up. Clearing the checkbox therefore restores nothing instantly; the engine must come back and see that noindex is gone.

Pacing the recrawl work

Practical suggestion: request recrawls for the home page, a few category pages and a few key articles, keep the sitemap current, and let internal links carry the rest. Requesting every URL one by one adds effort without changing the underlying wait. The steps for a refreshed page are described in recrawling an updated page with URL Inspection and sitemaps. Tool names change, so go by the labels on your screen.

Example scenario: a new honey producer

A honey producer builds its shop on a temporary address, and the developer ticks the box. On launch day the address changes and nobody opens the Reading screen. A month later the owner asks why the brand name finds nothing.

1. The developer opens Settings > Reading and finds the box still ticked.
2. The box is cleared and the cache purged.
3. Searching the home page source for "robots" shows the noindex tag is gone.
4. robots.txt is checked for a blocking line; none exists.
5. URL Inspection is used to request a recrawl of the home page.
6. The sitemap is confirmed current, and the Page Indexing report is watched for the following weeks.

The key point is expectation management. Because the documentation says a revisit can take months, the owner is told progress will be tracked, not that results arrive overnight.

Reading the page source

View the source of a page in a private window and search for "robots". The table maps what you find to what it means.

Open the source while logged out. An administrator session can show output that differs from what visitors and crawlers receive, and a caching plugin can serve stale output until it is purged.

Comparing your view with Google's

Google's noindex documentation says the URL Inspection tool can show the HTML Googlebot received while crawling, which helps test whether your noindex implementation works. When your browser source and Google's copy differ, the gap can sit in a cache, a CDN or a bot-handling rule. Practical suggestion: put the two robots tags side by side before you change anything.

Who owns the box in each phase

This table is a practical suggestion. A short handoff note can record the checkbox state on launch day, who checked it, the date, the SEO plugin in use and its site-wide index setting, where robots.txt lives, and who controls server headers. Months later, nobody has to argue about who ticked what.

Reporting to the client

Practical suggestion: write the finding, the cause, the change made, the evidence and the expectation on separate lines. For the expectation line, state that Google's documentation says a revisit can take months; that is an honest way to avoid promising results. If the client has no Search Console access, attach a short excerpt of the page source as evidence instead of a screenshot of an unrelated tool.

What to write down after the check

A short written record protects you and the client. It should carry these fields.

Update the record once in the first week after launch and once at the end of the first month. If nothing moves, assume a layer is still closed rather than that Google is slow, and run the launch gate again from the top.

Example scenario: a small architecture studio

A studio has an agency build its portfolio site. The agency ticks the box during development and the site goes live after client approval. Three months later the owner searches the studio name and finds only social profiles.

1. The agency's maintenance contact opens the Reading screen first. The box is cleared, so the cause lies elsewhere.
2. Searching the home page source for "robots" turns up a noindex tag. It traces back to an SEO plugin option that hides the whole site from results, left on after launch.
3. The option is switched off, the cache purged, and the source checked again.
4. The contact confirms robots.txt has no blocking rule and the server sends no X-Robots-Tag header.
5. URL Inspection shows no noindex in the HTML Google received, and a recrawl is requested.
6. The studio is told that the documentation gives no recovery timeline and that progress will be reported from the Page Indexing report.

The case shows the checkbox is only one possible source. Checking it is necessary but not sufficient; the source inspection proves which layer produces the noindex.

Side effects beyond the meta tag

The checkbox does more than print a tag. The WordPress Settings Reading page lists three further effects. It stops pings to Ping-O-Matic and any other RPC ping services configured under Update Services. It does this through a filter named privacy_ping_filter, which strips the ping sites from the list whenever the generic ping function asks for them. And it hides the "Update Services" option on the Writing screen, replacing it with the message "WordPress is not notifying any Update Services because of your blog's privacy settings."

That gives you a second, independent signal to check at launch. If the Writing screen still shows the privacy message instead of the Update Services field, the box is still ticked, whatever the cache or a plugin may be telling you about the front end.

The documentation does not describe a per-page or per-post-type version of this option. The top half of the same screen, where you pick a static front page or latest posts, is a different decision. Someone who says "I checked Reading settings and all is fine" may have looked only at that top half.

The physical robots.txt file in older versions

For 5.2 and earlier, the documentation adds a note: the robots.txt response works only when WordPress is installed in the site root and no robots.txt exists. Two risks follow from that sentence. If a physical robots.txt sits in the root, WordPress's virtual answer never runs, so the file wins even with the box ticked. And a WordPress copy installed in a subfolder cannot speak for the root file at all. From 5.3 the mechanism moved into the page, so those two limits no longer apply to the tag. A leftover file on your server can still cause trouble by itself.

Other robots directives and where the box stops

Google's robots meta tag page lists directives the box never writes. Knowing them helps you decide when a page-level rule is the better tool.

The page also says the name and content attributes are case-insensitive, so NOINDEX and noindex mean the same thing and your source search should not be case-sensitive. It adds that "googlebot" targets text results and "googlebot-news" targets news results. A theme or plugin that writes a noindex only for googlebot is a separate source from the generic tag the box produces. When you search the source, look for the word googlebot as well as robots.

One tag or several

Google says you can use multiple robots meta tags to address different crawlers individually. Two tags in the source are therefore not automatically a fault. The fault is two conflicting rules for the same crawler, and the page says the more restrictive one applies. Practical suggestion: when you find two tags, first read which crawler names they address, then find out which plugin or theme file wrote each one.

From symptom to layer

This table runs from what you see on the site to the layer you should inspect first. It is a working order, not a statement from either document.

Telling a header noindex from a tag noindex

Google's noindex page says an X-Robots-Tag header can return noindex or none. Because the WordPress box writes a meta tag, a header-based noindex never shows in the page source. You catch it only in the response headers. Open the network panel in your browser's developer tools, select the document request and look for an x-robots-tag line. No line means the header layer is clean. A line with noindex means someone must find the server rule, CDN setting or plugin that sends it before you sign off on the launch.

Example scenario: an online bookshop with a catalog import

Example scenario: a small online bookshop migrates its catalog to a new WordPress build. The developer ticked the box on the build server. On launch day the box was cleared, yet the book pages stay out of results while the blog posts appear.

1. The maintainer opens the Reading screen and sees the box cleared. The site-wide switch is not the cause.
2. She searches the source of a book page for robots and finds a noindex tag. A blog post has no such tag.
3. The difference comes from the catalog plugin, which marks products as hidden until a "published" flag is set during the import. The imported products still carry the hidden flag.
4. She checks response headers for X-Robots-Tag and finds none.
5. She fixes the flag in bulk, empties the cache and requests a recrawl for one product, one category and the catalog landing page.
6. The report states that the box was clean, that the cause was a per-product rule, and that Google's documentation gives no guaranteed recrawl time.

The lesson is narrow. A cleared box proves one switch is off. It does not prove that every page is open, because page-level rules sit beside the switch and the stricter rule wins when they disagree.

Pre-launch and post-launch checklist

• Is the "Search Engine Visibility" box cleared on the live site?
• Do the home, category and post sources contain no noindex robots tag?
• Does robots.txt avoid a rule that blocks the whole site?
• Is any X-Robots-Tag header absent from server and CDN responses?
• Is the SEO plugin's site-wide noindex option off?
• Was the cache purged?
• Does the HTML Google received show no noindex?
• Is the sitemap current and limited to indexable URLs?
• Was a recrawl requested, and is the Page Indexing report being watched?

Decision rule: when the box should stay ticked

Practical suggestion: let the publishing state decide.

1. While a site sits on a public address during development, keep the box ticked, and add access control if privacy is required.
2. On the day the site goes live, clear the box and run the six-step gate.
3. If you find the box ticked on a live site, clear it first, check robots.txt and headers second, request a recrawl third.
4. Track progress weekly; the documentation promises no timeline.

Put the checkbox at the top of your launch list. Looking at the Reading screen costs half a minute, while forgetting it can cost months of invisibility.

Frequently Asked Questions

Does ticking the box hide my site?

No. The WordPress documentation says these options do not block access, and honoring the request is up to the search engines. Use access control if you need privacy.

Which tag does the box add?

Since version 5.3, a meta robots tag with noindex,nofollow in the head section when the theme uses wp_head.

I cleared the box. Why am I still missing from Google?

Google's documentation says the page must be recrawled to see the change, which can take months. A robots.txt block or another layer adding noindex can also be the cause, so inspect the HTML Google received.

Is it safer to block the site in robots.txt as well?

No. Per Google, a page blocked by robots.txt never has its noindex rule seen, and it can still appear in results if other pages link to it.

My SEO plugin says index. Does the box still matter?

Yes. Google says that when robots rules conflict the more restrictive one applies, so a noindex tag keeps the page out.

Can I use the box to hide a single page?

No, it is a site-wide setting. A single page needs a page-level method such as the per-page field in an SEO plugin or the wp_robots filter.

Conclusion

The box is a request, not a barrier, and it travels easily from development into launch. On go-live day check the checkbox, the page source, robots.txt and the response headers, then ask for a recrawl and track progress. The documentation sets no timeline for the return to results.

When your new site is ready to publish, our content writing service can supply the articles it needs.