What you can check yourself
- Under Settings → Reading, ensure Discourage search engines from indexing this site is unchecked on production.
- Open representative pages while logged out. Confirm they return the actual page rather than maintenance mode, a security challenge or a login.
- In the active SEO plugin, check important pages and page types for unintended
noindexsettings. - Inspect the final live
/robots.txt. A physical file, SEO plugin, security plugin, host or CDN may alter it. - Check
/wp-sitemap.xml,/sitemap.xmlor the sitemap endpoint declared by the active SEO plugin. Confirm important pages are included. - Confirm essential information appears in readable page content rather than only in a slider, pop-up, image or third-party booking widget.
- Ask the host or security provider to review
403,429, CAPTCHA and bot-management behaviour if desired crawlers cannot fetch the site. - Clear relevant caches, publish the change and rerun the assessment.
What healthy looks like
- Production search discouragement is off.
- One clearly identified system owns page indexing, sitemap and crawler rules.
- Desired crawlers are not blocked by the final
robots.txt, host, CDN or security plugin. - Important pages are public, readable and listed in the authoritative sitemap.
Making a change
Ask your provider which plugin or hosting setting controls the rule, and make the change there. Keep one system responsible for each setting to avoid conflicting rules. Back up the configuration before editing it.
Message for the WordPress provider
Our crawler-access assessment found [EXACT RULE, NOINDEX TAG, STATUS OR CHALLENGE] on [PAGE URL] for [CRAWLER NAME]. We want this public page available for current search and AI-assisted retrieval. Please identify whether WordPress core, the active SEO/security plugin, the host, CDN or firewall owns the behaviour; make the smallest supported change; clear caches; and confirm the final live response.