A useful service or product page has to be reachable before it can help someone through search. The technical checks are about giving the right pages a clear path and avoiding conflicting instructions. Passing them does not guarantee that a page will be indexed or rank.
- Discover
- A search engine learns that an address exists.
- Crawl
- It requests the content at that address.
- Index
- It decides whether to include the page in its index.
Completing an earlier stage does not guarantee the next. Indexing is also different from receiving clicks or enquiries.
Discovery, crawling and indexing are separate steps.
A search engine may discover a URL through a link or a sitemap, fetch its content, and decide whether to include it in an index. A discovered URL is not automatically an indexed page. An indexed page is not automatically receiving useful visits.
That distinction matters in a report. A sitemap count, an inspection status and a search-click total answer different questions. Label them clearly so the owner can see what has actually been checked.
Check the instructions the website is sending.
A robots.txt file controls crawler access. It is not a privacy barrier or a reliable way to remove a page from search. A noindex instruction concerns indexing, and a crawler needs access to read it. Private information needs an appropriate access control.
Sitemaps help point search engines towards intended URLs. Internal links help both people and crawlers discover the site. A technical audit should check these together instead of treating a submitted sitemap as proof that every page is available.
Reading behind this section: Google Search Central
Give each intended page a clear address.
Duplicate or near-duplicate URLs can arise through filters, tracking parameters or old structures. A canonical link indicates a preferred version; it is not the same as moving a person to another page. A redirect has a different job when an address has changed.
Check the response, the destination, the internal links and the intended canonical page as a group. The useful question is whether a customer can reach the right information, including from an older saved link. Implementation needs to suit the actual platform.
Prioritise the pages that matter to the business.
Start with important service pages, buying information and the enquiry path. A crawler can list many resources, including images and scripts; that is not the same as a count of public content pages. Define the page inventory before reporting on it.
Aleyda Solis's audit roadmap is useful further reading for understanding the areas an investigation can cover. The priority on a real project comes from the business task, the source evidence and the effect of the issue, not the size of an exported list.
Reading behind this section: Aleyda Solis / LearningSEO.io
Put it to use
A useful check for your business.
- Name the intended public pages before counting them.
- Distinguish discovered, indexed and visited in reports.
- Test important old links and their destinations after a move.
The thinking behind this guide
Sources & further reading.
Our practical interpretation, with the references below for the specific principles and further detail.
- Introduction to robots.txt
Crawler access, noindex and the distinction from private access control.
- SEO Audits and Recommendations
A practitioner-curated audit learning path, used as further reading rather than proof that any specific issue affects rankings.
