Technical SEO Fundamentals: How to Audit Your Site From Scratch

Run any site through a crawler and it will hand you four thousand problems. Missing alt text. Meta descriptions too long. H1s in the wrong order. Redirect chains. It will colour them red and amber, and you will feel a sinking sensation, and then you will fix the easy ones, because the tool gave you no way to tell which of the four thousand actually matters.

That's the flaw in every technical SEO checklist ever published. A list is not a diagnosis. So instead of another list, here's the principle that lets you triage all four thousand in about ten minutes.

Technical SEO is a gate, not a dial

Content and links determine how well you rank. Technical SEO determines whether you're eligible to rank at all. It is not a contribution to your score. It's a series of doors, and a closed door isn't a smaller number, it's a zero.

This matters enormously for prioritisation, because it means technical problems are mostly binary and catastrophic, not gradual. A page carrying an accidental noindex isn't ranking slightly worse. It isn't ranking. Meanwhile, a meta description that's eleven characters too long costs you nothing whatsoever.

The triage question Does this stop a page that could earn money from being found, crawled, rendered, indexed, or understood? If yes, it's urgent. If no, it's housekeeping. Most of what your crawler flagged in red is housekeeping.

Audit in the order Google actually works

Here's the sequence, and it's the whole skill. Google must discover a page, crawl it, render it, index it, and understand it — in that order. A failure at any stage makes every later stage irrelevant.

Most audits begin at stage five, tinkering with title tags on pages that fail at stage two. It's the SEO equivalent of repainting a house with no foundations. Never optimise a page that isn't indexed.

Stage 1 — Discover

A page nobody links to and no sitemap mentions is, to a search engine, a page that doesn't exist. Google's guidance is that sitemaps don't restrict crawling but do help it prioritise, which makes them a recommendation letter rather than a permit.

What to check: is your sitemap current, does it list only canonical, indexable URLs (not redirects, not 404s, not noindexed pages), and is it declared in robots.txt and submitted in Search Console? Then hunt orphan pages — anything with no internal links pointing at it. If a page matters, link to it from somewhere that matters.

Stage 2 — Crawl, and the trap that undoes most sites

Now the single most damaging misunderstanding in this entire discipline, and Google states the correction plainly in its own documentation: do not use robots.txt to prevent indexing.

robots.txt controls crawling. noindex controls indexing. They are different things, and here is the part that catches almost everyone:

Beyond that: use robots.txt for what it's for. Google recommends blocking duplicate content, trivial resources that hammer your server, and URLs that change state — carts, comment posts, account creation. Those should never be crawled.

Stage 3 — Render

A page can be crawled and still be invisible, because Googlebot renders it and gets something different from what you see.

The classic cause is blocked resources. If your CSS or JavaScript is disallowed, Google may not be able to render the page as a user experiences it — and Google notes these blocks show up only at the individual URL level, in the URL Inspection tool, which is why site-wide crawls miss them.

So do this, today: take an important page, open URL Inspection in Search Console, and use Test Live URL to render it. Compare what Google receives with what you expect. If your content, prices, or reviews are missing from that render, nothing further down this article will save you. It's the same failure mode we set out in designing websites for AI search — content that doesn't render is content that doesn't count, and that's now true for AI assistants as well as crawlers.

Stage 4 — Index

The Page Indexing report in Search Console is the most useful screen in SEO, and the least read. It tells you, bluntly, which pages Google chose not to keep and why.

The recurring culprits:

• Accidental noindex. Usually a staging setting that shipped to production. Check first, always. It's the highest-damage, lowest-effort bug in the business.
• Canonical confusion. Filters, sort orders, tracking parameters and product variants each spawn near-identical URLs. Pick the version you want ranked and point the rest at it. Get this wrong on an e-commerce site and you can dilute an entire category, which is why we cover it in the product page optimisation guide.
• Soft 404s. A missing page returning a friendly "sorry, not found" message with a 200 OK status. Google is explicit: return a true 404. Lying to a crawler about a page's existence wastes everyone's time.
• Redirect discipline. 301 for permanent, 302 for temporary. Using 302 for a permanent move tells Google to keep the old URL, which is precisely what you didn't want.
• "Crawled, currently not indexed." The one nobody wants to hear: Google found it, looked at it, and decided it wasn't worth keeping. That's not a technical fault. That's a content verdict.

Stage 5 — Understand

Only now do the familiar items matter, and they matter far less dramatically than the checklists imply.

Google's advice here is refreshingly unglamorous: put key information in text, not graphics. Text remains the safest way for a machine to understand a page. Add structured data where it earns you something (products, FAQs, breadcrumbs, articles), not as a ritual. Use HTTPS. Make sure the mobile version is complete, since Google's default crawler is now a mobile one — if your mobile site hides content the desktop shows, the hidden version is the one that counts.

Speed belongs here too, but keep it in proportion: a fast page beats a slow one, and Core Web Vitals are worth fixing where they're bad. Chasing a perfect score is a hobby, not a strategy. The rest of the on-page work is covered in our on-page SEO checklist.

Three myths that eat audit budgets

Myth 1: "We need to fix our crawl budget"

Almost certainly you don't. Google frames crawl budget as a concern for sites with hundreds of millions of pages, or tens of millions changing frequently. If you have four hundred pages and someone is selling you crawl-budget optimisation, they are selling you a solution to a problem you do not have.

Myth 2: "The crawler found 4,000 errors, so we have 4,000 problems"

Audit tools flag deviations from best practice, not losses of revenue. They cannot tell which pages you care about. Ten minutes spent sorting the list by "does this page make money" will save you three months.

Myth 3: "Technical SEO will fix our traffic"

If your pages are indexed and ranking and traffic is still falling, the problem isn't technical, and no amount of schema will rescue it. Diagnose properly before you spend, as we argue in auditing your funnel end to end. It may well be that the traffic you lost was never worth much — the case we make in what AI Overviews actually took from you.

The free audit that beats most paid ones

You can do the high-value 80% of a technical audit this afternoon, with nothing but free tools.

If those six checks come back clean, your technical foundation is sound and your problem lies in content, links, or the market — which is a more expensive problem, but at least it's the right one. Google's own Search Central documentation is the authority for all of this, and it's free. And for AI-era retrievability specifically, our generative engine optimisation playbook picks up where this leaves off. If the fixes turn out to be structural, that's what SEO support is for.

What matters here

Technical SEO isn't a score to maximise, it's a set of doors to keep open — and once they're open, opening them further achieves nothing. So stop treating your crawler's four thousand red flags as a to-do list, and start asking the only question that ranks them: is this stopping a page that could make money from being found, fetched, rendered, stored, or understood? Work in Google's order, not the tool's. Check noindex before you touch a title tag. Learn the robots.txt trap properly, because half the industry is still using it to do the exact opposite of what it intends. And when the six free checks come back clean, believe them, and go and spend the budget on something that will actually move the number.