Technical SEO for Non-Developers: The Essentials

Technical SEO is the set of settings that lets Google access, load and understand your site's pages: robots.txt, sitemap, canonical, noindex, HTTPS, the mobile version and JavaScript rendering. You do not need to code to master the essentials. You need to know what to check, which mistakes are costly and what to ask your developer for, using the right words.

This guide is part of the complete SEO guide and builds on what you saw in how Google crawls, indexes and ranks: if one of the stages fails, the best content in the world does not appear.

Quick summary

• Technical SEO takes care of eligibility: the page must be crawlable, indexable and displayed correctly.
• robots.txt controls what Googlebot may crawl, but it is not a way to hide pages from Search. That is what noindex is for.
• The sitemap helps Google discover URLs, with no guarantee they will be crawled or indexed.
• The canonical indicates which version of a repeated page you prefer. It is a signal, not an order.
• Google uses the mobile version of the content to index and rank, and prefers HTTPS.
• The costliest technical mistake is usually a simple one: a block forgotten after a launch.

How to read this guide: if you are new to SEO, read the summary table, the hypothetical example and the list of what to ask the developer. If you already work in SEO, go to the "For SEO professionals" blocks and the interactions between robots.txt, noindex and canonical.

What is technical SEO and why does it come first?

Technical SEO is the part of SEO that guarantees the "infrastructure" for everything else to work. It comes first because technical failures cancel out content work: a page that is blocked, has noindex or does not load for the bot simply does not enter the index.

In plain English: think of a store. Technical SEO is the unlocked door (robots.txt), the updated mall directory (sitemap), the sign that says "the official store is this one, not the similar branch" (canonical), the "closed to the public" notice (noindex) and the floor that holds up the weight of everyone who walks in (server, HTTPS, speed). The customer does not even notice when everything is right, but gives up at the first locked door.

The minimum technical requirements are part of Google Search Essentials, and the documentation itself reminds you that meeting them does not guarantee that Google will crawl, index or show the page.

What does each item do? An overview

The table summarizes the essentials, how to check without coding and what to ask the developer.

Item	What it is for	How to check without coding	What to ask the dev
robots.txt	Tells Googlebot where it should not crawl	Open yourdomain.com/robots.txt in the browser	"Confirm the production robots.txt does not block important pages"
Sitemap	Lists URLs you want Google to know about	Open /sitemap.xml and check the Sitemaps report in Search Console	"Generate the sitemap with only 200, indexable URLs, and update it when we publish"
Canonical	Indicates the preferred version of repeated pages	URL Inspection: compare the declared and the selected canonical	"Every page should have an absolute canonical, pointing to itself or to the preferred version"
Noindex	Keeps the page from appearing in Search	URL Inspection: "Indexing allowed?"	"Remove the noindex from these URLs" (or apply it to those that should leave)
HTTPS	Secure connection	Padlock in the browser, no warnings	"Redirect all HTTP to HTTPS with a 301"
Mobile	Google indexes the mobile version	Open the page on your phone and compare the content	"The content and tags of the mobile version must match the desktop"
Rendering	Ensures Google sees the content	"Test live URL" in URL Inspection	"The main text and links must be in the HTML delivered by the server"

What is robots.txt for, and what does it not do?

robots.txt is a file at the root of the site that tells crawlers which URLs they may access. It is used to control bot traffic on the server. Google's documentation is direct: it is not a mechanism for keeping a page out of Google.

The reason: a URL blocked in robots.txt can still be indexed, just without a description, if other sites link to it. To take a page out of the results, use noindex or password protection.

A simple example:

User-agent: *
Disallow: /search/
Disallow: /cart/

Sitemap: https://www.example.com/sitemap.xml

Reading: all bots may crawl the site, except /search/ and /cart/. Beware of the line that takes everything down: Disallow: / blocks the entire site.

For SEO professionals: according to Google's specifications, the details that catch people out: the file must be at the root of the host (each subdomain has its own), in UTF-8, with case-sensitive rules; the limit is 500 KiB, and anything beyond that is ignored; Google caches the file for up to 24 hours. If the response is 4xx (except 429), Google assumes there are no restrictions. If it is 5xx, Googlebot stops crawling for up to 12 hours, uses the last cached version for up to 30 days and, after that, acts as if there were no robots.txt (on generally available sites). See more in robots.txt and AI crawlers.

What is a sitemap and when do I need one?

A sitemap is a file that lists the pages you want search engines to know about. It helps discovery, but it does not guarantee that the listed URLs will be crawled or indexed. Small, well-linked sites without rich media can work without one, and platforms like WordPress usually generate one automatically.

According to the documentation on building sitemaps, a sitemap is limited to 50 MB (uncompressed) or 50,000 URLs, and beyond that you need to split it into several files with an index. Google accepts XML, RSS/Atom and plain text, requires absolute URLs in UTF-8 and ignores the <priority> and <changefreq> tags. The <lastmod> tag is only used when it is consistent and verifiably accurate, and it should reflect significant changes, not a footer swap.

Submit it through the Sitemaps report in Search Console or through the Sitemap: line in robots.txt, including only URLs that return 200 and can be indexed.

What is a canonical and how do you avoid repeated pages?

A canonical is the indication of the preferred URL when the same content (or very similar content) exists at more than one address. This happens more often than it seems: http and https, with and without www, filter parameters, print versions.

The documentation on duplicate URLs describes the methods as signals, not orders. It recommends absolute URLs in rel="canonical", a canonical on the canonical page itself (self-referencing), consistency in internal links and no conflict between methods. Example:

<link rel="canonical" href="https://www.example.com/blog/how-to-choose-first-guitar" />

For non-HTML files, such as PDFs, the canonical can be sent as an HTTP header. Duplication itself is not penalized: the problem is splitting signals and letting Google choose for you.

For SEO professionals: the documentation ranks the canonicalization signals: redirects and rel="canonical" are strong, and inclusion in the sitemap is weak; Google also prefers HTTPS over HTTP by default. When Search Console shows that the canonical Google chose is different from the declared one, there is a contradiction among signals (internal links to another variation, redirects, very similar content). And if the canonical is injected by JavaScript, Google asks that it be identical to the one in the original HTML.

What is noindex and when should you use it?

Noindex is an instruction that asks Google not to put the page in the index. Use it on pages that should not appear in Search, such as the site's internal search results, thank-you pages and test environments (ideally, protect those with a password).

Google documents two ways, on the page about blocking indexing:

<meta name="robots" content="noindex">

X-Robots-Tag: noindex

The first goes in the HTML; the HTTP header also works for non-HTML files. Two important gotchas: the page must not be blocked in robots.txt, otherwise Googlebot never sees the noindex; and the effect is not immediate, because it can take months for Googlebot to revisit low-importance pages. Search Console's URL Inspection confirms whether it already sees the rule.

HTTPS, redirects and errors: what should you check?

The basics are: the whole site on HTTPS, old versions redirecting to the new one with a 301, and nonexistent pages returning 404, not 200. Google lists HTTPS as part of page experience, and the canonicalization system prefers HTTPS by default.

• 301 or 308 (permanent): tell Google that the destination should be the canonical version. Use them for domain changes, URL changes or when consolidating pages.
• 302 or 307 (temporary): do not send that signal, and the old URL may stay indexed. Use only if the change is temporary.
• Redirect chains: avoid several steps. Crawlers follow up to 10 hops, according to Google's HTTP status codes documentation, but each hop slows down users and bots.

In plain English: a 301 redirect is the permanent change-of-address notice stuck on the old store's door: "we are over there now". A 302 is the "back soon" sign, which makes nobody update the address in their contacts.

What does Google require from the mobile site?

Google uses the mobile version of the content to index and rank. The mobile-first indexing documentation asks that the mobile site have the same content as the desktop (putting text in tabs or accordions is acceptable, removing it is not), the same robots meta tags and the same structured data. It also warns that Google does not load content that depends on interaction, such as scrolling, clicking or typing.

A simple test: open your main pages on your phone. What exists only on the computer may be left out of what Google indexes.

How does JavaScript affect SEO?

If your site only shows the content after running JavaScript, Google can still see it, but in another stage and with some delay. According to the JavaScript SEO basics guide, the process has three phases (crawling, rendering, indexing), and the recommendation is to prefer server-side rendering or pre-rendering, which also benefits users and bots that do not run JavaScript.

The same guide asks for real links with <a href>, real URLs (the History API, not # fragments) and correct HTTP codes: a nonexistent page should return 404, not 200, or it becomes a "soft 404".

What should you ask the developer? A list and a message template

Ask by problem and by evidence, not by technology:

1. Confirm that the production robots.txt does not have Disallow: / and does not block important folders.
2. Remove noindex (meta or header) from the pages that should appear in Search.
3. Generate a sitemap with 200, canonical and indexable URLs, and declare it in robots.txt.
4. Set an absolute, self-referencing rel="canonical" on every indexable page.
5. Redirect HTTP to HTTPS and www or non-www to a single version, with a 301.
6. Make sure the main content and links are in the HTML delivered by the server.
7. Make the mobile version have the same content and tags as the desktop one.
8. Return 404 (or 410) for nonexistent pages, and never 200 with an error message.

Message template: "Search Console shows [status] for [URL], but the page should be indexed. Can you check whether there is a meta robots or X-Robots-Tag with noindex in production? Screenshot attached. I will inspect the URL again afterward."

Hypothetical example: the site that vanished after the relaunch

Hypothetical example: a store relaunches its site on a new platform. Ten days later, Google traffic drops sharply. The owner thinks "Google penalized us" and starts rewriting texts.

Someone who knows technical SEO follows a different sequence. They open /robots.txt and find Disallow: /, a rule inherited from the test environment. They open URL Inspection on a product page and see "Blocked by robots.txt". They ask the developer to swap the file for a production version and check that no meta noindex was left in the templates. They resubmit the sitemap and inspect the main URLs. Then they track indexing over the following weeks.

None of this required code, only knowing where to look. The guide to Google Search Console for beginners shows how to read these reports.

Frequently asked questions

What is technical SEO?

It is the set of adjustments that ensures Google can crawl, index and display the site's pages: robots.txt, sitemap, canonical, noindex, HTTPS, the mobile version, JavaScript rendering and speed. It does not improve the content itself, but without it the content may never be seen.

What is the difference between robots.txt and noindex?

robots.txt controls what Googlebot may crawl. Noindex asks for the page not to be indexed. To hide a page from the results, use noindex, and do not block the page in robots.txt, because Googlebot needs to access it to read the instruction. A blocked URL can still appear without a description.

Does every site need a sitemap?

Not necessarily. Google says small sites that are well linked internally and have no rich media can do without one. For large sites, new sites, sites with few external links or sites with many videos and images, the sitemap helps discovery. In any case, it does not guarantee indexing.

Is the canonical mandatory?

No, but it is recommended. Without it, Google chooses the main version among similar pages on its own, using its own signals. Declaring an absolute, self-referencing canonical reduces the risk of it choosing a variation you do not want, although the canonical is a signal, not an order.

How do I know whether Google is seeing my page correctly?

Use Search Console's URL Inspection and run the live test: it shows whether the page is crawlable, whether there is a block by robots.txt or noindex, which canonical Google chose and how the page was rendered. The indexed version may differ from the current one, because it reflects the last visit.

Next step

With the technical foundation checked, the next step is reviewing the whole site at once: the guide to SEO audit step by step gathers these checks in a complete walkthrough. To understand the speed and stability metrics, read Core Web Vitals explained.

Read also:

• Robots.txt and AI crawlers
• Google Search Console: a beginner's guide
• On-page SEO: a complete checklist for every page