Skip to content
Guide · SEO

Technical SEO: crawlability, indexing, speed, and structure

Technical SEO is the work that lets a search engine reach your content in the first place. Great writing is wasted if a crawler cannot find the page, cannot render it, or gives up waiting for it to load.

Last updated

Crawling and indexing

Before a page can rank it has to be crawled — visited by a search engine bot — and then indexed, meaning stored and considered for results. Anything that blocks a crawler, such as a stray robots rule or a broken link path, quietly removes pages from contention.

A site should make its important pages easy to discover through clean internal links and an accurate sitemap, and keep its low-value pages out of the index rather than letting them dilute the crawl. Managing what gets indexed is as much technical SEO as making sure the good pages do.

Speed and rendering

How quickly a page becomes usable affects both ranking and whether a visitor stays. Large images, blocking scripts, and slow servers all push the moment of usefulness later, and a reader who waits too long simply leaves. Core Web Vitals put numbers to this experience.

Rendering matters too. If key content only appears after heavy JavaScript runs, a crawler may see an empty shell. Serving meaningful HTML up front, and letting scripts enhance rather than construct the page, keeps content reliably visible.

Site structure and URLs

A sensible structure groups related pages so both people and crawlers can predict where things live. Shallow paths — important pages a click or two from the home page — get discovered and re-crawled more readily than pages buried deep in the tree.

Clean, stable URLs help as well. Readable paths, consistent use of one canonical version, and avoiding needless duplicate addresses all reduce the confusion that splits ranking signals across near-identical pages.

How Kirti approaches this

Kirti is being built to run technical checks as part of its audit stage — surfacing crawl blocks, slow pages, broken links, and indexing gaps — and to propose fixes a brand's team can approve rather than acting on the site unsupervised.

Because the agent is still in active development and not yet generally available, this guide focuses on the concepts, which hold true whatever tooling you use to catch these issues.

In active development

One agent for the whole brand.

Kirti is an AI-native brand-operations agent — one agent per brand that runs a brand's whole online presence as one closed loop across website, SEO, GEO, content, social, and paid ads. It is being built now and not yet generally available — read what Kirti is, or tell us about your brand.

Questions

People also ask

What is the difference between crawling and indexing?

Crawling is a search engine visiting a page; indexing is it deciding to store that page for possible results. A page can be crawled but not indexed if the engine judges it low-value or duplicate.

Does site speed really affect rankings?

Yes, though modestly. Speed is one signal among many, and its bigger effect is on user behaviour — slow pages lose visitors before they convert, which itself sends negative signals.

How do I know if my pages are indexed?

A search engine’s webmaster tools, such as Search Console, report index status per page. You can also search for a specific page URL to see whether it appears at all.