Definition
Technical SEO is the part of search engine optimization that configures a website’s server, code and structure so search engines can crawl, render and index its pages and interpret them correctly. It covers status codes, redirects, robots.txt, canonical tags, XML sitemaps, JavaScript, structured data, page speed and hreflang.
Key takeaways
- Technical SEO decides whether Google and Bing can crawl and index your pages. How high a page ranks depends mainly on its content.
- Your robots.txt only controls crawling. To keep a page out of search results, it needs noindex, and Google must be able to crawl it.
- Redirects, canonical tags and sitemaps should name the same URL. If the signals conflict, Google picks the canonical URL itself.
- Put content and links in the HTML your server delivers. Google runs JavaScript only after crawling, and not every crawler can.
- The key checks are free: Google Search Console, Bing Webmaster Tools and the page source in your browser.
How does a page get into Google Search?
Google works in three stages: crawling, indexing and serving search results. When crawling, Googlebot fetches the page; when indexing, Google analyzes it and stores it in its index. Technical SEO makes sure a page gets through both stages. Its ranking then depends mainly on relevance.
A crawler is a program that fetches pages automatically; Google’s is called Googlebot. There’s no central registry of web pages, so Googlebot finds new URLs through links from pages it already knows and through sitemaps, which list a site’s URLs (Google on how Search works).
For most sites, Google indexes the mobile version. Googlebot reads only the first 2 MB of an HTML file, measured uncompressed, and ignores the rest for indexing (Google, as of Feb 3, 2026).
When indexing, Google analyzes the text, title, images and other information. It groups pages with very similar content and picks a canonical URL from each group: the authoritative version that can appear in search results. Google doesn’t accept payment for crawling or rankings.
Google’s minimum technical requirements are short: Googlebot isn’t blocked, the page returns status code 200, and it has indexable content. Even a page that meets them may not be indexed (Google’s technical requirements).
| Step | What happens | Technical levers |
|---|---|---|
| Discovery and crawling | Googlebot finds URLs through links and sitemaps and fetches them. | robots.txt, status codes, redirects, internal links, XML sitemap |
| Rendering | Google runs JavaScript and reads the resulting HTML. | JavaScript, blocked CSS and script files, lazy-loaded content |
| Indexing | Google analyzes the page and picks the canonical URL. | noindex, canonical, hreflang, structured data |
| Serving (ranking) | Google ranks pages for a search query. | content and relevance, Core Web Vitals, display on smartphones, HTTPS |
According to Microsoft, search experiences in Bing and Microsoft Copilot rely on the same crawling, indexing and ranking foundation as traditional search (Bing’s guidelines). Google’s AI Overviews and AI Mode also link only to pages that are indexed and eligible to appear with a snippet (Google on AI features). So technical errors affect classic search and AI answers alike. What else counts for AI answers is covered in our guide to generative engine optimization (GEO).
If your site is missing from Google altogether, our how-to Why isn’t my website showing up on Google? takes you through the possible causes in order.
What’s part of technical SEO, and what isn’t?
Technical SEO covers everything that lets search engines access and understand your pages. What a page says is content. Links and mentions on other sites are off-page SEO. Generative engine optimization (GEO), optimizing for AI answers, builds on all three. A page that Google can’t crawl or index won’t appear, however good its content.
| Area | Key question | Examples | Goal |
|---|---|---|---|
| Technical SEO | Can search engines crawl, render and correctly interpret your pages? | status codes, redirects, robots.txt, noindex, canonical, sitemap, JavaScript, hreflang, page speed | pages are indexed |
| Content (on-page SEO) | Does the page answer the query in a helpful, reliable way? | title, headings, text, images, firsthand experience | pages match the query |
| Off-page SEO | Do other websites link to your pages? | links, directories, expert articles, press | links as a signal and a path to your pages |
| GEO (generative engine optimization) | Do AI systems mention your company and link to your pages? | access for AI crawlers, answers at the start of each section, consistent company facts | mentions and citations in AI answers |
The lines blur. Titles and headings are content, but a missing or duplicate title is also a technical finding. Google uses links to find new pages and judge their relevance (Google on links). So internal links belong to technical SEO, and links from other sites to off-page SEO.
If you want to improve your Google ranking, it helps to know that technical work clears obstacles out of the way. Google says Search always seeks to show the most relevant content, even if its page experience is poor. When many helpful pages answer the same question, good page experience can contribute to success (Google on page experience).
On the websites we build, technical SEO is part of the scope. For existing websites, we start with an audit; what it covers is on our website development page. Our cost guide How much does a website cost in Germany? compares what a new site costs, and our comparison GEO vs. AEO vs. SEO explains how SEO and GEO relate.
How do you control what crawlers fetch and what gets indexed?
Your robots.txt file controls which URLs crawlers may fetch. Whether a page gets indexed is controlled by a noindex rule in the page code or the HTTP header. Google and Bing both stress that robots.txt doesn’t keep a page out of search results. Protect anything nobody should see with a password.
| Goal | Method | What Google does | Common mistake |
|---|---|---|---|
| Keep unimportant or near-identical pages from being crawled, such as filter pages with many parameters | Disallow in robots.txt | doesn’t crawl the URLs; linked URLs can still appear in results without a description | The block from the build phase is still in place at launch. |
| Keep a page out of search results | noindex as a meta tag or an X-Robots-Tag header | drops the page from search results once Googlebot reads the noindex | robots.txt also blocks the page, so Google never sees the noindex. |
| Content for authorized users only | password or login | doesn’t crawl the page | A staging environment is publicly accessible without protection. |
| Merge duplicate versions | redirect or canonical | consolidates the versions and usually shows one of them | robots.txt or noindex is used instead of consolidating. |
robots.txt: This text file sits in the top-level directory of a site, for example at https://example.com/robots.txt. It applies only to that exact host and protocol, so each subdomain needs its own. Google reads at most 500 KiB of it (Google, as of Oct 23, 2026). Don’t block CSS and script files that Google needs to render the page (Google on robots.txt). The standard for the file, RFC 9309 from September 2022, makes clear that its rules aren’t a form of access authorization. Our article AI crawlers and robots.txt covers which rules make sense for the crawlers of AI providers.
noindex: The rule sits in the head of the page as <meta name="robots" content="noindex"> or in the HTTP header as X-Robots-Tag: noindex. The header is information that precedes the content of every server response, and for PDFs and images it’s the only option. Google only sees the noindex if it’s allowed to crawl the page, and it doesn’t support noindex rules in robots.txt (Google on noindex). Bing also has to crawl a page to read the noindex (Bing on robots meta tags). For Bing, the rule means exclusion from search, from Copilot and from the grounding results that AI answers draw on (Bing Webmaster Guidelines).
How to check: In Google Search Console, the robots.txt report shows which robots.txt files Google found, when it last crawled them and any warnings or errors (Search Console Help on the robots.txt report).
Which status codes does your website need?
Pages that should appear in search return status code 200. Permanently moved URLs return 301 or 308, deleted ones 404 or 410. Google only indexes pages that return 200 and removes pages with 4xx errors from its index. Server errors (5xx) slow down crawling, and if they persist, even indexed pages drop out.
The status code is the three-digit number a server uses to answer every request (RFC 9110). Your browser doesn’t show it.
| Status code | Meaning | What Google does | When to use it |
|---|---|---|---|
| 200 | The page is served. | passes the content on to indexing; indexing isn’t guaranteed | every real page |
| 301, 308 | moved permanently | follows it and treats it as a strong signal that the target is canonical | site move, redesign, HTTP or www versions |
| 302, 307 | moved temporarily | follows it but treats it only as a weak signal | a short detour, for example while an offer is temporarily unavailable |
| 404, 410 | not found or permanently removed | doesn’t index the URL and removes it from the index | a deleted page with no successor |
| 429 | too many requests | treats it like a server error and crawls more slowly | only under genuine overload |
| 500, 503 | server error or temporarily unavailable | crawls more slowly; if errors persist, indexed pages drop out | 503 with Retry-After for maintenance lasting a day or two |
| 200 with an error message | soft 404 | reports a soft 404 error in Search Console | never; serve error pages with a 404 |
Googlebot treats all 4xx errors except 429 the same: the content counts as nonexistent (Google, as of Feb 4, 2026). During maintenance, your robots.txt must not return a 503, or crawling stops for the whole site (Google on pausing a website).
How to check: curl -I https://example.com/ shows a URL’s status code in the first line of the response; on Windows, type curl.exe. The Crawl Stats report under Settings shows how your server responds to Googlebot. According to Google, sites with fewer than 1,000 pages shouldn’t need it (Search Console Help on Crawl Stats).
How do you set up redirects correctly?
Redirect a permanently moved URL on the server with a 301 or 308 to its new target, in a single step. Use temporary 302 or 307 redirects only briefly; Bing says for less than two days. After a site move, Google advises keeping the redirects in place, generally for at least a year.
Redirect on the server: Google interprets server-side redirects most reliably. It treats an instant meta refresh as permanent and a delayed one as temporary, and it only recommends JavaScript redirects if neither is possible (Google on redirects).
Settle on one URL: Many websites can be reached at several addresses, for example with HTTP and HTTPS or with and without www. Pick one version and redirect all others to it in a single step. Googlebot does follow up to 10 redirects in a row (Google, as of Feb 4, 2026), but every hop costs your visitors time. Choose the HTTPS version, because Google prefers it as the canonical URL. An invalid certificate or a redirect from HTTPS to HTTP, by contrast, makes Google prefer the HTTP version very strongly, and HSTS can’t override that (Google on canonical URLs). HSTS lets a website tell browsers that it can only be reached over a secure connection (RFC 6797).
Redesigns and site moves: Redirect every old URL to the new page with the same content. If many old URLs redirect to the home page, Google may treat that as a soft 404. Keep the redirects for as long as possible, generally at least a year (Google on site moves). Our website migration checklist covers the whole process.
How to check: curl -sIL https://example.com/old-page/ shows every hop in a chain with its status code and target. In the Page indexing report, redirected URLs appear under “Page with redirect” and broken chains under “Redirect error” (Search Console Help on page indexing). Google contradicts itself on whether its inspection tools follow redirects (as of Nov 8, 2026), so test the target there directly.
What is a canonical tag, and when does it help?
A canonical tag is an element in the head of a page that tells search engines the authoritative URL of content that’s available at several URLs. Google treats it as a strong hint and may still choose a different URL. It helps with duplicates from parameters, sorting and filtering. It does harm when it points to the wrong URL.
According to Google, redirects and canonical tags are strong signals for choosing the canonical URL and sitemap inclusion a weak one, and several signals together are stronger than one (Google on canonical URLs). The HTTP Archive found a canonical tag on 67% of pages crawled as mobile in July 2025.
Source: HTTP Archive (external site), , updated · Web Almanac 2025, data from July 2025
When it helps: Duplicates come from tracking parameters such as gclid, sorting and filtering, HTTP and HTTPS versions and staging sites accidentally left public (Google on canonicalization). Also put the canonical on the authoritative page itself; Google recommends this self-referencing canonical. For variants of your own URL, a redirect is better, because it also takes visitors to the right version.
When it does harm: Google names common causes: content management systems and plugins that set the canonical incorrectly, misconfigured servers and hacked sites where attackers insert a canonical pointing to other sites (Google on fixing canonicalization issues). So follow these rules:
- Every page names its canonical URL exactly once, in the
<head>, as a full URL that includes https. - Sitemap, internal links and canonical name the same URL.
- Neither robots.txt nor noindex can consolidate pages: a blocked URL can be indexed without its content, and noindex removes the page from search entirely.
How to check: In Search Console, URL Inspection shows “User-declared canonical” and “Google-selected canonical.” You can only see Google’s choice in the indexed data, not in the live test (Search Console Help on URL Inspection). After a fix, Google may keep pages in a duplicate cluster for up to two weeks (Google on fixing canonicalization issues).
How do you set up a multilingual website with hreflang?
Give each language version its own URL and connect the versions with hreflang annotations. Each version lists itself and all the others; if the return link is missing, Google ignores the annotation. Use x-default to name the page for all other languages, and don’t redirect visitors automatically based on their browser language.
The hreflang annotation sits in the page head, the HTTP header or the sitemap and points to a page’s versions in other languages or for other countries. Googlebot usually crawls from the US without sending a preferred language, so it may miss versions that switch automatically (Google on multilingual sites). Google detects a page’s language itself and doesn’t use the lang attribute or hreflang for that (Google, as of Sep 21, 2026). So keep each page in a single language, navigation and footer included.
Rules for hreflang:
- Each version lists itself and all the others, each with a full URL that includes https.
- The code gives the language in ISO 639-1 and, optionally, the region in ISO 3166-1 Alpha 2, such as
deorde-AT. A region on its own isn’t valid. x-defaultnames the page for every language you don’t offer, such as a language selector.- Page head, HTTP header and sitemap are equivalent for Google. Pick one method; using all three brings no benefit.
- The canonical of each version points to a page in the same language (Google on canonical URLs).
Example: A page in German and English carries these three lines in the head of both versions; x-default sends everyone else to the English version.
<link rel="alternate" hreflang="de" href="https://example.com/de/">
<link rel="alternate" hreflang="en" href="https://example.com/en/">
<link rel="alternate" hreflang="x-default" href="https://example.com/en/">Which URL structure? Google compares country-code domains such as example.de, subdomains such as de.example.com, subdirectories such as example.com/de/ and parameters such as ?loc=de, and advises against parameters. A country-code domain is a strong signal for one country but can only target that one (Google on multilingual sites).
How to check: Search Console no longer has an hreflang report. Google has deprecated the International Targeting report but still supports and uses hreflang. So check the page source of each version for complete, reciprocal annotations.
What is an XML sitemap, and what goes in it?
An XML sitemap is a file that lists the URLs on your website that should appear in search, with the date each one last changed. It helps search engines find your pages but doesn’t guarantee indexing. Include only canonical pages that return 200, and change the date only when the content really changes.
Do you need one? Google says it can usually find small, well-linked sites with up to about 500 pages without a sitemap. A sitemap matters most for large sites and for new sites with few links from other websites. In most cases it still helps, and many content management systems create one automatically (Google on sitemaps).
Rules: A sitemap holds at most 50,000 URLs or 50 MB uncompressed. Larger sites split it up and submit a sitemap index that lists the individual sitemaps. The file is UTF-8 encoded and uses full URLs. Google ignores the priority and changefreq fields and uses lastmod if it’s consistently and verifiably accurate. A new copyright year isn’t a significant change, but new main content, structured data or links are (Google, as of Jul 8, 2026). Google has shut down the ping endpoint once used to submit sitemaps; such requests now return a 404 error (Google, Jun 26, 2023).
Bing asks for the same: only canonical URLs, prompt removal of deleted and redirected URLs and an accurate lastmod (Bing Webmaster Guidelines). With the IndexNow protocol, you can report new, changed and deleted URLs right away to Bing, Naver, Seznam.cz, Yandex and Yep; Google isn’t among the participants (IndexNow). Our how-to Bing and IndexNow walks through the setup. For Google Search and its AI features, Google says you don’t need special text files for AI such as an llms.txt (Google on generative AI features). Our explainer llms.txt covers what the file does for other systems.
How to check: Submit your sitemap in the Sitemaps report in Search Console, and also list it in robots.txt with the line Sitemap: https://example.com/sitemap.xml. If the report shows a status other than “Success,” click the row to see the errors (Search Console Help on the Sitemaps report).
Can Google see content that JavaScript generates?
Yes, but with a delay. Google puts every page that returns 200 and may be indexed into a rendering queue, then runs its JavaScript later in a recent version of Chromium. That can take seconds or longer. Not every other crawler can run JavaScript, so content, links and metadata belong in the HTML your server delivers.
On sites that assemble their content in the browser, the initial HTML often contains only a skeleton, and Google sees the content only after rendering (Google on JavaScript SEO). For that to work, watch four points:
- Links as real links: Google can generally only extract links that are
<a>elements with anhrefin the code (Google on links). - No content behind clicks: Google doesn’t click or scroll, so it won’t find anything that only loads after an action (Google on lazy loading).
- noindex and canonical in the initial HTML: If the delivered HTML contains a noindex, Google may skip rendering, so a script that removes the noindex later may not work (Google on JavaScript SEO).
- Error pages with a real status code: Single-page apps, which render every view in one page, often return 200 even for missing content. Google then recommends redirecting to a URL that returns 404 or adding noindex to the error view (Google on JavaScript SEO).
Google recommends server-side rendering, static rendering or hydration. With hydration, a script makes the ready-made HTML from the server interactive (web.dev on rendering). Google calls dynamic rendering, a separate version just for crawlers, a workaround and doesn’t recommend it (Google on dynamic rendering). Bing also advises against hiding important content behind client-side rendering (Bing Webmaster Guidelines). In a December 2024 measurement by Vercel, none of the major AI crawlers ran JavaScript, including those from OpenAI, Anthropic and Perplexity (Vercel). Our how-to on getting recommended by ChatGPT shows how to check your pages for this.
How to check: Open the page source, in Chrome with Ctrl+U on Windows or Command+Option+U on a Mac (Chrome Help), and search for a key sentence from your page. If it’s there, no crawler needs JavaScript to read it. For a test for AI crawlers with commands and expected output, see our article JavaScript and AI crawlers.
What does structured data do, and how do you test it?
Structured data is machine-readable information in the page source, usually in JSON-LD format using the Schema.org vocabulary. Google uses it to understand content and can show enhanced results with it, known as rich results. That isn’t guaranteed. The markup must describe what’s visible on the page.
Google recommends JSON-LD and also accepts Microdata and RDFa (Google on structured data). Don’t block the marked-up page from Googlebot with robots.txt or noindex (Google’s structured data guidelines). Bing reads Schema.org in JSON-LD, Microdata, Microformats and RDFa, plus Open Graph, and doesn’t prefer any of these formats (Bing on structured data).
According to Google, generative search doesn’t require special Schema.org markup, but structured data remains useful for rich results (Google on generative AI features). Bing writes that structured data can support grounding but doesn’t guarantee visibility (Bing Webmaster Guidelines). Our explainer on structured data and schema markup covers what the markup does and how to build it.
How to check: The Rich Results Test shows which rich result types it finds on the page, plus any errors or suggestions for your structured data (Search Console Help on the Rich Results Test). The Schema Markup Validator from Schema.org checks syntax and conformance to the Schema.org standard for all types, with or without a rich result. It doesn’t check rich results (Google, Dec 15, 2020).
What role do Core Web Vitals play in rankings?
Core Web Vitals are three metrics from real visits: loading time of the largest content element (LCP), responsiveness to input (INP) and layout stability (CLS). Google uses them in its ranking systems. Good values are LCP up to 2.5 seconds, INP up to 200 milliseconds and CLS up to 0.1, at the 75th percentile of page loads.
The 75th percentile means at least three in four page loads reach the value, separately for mobile and desktop (web.dev on Web Vitals). INP replaced the earlier FID metric on March 12, 2024 (web.dev, Jan 31, 2024).
Google explicitly recommends good Core Web Vitals (Google on Core Web Vitals). It also says good values don’t guarantee a top spot (Google on page experience).
How to check: The Core Web Vitals report in Search Console shows field data, meaning values from real Chrome visits collected in the CrUX report. If your site has too few visits, it says “No data available.” Then test individual pages in the lab, for example with PageSpeed Insights or Lighthouse (Search Console Help on the Core Web Vitals report). Our explainer on Core Web Vitals shows how many sites reach the thresholds and how to improve yours. Our comparison of static sites and WordPress shows how they perform with real visitors.
How do you check whether your website is SEO-friendly?
Check in the order Google works: first availability and status codes, then blocks from robots.txt and noindex, then redirects, canonical and sitemap, and finally rendering, structured data, page speed and language versions. If a page is blocked, improving later points won’t help. Google or Bing has a free tool for almost every point.
| Point | Good result | Google Search Console and tests | Bing Webmaster Tools |
|---|---|---|---|
| 1 · Availability and HTTPS | The home page returns 200 over HTTPS; HTTP and www versions lead there in one step. | URL Inspection with live test; HTTPS report | URL Inspection, Live URL |
| 2 · robots.txt | No rule blocks important pages; the file lists the sitemap. | robots.txt report; URL Inspection, Crawl allowed? | robots.txt tester |
| 3 · noindex | only on pages that should stay out of search | URL Inspection, Indexing allowed? | URL Inspection |
| 4 · Status codes | 200 for pages, 404 or 410 for removed ones, no soft 404 | Page indexing report; Crawl Stats report | URL Inspection; notifications |
| 5 · Redirects | one step, 301 or 308 for permanent moves | Page indexing report | URL Inspection, Live URL |
| 6 · Canonical | self-referencing; Google selects the same URL | URL Inspection, Google-selected canonical | URL Inspection, the page’s HTML |
| 7 · Sitemap | only canonical pages that return 200, lastmod only after a real change | Sitemaps report | Sitemaps |
| 8 · JavaScript | Key sentence and links are in the page source. | URL Inspection, View tested page | URL Inspection, Live URL |
| 9 · Structured data | valid and consistent with the visible content | Rich Results Test; Schema Markup Validator | URL Inspection, Markup card |
| 10 · Core Web Vitals | LCP up to 2.5 s, INP up to 200 ms, CLS up to 0.1 | Core Web Vitals report; PageSpeed Insights | not specified |
| 11 · hreflang | reciprocal, with self-reference and x-default | page source of each version | not specified |
A dash means the Bing Webmaster Tools help center lists no dedicated tool for this (Bing’s help center). The Bing column draws on Bing’s help pages: Bing robots.txt tester, Bing URL Inspection, Bing sitemaps and Bing crawl error alerts. To check the whole site, Bing offers Site Scan, which crawls your site on demand and lists technical SEO issues by severity (Bing on Site Scan).
What changes when your site targets Germany?
The same rules apply as anywhere else. But a site aimed at Germany and the EU deserves a second look at four points: Bing’s share on desktop, the language codes for Germany, Austria and Switzerland, umlauts in URLs, and privacy or security settings that can lock out Googlebot.
Bing: In October 2026, Bing’s market share in Germany was 19.74% on desktop and 6.41% across all devices. Add your site to Bing Webmaster Tools as well: you can import sites from Search Console there, and they’re then verified automatically (Bing on adding a site).
Source: StatCounter (external site), · monthly figure, October 2026, desktop
Source: StatCounter, · monthly figure, October 2026, all devices
Language codes for German-speaking countries: If one German version serves Germany, Austria and Switzerland alike, de is enough. Separate codes such as de-AT or de-CH pay off when the content differs, for example with prices in another currency.
Umlauts and spelling: Google advises percent-encoding non-ASCII characters in links, for example gem%C3%BCse instead of gemüse, and treats /Leistungen/ and /leistungen/ as two different URLs (Google on URL structure). It’s simpler to spell out umlauts in URLs, writing ä as ae and ß as ss, and to use lowercase throughout. That’s how we do it on datafalke.de.
Country blocking and banners: Most of Google’s crawls come from the US, so country blocking in your firewall or hosting that stops US traffic also stops Googlebot. A consent banner should overlay the content. If a site instead redirects every URL to a separate consent page, all other pages drop out of search results (Google on dialogs).
From our work: the test pipeline behind datafalke.de
We treat datafalke.de as our own reference project for technical SEO. Every new version goes through a test pipeline of our own scripts, and our rule is that a version only goes live once every check on the finished build has passed. We deliver the pages as prebuilt HTML without JavaScript; the colophon lists the technology and hosting. The table shows what the pipeline requires. It doesn’t prove any effect, and we’ll only publish measurements with the measurement log.
| Item | What the check requires | When |
|---|---|---|
| Status codes and redirects | The home page via HTTP, www and the CDN address redirects to https://datafalke.de/ with a single 301. On https://datafalke.de, URLs without a trailing slash or ending in index.html redirect in one step. An unknown URL returns a 404 with its own error page set to noindex. Uppercase letters in the path never return 200. | after every upload, from outside |
| robots.txt | The wording is fixed: all crawlers may fetch everything, plus a reference to the sitemap. | in the build and after every upload |
| Meta robots | Every indexable page carries the same index directive, every other page a noindex. On the staging environment, every response carries noindex. | in the build and after every upload |
| Canonical | exactly one canonical in the head of every page, as the page’s own full URL | in the build |
| Sitemap | a sitemap index with one sitemap per language, listing every indexable, self-canonical page exactly once, with lastmod including time and time zone | in the build |
| hreflang | self-reference and x-default on every indexable page; annotations reciprocal and pointing to existing pages | in the build |
| JavaScript and size | no script except the structured data, no file over 2 MB | in the build |
| Internal links | Every link target exists; every indexable page has an incoming link and is at most three clicks from the home page. | in the build |
| Structured data | exactly one JSON-LD block with linked nodes; the visible breadcrumb matches the one in the structured data | in the build |
| Title, description, H1 | lengths within fixed limits, exactly one H1, no duplicates across pages | in the build |
| HTML | valid HTML under a fixed rule set | in the build |
| Crawler access | Twelve user agents of search and AI crawlers get status 200, no cookie and the same page as a regular request. | after every upload |
The pipeline checks what we deliver, not whether Google and Bing index the pages. To track that, datafalke.de has been registered in Search Console as a Domain property and in Bing Webmaster Tools since November 7, 2026, and we report new pages through IndexNow after each upload.
You can check three entries from the table yourself; on a Mac, type curl instead of curl.exe.
curl.exe -sI http://datafalke.de/
curl.exe -sI https://www.datafalke.de/
curl.exe -sI https://datafalke.de/gibt-es-nicht/The first two requests return 301 Moved Permanently and Location: https://datafalke.de/, the third 404 Not Found. To see the canonical, open https://datafalke.de/en/services/?utm_source=test and view the page source: the canonical names the URL without the parameter.