In short
Either Google hasn’t indexed your website yet, or it ranks low for your keywords. A site: search for your domain tells you which. If the site is missing, Google says the most common reason is that it’s simply too new. Often, robots.txt or a noindex rule blocks Google, or redirects and duplicates keep pages out of the index.
Key takeaways
- A site: search shows whether Google knows your site. If results appear, it’s indexed and probably ranks lower for your keywords.
- Google says the most common reason a site isn’t indexed is that it’s too new. Indexing typically takes a day or two to a few weeks.
- If URL Inspection reports a page as not indexed, Google names robots.txt and noindex blocks as the most common reasons.
- URL Inspection in Search Console shows whether Google may crawl and index a URL, and which version it treats as authoritative.
- Bing runs its own index, which Microsoft Copilot also relies on. In October 2026, Bing had a 19.74% desktop market share in Germany.
Is your website really missing from Google?
To see whether Google knows your website, search Google for site:example.com, using your own domain. If results appear, the site is indexed and probably just ranks lower for your keywords. If nothing appears, it’s missing from the index. The steps below help you find out why.
Why this comes first: Many people think their website is invisible when it simply ranks low. Google points this out itself in its help on missing pages (external site). Results also differ from person to person, because Google factors in things like location, language and device (external site). This question also decides whether you can show up in AI answers. According to Google, only an indexed page (external site) can be a source in AI Overviews and AI Mode. The page must also be eligible to be shown with a snippet. Our guide to generative engine optimization (GEO) explains what else counts. How crawling, indexing and the other technical basics fit together is covered in our guide to technical SEO.
How to do it:
- Turn off SafeSearch in Google, because the filter can hide results. Then search for
site:example.com. - Check your most important pages one by one, for example with
site:https://example.com/services/. Type the URL exactly as it appears in your browser: according to Google, the search returns different results (external site) with and withoutwww. - Also search for your company name, and for your service plus your location, the way your customers would.
Example: If site:example.com shows only your home page, Google knows the site but probably hasn’t indexed the other pages yet. Search Console is your next step.
Common mistake: You treat the site: results as a complete list of your indexed pages. According to Google, the list isn’t always exhaustive. To see the status of a single URL, use URL Inspection in Search Console.
Done when you know whether your home page and your most important pages are indexed. If they are, skip to the section on websites that are indexed but not showing up for your services.
Have you set up Google Search Console?
Google Search Console is a free service from Google that shows where your website stands in Google’s index. Its URL Inspection tool tells you for any URL whether it’s indexed and, if not, why. Google’s own troubleshooting help assumes you use Search Console, because it makes indexing problems much easier to find.
Why: You don’t have to sign up for Search Console (external site) to appear in Google Search. For troubleshooting, though, it’s the most important tool. URL Inspection (external site) shows under “Crawl allowed?” whether Google may fetch the page, and under “Indexing allowed?” whether the page permits indexing. “Google-selected canonical” tells you which URL Google treats as the authoritative version.
How to do it:
- Sign in to Search Console (external site) with a Google account and add a Domain property for
example.com. It covers all versions with and without www, on http and https. You verify it with a DNS record at your domain or DNS provider (Search Console Help (external site)). - Enter your home page URL in the search bar at the top of Search Console. If the result says “URL is not on Google,” expand the Availability section. It gives the reason.
- Click “Test live URL.” If the page passes the test, click “Request indexing.”
- Submit your sitemap in the Sitemaps report. A sitemap is a file that lists all the URLs you want Google to know about.
What the statuses mean: If your site has fewer than 500 pages, you probably don’t need the Page indexing report, according to Google (external site). The site: search and URL Inspection are enough. Many statuses are intentional, for example for URLs that redirect. You only need to act on pages that should appear on Google. The table uses the status names from Google’s Search Console Help; the wording in Search Console itself may differ slightly.
| Status | What it means | What to do |
|---|---|---|
| URL blocked by robots.txt | Google isn’t allowed to fetch the page. | remove the rule from robots.txt, see the robots.txt section |
| URL marked ‘noindex’ | The page doesn’t allow indexing. | remove noindex from the page code or header, see the noindex section |
| Page with redirect | The URL redirects. At most, the target gets indexed. | nothing, if the redirect is intentional |
| Redirect error | The redirect chain is too long, loops or contains an invalid URL. | redirect straight to the target, see the redirects section |
| Not found (404) | There’s no page at this URL (anymore). | nothing, if the page is gone; otherwise a 301 to the new URL |
| Soft 404 | The page looks like an error page but doesn’t return an error code. | serve error pages with a 404, or fill the page with content |
| Server error (5xx) | The server returned an error when Google requested the page. | sort it out with your host, see the status codes section |
| Blocked due to unauthorized request (401) | The page requires a login, such as a password. | remove the password protection if the page should be public |
| Blocked due to access forbidden (403) | The server refuses Googlebot access. | check access protection, firewall and country blocking, see the Germany and EU section |
| Duplicate without user-selected canonical | Google considers the page a duplicate and shows a different one. | set a canonical or make the content clearly different |
| Duplicate, Google chose different canonical than user | Your canonical and Google’s choice differ. | compare both URLs, see the canonical section |
| Discovered - currently not indexed | Google knows the URL and postponed crawling it so as not to overload the server. | wait; if many pages are affected, check server performance with your host |
| Crawled - currently not indexed | Google crawled the page but didn’t index it. | for important pages, check the content, see the section on indexed pages that don’t show up |
| Indexed, though blocked by robots.txt | Google knows the URL from links but isn’t allowed to read the content. | remove the block; if the page should stay out of Google, use noindex instead |
Common mistake: Instead of a Domain property, you add a URL-prefix property for http://example.com, even though your site runs on https. The property then shows hardly any data. Google lists this as a possible reason when a property still has no data (external site) several days after verification.
Done when your domain is verified, you’ve read the URL Inspection result for your home page and you’ve submitted a sitemap.
Did you launch your website recently?
A new website takes time for Google to find, crawl and index. According to Google, the most common reason a site isn’t indexed is that it’s simply too new. The whole process typically takes anywhere from a day or two to a few weeks. After requesting indexing, wait at least a week before assuming there’s a problem.
Why: Google finds new pages mainly through links from pages it already knows (Google’s SEO Starter Guide (external site)). In its crawling and indexing FAQ (external site), Google lists other common reasons why a site isn’t indexed. They include too few links from other websites and a site that was temporarily unavailable when Google tried to crawl it. If a new site has few links from other sites, a sitemap (external site) helps in particular, according to Google. A limited crawl allowance, often called crawl budget, isn’t an issue for most business websites. Google aims its guide on the topic at sites with 1 million pages or more, or 10,000 pages or more with content that changes daily (Google on crawl budget (external site)).
How to do it:
- Request indexing of your home page in URL Inspection. For a new website, that’s the best first step according to Google (external site), because Google can find the other pages from a well-linked home page.
- Submit your sitemap, and also list it in robots.txt with a line such as
Sitemap: https://example.com/sitemap.xml(Google on sitemaps (external site)). - Link every important page from your home page, in the menu or in the text. Google advises against links that only appear after a click.
- Get your first links from other websites, for example from your chamber of commerce, trade association or business partners. Google also discovers new sites through existing ones (external site).
Example: Your website goes live on Monday, and you request indexing of the home page the same day. If it’s still missing from a site: search a week later, start troubleshooting with the next section.
Common mistake: You request indexing again every day, or you pay a service to “submit your site to Google.” Repeated requests for the same URL don’t speed anything up (external site), and according to Google, getting into search results costs nothing (external site).
Inherited domain: If you took over a domain or website from someone else, check the “Manual actions” and “Security issues” reports in Search Console. They also show actions Google took against the site before it was yours (external site).
Done when you’ve requested indexing of your home page and submitted your sitemap, and at least a week has passed since.
Is your robots.txt blocking Googlebot?
Your robots.txt is a text file at example.com/robots.txt that tells crawlers which URLs they may fetch. If the rule Disallow: / applies to Googlebot there, Google isn’t allowed to read any page on your site. According to Google, a blocking robots.txt rule is one of the most common reasons why many pages aren’t indexed.
Why: Crawlers are programs that fetch websites automatically. Google’s crawler is called Googlebot, and it obeys robots.txt. If the file doesn’t exist, Google can crawl everything (external site). But if your server answers requests for robots.txt with a server error, Google first stops crawling the entire site for 12 hours (external site). To hide individual pages, robots.txt won’t work: blocked URLs can still end up in the index, just without a description (external site).
How to do it:
- Open
https://example.com/robots.txtin your browser. - Look for the group
User-agent: Googlebot. If there isn’t one, the groupUser-agent: *applies to Googlebot. If the group that applies containsDisallow: /, your entire site is blocked. - In URL Inspection, check that “Crawl allowed?” says Yes. The robots.txt report (external site) shows when Google last fetched the file.
- Google generally caches robots.txt for up to 24 hours, so a fix usually takes effect within a day. If it’s urgent, request a recrawl in the robots.txt report.
Example: These two lines block your site for all crawlers. Replace the second line with Allow: / to lift the block.
User-agent: *
Disallow: /Common mistake: During development, robots.txt blocks all crawlers, and the block is still in place at launch. A second mistake involves groups for individual crawlers. If there’s a group User-agent: Googlebot, only its rules apply to Googlebot, and the rules under User-agent: * no longer do (external site). Our article on AI crawlers and robots.txt explains which rules make sense for the crawlers of AI assistants.
Done when no rule blocks your important pages, robots.txt returns 200 or doesn’t exist at all, and URL Inspection shows Yes under “Crawl allowed?”
Are your pages set to noindex?
A noindex rule in a page’s code or HTTP header prevents search engines from indexing the page. When Googlebot finds it, Google drops the page from search results entirely, even if other sites link to it. In WordPress, a single setting applies it to your whole site.
Why: The rule sits either in a meta tag in the head of the page or in an X-Robots-Tag line in the HTTP header. The header is invisible information that the server sends along with every page. You can’t see either on the page itself. Google only sees the noindex if it’s allowed to fetch the page. If robots.txt blocks the page as well, the noindex goes unnoticed (external site).
How to do it:
- Open an important page and view its page source: in Chrome on Windows, press Ctrl+U, and on a Mac, Command+Option+U (Chrome Help (external site)). Search it for noindex with Ctrl+F (Command+F on a Mac).
- Check the header with the first request in the “From our work” section. A line
x-robots-tag: noindexhas the same effect as the meta tag. - In WordPress, you’ll find the setting under Settings › Reading › Search engine visibility. If “Discourage search engines from indexing this site” is checked, WordPress 5.3 and later sets
noindex,nofollowfor the entire site (WordPress documentation (external site)). - Once you’ve fixed it, click “Test live URL” and then “Request indexing” in URL Inspection. Without a request, Google only sees the change on its next crawl, which can take months (external site) for pages that get little attention.
Example: If this line is in the head of a page, Google won’t index it.
<meta name="robots" content="noindex">Common mistake: The new website is built with noindex so Google doesn’t index the unfinished site, and the noindex is still there at launch. Google’s guide to site moves (external site) specifically reminds you to remove it. Also watch out for a trap in URL Inspection. If robots.txt blocks the page, “Indexing allowed?” always says Yes, because Google can’t read the noindex (external site).
Done when neither the source code nor the header of your important pages contains noindex, and URL Inspection shows Yes under “Indexing allowed?”
Are your redirects and status codes working?
Google only indexes pages that return status code 200, meaning they’re served without an error. Every other version of your URL, for example with http or www, should lead to that page with a permanent redirect. Error pages, server errors and redirect errors keep pages out of the index.
Why: The status code is the three-digit number (external site) a server uses to answer every request. Google doesn’t index pages with 4xx codes such as 404, and it removes them from the index (Google on status codes (external site)). With server errors (5xx), Google slows down its crawling. If the errors persist, even indexed pages drop out. If a DNS or network error makes your server unreachable altogether, Google removes indexed URLs within days (external site). A permanent redirect (301 or 308) shows the target in search results, a temporary one (302 or 307) the old URL (Google on redirects (external site)).
How to do it:
- Open
http://example.com,http://www.example.comandhttps://www.example.com. All three should lead to the same https URL. Requests 2 and 3 in the “From our work” section show the status code and the target. - Open a URL that doesn’t exist. The response must be 404 or 410. If the server delivers the error page with a 200, Search Console reports a soft 404.
- After a redesign, redirect every old URL with a 301 to the new page with the same content. Google advises keeping these redirects for as long as possible, generally at least a year (external site). Our website migration checklist shows how to plan a redesign step by step.
- If your browser shows a certificate warning, have the certificate renewed. A faulty certificate makes Google strongly prefer (external site) the http version.
Example: http://www.example.com/ redirects to https://www.example.com/, and only from there to https://example.com/. That’s two redirects in a row. Googlebot follows up to 10 (external site), but each one adds load time for your visitors. So set up a direct redirect to the target for every version.
Common mistake: After a redesign, all old URLs redirect to the new home page. Google may treat this as a soft 404 (external site) because the target doesn’t match the old content.
Done when every version of your URL leads to the https version with a single 301, that version returns 200, and a made-up URL returns 404.
Does your canonical point to a different URL?
A canonical tag in the head of a page tells Google which URL is the authoritative version of its content. If it points to another domain, a staging site or the old http URL, it steers Google away from your page. Google treats the tag as a hint and may choose a different URL.
Why: Google groups identical or very similar content found at several URLs into a cluster and usually shows only one of them (Google on canonicalization (external site)). Duplicates like these arise from the http and https versions of a site, for example, or from a staging site that was accidentally left public. Some content management systems and plugins set the canonical incorrectly (external site).
How to do it:
- Search the page source for canonical. The entry
<link rel="canonical" href="…">should name the page’s own URL. Google recommends (external site) this self-referencing canonical. - In URL Inspection, compare “User-declared canonical” with “Google-selected canonical.” If they differ, Google considers a different page authoritative.
- Protect staging sites with a password. Google recommends this for pages that shouldn’t appear in search (external site).
- Be patient after a fix: Google may keep pages in a duplicate cluster for up to two weeks (external site).
Example: An agency builds the new website at test.example.com and copies it over at launch. Every page still names the test address as its canonical. For every page, Google therefore gets a hint to index a different URL.
Common mistake: The sitemap and the canonical name different URLs for the same page, for example once with and once without www. Google advises never to specify different URLs (external site) as canonical for the same page.
Done when every important page names itself as canonical and URL Inspection shows that Google selected the same URL.
Is your website indexed but not showing up for your services?
If your site is indexed but doesn’t show up when people search for your services, Google mainly points to the content. Either it isn’t relevant to the search, or its quality is low. A manual action, a security issue or a removal request can also hide pages.
Why: Google doesn’t index every page it knows about. According to Google, “Crawled - currently not indexed” is a common status (external site) and doesn’t necessarily mean something is wrong. Google advises using the words people would use to look for what you offer. They belong in prominent places, such as the title and the main heading (Google Search Essentials (external site)). For indexing and ranking, Google uses the smartphone version (external site) of your site. So Google can hardly take into account content that only exists in the desktop version.
How to do it:
- Give each service its own page that describes it in your customers’ words, with location, process and a point of contact.
- Open the “Manual actions” and “Security issues” reports in Search Console. A human reviewer at Google issues a manual action when pages violate the spam policies (external site).
- In the Removals tool, check whether anyone has hidden URLs from your site. Such a removal lasts about six months (external site), and you can revoke it.
- Search for
site:example.com casino. Results on topics you’ve never covered point to hacked pages. Google names this kind of search as a way to find spam on your own site (external site).
Example: A painting contractor describes all its services on one page headed “What we offer.” If someone searches for “exterior house painting” plus their town, Google finds neither the word “exterior” nor a place name on that page.
Common mistake: A nearly identical page is created for every town in the area, with only the place name swapped. Google counts pages targeted at specific cities that funnel visitors to a single page as spam (external site).
Google Maps: If by “showing up on Google” you mean the map, your Google Business Profile is what matters. According to Google, local results there depend mainly on relevance, distance and popularity (external site).
Your website shows up for your services, and inquiries still don’t come? Then see our how-to Why isn’t my website generating leads?Done when every service has its own meaningful page and the reports on manual actions, security issues and removals show no entries.
Why is your website missing from Bing?
Bing runs its own index, which Microsoft Copilot also relies on. If your website is missing there, the reasons are similar to Google’s. It’s new or has few links pointing to it, it’s blocked by robots.txt or noindex, or it doesn’t meet Bing’s quality threshold. The free Bing Webmaster Tools help you find out which.
Why: According to Microsoft, Bing and Copilot rely on the same core crawling, indexing and ranking foundation (external site) as traditional search. In Germany, Bing had a 19.74% desktop market share in October 2026, and 6.41% across all devices. Our how-to on getting recommended by ChatGPT explains why the Bing index also matters for other AI assistants.
Source: StatCounter (external site), · monthly figure, October 2026, desktop
Source: StatCounter, · monthly figure, October 2026, all devices
How to do it:
- Search Bing for
url:https://example.com/. If the page appears, it’s in the Bing index (external site). - Add your website to Bing Webmaster Tools (external site). If you’ve already set up Search Console, import your site and sitemaps from there (external site); the site is then verified automatically. The first data usually appears after 48 hours.
- Check your home page there with URL Inspection, and see whether anyone has blocked URLs with the Block URLs tool.
- Report new and changed pages through IndexNow, a protocol that tells search engines about changes right away. It doesn’t guarantee immediate indexing. According to IndexNow (external site), the participating search engines are Bing, Naver, Seznam.cz, Yandex and Yep; Google isn’t on the list. Our how-to Bing and IndexNow shows how to set up IndexNow and check your submissions.
Common mistake: A temporary redirect (302) stays in place for months. Bing advises using 302 only for changes lasting less than two days (external site).
Done when your site is verified in Bing Webmaster Tools, your sitemap is listed there and a url: search finds your home page.
What’s different for websites in Germany and the EU?
Websites in Germany and the EU, or aimed at audiences there, follow the same rules as anywhere else. Watch out, though, for privacy and security settings that can lock Googlebot out. These include country blocking in your firewall or at your host, consent banners that replace your content, and automatic redirects based on browser language.
- Country blocking: By default, Googlebot crawls from IP addresses in the US. According to Google, site owners should treat it like any other visitor from that country (external site). So if your firewall or your host blocks traffic from the US, it blocks these crawls too. Google advises checking your firewall rules for overly broad blocks (external site) of Google’s IP addresses.
- Consent banners: A banner that asks for consent should overlay your content. If your site instead redirects every request to a separate consent page, Googlebot can only fetch that one page. All the others then drop out of search results (external site). Google doesn’t see content that only loads after a click, because Search doesn’t interact with the page (external site).
- Language redirects: Googlebot doesn’t send a preferred language with its requests (Google on locale-adaptive pages (external site)). If your site automatically sends visitors to one version based on their browser language, Google may only see one of your versions. Google recommends a separate URL for each language, with hreflang annotations that link the language versions to each other.
Done when no country block stops requests from the US, your consent banner only overlays the content, and every language version has its own URL.
From our work: our indexing checks, step by step
For datafalke.de, we wrote a small Python script that runs the technical checks from this how-to one after another. It only fetches the website and changes nothing on it. The eight requests below are its core. You or your service provider can run them without the script: curl is already installed (external site) on every computer with Windows 10 or 11. Type curl.exe, because in Windows PowerShell, curl on its own starts a different command.
Replace example.com with your domain. If your main address is the www version, requests 2 and 3 work the other way around. On a Mac, type curl instead of curl.exe, and for request 5 use curl -s https://example.com/ | grep -iE "noindex|canonical".
curl.exe -sI https://example.com/
curl.exe -sI http://example.com/
curl.exe -sI https://www.example.com/
curl.exe -s https://example.com/robots.txt
curl.exe -s https://example.com/ | findstr /i "noindex canonical"
curl.exe -sI https://example.com/does-not-exist-4711/
curl.exe -s https://example.com/sitemap.xml
curl.exe -sI -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" https://example.com/| Request | Good result | Otherwise, see section |
|---|---|---|
| 1 · Home page | first line with status 200, no x-robots-tag line with noindex | Redirects and status codes; noindex |
| 2 · URL with http | status 301 or 308, location line with https://example.com/ | Redirects and status codes |
| 3 · URL with www | status 301 or 308 to the same URL as in request 2 | Redirects and status codes |
| 4 · robots.txt | no Disallow: / in the group that applies to Googlebot, plus a Sitemap line | robots.txt |
| 5 · noindex and canonical | no noindex; the canonical entry names the page’s own URL | noindex; canonical |
| 6 · Made-up URL | status 404 or 410 | Redirects and status codes |
| 7 · Sitemap | only https URLs that load without a redirect | New website |
| 8 · Request as Googlebot | the same result as request 1 | Germany and the EU |
If request 5 prints very long lines, search the page source as described in the noindex section. Request 8 only reveals blocks that target the crawler’s name. The real Googlebot crawls from IP addresses that Google publishes, and its user agent string can be spoofed (external site). So a 403 here may also come from protection against fake Googlebots. The live test in URL Inspection shows whether Google itself gets through.
For the websites we build, setting up Search Console and Bing Webmaster Tools is part of the launch. We’ll only publish our own figures, such as how many pages of datafalke.de are indexed, with the measurement log. Our methodology describes what it will include.