In short
You can track AI traffic without cookies or scripts in your server’s or CDN’s access logs. Visits from AI answers show up there with a referrer such as chatgpt.com or the parameter utm_source=chatgpt.com. Fetches by AI systems carry user agents such as ChatGPT-User; verify each fetch against the operator’s IP list. Then keep only the totals.
Key takeaways
- AI traffic covers two things: visits from people who click links in AI answers, and fetches by AI systems. Count them separately.
- You can spot visits by the referrer, such as chatgpt.com or www.perplexity.ai. ChatGPT also adds utm_source=chatgpt.com to its links.
- User-initiated fetches carry user agents like ChatGPT-User, Claude-User or Perplexity-User. Check each against the operator’s IP list.
- Google Analytics (GA4) automatically excludes known bots, so you’ll find fetches by AI crawlers in your server and CDN logs.
- Logs give a lower bound. Answers without a click are missing, and clicks from Google’s AI features blend in with the rest of Google Search.
What counts as AI traffic?
AI traffic covers two different things: visits from people who clicked a link in an AI answer, and fetches in which AI systems retrieve your pages themselves. You can tell visits by where they came from, and fetches by the user agent of the program making the request. Count the two separately, because a fetch isn’t a visit.
Fetches break down further by purpose:
| Type | What happens | How to spot it | Where it shows up |
|---|---|---|---|
| Visit from an AI answer | A person clicks a link to your website in ChatGPT, Perplexity, Microsoft Copilot, Gemini or Claude. | referrer; for ChatGPT, also the parameter utm_source=chatgpt.com | server and CDN logs, web analytics |
| User-initiated fetch | An AI assistant opens your page while answering someone’s question. | user agent such as ChatGPT-User, Claude-User or Perplexity-User | server and CDN logs |
| Agent | A browser on an AI provider’s servers completes a task on your website on someone’s behalf. | user agent such as Claude-Agent or Google-Agent; at OpenAI, a signature in separate headers | server and CDN logs; the standard log format doesn’t record OpenAI’s signature |
| Search crawler | A program builds the search index from which an AI assistant later picks its sources. | user agent such as OAI-SearchBot, Claude-SearchBot or PerplexityBot | server and CDN logs |
| Training crawler | A program collects content that may go into future language models. | user agent such as GPTBot or ClaudeBot | server and CDN logs |
The details on fetches come from the operators’ documentation:
- OpenAI’s crawler overview (external site) and ChatGPT Work’s Cloud browser (external site)
- Anthropic’s crawlers (external site) and Anthropic’s browsers acting on someone’s behalf (external site)
- Perplexity’s crawlers (external site) and Google’s user-triggered fetchers (external site)
Why keep them apart: A user-initiated fetch shows that an AI assistant opened your page for an answer. It doesn’t show whether the answer mentions you or whether anyone clicks. A visit shows the click, but not the question that came before it. Answers nobody clicks on leave no trace in your logs at all. To find out how often AI systems mention you in their answers, you therefore need a fixed set of questions, as our how-to on measuring AI visibility describes. Our guide to generative engine optimization (GEO) explains how AI systems choose their sources. What each crawler may do and how you control it is covered in our overview of AI crawlers.
Common mistake: Crawler fetches get counted as visits. A page then looks popular when it’s mostly programs fetching it.
Done when you’ve decided which types you count and you report each one separately.
Where can you find the data without cookies or scripts?
The data is in your server’s access logs and, if a CDN delivers your pages, in the CDN’s logs. Each line describes one request with its time, address, status, referrer and user agent. These logs are written without cookies or browser scripts. They also contain the bot fetches that web analytics tools such as Google Analytics filter out.
What a line contains: In nginx, the combined format is the default; in Apache, it’s one of the commonly used formats (see the documentation for nginx (external site) and Apache (external site)). This line shows a visit from ChatGPT, with made-up values:
192.0.2.10 - - [06/Nov/2026:09:12:01 +0100] "GET /?utm_source=chatgpt.com HTTP/2.0" 200 57159 "https://chatgpt.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36"| Field | Value in the example | What you need it for |
|---|---|---|
| IP address | 192.0.2.10 | checking whether a fetch really comes from its operator |
| Time | 06/Nov/2026:09:12:01 +0100 | assigning the request to a day and month |
| Request | GET /?utm_source=chatgpt.com | the page requested; for ChatGPT, with the parameter |
| Status | 200 | only pages delivered successfully count |
| Referrer | https://chatgpt.com/ | the referring website of a visit |
| User agent | Mozilla/5.0 (Windows NT 10.0 … | identifying a crawler, user-initiated fetch or agent |
Why the CDN matters: A CDN is the network that delivers your pages from servers close to your visitors. If a page comes from its cache, the request doesn’t need to reach your server (MDN on HTTP caching (external site)). The request then appears only in the CDN’s log. Bunny, the EU-based provider whose CDN delivers our website, keeps its logs for three days. For each request, the logs also show the cache status, country and network operator (Bunny’s documentation (external site)).
Why script-based analytics isn’t enough: Google Analytics 4 (GA4) automatically excludes traffic from known bots, and you can’t turn this off (Google Analytics Help on known bots (external site)). For fetches by AI crawlers, you therefore need the logs. GA4 groups visits from AI assistants in its “AI Assistant” channel. Google names ChatGPT, Gemini, DeepSeek, Copilot and Grok as examples. Google’s AI Overviews and AI Mode aren’t part of this channel (default channel groups in Google Analytics (external site)).
How to do it:
- Ask your hosting provider for your website’s access log, its format and how long it’s kept. If your host doesn’t provide logs, you’re left with the reports from Google and Bing, web analytics and asking people how they found you.
- If a CDN delivers your pages, turn on logging there and analyze those logs too.
- Check whether the logs contain the full IP address, a shortened one or none at all. That determines how you can verify fetches (step 5).
- Decide when your analysis runs. If your provider keeps the logs for only three days, run it daily.
Common mistake: Only the server log gets analyzed, even though the CDN delivers pages from its cache. Those requests are then missing.
Done when you know where your logs are, which fields they contain and how long they’re kept.
How can you spot visits from AI answers?
You can spot visits from AI answers by the referrer, meaning the referring website, such as chatgpt.com or www.perplexity.ai. ChatGPT also adds the parameter utm_source=chatgpt.com to links in its answers, and the parameter stays in the address even without a referrer. Clicks from Google’s AI Overviews and AI Mode can’t be separated from other Google Search clicks this way.
| Service | Referrer | Parameter in the address | Note |
|---|---|---|---|
| ChatGPT | chatgpt.com | utm_source=chatgpt.com | chat.openai.com redirects to chatgpt.com |
| Perplexity | www.perplexity.ai | not specified | perplexity.ai redirects to www.perplexity.ai |
| Microsoft Copilot | copilot.com, copilot.microsoft.com | not specified | copilot.microsoft.com redirects to copilot.com |
| Gemini | gemini.google.com | not specified | not specified |
| Claude | claude.ai | not specified | not specified |
| Google’s AI Overviews and AI Mode | same as Google Search | not specified | Google Search Console counts the clicks under the “Web” search type |
Measured · curl 8.21.0, requests to each service’s start address, redirects and Referrer-Policy header
Our data page on AI search in Germany shows how many visits AI assistants have sent to websites in Germany so far.
What the referrer reveals: If a website doesn’t set a policy of its own, the browser tells other websites only the referring site’s address, such as https://chatgpt.com/, without the path. That’s how the default policy, strict-origin-when-cross-origin, works, as MDN’s page on the Referrer-Policy header (external site) explains. When we checked, chatgpt.com and copilot.com sent exactly this policy. So you can see that someone came from ChatGPT, but not from which conversation. Individual links can override the policy and leave out the referrer, for example with rel="noreferrer".
What the parameter shows: In October 2026, OpenAI’s help page for publishers still said that ChatGPT automatically adds the parameter to its links (archived version, Oct 4, 2026 (external site)). On November 7, 2026, that page redirected to a different article. The parameter is part of the requested address itself, so it arrives even when the browser sends no referrer. Still, don’t rely on it alone. Profound, a company that sells measurement tools, observed that it was missing from many links from October 8 to 16, 2026. For about 27 hours, only 5–12% of answers with source links included a tagged link, compared with around 99% before and after.
Source: Profound (external site), · vendor study · answers with source links, analyzed in 30-minute intervals
What’s different with Google: Search Console counts clicks on links in AI Overviews and AI Mode as regular clicks in its Performance report, under the “Web” search type (Google Search Central on AI features (external site); Search Console Help on how clicks are counted (external site)). Google doesn’t name a separate identifier in the referrer or address for these clicks. In your logs, they therefore can’t be separated from other clicks from Google Search. How often your pages appear in these answers is shown in the generative AI performance report, which our section on the Google and Bing reports describes.
How to do it:
- Count page views with a referrer from the table, per service and per page.
- Also count requests with utm_source=chatgpt.com but no matching referrer as visits from ChatGPT. Each request counts only once.
- Only analyze pages delivered with status 200 or 304, not images or stylesheets. Status 304 means the browser already had the page and only checked whether it had changed (RFC 9110 on status 304 (external site), section 15.4.5).
- Review your list of services every few months. Addresses change, as Copilot’s move to copilot.com shows.
For a first look, three commands in a Linux or macOS terminal are enough:
grep -c '"https://chatgpt.com/' access.log
grep -c 'utm_source=chatgpt.com' access.log
grep -c 'ChatGPT-User' access.logThese commands count every matching line, including error pages. On Windows, (Select-String -Path access.log -Pattern 'ChatGPT-User').Count does the same in PowerShell. The script under “From our work” counts more precisely.
Common mistake: Only the referrer gets analyzed, so visits where only the parameter arrives are missing. If, on the other hand, you add up referrers and parameters separately, you count some visits twice.
Done when your analysis counts visits per AI service and entry page, and counts each request only once.
How can you spot fetches by AI systems?
You can recognize fetches by AI systems from the name in the user agent, the field in which a program identifies itself. For fetches triggered by someone’s question, OpenAI, Anthropic and Perplexity use ChatGPT-User, Claude-User and Perplexity-User. Search for the name without the version number, because other parts of the string can change.
| User agent | Operator | When the fetch happens | Operator’s IP list |
|---|---|---|---|
| ChatGPT-User | OpenAI | when ChatGPT or a Custom GPT opens a page to answer a question | openai.com/chatgpt-user.json |
| Claude-User | Anthropic | when Claude opens a page to answer a question | claude.com/crawling/bots.json |
| Perplexity-User | Perplexity | when Perplexity visits a page for an answer | www.perplexity.ai/perplexity-user.json |
| Claude-Agent | Anthropic | when an Anthropic-hosted browser completes a task on someone’s behalf, for example in Claude Duet | claude.com/.well-known/sail |
| Google-Agent | when an agent on Google’s infrastructure navigates the web and takes actions on someone’s behalf | user-triggered-agents.json under developers.google.com/static/crawling/ipranges/ | |
| Signature-Agent header set to https://chatgpt.com | OpenAI | when ChatGPT Work’s Cloud browser opens a page; to identify it, OpenAI names the signature, not a user agent | keys under chatgpt.com/.well-known/http-message-signatures-directory |
Why user-initiated fetches are revealing: They show which pages AI assistants are currently opening for answers. In our assessment, no other trace in your logs shows as directly which content AI answers draw on. But a fetch doesn’t prove a citation: it doesn’t tell you whether the answer names your page. Whether a fetch stands for one question or several isn’t described in the documentation from OpenAI, Anthropic or Perplexity. So the number of fetches isn’t a measure of the number of questions.
Search and training crawlers: OAI-SearchBot, Claude-SearchBot and PerplexityBot build the assistants’ search indexes, and GPTBot and ClaudeBot collect content for training. For the AI features in Google Search, Googlebot (external site) fetches your pages. Microsoft also uses content its crawlers collect for answers in Copilot (Microsoft Bing, Oct 30, 2026 (external site)); its standard crawler is Bingbot (external site). More user agents, with their purpose and what blocking them does, are in our list of AI crawlers.
How to do it:
- Search the user agent string for the name, ignoring case.
- Assign each user agent to a type: user-initiated fetch, agent, search or training.
- Count user-initiated fetches and agents per page. Compare the list with the pages you want AI assistants to cite.
- Verify each fetch as described in step 5.
Example: This is how ChatGPT-User identifies itself, according to OpenAI. Search only for “ChatGPT-User”:
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/botCommon mistake: You search for the full string, version number included. With the next version, every fetch drops out of your count.
Done when every user agent is assigned to a type and your analysis shows user-initiated fetches per page.
How can you verify that a fetch is genuine?
Check the IP address of every fetch against the list its operator publishes. Anyone can fake the name in the user agent, as Google, Microsoft and Anthropic point out themselves. If your logs don’t contain the IP address, a fetch can’t be verified. Report such fetches separately.
The lists share a format: Google, OpenAI, Perplexity, Anthropic and Microsoft publish their IP ranges as JSON files. Each file contains a list called prefixes, and each entry gives a range under ipv4Prefix or ipv6Prefix. The addresses for Claude-Agent are in a separate file in the SAIL format. According to Anthropic, this format is still a draft and not yet a standard. Download the lists again before every analysis. Microsoft advises refreshing the Bingbot list daily because it can change at any time (Bing on verifying Bingbot (external site)). For Claude-Agent, Anthropic recommends updating your rules from its file automatically.
Measured · curl 8.21.0 and Python 3.12, compared the structure of 10 IP lists and the SAIL file
Without an IP address: Bunny, for example, can shorten IP addresses or remove them entirely when you retrieve the logs. That leaves the network operator as a clue: for each request, Bunny provides the autonomous system number (ASN) of the network and the organization that owns it. OpenAI publishes its own address ranges for ChatGPT-User. If a fetch with this user agent comes from a residential internet provider’s network, it’s unlikely to be from OpenAI. This test doesn’t replace a check against the IP list.
For Googlebot and Bingbot, you can also verify requests with a reverse DNS lookup (Google on verifying its crawlers (external site)). How this works and where every operator publishes its lists is covered in our section on verifying genuine AI crawlers.
How to do it:
- Download the IP lists of the operators whose user agents appear in your logs.
- During the analysis, check whether the IP address falls within one of the ranges. Record only the result: match, no match or can’t be verified.
- Then discard the IP address.
- Report the three results separately. Only report fetches as coming from a provider if their IP address matches.
Common mistake: A fetch is counted on the strength of its user agent alone. Faked fetches then end up in your statistics.
Done when every fetch you count carries one of the three results.
How can you analyze logs with as little data as possible?
Analyze the logs regularly and keep only totals afterward: visits per AI service and page, and fetches per user agent, page and check result. You need the IP address only to verify fetches, and the referring website only by its name. Don’t count visitors or sessions, because that would mean linking requests to each other.
This analysis stores nothing in your visitors’ browsers and reads nothing from them. It works with the information that the browser sends to your server with every request.
How to do it:
- IP address: Verify fetches during the analysis, then discard the address. If you’d rather not handle IP addresses at all and your CDN offers it, have them shortened or removed when you retrieve the logs. You can then verify fetches only by their network operator.
- Referrer: Keep only the name of the referring website, such as chatgpt.com, without the path.
- Parameters: Discard all parameters in the address except utm_source with a value that an AI service verifiably sets. Other parameters can contain identifiers, for example from partner or campaign links (Guidelines 2/2023 of the European Data Protection Board (external site), para. 49).
- Raw data: Store daily totals and delete the raw lines after the analysis.
- No linking: Don’t create an identifier for visitors, not even a hash of the user agent and other browser details. The German Data Protection Conference (DSK) describes such a value as fingerprinting (guidance for digital services (external site), para. 23, in German).
What belongs in your monthly report:
- visits from AI answers per service, and the 10 most common entry pages
- user-initiated fetches and agents per user agent and page, split by check result
- fetches by search and training crawlers per user agent
- the share of fetches that couldn’t be verified
Compare months with caution. A rise can also come from more people using AI assistants. How to separate this from the effect of your own changes is explained in the section on visits and fetches of our how-to on AI visibility.
Common mistake: Raw logs are kept for months, in case you want to analyze them later.
Done when your analysis stores only totals and you know when the raw data gets deleted.
What don’t the logs show?
Logs only show what reaches your server or CDN. Anyone who reads an AI answer and doesn’t click is missing from them. Visits without a referrer or parameter look like direct visits, and some agents can’t be recognized in standard logs. So treat figures from logs as a lower bound.
- No referrer: Some addresses come from a source that has no web address of its own, such as a bookmark or the keyboard. In that case, the HTTP standard says the browser doesn’t send a usable referrer (RFC 9110 on the referrer (external site), section 10.1.3). In our assessment, this also applies to links that an app hands over to the browser. A service can also suppress the referrer with its Referrer-Policy.
- Known gaps: ChatGPT’s parameter was missing from many links from October 8 to 16, 2026, and clicks from Google’s AI features can’t be separated from the rest of Google Search (step 3). So a drop in your statistics can be a measurement error.
- Agents you can’t spot by user agent: For ChatGPT Work’s Cloud browser, OpenAI only describes identification through a signature in separate headers. The combined format stores none of this. The browser only becomes visible once your server or CDN checks or logs the signature.
- No visitor numbers: Without linking requests, you count page views, not people.
What happens without a click is best captured by a fixed set of questions for AI systems and by asking new contacts how they found you. Our how-to on AI visibility describes both.
Common mistake: The number of visits from AI answers is taken as a measure of the entire impact of AI answers.
Done when your report labels its figures as a lower bound and says which sources are missing.
From our work: what datafalke.de logs and how to run your own count
Our website sets no cookies, uses no analytics or tracking scripts and loads no content from other providers. That’s why it has no cookie banner. Our pages send the security policy script-src 'none', which tells the browser not to allow any JavaScript on the page (MDN on script-src (external site)). You can check this yourself:
curl -s -D - -o /dev/null https://datafalke.de/On Windows, type curl.exe instead of curl in PowerShell and replace /dev/null with NUL. On November 8, 2026, the response contained these lines and no Set-Cookie line:
Content-Security-Policy: script-src 'none'; object-src 'none'; base-uri 'self'; form-action 'self'; frame-ancestors 'none'; upgrade-insecure-requests
Referrer-Policy: strict-origin-when-cross-originMeasured · curl 8.21.0, headers of the German home, contact, methodology, privacy and insights pages
What our host logs: Each time a page is requested, Bunny processes the IP address and other connection data, and it deletes its logs after three days. Logging is set up so that every log we retrieve comes without IP addresses: Bunny removes the IP address completely before we receive the data. We haven’t turned on a log archive or forwarding to other services.
What isn’t running yet: We don’t currently use any audience measurement. Before we start counting from server logs, we’ll add this to our privacy policy. How we’ll count is set out in our measurement principles. We’ll only publish figures in our measurement log. Our contact form already asks, as an optional question, how you heard about us.
How to run your own count: The following Python script counts AI traffic in access logs in the combined format. It needs only the standard library and outputs nothing but totals. It checks IP addresses against the operators’ IP lists in memory only. Save the lists under their file names in one folder; steps 4 and 5 say where to get them.
#!/usr/bin/env python3
"""Counts AI traffic in access logs in the "combined" format (Apache, nginx).
Usage: python3 count-ai-traffic.py access.log [more.log.gz ...] [--lists FOLDER]
Prints totals only. The script checks IP addresses in memory against the operators'
IP lists in the --lists folder; it neither prints nor stores them.
Download the lists from the operators; the file names are in USER_AGENTS.
User agents and addresses as of Nov 7, 2026. License: MIT-0 (datafalke.de).
"""
import argparse
import gzip
import ipaddress
import json
import re
from collections import Counter
from pathlib import Path
from urllib.parse import parse_qs, urlsplit
LINE = re.compile(r'(\S+) \S+ \S+ \[[^\]]*\] "(\S+) (\S+)[^"]*" (\d{3}) \S+ "([^"]*)" "([^"]*)"')
# Addresses at which AI assistants run in the browser (referrer)
SERVICES = {"chatgpt.com": "ChatGPT", "perplexity.ai": "Perplexity", "gemini.google.com": "Gemini",
"copilot.microsoft.com": "Copilot", "copilot.com": "Copilot", "claude.ai": "Claude"}
# Token in the user agent string: type of fetch and file with the operator's IP ranges
USER_AGENTS = {
"ChatGPT-User": ("user-initiated", "chatgpt-user.json"),
"Claude-User": ("user-initiated", "bots.json"),
"Perplexity-User": ("user-initiated", "perplexity-user.json"),
"Claude-Agent": ("agent", "sail"),
"Google-Agent": ("agent", "user-triggered-agents.json"),
"OAI-SearchBot": ("search", "searchbot.json"),
"Claude-SearchBot": ("search", "bots.json"),
"PerplexityBot": ("search", "perplexitybot.json"),
"Googlebot": ("search", "common-crawlers.json"),
"bingbot": ("search", "bingbot.json"),
"GPTBot": ("training", "gptbot.json"),
"ClaudeBot": ("training", "bots.json"),
}
def load_networks(file):
"""IP ranges from an operator's JSON list or from Anthropic's SAIL file."""
text = file.read_text(encoding="utf-8")
if file.suffix == ".json":
entries = [p.get("ipv4Prefix") or p.get("ipv6Prefix") for p in json.loads(text)["prefixes"]]
else:
entries = re.findall(r'"([0-9A-Fa-f:.]+(?:/\d+)?)"', text)
networks = []
for entry in entries:
try:
networks.append(ipaddress.ip_network(entry, strict=False))
except (TypeError, ValueError):
pass
return networks
def check(ip, networks):
if networks is None:
return "no list"
try:
address = ipaddress.ip_address(ip)
except ValueError:
return "no IP"
if address.is_unspecified: # 0.0.0.0 or :: if the CDN has removed the IP
return "no IP"
return "IP match" if any(address in network for network in networks) else "IP mismatch"
def is_page(path):
last = path.rsplit("/", 1)[-1]
return path.endswith("/") or last.endswith(".html") or "." not in last
def main():
parser = argparse.ArgumentParser(description="Count AI traffic in access logs")
parser.add_argument("logs", nargs="+")
parser.add_argument("--lists", type=Path, help="folder with the operators' IP lists")
args = parser.parse_args()
networks = {}
for name in {ip_list for _, ip_list in USER_AGENTS.values()}:
if args.lists and (args.lists / name).is_file():
networks[name] = load_networks(args.lists / name)
visits, visited_pages, fetches, fetched_pages = Counter(), Counter(), Counter(), Counter()
for log in args.logs:
opener = gzip.open if log.endswith(".gz") else open
with opener(log, "rt", encoding="utf-8", errors="replace") as file:
for line in file:
match = LINE.match(line)
if not match:
continue
ip, method, target, status, referrer, user_agent = match.groups()
url = urlsplit(target)
token = next((t for t in USER_AGENTS if t.lower() in user_agent.lower()), None)
if token:
kind, ip_list = USER_AGENTS[token]
fetches[(kind, token, check(ip, networks.get(ip_list)))] += 1
if kind in ("user-initiated", "agent"):
fetched_pages[(token, url.path)] += 1
continue
if method != "GET" or status not in ("200", "304") or not is_page(url.path):
continue
host = urlsplit(referrer).hostname or ""
service = next((s for a, s in SERVICES.items() if host == a or host.endswith("." + a)), None)
if service:
visits[(service, "referrer")] += 1
elif "chatgpt.com" in parse_qs(url.query).get("utm_source", []):
visits[("ChatGPT", "utm_source only")] += 1
else:
continue
visited_pages[url.path] += 1
print("Visits from AI answers (page views)")
for (service, route), count in sorted(visits.items()):
print(f" {service:<12} {route:<15} {count:>6}")
print("Pages with the most visits from AI answers")
for path, count in visited_pages.most_common(10):
print(f" {count:>6} {path}")
print("Fetches by AI systems and search engines")
for (kind, token, result), count in sorted(fetches.items()):
print(f" {kind:<15} {token:<17} {result:<11} {count:>6}")
print("Pages with the most user-initiated fetches and agents")
for (token, path), count in fetched_pages.most_common(10):
print(f" {count:>6} {token:<16} {path}")
if __name__ == "__main__":
main()This is what the output looked like for a test file with 18 made-up lines, including the line from step 2:
python3 count-ai-traffic.py access.log --lists lists
Visits from AI answers (page views)
ChatGPT referrer 1
ChatGPT utm_source only 1
Copilot referrer 1
Gemini referrer 1
Perplexity referrer 1
Pages with the most visits from AI answers
2 /
1 /services/geo/
1 /insights/
1 /contact/
Fetches by AI systems and search engines
agent Claude-Agent IP match 1
agent Google-Agent IP match 1
search Googlebot IP match 1
search OAI-SearchBot IP match 1
training ClaudeBot IP match 1
training GPTBot IP match 1
user-initiated ChatGPT-User IP match 1
user-initiated ChatGPT-User IP mismatch 1
user-initiated Perplexity-User IP match 1
user-initiated Perplexity-User no IP 1
Pages with the most user-initiated fetches and agents
1 ChatGPT-User /services/geo/
1 ChatGPT-User /insights/
1 Perplexity-User /
1 Perplexity-User /services/geo/
1 Claude-Agent /contact/
1 Google-Agent /services/The test file also contains a faked user agent (IP mismatch), a line without an IP address and a request from ChatGPT with status 404, which doesn’t count as a visit. The script only reads the combined format. If your CDN delivers logs as JSON, as some do, convert them first. The script doesn’t run reverse DNS lookups or download the IP lists itself. Add new user agents to USER_AGENTS.