Bot detection

​

Overview

// What the site does (runs very early, before your code):
const _fetch = window.fetch;
window.fetch = function(...args) {
  const stack = new Error().stack;
  if (!isExpectedCallSite(stack)) {
    reportAnomaly({ url: args[0], stack });
  }
  return _fetch.apply(this, args);
};

​

Identifying bot detection on your target site

// Check if fetch has been wrapped
window.fetch.toString()
// Native (safe):     "function fetch() { [native code] }"
// Patched (flagged): will show actual JavaScript source

// Check XMLHttpRequest
XMLHttpRequest.prototype.open.toString()
// Native: "function open() { [native code] }"

// Check for property descriptor tampering
Object.getOwnPropertyDescriptor(window, 'fetch')
// Native: { value: ƒ, writable: true, enumerable: true, configurable: true }
// Tampered: may have getters/setters or different configurability

try {
  const desc = Object.getOwnPropertyDescriptor(window, 'fetch');
  console.log('configurable:', desc.configurable);
  console.log('writable:', desc.writable);
  console.log(window.fetch instanceof Function); // should be true
  console.log(window.fetch.prototype); // native fetch has no prototype
} catch (e) {
  console.log('fetch access is trapped');
}

// Check if common event listeners are heavily registered
getEventListeners(document)
// In Chrome DevTools, this shows all listeners. An unusually large number
// of mousemove, keydown, scroll, and touchstart listeners suggests telemetry.

// Check for known telemetry globals
// PerimeterX:
typeof window._pxAppId !== 'undefined'
// Akamai:
typeof window.bmak !== 'undefined'
// DataDome:
typeof window.ddjskey !== 'undefined'

import { chromium } from 'playwright';
const browser = await chromium.launch({ headless: false });
const page = await browser.newPage();
await page.goto('https://target-site.com');
// If you get a challenge page, CAPTCHA, or block — bot detection is active.

​

Infrastructure considerations

• Cloud provider IP ranges (AWS, GCP, Azure) are well-known and flagged by most bot protection services. Requests from these ranges face higher scrutiny or outright blocking regardless of browser fingerprint quality.
• Even without bot detection, sites enforce per-IP request limits. Hitting the same site too frequently from one IP triggers throttling or temporary bans.
• If your IP geolocates to one region but your browser reports a timezone and locale from another, that inconsistency is a signal.
• Residential proxy services provide IP addresses from real ISPs, making requests appear to originate from normal households. Rotating proxies distribute requests across many IPs to avoid rate limits.

• reCAPTCHA v2: The checkbox or image-selection challenge. Can sometimes be bypassed in automated browsers if the risk score is low enough (it evaluates browser fingerprint and behavior first).
• reCAPTCHA v3: Invisible — returns a score from 0.0 to 1.0 with no user interaction. A well-fingerprinted browser with natural behavior scores higher.
• hCaptcha: Similar to reCAPTCHA v2. Cloudflare uses it as a fallback.
• Cloudflare Turnstile: Non-interactive challenge that evaluates browser signals. Replaces traditional CAPTCHAs on many Cloudflare-protected sites.

• Soft blocks: The site returns degraded results (fewer items, missing data, slower responses) without an explicit error. These are hard to detect — you may not realize you’re getting incomplete data.
• Hard blocks: HTTP 403, CAPTCHA pages, “Access Denied” responses, or redirects to a challenge page.
• Cookie consent and GDPR banners: Not bot detection, but a common obstacle. These overlays block interactions with the underlying page and must be detected and dismissed before proceeding.

​

Related pages