Your Scraper's TLS Handshake Is a Confession: JA4, Akamai, and the Fix That Actually Works
Your scraper sends a perfect User-Agent header, a perfect header order, residential proxies, the works. The request still gets a 403 from a site that loads fine in Chrome. The usual advice — “rotate your proxies” — misses the actual mechanism, and understanding the mechanism is worth real money, because it decides whether your data pipeline costs you a $1,500/month unblocker API or a few cents of compute. The mechanism is TLS fingerprinting. Before your scraper sends a single HTTP header — before the server even knows which URL you want — your client completes a TLS handshake, and that handshake describes your software in detail. Akamai, Cloudflare, AWS WAF, and Google Cloud Armor all read it. In this article we’ll reproduce the JA4 algorithm from the official spec byte-for-byte, capture the fingerprints of real Chrome, python-requests, and curl-impersonate on live requests, and run the decisive experiment: identical HTTP requests to a real Akamai-protected site where the only variable is the handshake — one gets 403, the other 200, every time. ...