Your Playwright Fleet Loses 92 MB per Task — a Supervisor That Fixes Leaks, Crashes, and Zombie Browsers

Every guide about Playwright at scale talks about anti-bot detection. Almost none talk about what actually kills long-running browser automation on a $5 VPS: the fleet rotting. Contexts that never quite get closed. Browsers that survive their own crashes as half-orphaned process trees eating half a gigabyte. Tasks that hang forever because the sync API has no cancellation. This article builds the piece that production setups have and tutorials skip: a fleet supervisor — and every number in it comes from a live run on this exact class of box (4 GB RAM, 2 vCPU). ...

November 13, 2026 · 17 min · 3577 words

The Cheapest Loan Nobody Shops For: Measuring SPX Box-Spread Financing Against Broker Margin (Tested Python)

If you ask a retail trader what their money costs, they’ll quote their broker’s margin rate and stop thinking. Nobody shops for the loan because they don’t know the loan exists as a market. It does: a box spread on SPX is a bond you assemble from four options, repayable in cash at a date you pick, priced continuously by the deepest option market on earth. On November 12, 2026, I pulled the entire SPX chain from CBOE’s free delayed feed — 31,016 contracts, no API key — and measured what it charges. The short end printed 3.86%. The long end, 435 days out, printed 5.16%, versus roughly 5.39% for the first tier of Interactive Brokers’ margin schedule. The gap is thin, and the interesting part is exactly where it opens and closes. ...

November 12, 2026 · 11 min · 2317 words

The Other Side of the Anti-Bot Arms Race: A Tripwire Layer That Fingerprints Bots by What They Can't Resist (Tested Python)

If you read this blog, you’ve probably read the other side of this arms race from us: the JA4 TLS fingerprint that outed a scraper’s handshake, the HTTP/2 SETTINGS layer Akamai checks when JS is clean, the session pool that keeps a login alive across hundreds of requests. Those posts are written from the attacker’s chair because that’s where the engineering leverage is when you collect data. There’s a second market on the same shelf, and almost nobody writes for it: every business that’s being scraped, form-spammed, and path-probed and has no idea which of the 53% of requests hitting it are not customers. Imperva’s 2026 Bad Bot Report puts automated traffic at more than 53% of all web traffic in 2025, up from 51% the year before (imperva.com/blog/bad-bot-report-2026); the 2025 edition already counted 37% as bad bots — the hostile slice, not the search crawlers (imperva.com 2025 Bad Bot Report). If you sell engineering services, that’s a bigger addressable market than scraping itself: every scraped site is a potential client for the thing this article builds. ...

November 12, 2026 · 11 min · 2341 words

Your LLM Pipeline Pays Full Price for Documents It Has Already Read: A Near-Duplicate Gate That Doesn't

Every pipeline that feeds an LLM — agent task feeds, RAG ingestion, scraped corpora, per-item classification queues — leaks money the same way: the same content, or content within a few percent of it, arrives twice, and you pay model prices for both copies. Retry loops re-submit the same payload. Crawlers re-fetch pages whose only change is a rotated ad slot. Syndicated content enters through three feeds with different boilerplate wrappers. Teams that run agent-style workflows get hit hardest, because an agent’s own intermediate outputs cycle back into its input feed. Gartner expects over 40% of agentic AI projects to be scrapped by the end of 2027, with escalating cost the number-one reason — and “paying flagship token prices for content you have already fingerprinted once” is exactly the kind of cost failure they mean. ...

November 12, 2026 · 18 min · 3802 words

Your Scraper Doesn't Need More Retries. It Needs a Congestion Controller.

Most scrapers don’t fail because they get blocked. They fail because they retry their way into a ban. The developer adds exponential backoff with jitter — the AWS-approved pattern — and the scraper still ends up with a permanently 403’d identity two hours into the run. The backoff formula isn’t the problem. The problem is that backoff is a per-request reflex, while bans are a session-level judgment. Two different control loops, and most codebases only implement the first one. ...

November 12, 2026 · 12 min · 2458 words

Your Scraper Is Throttled by the Client, Not the Target: A Throughput Engineering Guide (Tested aiohttp vs httpx vs requests)

Every scraping article talks about TLS fingerprints, proxy rotation, and captcha solvers. Almost none talk about the thing that kills 80% of scrapers before anti-bot even notices them: the HTTP client itself. You can have a perfect JA4 hash, a residential IP, and a clean browser signature — and still crawl at one-quarter of your machine’s capacity because your client library is the bottleneck. This article benchmarks the three Python HTTP clients that matter — requests, aiohttp, and httpx — on a controlled local server, then builds a production-grade async fetcher with bounded concurrency, per-host rate limiting, exponential backoff, and a circuit breaker. Every line of code was executed on the machine writing this article before it went in. No invented benchmarks. ...

November 11, 2026 · 10 min · 1950 words

Gmail, Yahoo and Microsoft Now Reject Non-Compliant Senders — This Python Auditor Checks Any Domain in Half a Second

Google, Yahoo and Microsoft now permanently reject non-compliant bulk senders: 5,000+ messages/day to personal accounts requires SPF + DKIM, a DMARC record, complaints under 0.3%, one-click unsubscribe — Google from February 2024, Microsoft May 5, 2025. The FBI logged $3.05 billion in business email compromise losses in 2025; per a 5.5-million-domain study (October 2026), only 30.4% of domains publish DMARC — 12.8% enforce a policy that blocks spoofing. That gap is a business. Agencies charge $2,000–$5,000 for deliverability audits (InboxArmy’s published rates, cited by ITWire, October 2026); the work is mostly reading DNS against a rulebook, and DNS is free. This article builds the auditor: one Python script, no API keys, that parses SPF including the redirect= trap, measures DKIM key strength by walking the DER, parses DMARC per RFC 7489 with org-domain fallback, and scores it against the 2024–2026 rulebook. Every line ran clean before publication — mock tests, 15 unit assertions, live DNS queries, all exit 0. ...

November 11, 2026 · 12 min · 2389 words

Your Scraper Needs Japan at 3 A.M. Self-Hosted Geo Egress With WireGuard and HAProxy

Most teams that need geo-specific egress never do the math. They sign up for a residential proxy panel, click “Germany”, and pay the meter. Bright Data lists residential proxies starting at $5.88/GB [1]. At 2 TB/month that is not an infrastructure line item, it is a second payroll. But “geo-targeting” and “residential” are two different problems that proxy vendors bundle into one invoice. If your target is an e-commerce site, a search frontend, or a media catalog, what it actually checks is: does this IP’s registered country match the catalog the user expects, and does the IP look like a hosting customer instead of a credential-stuffing farm? The first half is pure topology — an IP registered to a German netblock, geolocating to Germany. That you can own outright. ...

November 11, 2026 · 16 min · 3277 words

HTTP 402 Went Live: Build a Machine-Payable API with x402 (Tested Python + the Fee Math Cards Can't Touch)

In 1991, the authors of what became HTTP reserved status code 402 — Payment Required — and wrote three words that would sit unused for a third of a century: “reserved for future use.” In May 2025, Coinbase and Cloudflare shipped the thing that finally used it: x402, a protocol that turns a plain 402 response into a machine-readable price quote and settles the payment in USDC on the retry. By July 2026 the x402 Foundation was operating under the Linux Foundation with 40 member organizations — Visa, Mastercard, Stripe, Google, AWS, Cloudflare and American Express among the 17 premier members (finance.yahoo.com). ...

November 11, 2026 · 12 min · 2549 words

Your REST Poller Misses Half the Tape: WebSocket and gRPC Market-Data Streams, Built and Benchmark-Tested

Every data product I’ve inherited from a client has the same architecture: a Python loop that hits a REST endpoint every N seconds, deduplicates, writes to Postgres, and quietly loses money in the gap between polls. The gap looks harmless in dev, where the feed is slow and nobody’s trading against it. In production, it’s the difference between capturing an event and reading about it two seconds later — and it costs you 15x the request volume for the privilege. ...

November 11, 2026 · 13 min · 2730 words

Your Scraper Logged In Once and Never Again: A Session Pool That Survives Login Walls, Challenges, and Token Rot

Every scraping guide covers proxy rotation. Almost none cover what happens after the login succeeds — because most scrapers never get that far. The data that pays — B2B pricing feeds, marketplace inventory behind seller dashboards, authenticated API endpoints with rate limits tied to accounts — sits behind a login wall. Keeping 50 authenticated sessions alive across a six-hour scrape, through anti-bot challenges and token expirations, is where production scrapers die. ...

November 10, 2026 · 11 min · 2274 words

The Hedge Budget: Google's Tail-Fix for Latency, Priced So It Doesn't Melt Your API Bill

Your automation pipeline has 50,000 requests to get through tonight and a hard deadline at 6 a.m. The median latency is 28ms. The math says done by 2 a.m. At 4:40 a.m. you’re still waiting, because 0.1% of your calls — 50 of them — took two seconds each, they were all on your critical path, and no dashboard you configured ever showed them. The average is a lie your slowest requests tell in unison. ...

November 10, 2026 · 18 min · 3656 words