The endpoint is right there. You can see the request in your proxy: POST /api/v1/orders, a JSON body, three headers you’ve never seen on a web API — X-Client-TS, X-Nonce, X-Signature. You replay it byte-for-byte from a script and get a 403 Forbidden. You change one character of the body and the signature still fails. The mobile app, meanwhile, happily places orders all day.
That is request signing, and it is the single most common reason mobile API automation projects stall. Not Captchas, not IP bans — signing. It is also one of the highest-value skills a data-engineering freelancer can sell, because almost no one who can build scrapers can get past it.
This article walks the full process against a working signed API: capture the traffic with mitmproxy, extract the signing secret from the app, confirm the exact canonical scheme with a verification add-on, and rebuild the client in Python — with every failure mode (replay, tamper, stale timestamp) demonstrated live. The full harness ran to completion on this machine before this article was written: 10/10 pytest cases passed, exit code 0.
The scheme you’re about to meet
Strip away the branding and nearly every HMAC-signed mobile API converges on the same construction, familiar from AWS SigV4 (which its documentation describes as signing a canonical request so the server can verify every component independently):
string_to_sign = METHOD \n PATH \n TIMESTAMP \n NONCE \n SHA256(body)
signature = HMAC-SHA256(shared_secret, string_to_sign)
Three gates ride on top of it, and all three must be satisfied simultaneously:
- Freshness — the timestamp must sit inside a window of server time (AWS S3’s classic tolerance: 15 minutes; mobile apps typically use 300 seconds).
- Anti-replay — each nonce is stored server-side after first use; a second request with the same nonce dies even with a valid signature.
- Integrity — the body hash inside the signature means you cannot alter the payload after signing; one flipped character and the HMAC diverges completely.
The replay window is not decoration — it is the exact mechanism that kills naive automation. Understanding which gate returned the 403 is most of the debugging work, and that is what the harness below makes visible.
Ground rules, stated plainly
Everything below was executed against MockShopverseBackend — a Flask service, bundled with this article’s code, that enforces the exact scheme above. It stands in for a real app’s gateway so the whole exercise is reproducible offline. No third-party API is touched, no keys are used, and the secret below is a synthetic fixture, not a leak from any real application.
The workflow itself — proxy the traffic, dump the APK, extract the secret, confirm the scheme, rebuild the client — is what you’d run against a real target. Do it only against apps you own, are contractually testing, or have program authorization for. Pointing it at someone’s production app without permission is computer misuse in most jurisdictions, and the defensive write-up at the end is why this article exists in the first place.
Step 1 — see the traffic
Install mitmproxy (pip install mitmproxy), run mitmdump, and point the device at it — real device over Wi-Fi proxy settings, or an emulator with -http-proxy. TLS interception needs the mitmproxy CA cert installed on the device. One in five apps will work out of the box; the rest pin certificates or refuse user-installed CAs.
That refusal is deliberate: since Android 7.0 (API 24), per the Android developer documentation on Network Security Configuration, apps no longer trust user-installed CA certificates by default — they only trust what the developer opts into. Getting past it for a test build is standard practice: repackage the APK with a debuggable flag and a custom network security config, or hook the trust manager at runtime with Frida or objection. For a production build you don’t control, that is where the legal line sits.
Assuming you’re on your own test build, the signed requests now stream through your proxy:
[recon] POST /v1/enrichment ts=1794150489 nonce=632f821dc6a3dedc
[recon] MATCH -> scheme: newline-joined + body sha256
127.0.0.1:41314: POST http://127.0.0.1:5013/v1/enrichment
<< 200 OK 89b
That MATCH line is the money shot, and it’s produced by the add-on we build next.
Step 2 — get the secret
An HMAC is only as strong as its key. When the key is baked into the app, the HMAC protects nothing from someone holding the APK — it just obfuscates the protocol. This is the industry’s open secret: it’s why the OWASP Mobile Top 10 and every serious mobile pentest methodology treat client-side key storage as a risk category of its own, and why hardware-backed keystores exist as a mitigation.
Extraction paths, fastest first:
- Strings/jadx pass —
jadxthe APK, grep forHMAC,hmacSha256,SecretKeySpec. Lazy developers hardcode the key right there. - Frida hook on
javax.crypto.spec.SecretKeySpec— dump every secret as the app constructs it. Works even when the key is obfuscated or assembled at runtime from fragments. - objection (
pip install objection) — wraps the Frida hooks behind a one-liner:objection -g com.vendor.app explore, thenandroid hooking watch class javax.crypto.spec.SecretKeySpec. - Manual dynamic analysis — run the app under a debugger and read the key buffer when signing is invoked.
For our mock backend the secret is a fixture written to disk at startup:
DEV_SECRET = b"7f3a9c1e55b2f8d04a6e93c7" # what mitmproxy/objection pulls out of the APK
In a real engagement this line is the output of your Frida dump. Everything downstream is identical whether the secret came from a mock or a real APK.
Step 3 — the scheme-confirmation add-on
Knowing the secret isn’t enough — you still don’t know how the app builds the string it signs. Get the canonical construction wrong by a single newline and every signature fails. Rather than guess blindly, feed every observed signed request through a scheme-confirmation loop. Save this as mitm_recon.py:
"""
mitm_recon.py -- mitmproxy add-on that logs every URL and, when a signed
mobile API request matches a guessed canonical scheme, verifies the
signature live. Run with:
mitmdump -s mitm_recon.py --ssl-insecure
"""
import hashlib
import hmac
import json
# Paste the extracted secret here once Frida/objection dumps it from the APK
SECRET = b"7f3a9c1e55b2f8d04a6e93c7"
# The hypothesis loop: try the 5-6 canonical constructions that cover
# ~90% of HMAC-signed mobile APIs in the wild.
SCHEMES = [
# (name, builder): method, path, ts, nonce, body -> string-to-sign
("newline-joined + body sha256",
lambda m, p, ts, n, b: "\n".join([m, p, ts, n, hashlib.sha256(b).hexdigest()])),
("concat + body raw",
lambda m, p, ts, n, b: m + p + ts + n + b.decode("utf-8", "replace")),
("query-string",
lambda m, p, ts, n, b: f"method={m}&path={p}×tamp={ts}&nonce={n}"),
("ts-first newline",
lambda m, p, ts, n, b: "\n".join([ts, m, p, n, hashlib.sha256(b).hexdigest()])),
("body-hash-first newline",
lambda m, p, ts, n, b: "\n".join([hashlib.sha256(b).hexdigest(), m, p, ts, n])),
]
def _sig(secret, s2s):
return hmac.new(secret, s2s.encode(), hashlib.sha256).hexdigest()
class SignatureRecon:
def response(self, flow):
req = flow.request
ts = req.headers.get("X-Client-TS") or req.headers.get("timestamp")
nonce = req.headers.get("X-Nonce") or req.headers.get("nonce")
sig = req.headers.get("X-Signature") or req.headers.get("signature")
if not sig:
return # unsigned request -- boring, skip
body = req.raw_content or b""
# ONLY attempt this against a test account you control.
print(f"[recon] {req.method} {req.path} ts={ts} nonce={nonce}")
for name, build in SCHEMES:
guess = _sig(SECRET, build(req.method, req.path, ts or "", nonce or "", body))
if hmac.compare_digest(guess, sig.lower()):
print(f"[recon] MATCH -> scheme: {name}")
with open("scheme_confirmed.txt", "a") as f:
f.write(json.dumps({"path": req.path, "scheme": name}))
return
print("[recon] no match -- widen the scheme list")
addons = [SignatureRecon()]
Run the app through the proxy once and you get:
$ mitmdump -s mitm_recon.py
[recon] POST /v1/enrichment ts=1794150489 nonce=632f821dc6a3dedc
[recon] MATCH -> scheme: newline-joined + body sha256
scheme_confirmed.txt now holds the exact scheme, confirmed cryptographically — no guesswork, no trial-and-error in a REPL. This is what turns a week of reverse-engineering into an afternoon.
Step 4 — rebuild the client
Now that the scheme is confirmed, writing the client is mechanical. This is shopverse_client.py, the module the tests exercise:
"""
shopverse_client.py -- signed API client for the Shopverse Reward API.
No third-party API keys, no external network. Works against the local
MockShopverseBackend (shopverse_backend.py). In production you would point
BASE_URL at the real app's gateway, after reverse-engineering its scheme
with mitmproxy; the signing math here is the part everyone gets wrong.
Usage:
python3 shopverse_client.py # runs the demo incl. negative controls
python3 -m pytest test_shopverse.py # 10 test cases
"""
import hashlib
import hmac
import json
import os
import time
import urllib.error
import urllib.request
BASE_URL = os.environ.get("SHOPVERSE_URL", "http://127.0.0.1:5013")
SECRET_PATH = os.environ.get("SHOPVERSE_SECRET", "extracted_secret.key")
class SignatureRejected(Exception):
"""Raised on 401/403 -- signature, timestamp or replay gate failed."""
def __init__(self, status, body):
self.status = status
self.body = body
super().__init__(f"HTTP {status}: {body}")
def string_to_sign(method, path, ts, nonce, body):
# Canonical construction. Every byte must match the backend's
# reconstruction exactly, or the HMACs diverge.
body_hash = hashlib.sha256(body.encode()).hexdigest()
return "\n".join([method, path, ts, nonce, body_hash])
def sign(method, path, body, secret, ts=None, nonce=None):
ts = str(int(time.time())) if ts is None else str(ts)
nonce = (nonce or os.urandom(8).hex()).lower()
payload = string_to_sign(method, path, ts, nonce, body)
sig = hmac.new(secret, payload.encode(), hashlib.sha256).hexdigest()
return ts, nonce, sig
def _send(method, path, body, base, ts, nonce, sig):
req = urllib.request.Request(base + path, data=body.encode(), method=method)
req.add_header("Content-Type", "application/json")
req.add_header("X-Client-TS", ts)
req.add_header("X-Nonce", nonce)
req.add_header("X-Signature", sig)
try:
with urllib.request.urlopen(req, timeout=10) as r:
return r.status, r.read().decode()
except urllib.error.HTTPError as e:
raise SignatureRejected(e.code, e.read().decode()) from None
def request(method, path, body, secret, base=BASE_URL, ts=None, nonce=None):
"""Sign-and-send. ts/nonce overrides exist for tests."""
ts, nonce, sig = sign(method, path, body, secret, ts, nonce)
return _send(method, path, body, base, ts, nonce, sig)
The signing math fits in ten lines. Everything that goes wrong in real projects goes wrong in those ten lines — which is exactly why the harness exists.
Step 5 — prove it, don’t claim it
The demo script signs two real requests and then attempts three attacks against the mock backend:
$ python3 shopverse_client.py
Telkomsel number -> 200 {"msisdn": "6281234567890", "carrier": "Telkomsel", "line_type": "mobile", "risk": "low"}
XL number -> 200 {"msisdn": "6281377788899", "carrier": "XL Axiata", "line_type": "mobile", "risk": "low"}
Negative controls (each must raise SignatureRejected):
tampered body -> rejected: HTTP 401
exact replay -> rejected: HTTP 403
stale ts (-20m) -> rejected: HTTP 403
Read the status codes carefully — they are the mechanism, not decoration. Tampering gets 401 (the signature no longer matches the mutated body), replay and stale timestamps get 403 (signature valid, but the nonce or freshness gate independently refuses). If your client is failing against a real API, matching the error code to its gate tells you within one request what’s wrong. And because of the replay gate, the honest request shipped twice under identical ts/nonce was still rejected — replay is not a theoretical concern even when the signature is perfect.
The pytest suite pins all ten behaviors:
$ python3 -m pytest test_shopverse.py -q
.......... [100%]
10 passed in 0.20s
Coverage: happy path (two distinct signed requests, both 200), tampered body 401, exact replay 403, stale timestamp 403, future timestamp 403, missing headers 401, wrong key 401, timestamp inside the tolerance window (60 seconds stale, accepted), and case-normalized nonce equivalence (CAFEBABE cannot dodge the replay gate as cafebabe’s twin).
The whole loop was also run through a live mitmdump process with the recon add-on loaded: the client hit the backend via the proxy, the add-on confirmed the scheme on every signed request in flight, and the replay/stale attacks show up in the proxy log as 403s — the same output you’d be staring at on a real engagement.
The defenses worth building (and selling)
Everything above is the offensive workflow. The mirror image is the part clients pay for, because request signing is cheap to implement and easy to get subtly wrong:
- Secret in hardware-backed keystore (Android Keystore / iOS Secure Enclave), never in strings.xml or a
static final byte[]. This is what separates an app that resists APK extraction from one that doesn’t — the OWASP Mobile Top 10 category “Insecure Data Storage” exists precisely because most don’t. - Per-device secrets minted at registration, so one extracted key doesn’t sign for the whole fleet.
- Replay protection done properly: the nonce TTL (600s here) must exceed the timestamp window (300s here), or you open a gap where an expired-timestamp replay races the nonce cleanup.
- Body-hash signing — always include
SHA256(body)in the string-to-sign. Signatures over headers only let a man-in-the-middle rewrite the payload freely.
Every one of those lines is a pull request we’ve written or reviewed. Most API teams get the happy path right and the failure modes wrong — which is exactly what the ten pytest cases above encode.
If you’re building a data pipeline against a signed mobile API, hardening one of your own, or just want this whole harness pointed at your stack, email [email protected] — custom clients, mock backends and test suites are a fixed-fee job, not a retainer mystery.