GoKawiilTechnical guide · 24 endpoints · Measured Scraping TikTok's Mobile API TikTok's Android app talks to a private HTTP+JSON API that is faster than the web endpoints and returns considerably more. This is a technical guide to reaching it: how devices are registered, how requests are signed, how the regional hosts are partitioned, and how the TLS handshake is fingerprinted. A system built on it collected 3.23 billion creator profiles, 5.94 billion videos and 2.8 billion comments in three weeks. Get the full code Free dataset. I uploaded 4.5 billion of those videos to Hugging Face: captions, view, like, comment and save counts, the sound, the country and the posting time. huggingface.co/datasets/kuben-developer/tiktok-videos-4b What you can pull. Creator profiles, every video a creator has posted, followers and following lists, and TikTok's own similar-creator graph. Full video detail with the complete statistics block. Comments and comment replies, each with the commenter's account. Sounds, the videos using them, and the trending sounds chart. Hashtags and their videos, newest or most popular. Keyword search across videos, creators and sounds. Trending shelves and camera effects. 24 endpoints in all, each with a measured success rate.
What this is Almost every TikTok scraper you will find drives a headless browser or hits the public web endpoints. Both are the wrong layer: slow, fragile, and missing most of the interesting fields. The Android app does not use either. It talks to a private HTTP+JSON API, the same one com.zhiliaoapp.musically hits when you scroll, and that API is fast, stable, and returns far more. Getting into it is the hard part, and it is hard in a specific way. Four completely unrelated things have to be right at once: a device credential TikTok issued, a valid request signature, the correct regional host, and a TLS handshake that looks like a phone. Get any one of them wrong and you receive the identical response: a clean HTTP 200 with an empty body. No error message. No status code. Your HTTP client reports success, your logs stay green, and your database fills with nothing. There is no signal telling you which of the four you are standing at. This article walks through all four, then documents the 24 endpoints that come out the other side. It names the primitives, shows the real pipeline and gives measured numbers rather than claims. None of the four has a feedback loop. A wrong rotation constant, a wrong byte order, a wrong host and a wrong cipher suite in the handshake all produce the same well-formed request and the same empty response, so there is no error to bisect on and no partial credit. Scope Everything below is anonymous device traffic. There is no login anywhere in this system, no account, no session cookie. That also means anything genuinely account-gated (your own DMs, private videos, who liked what) is out of reach and stays out of reach. No amount of tuning gets you there.
Anatomy of a request Before anything else, here is what one of these requests actually looks like. This is a real call, with the identifying values shortened: GET /aweme/v1/aweme/post/ ? # ── what you are asking for ────────────────────────────── source=0 &user_id=6744630345964389381 &count=20 &max_cursor=1751028792000 &sort_type=0 & # ── who is asking: 38 params, order matters ───────────── ts=1788361402&ac=mobile&ac2=lte &aid= 473824 # app id: TikTok Lite &iid= 7680617333853718293 # install id ← from register &device_id= 7680616891110524437 # device id ← from register &cdid= 4a1d... # client-generated uuid &openudid= 8f2c... # client-generated 16-hex &device_brand=Samsung&device_type=SM-A136U&os_version=12&os_api=30 &resolution=1080*2280&dpi=440&host_abi=arm64-v8a ®ion=SG&carrier_region=SG&sys_region=SG&mcc_mnc=52506 &language=ja&app_language=ja&locale=ja-SG&timezone_name=Asia%2FSingapore &version_name=32.8.2&version_code= 320820 &manifest_version_code=320820 &_rticket=1788361402193&channel=googleplay&app_type=normal Headers: user-agent: com.ss.android.ugc.tiktok.lite/320802 (Linux; U; Android 12; ...) x-tt-trace-id: 00-6a9f...-6a9f...-01 x-ss-req-ticket: 1788361402193 x-khronos : 1788361402 # timestamp x-ladon : XKp9... # Speck-128/256 x-argus : cQqbRZm8k1x... # the hard one x-gorgon : 0404b0d30000... # legacy digest A single creator-timeline request. Everything below the fold is device identity. Three things to notice, because each one bites later: Two thirds of the URL is device identity. Thirty-eight common parameters describe the handset, the carrier, the region and the app build. They are not decoration. The signature covers them.
Thirty-eight common parameters describe the handset, the carrier, the region and the app build. They are not decoration. The signature covers them. device_id and iid are issued by TikTok , not chosen by you. cdid and openudid you generate and submit at registration. Getting the distinction wrong is the first wall.
, not chosen by you. and you generate and submit at registration. Getting the distinction wrong is the first wall. Parameter order is fixed. The signature hashes the query string as a literal, so url.Values.Encode() , which sorts keys alphabetically, silently produces an invalid signature. In Go you have to build the query by hand. The vocabulary, since it recurs throughout: Field What it is Origin aid Application id. 1233 is the main app (musically), 473824 is Lite, 1340 is musically_go. Different aid means a different signing key and a different endpoint set. Constant device_id The durable device identity. 19 digits. TikTok, at register iid Install id. Pairs with device_id . TikTok, at register cdid Client device id. A UUID you generate. You openudid 16 hex characters you generate. You license_id Feeds the X-Ladon key schedule. Constant per app version_code App build. Gates which endpoints answer at all. You choose
The first empty 200 A correctly implemented signer produces output that verifies against captured traffic, with parameters matching byte for byte. The response is still this: $ curl -sD- -o /tmp/body "https://api16-normal-c-alisg.tiktokv.com/aweme/v1/user/profile/other/?..." HTTP/1.1 200 OK content-type: application/json content-length: 0 x-tt-logid: 2026090117... server: TLB $ wc -c /tmp/body 0 /tmp/body Two hundred. Zero bytes. No status_code , because there is no body to put one in. This is TikTok's soft block, and it is the single most important thing to understand about this API. It is not a 403. It is not a 429. It is not a challenge page. It is a successful HTTP response containing nothing. Which means this code, which is what everyone writes first, is silently broken: res = requests.get(url, headers=signed) if res.ok: # True. Always true. store(res.json()) # {} stored, no exception # six hours later: 400,000 rows in the database, all empty, # nothing in the error log, dashboard green It is expensive to debug because four unrelated failures produce it: Your device was never activated (§ activation) Your signature is wrong (§ X-Argus) You are talking to the wrong regional host (§ regions) Your TLS handshake looks like a server, not a phone (§ JA3) There is nothing in the response to tell you which. You cannot bisect it by reading errors, because there are none. The only way through is to fix all four and measure each one in isolation.
Where device IDs come from You cannot invent a device_id . TikTok issues it, from /service/2/device_register/ on its logging host, in exchange for a plausible handset. The request body is a JSON document (app header, device header, custom block) encrypted with TTEncrypt (TikTok's own body cipher, a simple byte-level transform with a fixed key schedule) and posted as application/octet-stream;tt-data=a . It goes out with the full signature set, so you need working signing before you can get a device, and the signing needs a device. You bootstrap with the client-generated fields and zeros where the issued ones go. The body's shape, with the parts that matter: { "magic_tag" : "ss_app_log" , "header" : { // app identity: must agree with the aid in the query string "aid" : 473824, "package" : "com.ss.android.ugc.tiktok.lite" , "app_version" : "32.8.2" , "version_code" : 320820, "sdk_version" : "..." , "git_hash" : "..." , "sig_hash" : "..." , // hardware: every field here has to be internally consistent "device_model" : "SM-A136U" , "device_brand" : "Samsung" , "device_manufacturer" : "samsung" , "cpu_abi" : "arm64-v8a" , "os_version" : "12" , "os_api" : 30, "resolution" : "2280*1080" , "density_dpi" : 440, "rom" : "..." , "rom_version" : "..." , // identity you generate and are about to trade in "cdid" : "<uuid4>" , "openudid" : "<16 hex>" , "clientudid" : "<uuid4>" , "google_aid" : "<uuid4>" , // region: carrier must plausibly exist in this country "region" : "SG" , "sim_region" : "sg" , "carrier" : "Singtel" , "mcc_mnc" : "52506" , "tz_name" : "Asia/Singapore" , "tz_offset" : 25200, "custom" : { "screen_width_dp" : 408, "screen_height_dp" : 883, "web_ua" : "Dalvik/2.1.0 (Linux; U; Android 12; SM-A136U Build/...)" , "apk_last_update_time" : 1788361409271 }, "apk_first_install_time" : 1788360902118 }, "_gen_time" : 1788361402240 } Every field there is checked against the others. A Samsung SM-A136U has a specific screen resolution, a specific DPI, a specific ABI, and shipped with a specific range of Android versions. It is sold on carriers in some countries and not others. A flagship handset on a network that never carried it is not a real phone, and the registration is refused. Rather than generating these procedurally, I build them from a catalogue of ~250 real Android device profiles crossed with a carrier table of MCC/MNC pairs (roughly 2,000 rows, derived from public numbering-plan data). Pick a handset, pick a carrier that actually exists in the target country, fill in the coherent values. A successful registration comes back with the two ids you needed: { "device_id_str" : "7680616891110524437" , "install_id_str" : "7680617333853718293" , "new_user" : 1 } Most implementations stop here.
The activation call With registration working, most endpoints answered. Video listings, search, hashtags, sounds, all fine. But /aweme/v1/user/profile/other/ , the full profile record, returned the empty 200 every single time, on every device I made, forever. The obvious suspect is the signature, and it is the wrong one. The tell is that an older pool of devices, generated months earlier by different code, worked fine on that same endpoint with the same signer and the same parameters. The only difference was in how the devices had been created, and it came down to one extra HTTP call: GET /service/2/app_alert_check/?<common params> &cronet_version=...&ttnet_version=... &tt_info=<base64url(TTEncrypt(<60-field key=value blob>))> → {"message":"success"} That is it. It returns nothing you need. It looks like telemetry, and functionally it is telemetry. It is the call the real app makes on launch, before it requests any data. That is what the call is for. A device that registered and then immediately started querying the API is, from ByteDance's side, an install that never launched. Registration alone does not make you a running app. The startup call does. Device generation Profile endpoint Register only 0 / 360 Correct signature. Empty body, every time, indefinitely. Register + startup call 100 / 100 Same code, same signature, one extra request. 0 / 360 to 100 / 100 Zero to a hundred percent, from a call whose response you throw away. It is not documented anywhere. It is not visible in a signature dump. It does not fail loudly. And because the symptom is the empty 200, it is indistinguishable from a broken signer. The tt_info blob is the interesting part of the request: about sixty key=value pairs (GAID, timezone, install id, device id, carrier, screen, ABI, locale, a request UUID) TTEncrypt-ed and base64url-encoded. It is the app reporting its full environment on startup. My guess, and it is only a guess, is that this is where the device gets marked as a real install rather than a bare registration; I have not tried to prove it, because the empirical result is unambiguous.
Proving a device before you use it The activation fixed the profile endpoint, but it introduced a second-order problem: activation itself sometimes fails silently, and a device that failed activation looks exactly like a device that succeeded until you use it. So generation does not end at activation. It ends with a real read against a known creator. If real content comes back, the device joins the pool. If not, it is thrown away. Not retried, not quarantined. Discarded. func GenerateDevice(client *http.Client, country string ) (map[ string ]any, error) { tmpl, err := NewAndroidTemplate() // handset × carrier ... if err := registerDevice(client, tmpl); err != nil { return nil, fmt.Errorf( "register: %w" , err) } // Without this TikTok will not serve profile detail to a fresh device. if err := appAlertCheck(client, tmpl); err != nil { return nil, fmt.Errorf( "activate: %w" , err) } // Survivorship filter: only provably-capable devices enter the pool. if !profileCapable(client, tmpl) { return nil, errors.New( "profile probe failed: device not capable" ) } return tmpl, nil } The three-stage pipeline. Roughly 60-95% of attempts survive it, depending almost entirely on proxy quality. Without the filter you get a pool that is a mixture of working and quietly dead devices, and because dead devices return the empty 200, the same as every other failure, the pool degrades invisibly. Your success rate drifts down over days and there is nothing in the logs to explain it. With the filter, the pool is uniformly capable by construction. Live health is visible from the running server: $ curl -s localhost:8080/v1/devices | jq { "live" : 43, "generated_total" : 43, "rejected_total" : 2, "evicted_total" : 0, "success_total" : 177, "failure_total" : 74, "generation_survival_rate" : 0.9555 }
The five headers Every request carries a family of headers that TikTok verifies before it looks at your query. The names are public; knowing them is worth nothing. Header Binds Difficulty X-Khronos Unix seconds. Bounds replay. None, it is a timestamp X-Ss-Stub MD5 of the request body None, and only on POSTs X-Gorgon Legacy digest over URL, body, time Low. Public write-ups exist. X-Ladon Timestamp + license id + app id Moderate. Speck-128/256. X-Argus Everything, bound to the device High. Protobuf, two cipher layers, no feedback. Two properties of the scheme shape everything downstream: The signature covers the query string, not just the path. There is no signing a template and varying the arguments. Change count=20 to count=21 and you recompute from scratch. The upside is that a retry is a genuinely fresh cryptographic operation and never a replay. The signature is bound to one device. You cannot sign with device A and send device B's identifiers. Every retry against a different device re-signs.
... continue reading