perf: gzip and HTTP/2 for the site clients

x-media's reqwest had exactly `["json", "rustls-tls"]` — no decompression, no
HTTP/2 — so every adapter fetched its JSON as identity over HTTP/1.1. The
site APIs compress: twitter's syndication body measures 4469 bytes identity
against 1066 gzipped (4.2x) for a single-tweet response, and the fetch is on
every twitter link. The CDNs all negotiate h2 (the shared client reports
HTTP/2.0 against cdn.syndication.twimg.com, api.bilibili.com and
public.api.bsky.app), which also multiplexes the concurrent media downloads
that used to open a connection each.

Both features are one crate-level switch, so the pages' compression is what
the APIs answer with and requires no adapter change: reqwest adds
`accept-encoding: gzip` and decompresses transparently. `download_media_limited`
keeps a correct size cap either way — it checks the accumulated *body* while
streaming, not the declared Content-Length, which for a compressed response
is the compressed size.

Verified: a local echo server recorded `accept-encoding: gzip` on a request
from the shared client (both clients come from `build_client`), and a probe
over the shared `CLIENT` reported `HTTP/2.0` for the three site hosts above.
`cargo test -p x-media -- --ignored live` (14 passed) covers the real
endpoints with the new transport.

Cargo.lock gains async-compression (+codecs/core), fnv and h2.

`cargo fmt --check`, `cargo clippy --workspace --all-targets --locked -- -D
warnings` and `cargo test --workspace --locked` clean.
This commit is contained in:
2026-09-21 13:09:32 +08:00
parent a501a17519
commit 994734b001
3 changed files with 63 additions and 2 deletions
+1 -1
View File
@@ -4,7 +4,7 @@ version = "1.8.0"
edition = "2024"
[dependencies]
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls", "gzip", "http2"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
regex = "1.12"