Files
TelegramTwitterMediaBot/crates/x-media/src/site
YoursFunny 818a44d697 perf: fetch bsky HLS segments concurrently
A long bluesky video can be 500 segments, and the remux downloaded them
strictly one at a time: the user waited for every round trip in turn, which
is the dominant cost of the whole remux (the ffmpeg concat itself is local).
Each segment's multi-megabyte body also went to disk through a blocking
`std::fs::write` on an executor thread.

Segments now download and write under a small bound (`SEGMENT_CONCURRENCY`,
4 — a segment can be 20 MiB and the playlist is capped at 256 MiB, so this is
also what bounds the remux's peak memory) and the write goes through
`tokio::fs`. Concurrent downloads complete in completion order, and ffmpeg
concatenates the list in whatever order it holds — an out-of-order list is a
*silently* scrambled video, not an error — so `concat_list` sorts by segment
index and carries its own test.

x-media's own tokio features gain `rt` (JoinSet) and `fs`: the library
already used `spawn_blocking` on the strength of the bot crate's features.

Verified against a local HLS fixture — 12 one-second segments of solid
red/green/blue, each served with a 150 ms delay, reached through a name that
resolves to loopback (the guard refuses a literal 127.0.0.1) — with a
temporary in-module test: all 12 sampled frames come back in the right colour
order, and the server recorded a peak of 4 requests in flight, where the
serial version showed 1.

`cargo fmt --check`, `cargo clippy --workspace --all-targets --locked -- -D
warnings` and `cargo test --workspace --locked` clean.
2026-09-21 12:55:12 +08:00
..