perf: stop probing a media item's size before downloading it

The upload fallback asked `x_media::site::media_size` for every remote item
before downloading it. That call is a real GET (not a HEAD) on the *un-
guarded* `CLIENT` — so every fallback item cost two requests where one would
do, the response body was never consumed (the connection cannot return to
the pool), and for photos the answer was discarded outright
(`too_large && !matches!(item, Photo { .. })` still fired the request). It
bypassed `media_request`'s private-network guard as well, the one choke
point every other egress goes through.

For videos the probe was redundant twice over: `download_media_limited`
reads the declared Content-Length before any body byte and aborts with
`FetchError::TooLarge`, which the call site already turns into the item's
smaller URL (`FallbackError::MediaTooLarge` → `fallback_url`).

`media_size` is deleted (no other caller) and the download's own cap is the
only size gate. The video cap is now exactly `MAX_UPLOAD_BYTES` instead of
`MAX_UPLOAD_BYTES + 1`, so the boundary the probe drew survives byte for
byte: a file of exactly the cap is admitted (`len > max_bytes` is false),
one byte over degrades to the smaller URL.

Verified with a throwaway harness (a local HTTP server reached through
`TELOXIDE_PROXY`, the one LAN egress the guard allows): a small video, a
photo and an oversized video each cost 1 request where the probe made it 2,
and the oversized one still lands on `/fallback.mp4` without fetching it.

`cargo fmt --check`, `cargo clippy --workspace --all-targets --locked -- -D
warnings` and `cargo test --workspace --locked` clean.

AGENTS.md

@@ -36,7 +36,7 @@ The `x-media` library: `site::fetch(url)` dispatches through the `SITES` registr
 
 | Path | Purpose |
 |---|---|
-| `crates/x-media/src/` | Fetch library. `site/mod.rs` = dispatcher + `Fetched`/`FetchError`/`download_media`/`media_size`; `media.rs` = `Media` enum; `examples/fetch.rs` = end-to-end usage sample |
+| `crates/x-media/src/` | Fetch library. `site/mod.rs` = dispatcher + `Fetched`/`FetchError`/`download_media*` (the streaming `download_media_to_file` and the capped `download_media_limited`, which is where a download's size is checked); `media.rs` = `Media` enum; `examples/fetch.rs` = end-to-end usage sample |
 | `crates/x-media/src/site/<twitter\|pixiv\|bsky\|misskey\|bilibili>/` | One directory per site: `mod.rs` (re-exports), `interface.rs` (PATTERN, `enabled()`, `fetch_from_url()`, `cache_key`/`is_retryable`/`media_headers`, unit struct `<Name>Site` implementing `site::Site`, `From<SiteStruct> for Fetched`), `model.rs` (serde DTOs). Pixiv adds `api.rs` (auth + transport); twitter adds `auth.rs` (logged-in GraphQL `TweetDetail` fallback for NSFW tweets, gated on `TWITTER_AUTH_TOKEN`; without the token a withheld tweet stays `FetchError::Sensitive` and the bot reports it as age-restricted instead of "no media"). Misskey targets misskey.io only (`POST /api/notes/show`, 400+`NO_SUCH_NOTE` → NotFound). Bilibili fetches dynamics (images/animated images only — an attached video degrades to its cover, and its title stands in for the post text, which AV dynamics do not have) from `/x/polymer/web-dynamic/v1/detail` sent with `features=itemOpusStyle` (without that flag the legacy serialization drops an image/text post's body and headline entirely — `desc` comes back `null`; the adapter still parses the legacy `major.draw`/`desc`/`archive` shapes as a fallback). No WBI signature is involved; device cookies `buvid3`/`buvid4` are fetched automatically from `/x/frontend/finger/spi` because bilibili's `-352` risk control starts rejecting plain requests, `BILIBILI_COOKIE` is the escalation when an IP stays blocked; `b23.tv` short links are deliberately unmatched. Twitter's `from_syndication_json` HTML-decodes the API text — syndication and GraphQL `full_text` both arrive pre-escaped (`&gt;` `&lt;` `&amp;` `&#39;`) — so the stored text is raw and the caption escap…
 | `crates/xmedia-bot/src/main.rs` | Entry point: env/log init, command registration (`register_commands` — `setMyCommands` plus the profile description texts), shared `send::BOT` force-init, startup sweep of this project's leftover temp files (`x_media::TEMP_FILE_PREFIX` + an age gate, since a killed process runs no destructors), startup repair of queued retries whose local media did not survive a restart (`handlers::repair_lost_local_media`, before any worker can lease: those rows are re-fetched from their `source_url`), queue worker start, site login validation (`site::validate_all`), `periodic_sweep` (`SWEEP_INTERVAL` 300 s): expired prompts are rewritten in place to `EDIT_PROMPT_EXPIRED_TEXT` with an empty keyboard — an edit, never a new message, so a background timer cannot wake a chat — plus the link-cache prune, the idle rate-limit buckets and the idle inline-query entries, and the queue backlog line (only when non-empty). Takes its collaborators rather than the statics so its loop is testable with a paused clock, dptree handler tree, webhook vs polling dispatch |
 | `crates/xmedia-bot/src/config.rs` | Manual env parsing into `Config` |
@@ -82,7 +82,7 @@ Docker: `docker build -t tgxmb .` then `docker run --rm -d --name tgxmb --env-fi
 |---|---|
 | `crates/xmedia-bot/src/main.rs` | Startup sequence, webhook vs polling, graceful shutdown (SIGINT via teloxide ctrlc / SIGTERM via `stop_token` for docker, → sweep stop → admin msg → queue stop) |
 | `crates/xmedia-bot/src/handlers/` | `statics.rs` = `CHAT_STORE`/`TASK_QUEUE`/`CONFIG` singletons (open `$DATA_DIR/task_queue.db`, default `data/` **relative to CWD**, dir auto-created); `mod.rs` also holds `apply_caption_edit`, the one place a caption edit is applied and its failure classified: a short retryable delay is retried once, anything else is reported to the user instead of being swallowed (`callback.rs`'s template button answers its toast with the failure and leaves the record alone); `commands.rs` = command dispatch (incl. `/test <url>` send-only, `/debug <url>` parse-only, the read-only `/settings` every chat member can read — unlike the admin-only `/bot_dict` raw dump — and template removal; `/start`/`/help` carry the guidance teloxide's `descriptions()` cannot render, and `/set_format` rejects unknown `{…}` placeholders, resetting with `-`); `urls.rs` = URL extraction + the per-URL pipeline (`url_media` takes a `PostSend` mode: chat settings vs `/test`'s suppressed actions); `inline.rs` = debounced inline queries (hotlink-protected and local media skipped); `callback.rs` = edit-before-forward buttons (dptree entry + testable `handle_callback` core, incl. `skip`) |
-| `crates/xmedia-bot/src/send/` | `mod.rs`: constants `MAX_MEDIA_GROUP = 10`; `classify_request_error` (5xx/non-JSON bodies retry, see the Retries bullet) and the media-fetch markers that route a URL send into the reupload fallback — including `failed to get HTTP url content`, the description single-media URL sends answer with; the senders. `upload.rs`: download-and-reupload fallback triggered only by Telegram API errors (`is_media_fetch_failure` / `is_size_error`), with a download's class from `classify_download_error` (transport/429/5xx retry; 4xx is permanent — the media itself is gone or refused — and a temp-file *write* failure retries, being resource exhaustion far more often than a broken temp dir). `post_send.rs`: settlement (`settle_task`), cache write, post-send actions (dead-letter text via `failure_text`: post key + cause, since the raw error alone does not say which link died), queue handlers. `input_media.rs`: payload → `InputMedia` |
+| `crates/xmedia-bot/src/send/` | `mod.rs`: constants `MAX_MEDIA_GROUP = 10`; `classify_request_error` (5xx/non-JSON bodies retry, see the Retries bullet) and the media-fetch markers that route a URL send into the reupload fallback — including `failed to get HTTP url content`, the description single-media URL sends answer with; the senders. `upload.rs`: download-and-reupload fallback triggered only by Telegram API errors (`is_media_fetch_failure` / `is_size_error`), with a download's class from `classify_download_error` (transport/429/5xx retry; 4xx is permanent — the media itself is gone or refused — and a temp-file *write* failure retries, being resource exhaustion far more often than a broken temp dir). The check that routes an oversized item to `fallback_url` is the download's own declared-Content-Length abort (`FetchError::TooLarge` → `MediaTooLarge`) — there is no separate size probe, which used to cost a second request per item. `post_send.rs`: settlement (`settle_task`), cache write, post-send actions (dead-letter text via `failure_text`: post key + cause, since the raw error alone does not say which link died), queue handlers. `input_media.rs`: payload → `InputMedia` |
 | `crates/xmedia-bot/src/photo.rs` | Pure-Rust photo processing (no ffmpeg): `png` (image-png) decode/encode + `zune-jpeg` decode + `fast_image_resize` Lanczos3 downscale + `jpeg-encoder`. Photos over Telegram's limits (width + height > 10000 px → `PHOTO_INVALID_DIMENSIONS`; bytes > 10 MiB) are decoded, downscaled keeping the format, PNG bit depth > 24 (RGBA 32-bit / 16-bit per channel) reduced to 24-bit RGB with alpha flattened white (≤24-bit untouched, never upconverted), and transcoded to JPEG only if still over the cap; memory budget guarded, otherwise the item's smaller fallback URL |
 | `crates/x-media/src/site/mod.rs` | Dispatcher, `Fetched`/`FetchError`, shared `CLIENT`, `download_media` (adds `Referer: https://www.pixiv.net/` for `pximg.net` hotlink protection), `needs_media_headers` (the same per-site rule, asked by the inline path to skip what Telegram cannot fetch) |
 | `crates/x-media/src/site/pixiv/api.rs` | OAuth token exchange (hardcoded app client id/secret), access-token cache, ugoira zip→MP4 via ffmpeg in `spawn_blocking` |

crates/x-media/src/site/mod.rs

@@ -656,21 +656,6 @@ fn apply_media_headers(mut request: reqwest::RequestBuilder, url: &str) -> reqwe
     request
 }
 
-/// Downloads media bytes for the bot's upload fallback: when Telegram's own
-/// fetch of a media URL is blocked (hotlink protection), the bot downloads
-/// the file itself and uploads it via multipart. Site-appropriate headers
-/// come from each site's `media_headers` (pixiv image hosts need `Referer`).
-/// Returns the Content-Length of a media URL, or `None` when the server does
-/// not report one. Used to check whether a file fits Telegram's size limits
-/// before downloading/uploading it.
-pub async fn media_size(url: &str) -> Result<Option<u64>, FetchError> {
-    let response = apply_media_headers(CLIENT.get(url), url)
-        .send()
-        .await?
-        .error_for_status()?;
-    Ok(response.content_length())
-}
-
 /// Maps a media download's HTTP status onto the same classes the site
 /// adapters use, so callers can tell "try again" from "this URL is dead":
 /// 4xx is a property of the media (gone, refused by the host), while 429/5xx
@@ -687,7 +672,13 @@ fn download_status_error(status: reqwest::StatusCode) -> FetchError {
 /// Downloads a media file with a hard size cap: the body is streamed and the
 /// download aborts with [`FetchError::TooLarge`] the moment the cap is
 /// crossed (or when a declared Content-Length already exceeds it). Keeps the
-/// bot from buffering arbitrarily large bodies into memory.
+/// bot from buffering arbitrarily large bodies into memory — the size check
+/// the bot's upload fallback needs is the one here, not a probe of its own.
+///
+/// This is the bot's download path for the upload fallback: when Telegram
+/// cannot fetch a media URL itself (hotlink protection), the bot downloads
+/// the file and uploads it via multipart. Site-appropriate headers come from
+/// each site's `media_headers` (pixiv image hosts need `Referer`).
 pub async fn download_media_limited(url: &str, max_bytes: u64) -> Result<bytes::Bytes, FetchError> {
     let response = send_download(media_request(url)?).await?;
     if let Some(len) = response.content_length()

crates/xmedia-bot/src/send/upload.rs

@@ -62,12 +62,14 @@ async fn download_to_temp(
         | MediaItemPayload::Animation { media, .. } => media,
     };
     // Photos are downloaded even over the upload cap so `prepare_photo` can
-    // downscale / transcode them (cap = decode budget); videos/animations
-    // abort as soon as the upload cap is crossed mid-stream.
+    // downscale / transcode them (cap = decode budget); videos and animations
+    // are refused as soon as the declared size crosses the upload cap — the
+    // boundary the size probe this replaced drew: a file of exactly the cap is
+    // admitted (`len > max_bytes` is false), one byte over is not.
     let limit = if matches!(item, MediaItemPayload::Photo { .. }) {
         photo::MAX_DECODE_BYTES
     } else {
-        MAX_UPLOAD_BYTES + 1
+        MAX_UPLOAD_BYTES
     };
     let bytes = match x_media::site::download_media_limited(media_url, limit).await {
         Ok(bytes) => bytes,
@@ -197,28 +199,13 @@ pub(super) async fn prepare_upload_item(
             keep_alive: None,
         });
     }
-    // Size check before downloading/uploading: over the cap, use the
-    // smaller URL instead of the file. Photos are exempt — they are
-    // downloaded and processed (downscale / PNG→JPEG) before uploading.
-    let too_large = match x_media::site::media_size(media_url).await {
-        Ok(Some(size)) => size > MAX_UPLOAD_BYTES,
-        _ => false,
-    };
-    let too_large = too_large && !matches!(item, MediaItemPayload::Photo { .. });
-    if too_large {
-        let url = item
-            .fallback_url()
-            .ok_or_else(|| FallbackError::Permanent {
-                message: "media too large".into(),
-            })?;
-        let media = media_from_url(&item, url, caption, item.thumbnail_url())
-            .map_err(|message| FallbackError::Permanent { message })?;
-        return Ok(PreparedItem {
-            index,
-            media,
-            keep_alive: None,
-        });
-    }
+    // Whether a file is over the cap is settled by the download itself:
+    // `download_media_limited` reads the declared Content-Length before any
+    // body byte and aborts with `FetchError::TooLarge`, which arrives here as
+    // `FallbackError::MediaTooLarge` — turned into the item's smaller URL by
+    // the match below. A separate size probe used to issue a second GET of the
+    // same URL for an answer this path already has (and issued it for photos,
+    // whose answer was discarded one line later).
     match download_to_temp(&item).await {
         Ok((file, bytes)) => {
             if matches!(item, MediaItemPayload::Photo { .. }) {