chore: bump version to 1.2.1

Deleted tweets answer the syndication endpoint with HTTP 200 and a
TweetTombstone (no `errors`, no `id_str`). The body classifier only
knew the `errors` shape, so tombstones fell through to the
`no id_str -> Sensitive` branch and degraded to an empty result,
making the bot reply "No media found" for a deleted tweet.

- extract parse_syndication_body(); tombstone shape -> NotFound
- propagate NotFound from the TWITTER_AUTH_TOKEN GraphQL fallback
  instead of swallowing it into empty_fetched
- unit tests for all body classes + live regression test on a real
  tombstoned tweet

AGENTS.md

@@ -4,7 +4,7 @@
 
 Telegram bot (teloxide) that turns post links from X/Twitter, Pixiv, and Bluesky into media messages (images, video, GIF) with the post's title, author, and tags. It supports batch media splitting, retry with persistence, inline queries, forward-channel rebinding with caption templates, and Pixiv ugoira→MP4 transcoding. README and user-facing strings are in Chinese. The project is a Rust port of a Python predecessor (see `queue.rs` comments referencing `utils/task_queue.py`).
 
-Two-crate Cargo workspace (both v1.2.0, edition 2024, resolver 3):
+Two-crate Cargo workspace (both v1.2.1, edition 2024, resolver 3):
 
 - **`crates/x-media`** — library that fetches and normalizes media from the three sites. Pure, no Telegram knowledge.
 - **`crates/xmedia-bot`** — the bot binary: teloxide dispatcher, SQLite-backed chat state, persistent task queue.
@@ -60,7 +60,7 @@ Docker: `docker build -t tgxmb .` then `docker run --rm -d --name tgxmb --env-fi
 - **Serde**: per-site `model.rs` are pure `Deserialize` DTOs mirroring API JSON; site structs in `interface.rs` have private fields, a `caption()` builder, and `impl From<SiteStruct> for Fetched`. Persisted payloads use internally-tagged enums (`#[serde(tag = "kind")]` / `type`).
 - **Naming**: module-per-concern, snake_case files, `CamelCase` types, `snake_case` fns. `//!` module docs and `///` docs on non-obvious logic (syndication token, ugoira encoding, `display_text_range`).
 - **Retries**: only `x-media::site::fetch` retries (3 attempts, `1 << attempt` backoff, HTTP errors only). Queue retries are explicit `QueueError::Retryable` with computed delay (`retry_delay_seconds`).
-- Logging via `log` macros (`pretty_env_logger`, level from `RUST_LOG`).
+- Logging via `log` macros (`pretty_env_logger`, level from `RUST_LOG`). Level convention: `info` = lifecycle + per-post business results (`sent`/`forwarded`/`copied`), admin/operator actions and anomalies (fallback, retry enqueue, dead-letter is `error`); `debug` = per-request detail (message/command/URL extraction, `fetching`/`fetched`, batch sends, queue processing, photo processing, inline queries). Full user-submitted URLs and message text only appear at `debug`; at `info` and above links are printed via the normalized cache key (`handlers::log_key`, e.g. `[key=twitter:123...]`) so logs stay short and do not echo user data.
 
 ## Important Files
 

Cargo.lock

@@ -2925,7 +2925,7 @@ checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4"
 
 [[package]]
 name = "x-media"
-version = "1.2.0"
+version = "1.2.1"
 dependencies = [
  "bytes",
  "dotenv",
@@ -2944,7 +2944,7 @@ dependencies = [
 
 [[package]]
 name = "xmedia-bot"
-version = "1.2.0"
+version = "1.2.1"
 dependencies = [
  "bytes",
  "dotenv",

crates/x-media/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "x-media"
-version = "1.2.0"
+version = "1.2.1"
 edition = "2024"
 
 [dependencies]

crates/x-media/src/site/mod.rs

@@ -299,30 +299,53 @@ pub(crate) fn log_once_ffmpeg_missing() {
 /// Fetches a post from its URL. Returns `Ok(None)` when no site pattern
 /// matches (unsupported links are silently ignored by the bot).
 ///
-/// Transient network failures are retried: 3 total attempts with 1s then 2s
-/// delays. Retried classes: bare HTTP errors, [`FetchError::Transient`]
-/// (429/5xx from any site), and pixiv errors (its network failures arrive
-/// wrapped as `PixivError`). Non-retried: Json/NotFound/Blocked/Sensitive.
+/// Transient failures are retried: 3 total attempts with 1s then 2s delays.
+/// Retried classes: bare HTTP errors, [`FetchError::Transient`] (429/5xx
+/// from any site), pixiv network errors, and pixiv HTTP statuses that are
+/// actually transient (429 / 5xx). Permanent classes are returned
+/// immediately: Json, NotFound, Blocked, Sensitive, pixiv 4xx statuses
+/// (bad/expired token, forbidden, not found) and pixiv API/auth errors.
+/// Whether [`fetch`] should retry `err` (3 total attempts, 1s then 2s
+/// backoff). Permanent classes — 4xx statuses, invalid tokens, unparseable
+/// bodies, not-found/blocked/sensitive — are returned immediately; retrying
+/// them only wastes attempts against the source site.
+fn fetch_error_is_retryable(err: &FetchError) -> bool {
+    match err {
+        FetchError::Http(_) | FetchError::Transient(_) => true,
+        FetchError::Pixiv(e) => match e {
+            PixivError::Http(_) => true,
+            PixivError::Status(code) if *code == 429 || *code >= 500 => true,
+            // 4xx, invalid token, unparseable body: retrying cannot help.
+            PixivError::Status(_)
+            | PixivError::Api(_)
+            | PixivError::Json(_)
+            | PixivError::NoAuth => false,
+        },
+        _ => false,
+    }
+}
+
 pub async fn fetch(url: &str) -> Result<Option<Fetched>, FetchError> {
     for attempt in 0..3u32 {
         match fetch_once(url).await {
             Ok(Some(fetched)) => {
-                log::info!(
-                    "fetched {url}: site {} returned {} media",
+                // Per-request detail: debug only, keyed by the post id.
+                log::debug!(
+                    "fetched [key={}]: site {} returned {} media",
+                    cache_key(url).unwrap_or_else(|| "?".into()),
                     fetched.site_name(),
                     fetched.media.len()
                 );
                 return Ok(Some(fetched));
             }
             Ok(None) => return Ok(None),
-            Err(e @ (FetchError::Http(_) | FetchError::Transient(_) | FetchError::Pixiv(_))) => {
-                if attempt < 2 {
+            Err(err) => {
+                if fetch_error_is_retryable(&err) && attempt < 2 {
                     tokio::time::sleep(Duration::from_secs(1 << attempt)).await;
                 } else {
-                    return Err(e);
+                    return Err(err);
                 }
             }
-            Err(other) => return Err(other),
         }
     }
     unreachable!("retry loop always returns")
@@ -453,6 +476,51 @@ mod tests {
         assert_eq!(cache_key("https://example.com/not-a-post"), None);
     }
 
+    #[test]
+    fn fetch_error_retryability_classification() {
+        // Transient: network errors, explicit transient, pixiv 429/5xx.
+        assert!(fetch_error_is_retryable(&FetchError::Transient(
+            "429".into()
+        )));
+        assert!(fetch_error_is_retryable(&FetchError::Pixiv(
+            PixivError::Status(429)
+        )));
+        assert!(fetch_error_is_retryable(&FetchError::Pixiv(
+            PixivError::Status(500)
+        )));
+        assert!(fetch_error_is_retryable(&FetchError::Pixiv(
+            PixivError::Status(503)
+        )));
+        // Permanent: pixiv 4xx (bad/expired token, forbidden, not found),
+        // api/auth errors, unparseable bodies, not-found/blocked/sensitive.
+        assert!(!fetch_error_is_retryable(&FetchError::Pixiv(
+            PixivError::Status(400)
+        )));
+        assert!(!fetch_error_is_retryable(&FetchError::Pixiv(
+            PixivError::Status(401)
+        )));
+        assert!(!fetch_error_is_retryable(&FetchError::Pixiv(
+            PixivError::Status(403)
+        )));
+        assert!(!fetch_error_is_retryable(&FetchError::Pixiv(
+            PixivError::Status(404)
+        )));
+        assert!(!fetch_error_is_retryable(&FetchError::Pixiv(
+            PixivError::Api("invalid_grant".into())
+        )));
+        assert!(!fetch_error_is_retryable(&FetchError::Pixiv(
+            PixivError::NoAuth
+        )));
+        let json_err = serde_json::from_str::<serde_json::Value>("x").unwrap_err();
+        assert!(!fetch_error_is_retryable(&FetchError::Pixiv(
+            PixivError::Json(json_err)
+        )));
+        assert!(!fetch_error_is_retryable(&FetchError::NotFound));
+        assert!(!fetch_error_is_retryable(&FetchError::Blocked));
+        assert!(!fetch_error_is_retryable(&FetchError::Sensitive));
+        assert!(!fetch_error_is_retryable(&FetchError::TooLarge));
+    }
+
     #[test]
     fn caption_from_fields_substitutes_and_escapes() {
         // The format string is escaped, the field values are substituted

crates/x-media/src/site/pixiv/api.rs

@@ -29,6 +29,10 @@ pub enum PixivError {
     NoAuth,
     Http(reqwest::Error),
     Json(serde_json::Error),
+    /// Non-2xx HTTP status from the app API. The code lets [`crate::site::fetch`]
+    /// retry only transient classes (429 / 5xx) instead of burning attempts on
+    /// permanent 4xx (bad token, forbidden, not found).
+    Status(u16),
     Api(String),
 }
 
@@ -38,6 +42,7 @@ impl fmt::Display for PixivError {
             PixivError::NoAuth => write!(f, "pixiv: no authentication"),
             PixivError::Http(e) => write!(f, "pixiv http error: {e}"),
             PixivError::Json(e) => write!(f, "pixiv json error: {e}"),
+            PixivError::Status(code) => write!(f, "pixiv status {code}"),
             PixivError::Api(message) => write!(f, "pixiv api error: {message}"),
         }
     }
@@ -137,7 +142,7 @@ impl PixivAPI {
             .send()
             .await?;
         if !response.status().is_success() {
-            return Err(PixivError::Api(format!("status {}", response.status())));
+            return Err(PixivError::Status(response.status().as_u16()));
         }
         let json: serde_json::Value = serde_json::from_str(&response.text().await?)?;
         if json.get("error").is_some() {
@@ -190,7 +195,7 @@ impl PixivAPI {
             .send()
             .await?;
         if !response.status().is_success() {
-            return Err(PixivError::Api(format!("status {}", response.status())));
+            return Err(PixivError::Status(response.status().as_u16()));
         }
         let json: serde_json::Value = serde_json::from_str(&response.text().await?)?;
         if json.get("error").is_some() {

crates/x-media/src/site/twitter/interface.rs

@@ -29,13 +29,19 @@ pub async fn fetch_from_url(url: &str) -> Result<Fetched, FetchError> {
             if super::auth::enabled() {
                 match super::auth::fetch(id).await {
                     Ok(tweet) => Ok(tweet.into()),
+                    // The tweet is genuinely gone (deleted / suspended /
+                    // tombstoned): report it instead of degrading to an
+                    // empty result ("No media found"). Only unexpected
+                    // fallback failures (network, parse) keep the NSFW
+                    // placeholder.
+                    Err(FetchError::NotFound) => Err(FetchError::NotFound),
                     Err(e) => {
                         log::warn!("twitter auth fallback failed for {id}: {e}");
                         Ok(empty_fetched(url))
                     }
                 }
             } else {
-                log::info!("tweet {id} is sensitive; set TWITTER_AUTH_TOKEN to fetch NSFW media");
+                log::debug!("tweet {id} is sensitive; set TWITTER_AUTH_TOKEN to fetch NSFW media");
                 Ok(empty_fetched(url))
             }
         }
@@ -59,8 +65,8 @@ fn empty_fetched(url: &str) -> Fetched {
     }
 }
 
-/// Fetches a tweet from the syndication endpoint. Deleted/blocked tweets
-/// surface as `FetchError::NotFound`.
+/// Fetches a tweet from the syndication endpoint. Deleted/blocked/tombstoned
+/// tweets surface as `FetchError::NotFound`.
 pub async fn fetch(id: &str) -> Result<Tweet, FetchError> {
     let id_num = id.parse::<u64>().map_err(|_| FetchError::NotFound)?;
     let response = crate::site::CLIENT
@@ -79,23 +85,33 @@ pub async fn fetch(id: &str) -> Result<Tweet, FetchError> {
         };
     }
     let text = response.text().await?;
-    // Deleted tweets answer with {"errors": [...]} instead of a tweet.
-    if serde_json::from_str::<serde_json::Value>(&text)
-        .map(|v| v.get("errors").is_some())
-        .unwrap_or(false)
-    {
+    // Deleted/blocked tweets answer with an `errors` array or a
+    // TweetTombstone (HTTP 200, no `id_str`); NSFW withholding is an empty
+    // `{}`. Both classes are permanent — classify before parsing the tweet.
+    parse_syndication_body(&text)?;
+    Tweet::from_syndication_json(&text).map_err(FetchError::Json)
+}
+
+/// Parses and classifies a syndication response body. `Ok` means the body is
+/// a real tweet payload; `Err` carries the permanent error class:
+/// - `NotFound`: an `errors` array (deleted/blocked) or a `TweetTombstone`
+///   (deleted by the author / suspended — HTTP 200, no `errors`, no `id_str`).
+/// - `Sensitive`: an empty `{}` (NSFW / age-restricted withholding).
+/// - `Json`: an unparseable body.
+///
+/// The tombstone shape must NOT fall through to `Sensitive`: the bot would
+/// otherwise answer "No media found" for a deleted tweet instead of failing.
+fn parse_syndication_body(text: &str) -> Result<serde_json::Value, FetchError> {
+    let body: serde_json::Value = serde_json::from_str(text)?;
+    let tombstoned = body.get("tombstone").is_some()
+        || body.get("__typename").and_then(|t| t.as_str()) == Some("TweetTombstone");
+    if body.get("errors").is_some() || tombstoned {
         return Err(FetchError::NotFound);
     }
-    // NSFW / age-restricted tweets exist but are served as an empty `{}` —
-    // they surface as FetchError::Sensitive so the caller can retry as a
-    // logged-in user.
-    if serde_json::from_str::<serde_json::Value>(&text)
-        .map(|v| v.get("id_str").is_none())
-        .unwrap_or(false)
-    {
+    if body.get("id_str").is_none() {
         return Err(FetchError::Sensitive);
     }
-    Tweet::from_syndication_json(&text).map_err(FetchError::Json)
+    Ok(body)
 }
 
 /// The syndication token: JS `((id / 1e15) * PI).toString(36)` (the
@@ -572,6 +588,48 @@ mod tests {
         assert!(token.starts_with("236.v"), "got {token}");
     }
 
+    #[test]
+    fn syndication_tombstone_maps_to_not_found() {
+        // Deleted tweets answer HTTP 200 with a TweetTombstone (no `errors`,
+        // no `id_str`); it must not fall through to Sensitive, which would
+        // make the bot reply "No media found" for a deleted tweet.
+        let raw = serde_json::json!({
+            "__typename": "TweetTombstone",
+            "tombstone": {
+                "text": { "rtl": false, "text": "This Post was deleted by the Post author. Learn more" }
+            }
+        });
+        assert!(matches!(
+            parse_syndication_body(&raw.to_string()),
+            Err(FetchError::NotFound)
+        ));
+    }
+
+    #[test]
+    fn syndication_errors_maps_to_not_found() {
+        // The classic gone shape: {"errors": [...]}.
+        let raw = serde_json::json!({ "errors": [{ "message": "Couldn't find Tweet" }] });
+        assert!(matches!(
+            parse_syndication_body(&raw.to_string()),
+            Err(FetchError::NotFound)
+        ));
+    }
+
+    #[test]
+    fn syndication_empty_object_maps_to_sensitive() {
+        // NSFW / age-restricted withholding: an empty `{}`.
+        assert!(matches!(
+            parse_syndication_body("{}"),
+            Err(FetchError::Sensitive)
+        ));
+    }
+
+    #[test]
+    fn syndication_tweet_body_passes() {
+        let raw = fixture(serde_json::json!([]));
+        assert!(parse_syndication_body(&raw.to_string()).is_ok());
+    }
+
     #[tokio::test]
     #[ignore = "live network: requires outbound HTTPS to cdn.syndication.twimg.com"]
     async fn live_fetch_with_photos() {
@@ -596,4 +654,17 @@ mod tests {
             "got {result:?}"
         );
     }
+
+    #[tokio::test]
+    #[ignore = "live network: requires outbound HTTPS to cdn.syndication.twimg.com"]
+    async fn live_fetch_tombstone_deleted_tweet_is_not_found() {
+        // Regression: a real deleted tweet answering with a TweetTombstone
+        // (HTTP 200, no errors/id_str) used to surface as Sensitive and
+        // degrade to an empty result ("No media found").
+        let result = fetch("2085948045967986859").await;
+        assert!(
+            matches!(result, Err(FetchError::NotFound)),
+            "got {result:?}"
+        );
+    }
 }

crates/xmedia-bot/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "xmedia-bot"
-version = "1.2.0"
+version = "1.2.1"
 edition = "2024"
 
 [dependencies]

crates/xmedia-bot/src/handlers.rs

@@ -26,6 +26,10 @@ static URL_JOBS: LazyLock<parking_lot::Mutex<Option<tokio::sync::mpsc::Sender<Ur
 /// Set by main's shutdown sequence; workers stop pulling new jobs.
 static URL_STOP: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false);
 
+/// JoinHandles of the URL workers, awaited by [`stop_url_workers`].
+static URL_WORKER_HANDLES: LazyLock<parking_lot::Mutex<Option<Vec<tokio::task::JoinHandle<()>>>>> =
+    LazyLock::new(|| parking_lot::Mutex::new(None));
+
 /// Worker count draining URL jobs; keeps the old 8-permit concurrency cap
 /// while bounding how many jobs can be queued at all.
 const URL_WORKERS: usize = 8;
@@ -40,9 +44,10 @@ pub async fn start_url_workers() {
     let (tx, rx) = tokio::sync::mpsc::channel::<UrlJob>(256);
     *URL_JOBS.lock() = Some(tx);
     let rx = std::sync::Arc::new(tokio::sync::Mutex::new(rx));
+    let mut handles = Vec::with_capacity(URL_WORKERS);
     for _ in 0..URL_WORKERS {
         let rx = std::sync::Arc::clone(&rx);
-        tokio::spawn(async move {
+        handles.push(tokio::spawn(async move {
             while !URL_STOP.load(std::sync::atomic::Ordering::Relaxed) {
                 let job = rx.lock().await.recv().await;
                 match job {
@@ -50,13 +55,28 @@ pub async fn start_url_workers() {
                     None => break,
                 }
             }
-        });
+        }));
     }
+    *URL_WORKER_HANDLES.lock() = Some(handles);
 }
 
-/// Stops URL workers (drains up to the 256 queued jobs, then exits).
-pub fn stop_url_workers() {
+/// Stops the URL workers: sets the stop flag, drops the job channel (so
+/// workers blocked in \`recv()\` wake with \`None\` and exit) and awaits the
+/// worker tasks. Each worker finishes its in-flight job first; jobs still
+/// queued in the channel are abandoned (the old implementation neither
+/// drained them nor woke blocked workers — it only set a flag checked
+/// between jobs).
+pub async fn stop_url_workers() {
     URL_STOP.store(true, std::sync::atomic::Ordering::Relaxed);
+    // Dropping the sender makes every worker's recv() return None.
+    *URL_JOBS.lock() = None;
+    // Take the handles first so the lock guard drops before the awaits.
+    let handles = URL_WORKER_HANDLES.lock().take();
+    if let Some(handles) = handles {
+        for handle in handles {
+            let _ = handle.await;
+        }
+    }
 }
 
 pub static CHAT_STORE: LazyLock<ChatStore> =
@@ -111,6 +131,14 @@ where
         .await
 }
 
+/// Log prefix tying the whole lifecycle of one link (fetch → send → cache →
+/// forward) together: the normalized cache key (`twitter:123…`, `pixiv:123`,
+/// `bsky:handle/rkey`) instead of the raw URL, so logs stay short and do not
+/// echo full user-submitted URLs at info level.
+pub fn log_key(url: &str) -> String {
+    x_media::site::cache_key(url).unwrap_or_else(|| "<unsupported>".to_string())
+}
+
 /// Extracts URL and text-link entities (text + caption), deduped in order.
 pub fn extract_urls(message: &Message) -> Vec<String> {
     let mut urls = Vec::new();
@@ -514,7 +542,11 @@ async fn dispatch_send(bot: Bot, message: &Message, task: &Task, url: &str) {
     };
     match result {
         Ok(message_ids) => {
-            log::info!("sent {} message(s) for {url}", message_ids.len());
+            log::info!(
+                "sent {} message(s) for [key={}]",
+                message_ids.len(),
+                log_key(url)
+            );
             send::post_send_actions(&bot, task, message_ids).await;
             // The task settled: drop any keep-alive temp media.
             send::release_keep_alive(task);
@@ -523,7 +555,10 @@ async fn dispatch_send(bot: Bot, message: &Message, task: &Task, url: &str) {
             delay_seconds,
             task,
         }) => {
-            log::info!("send for {url} failed, queued for retry in {delay_seconds:.1}s");
+            log::info!(
+                "send for [key={}] failed, queued for retry in {delay_seconds:.1}s",
+                log_key(url)
+            );
             enqueue_retry(task, delay_seconds).await;
             let _ = reply(bot, message.clone(), "Send failed. Task queued for retry.").await;
         }
@@ -599,7 +634,7 @@ async fn url_media(bot: Bot, message: &Message, url: &str) {
     if let Some(key) = x_media::site::cache_key(url)
         && let Some(cached) = LINK_CACHE.get(&key, CONFIG.link_cache_ttl).await
     {
-        log::info!("link cache hit for {url}");
+        log::debug!("link cache hit for {key}");
         let chat_data = CHAT_STORE.get(chat_id).await;
         let site = key.split(':').next().unwrap_or("unknown");
         let format = chat_data
@@ -656,11 +691,11 @@ async fn url_media(bot: Bot, message: &Message, url: &str) {
         return;
     }
 
-    log::info!("fetching {url}");
+    log::debug!("fetching {url} [key={}]", log_key(url));
     match x_media::site::fetch(url).await {
         // Unsupported links are ignored silently (Python parity).
         Ok(None) => {
-            log::info!("no site pattern matches {url}; ignoring");
+            log::debug!("no site pattern matches {url}; ignoring");
         }
         // Retries exhausted: notify the user (Rust-only requirement 3).
         Err(e) => {
@@ -743,7 +778,8 @@ pub async fn message_handler(bot: Bot, message: Message) -> Result<(), RequestEr
             &t[..end]
         })
         .unwrap_or("<no text>");
-    log::info!(
+    // Per-request detail: debug only (message text is user data).
+    log::debug!(
         "message from {sender} in {} (private={is_private}): {text_preview}",
         message.chat.id
     );
@@ -754,14 +790,16 @@ pub async fn message_handler(bot: Bot, message: Message) -> Result<(), RequestEr
     if let Some(text) = message.text()
         && let Ok(command) = Command::parse(text, "")
     {
-        log::info!("command from {}: {text_preview}", message.chat.id);
+        log::debug!("command from {}: {text_preview}", message.chat.id);
         execute_command(&bot, &message, command).await?;
         return respond(());
     }
     if is_private {
         let urls = extract_urls(&message);
         if !urls.is_empty() {
-            log::info!("extracted {} URL(s): {urls:?}", urls.len());
+            // Debug only, and echo the normalized keys instead of the raw URLs.
+            let keys: Vec<String> = urls.iter().map(|u| log_key(u)).collect();
+            log::debug!("extracted {} URL(s): {keys:?}", urls.len());
         }
         for url in urls {
             // Clone out of the lock: the parking_lot guard is !Send and must
@@ -855,7 +893,11 @@ pub async fn inline_query_handler(bot: Bot, query: InlineQuery) -> Result<(), Re
 /// Fetches the post behind an inline query and answers it. The caller has
 /// already applied the debounce. Returns `true` when an answer was sent.
 async fn answer_inline_query(bot: Bot, query: InlineQuery) -> Result<bool, RequestError> {
-    log::info!("inline query: {}", query.query);
+    log::debug!(
+        "inline query: {} [key={}]",
+        query.query,
+        log_key(&query.query)
+    );
     match x_media::site::fetch(&query.query).await {
         Ok(Some(fetched)) => {
             let mut results: Vec<InlineQueryResult> = Vec::new();
@@ -932,7 +974,7 @@ pub async fn callback_query_handler(bot: Bot, query: CallbackQuery) -> Result<()
     let chat_data = CHAT_STORE.get(chat_id).await;
     let edit = chat_data.edit_message.get(&prompt_message_id).cloned();
     let Some(edit) = edit else {
-        log::info!(
+        log::debug!(
             "callback from {}: no edit record for prompt {prompt_message_id}",
             chat_id
         );
@@ -1008,7 +1050,7 @@ pub async fn callback_query_handler(bot: Bot, query: CallbackQuery) -> Result<()
                 }
             }
             None => {
-                log::info!("forward callback without a forward channel set");
+                log::debug!("forward callback without a forward channel set");
                 bot.answer_callback_query(callback_query_id)
                     .text("No forward channel set.")
                     .await?;

crates/xmedia-bot/src/main.rs

@@ -185,7 +185,7 @@ async fn main() {
     const SHUTDOWN_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(30);
     let shutdown = async {
         let _ = stop_tx.send(true);
-        handlers::stop_url_workers();
+        handlers::stop_url_workers().await;
         if let Some(admin) = CONFIG.admin_ids.first() {
             let _ = bot.send_message(ChatId(*admin), "Shutting down...").await;
         }

crates/xmedia-bot/src/photo.rs

@@ -223,7 +223,7 @@ fn prepare_png(file: NamedTempFile, bytes: &[u8]) -> Result<PhotoPrep, String> {
     if w + h <= PHOTO_MAX_DIMENSION_SUM && !size_over {
         return Ok(PhotoPrep::Upload(file));
     }
-    log::info!(
+    log::debug!(
         "photo {w}x{h} ({_bit_depth:?} {color_type:?}, {} bytes) needs processing",
         bytes.len()
     );
@@ -269,7 +269,7 @@ fn prepare_png(file: NamedTempFile, bytes: &[u8]) -> Result<PhotoPrep, String> {
         let (nw, nh) = target_dims(w, h);
         pix = resize_pix(pix, w, h, nw, nh)?;
         (w, h) = (nw, nh);
-        log::info!("downscaled photo to {w}x{h} (Lanczos3)");
+        log::debug!("downscaled photo to {w}x{h} (Lanczos3)");
     }
 
     let mut png_bytes = Vec::new();
@@ -277,7 +277,7 @@ fn prepare_png(file: NamedTempFile, bytes: &[u8]) -> Result<PhotoPrep, String> {
     if png_bytes.len() as u64 <= MAX_UPLOAD_BYTES {
         return Ok(PhotoPrep::Upload(write_temp(&png_bytes, "png")?));
     }
-    log::info!("PNG still over the upload cap after processing; transcoding to JPEG");
+    log::debug!("PNG still over the upload cap after processing; transcoding to JPEG");
     let jpeg_bytes = encode_jpeg(&pix, w, h)?;
     if jpeg_bytes.len() as u64 <= MAX_UPLOAD_BYTES {
         return Ok(PhotoPrep::Upload(write_temp(&jpeg_bytes, "jpg")?));
@@ -311,7 +311,7 @@ fn prepare_jpeg(file: NamedTempFile, bytes: &[u8]) -> Result<PhotoPrep, String>
         let (nw, nh) = target_dims(w, h);
         pix = resize_pix(pix, w, h, nw, nh)?;
         (w, h) = (nw, nh);
-        log::info!("downscaled jpeg to {w}x{h} (Lanczos3)");
+        log::debug!("downscaled jpeg to {w}x{h} (Lanczos3)");
     }
     let jpeg_bytes = encode_jpeg(&pix, w, h)?;
     if jpeg_bytes.len() as u64 <= MAX_UPLOAD_BYTES {

crates/xmedia-bot/src/queue.rs

@@ -188,7 +188,7 @@ impl PersistentTaskQueue {
             self.counter.fetch_add(1, Ordering::Relaxed)
         );
         let payload = payload.to_string();
-        log::info!("enqueued {id} (run_after {run_after:.1})");
+        log::debug!("enqueued {id} (run_after {run_after:.1})");
         self.pool.with_conn(move |conn| {
             conn.execute(
                 "INSERT OR REPLACE INTO tasks (id, payload, run_after, attempts, status, locked_until, created_at) \
@@ -339,10 +339,10 @@ impl QueueWorker {
                 return;
             }
         };
-        log::info!("processing {} (attempt {})", row.id, row.attempts + 1);
+        log::debug!("processing {} (attempt {})", row.id, row.attempts + 1);
         match (self.handler)(payload).await {
             Ok(()) => {
-                log::info!("task {} completed", row.id);
+                log::debug!("task {} completed", row.id);
                 self.delete_row(&row.id).await;
             }
             Err(QueueError::Retryable {
@@ -356,7 +356,7 @@ impl QueueWorker {
                     (self.dead_letter)(payload, message).await;
                 } else {
                     let delay = scaled_retry_delay(delay_seconds, row.attempts);
-                    log::info!(
+                    log::debug!(
                         "task {} rescheduled in {delay:.1}s (attempt {})",
                         row.id,
                         row.attempts + 1

crates/xmedia-bot/src/send.rs

@@ -3,7 +3,7 @@
 //! URL is blocked by hotlink protection; the bot downloads the file itself
 //! and uploads it via multipart).
 
-use crate::handlers::{CHAT_STORE, LINK_CACHE, TASK_QUEUE};
+use crate::handlers::{CHAT_STORE, LINK_CACHE, TASK_QUEUE, log_key};
 use crate::link_cache::{CachedMedia, CachedMediaKind, CachedPost};
 use crate::photo::{self, MAX_UPLOAD_BYTES, PhotoPrep};
 use crate::queue::QueueError;
@@ -232,7 +232,7 @@ async fn cache_sent_task(task: &Task, media: Vec<CachedMedia>) {
     post.media = media;
     if let Some(key) = x_media::site::cache_key(&post.url) {
         LINK_CACHE.put(&key, &post).await;
-        log::info!("cached send for {}", post.url);
+        log::debug!("cached send for [key={}]", log_key(&post.url));
     }
 }
 
@@ -257,7 +257,7 @@ pub async fn invalidate_cache(task: &Task) {
         && let Some(url) = task.source_url()
         && let Some(key) = x_media::site::cache_key(url)
     {
-        log::info!("removing stale link cache entry for {url}");
+        log::debug!("removing stale link cache entry for [key={}]", log_key(url));
         LINK_CACHE.remove(&key).await;
     }
 }
@@ -941,7 +941,7 @@ pub async fn send_media_sequence(bot: &Bot, task: &Task) -> Result<Vec<i64>, Sen
             .await
         {
             Ok(messages) => {
-                log::info!(
+                log::debug!(
                     "media group batch {idx}/{} sent ({} item(s))",
                     media_batches.len(),
                     batch.len()
@@ -952,7 +952,11 @@ pub async fn send_media_sequence(bot: &Bot, task: &Task) -> Result<Vec<i64>, Sen
             Err(RequestError::Api(api)) if is_media_fetch_failure(&api) || is_size_error(&api) => {
                 log::info!(
                     "Telegram could not fetch media for batch {idx} ({}), downloading and reuploading",
-                    batch.first().map(item_url).unwrap_or("?")
+                    batch
+                        .first()
+                        .map(item_url)
+                        .map(log_key)
+                        .unwrap_or_else(|| "?".into())
                 );
                 match send_batch_via_upload(bot, chat_id, reply_to, batch, caption).await {
                     Ok(messages) => {
@@ -1048,8 +1052,8 @@ pub async fn send_animation(bot: &Bot, task: &Task) -> Result<Vec<i64>, SendErro
         }
         Err(RequestError::Api(api)) if is_media_fetch_failure(&api) || is_size_error(&api) => {
             log::info!(
-                "Telegram could not fetch animation URL, downloading and reuploading: {}",
-                media_url
+                "Telegram could not fetch animation URL, downloading and reuploading: [key={}]",
+                log_key(media_url)
             );
             match download_to_temp(animation).await {
                 Ok((file, _bytes)) => {

docs/site-registry-refactor.md

@@ -0,0 +1,215 @@
+# 站点适配器重构方案：让新增站点变成"新模块 + 注册一行"
+
+> 状态：设计稿（未实施）。目标：把"加一个新站点"从改 8-9 处收敛到 3 处，
+> 并让站点身份、重试策略、下载 header 等站点能力归位到站点模块自身。
+> 本文只改文档，不动代码；每阶段均可独立合入、独立回滚。
+
+---
+
+## 1. 现状摩擦清单
+
+以现有三站（twitter / bsky / pixiv）为基线，新增第 4 个站点（代号 `example`）
+今天需要触碰的位置：
+
+| # | 位置（当前行号） | 改动 | 必改? |
+|---|---|---|---|
+| 1 | 新目录 `crates/x-media/src/site/example/{mod,interface,model}.rs` | 新模块 | 必改 |
+| 2 | `site/mod.rs:354-365` `fetch_once` | 加一个 `if` 分派分支 | 必改 |
+| 3 | `site/mod.rs:167-178` `cache_key` | 加一个 `if` 分支 + 约定 key 前缀 `"example:..."` | 必改 |
+| 4 | `site/mod.rs:53-63` `site_name()` | 加一个 URL `contains` 嗅探分支 | 必改 |
+| 5 | `handlers.rs:405` `SetFormat` 白名单 | `["twitter","bsky","pixiv"]` 加字符串 | 必改 |
+| 6 | `config.rs` / `main.rs:74-84` | 仿 pixiv 加启动校验（token、`disable()`） | 视站点 |
+| 7 | `site/mod.rs:312-326` `fetch_error_is_retryable` | 若重试策略特殊，改中央分类函数 | 视站点 |
+| 8 | `site/mod.rs:377/391/430` 三个下载函数 | 若媒体有防盗链，加 header（现在是硬编码 pximg 判断） | 视站点 |
+| 9 | `site/mod.rs:16,180-197` `FetchError` | 若错误类型特殊，加嵌套 variant（仿 `Pixiv(PixivError)`） | 视站点 |
+
+**根因**：仓库里没有"站点"这个实体。站点的四类能力——URL 识别（PATTERN +
+cache_key）、抓取、重试策略、下载 header——分别散落在中央 if 链、URL 字符串嗅探、
+魔法字符串 key 和 bot crate 的白名单里。`AGENTS.md` 现行约定 "no trait, no enum
+dispatch" 是刻意的简单性选择；本方案的目标是在**不推翻它精神的前提下**收敛摩擦，
+并在阶段 3 提供完整的 trait 注册表选项。
+
+## 2. 目标架构
+
+```
+crates/x-media/src/site/mod.rs
+  ├─ SITES: LazyLock<Vec<Box<dyn Site>>>      ← 注册表（唯一的"站点列表"）
+  ├─ find_site(url) / fetch(url) / cache_key(url) / site_ids()
+  └─ 通用类型：Fetched { site_id, ... } / FetchError（通用类 + Site 变体）
+        │
+        ├─ site/twitter/{mod,interface,model}.rs  impl Site
+        ├─ site/bsky/…                            impl Site
+        └─ site/pixiv/…                           impl Site  (download_headers: pximg Referer)
+                                                    (validate: token 校验)
+
+crates/xmedia-bot
+  ├─ handlers.rs  SetFormat 白名单 ← x_media::site::ids()（不再写死）
+  ├─ handlers.rs  site 格式查找 ← fetched.site_id（缓存/新鲜两条路径同口径）
+  └─ main.rs      启动校验 ← site::validate_all()（不再特判 pixiv）
+```
+
+## 3. 分阶段迁移
+
+每个阶段是一个独立提交，保持 `cargo fmt` / `cargo clippy -- -D warnings` /
+`cargo test --workspace` 全绿；行为完全不变，只挪代码、不换语义。
+
+### 阶段 1：站点身份单一来源（低风险，推荐先做）
+
+**动机**：同一概念目前有两个来源——缓存命中路径用 `key.split(':').next()`
+（`handlers.rs:639`），新鲜抓取路径用 `fetched.site_name()`（`handlers.rs:724`）；
+`site_name()` 又是对 `source_url` 的 `contains` 字符串嗅探，还有 `"unknown"`
+兜底分支。
+
+**改动**：
+
+1. `site/mod.rs`：`Fetched` 增加字段 `site_id: &'static str`（由各站点的
+   `impl From<SiteStruct> for Fetched` 填充；`empty_fetched` 同步填）。
+   `Fetched::site_name()` 改为 `return self.site_id`（保留方法名，删除
+   `source_url.contains` 嗅探与 `"unknown"` 分支）。
+2. `site/mod.rs`：新增 `pub fn site_id_from_key(key: &str) -> &'static str`
+   （解析 `"example:..."` 前缀，未知前缀返回 `"unknown"`），bot 缓存命中路径改用它，
+   与 `fetched.site_id` 口径统一。
+3. `handlers.rs:405`：`SetFormat` 白名单改为 `x_media::site::ids()`——阶段 1 先实现
+   `ids()` 为 `["twitter","bsky","pixiv"]` 的常量函数（数据源仍集中，行为不变），
+   阶段 3 再改为遍历注册表。
+4. `twitter/interface.rs:48-60` / `bsky` / `pixiv` 的 `From<SiteStruct> for Fetched`
+   各补 `site_id` 字段。
+
+**风险**：低。纯增量字段；`site_name()` 语义不变（测试 `pixiv/interface.rs:355`
+  已断言 `"pixiv"`）。
+**验证**：现有全部单测；`cache_key_normalizes_domain_variants` 等不变。
+**回滚**：revert 该提交。
+
+### 阶段 2：站点能力下沉（不引入 trait，静态分派）
+
+**动机**：把"每个站点自己才知道"的逻辑搬回站点模块，中央只做迭代。这是
+`AGENTS.md` 现有约定（无 trait）与完整注册表之间的折中，可独立交付。
+
+**改动**：每个站点模块新增并 `mod.rs` 重新导出：
+
+```rust
+// site/twitter/interface.rs（bsky/pixiv 同构）
+pub fn cache_key(url: &str) -> Option<String>;   // 用自身 PATTERN，返回 "twitter:<id>"
+pub fn is_retryable(err: &FetchError) -> bool;   // 默认 Http|Transient；pixiv 覆盖 PixivError 分支
+pub fn media_headers(url: &str) -> Option<Vec<(&'static str, String)>>;
+                                                 // pixiv: url 含 "pximg.net" → Referer
+```
+
+`site/mod.rs` 相应改为迭代三站：
+
+- `cache_key`：逐个调 `site::cache_key`，不再自己写 key 格式；
+- `fetch_error_is_retryable`：删除，`fetch()` 重试循环改调 `current_site::is_retryable`
+  （`fetch_once` 已能确定站点，把站点传下去）；
+- `media_size` / `download_media_limited` / `download_media_to_file` 里的
+  `pximg.net → Referer` 硬编码删除，改为遍历 `SITES`（阶段 2 是遍历
+  `[twitter, bsky, pixiv]` 静态列表）取 `media_headers(url)` 合并。
+
+**注意**：Referer 判定依据是媒体 URL 的 host（`pximg.net`），**不是**站点
+PATTERN（pixiv 的 PATTERN 只匹配 `pixiv.net/artworks/...`），所以 `media_headers`
+不能挂在 PATTERN 匹配上，必须按 URL 独立匹配——这正是把它做成独立函数的原因。
+
+**风险**：中。下载函数签名不变，行为必须逐字节不变；新增单元测试覆盖
+`media_headers("https://i.pximg.net/...") == Some(Referer)` 与
+`cache_key` 等价性（对全部既有用例断言新旧结果一致）。
+**回滚**：revert。
+
+### 阶段 3：Site trait + SITES 注册表（完整方案，可选）
+
+**动机**：加站点时 bot crate 与中央分派零改动；站点列表成为唯一注册点。
+
+**新增**（`site/mod.rs`）：
+
+```rust
+pub trait Site: Send + Sync {
+    fn id(&self) -> &'static str;
+    fn pattern(&self) -> &'static Regex;
+    fn enabled(&self) -> bool;
+    fn cache_key(&self, url: &str) -> Option<String>;   // 默认: id + 捕获组1
+    fn fetch_from_url(&self, url: &str)
+        -> Pin<Box<dyn Future<Output = Result<Fetched, FetchError>> + Send>>;
+    fn is_retryable(&self, err: &FetchError) -> bool;   // 默认: Http|Transient
+    fn media_headers(&self, url: &str) -> Option<Vec<(&'static str, String)>>; // 默认: None
+    fn validate(&self) -> Option<BoxFuture<'static, Result<(), String>>>;      // 默认: None
+}
+
+static SITES: LazyLock<Vec<Box<dyn Site>>> = LazyLock::new(|| vec![
+    Box::new(twitter::TwitterSite), Box::new(bsky::BskySite), Box::new(pixiv::PixivSite),
+]);
+```
+
+- `fetch_once` → `find_site(url)`（首个 PATTERN 命中且 `enabled()` 的站点）
+  → `site.fetch_from_url(url).await`；
+- `cache_key` / `site_ids()` / `media_headers` / `validate_all()` 全部遍历 `SITES`；
+- `fetch_error_is_retryable` 删除，重试判定走 `site.is_retryable`；
+- `main.rs:74-84` 的 pixiv 特判 → `site::validate_all()`（pixiv 的 `validate` 失败时
+  内部调用现有 `pixiv::disable()`，行为保持）；
+- 保留各站点的 `PATTERN`/`enabled()`/`fetch_from_url()` 顶层导出（兼容现有
+  `fetch_once` 及测试），trait 只是包一层薄壳。
+
+**async 形态**：仓库没有 `async-trait` 依赖。两个选择：
+(a) 手写 `Pin<Box<dyn Future>>` 返回类型（零新依赖，契合仓库手写风格，签名略丑）；
+(b) 引入 `async-trait`（可读性好，新增一个依赖）。
+建议先 (a)，理由：仓库显式偏好手写错误/状态机，且 `BoxFuture` 已有先例
+（`queue.rs:38` 的 `BoxFuture`）。
+
+**风险**：中。动中央分派，但每站点行为不变；注册表迭代 + `find_site` 补单测
+（`fetch`/`cache_key` 对既有 URL 集合的结果与阶段 2 完全一致）。
+**回滚**：revert。
+
+### 阶段 4：FetchError 泛化（可选，配合阶段 3）
+
+**动机**：`FetchError::Pixiv(PixivError)`（`site/mod.rs:16,184,241-245`）是站点特有
+错误嵌进通用枚举；第 4 个站点要么再加变体，要么用泛化变体。
+
+**改动**：`FetchError` 增加 `Site { site: &'static str, error: Box<dyn std::error::Error + Send + Sync> }`，
+`Pixiv(PixivError)` 变体保留但内部迁移到 `Site`（或直接替换并更新
+`is_retryable`/`Display`/`source()` 与测试）。重试判定在阶段 3 已归站点，
+中央枚举只剩通用类（Http/Json/NotFound/Blocked/Sensitive/TooLarge/Transient/Io）。
+
+**风险**：中。`Display`/`source()`/`From<PixivError>` 与 `fetch_error_is_retryable`
+测试（`site/mod.rs:480-522`）需同步。
+**回滚**：revert。
+
+### 阶段 5：收尾
+
+- 更新 `AGENTS.md` 的 "Site adapter convention" 段：写新约定（注册表 + `impl Site` +
+  每站点 `cache_key`/`is_retryable`/`media_headers`），删除 "no trait" 表述；
+- `examples/fetch.rs` 不变（走 `site::fetch`）；
+- 新增站点 checklist 见 §4。
+
+## 4. 重构后新增站点 checklist
+
+```
+1. crates/x-media/src/site/example/{mod,interface,model}.rs   // 新模块
+2. impl Site for ExampleSite 并注册进 SITES                   // 注册一行
+3. （可选）token 读取 + validate() 实现                         // 启动校验自动生效
+── bot crate 零改动 ──
+```
+
+对比现状的 8-9 处，bot crate 完全不碰：`SetFormat` 白名单、格式查找口径、
+缓存 key、启动校验全部自动跟随注册表。
+
+## 5. 权衡与明确不做的事
+
+- **不做**：Media 类型扩展（`media.rs` + `MediaItemPayload` + `CachedMediaKind` +
+  send.rs 约 10+ 处 match 的 blast radius）——这是"新增媒体类型"的摩擦，与"新增
+  站点"正交，优先级低，保持现状。
+- **不做**：DI/全局注入改造（`CHAT_STORE`/`TASK_QUEUE`/`CONFIG` 的 `LazyLock` 静态
+  模式是仓库惯例，与站点扩展无关）。
+- **不做**：schema 迁移——新站点只产生新的 cache key 前缀与 `message_format` JSON
+  key，`link_cache`/`chat_state` 表结构均无需变化。
+- **代价**：阶段 3 引入 `dyn Site` 与（选择 (a) 时）手写 `BoxFuture` 签名；若站点
+  数量长期 ≤5 且无新增迹象，阶段 2 的折中方案已够用，阶段 3/4 可无限期推迟。
+
+## 6. 建议的提交序列
+
+| 阶段 | 提交消息（建议） |
+|---|---|
+| 1 | `refactor(site): carry site_id on Fetched; unify cache-key site lookup` |
+| 2 | `refactor(site): move cache_key/is_retryable/media_headers into site modules` |
+| 3 | `refactor(site): introduce Site trait and SITES registry` |
+| 4 | `refactor(site): genericize FetchError::Site` |
+| 5 | `docs: update site adapter convention in AGENTS.md` |
+
+每阶段独立合入、独立回滚；阶段 2 完成后即可认为"加站点"摩擦已收敛，
+3/4 为可选深化。