Compare commits

...
5 Commits
Author SHA1 Message Date
YoursFunny 9e873131d4 chore: bump version to 1.8.0 2026-09-20 17:41:20 +08:00
YoursFunny 5b77d14497 fix(commands): make /debug preview the caption a link would actually send
`/debug` reported `Fetched::caption`, the site's built-in caption, so a
chat's `/set_format` override never showed up in the preview — the command
looked like a no-op, and the `/set_format` success reply now tells users to
preview with `/debug`, which made that advice wrong.

`preview_caption` mirrors the send paths instead: the per-site format
override (`caption_from_fields`, empty → built-in) plus the long-post
quoting. Reported from a live run against the test bot.
2026-09-20 17:30:55 +08:00
YoursFunny d4c36feb9a fix(commands): make /set_format and /clear_cache actually parse
teloxide's `split` parser takes exactly one space-separated token per
field, so `/set_format <site> <format>` — a two-token command — never
parsed: `Command::parse` failed, `message_handler` fell through to the URL
flow, and the user got silence. `/clear_cache` without its optional link
failed the same way ("too few arguments"), so clearing everything was
unreachable. Both now use the crate's `parse_arg_remainder` (whole
remainder, trimmed), which is what their executors were already written
against (`split_once(char::is_whitespace)`).

Found by driving the real binary against a scripted fake Bot API: the
`/set_format` replies never appeared while `/set_template` and the other
single-token commands did. `every_documented_invocation_parses` now pins
every documented form, which is what should have caught it.

The placeholder validation and `-` reset added earlier only work now that
the command reaches its executor at all.
2026-09-20 16:42:40 +08:00
YoursFunny 5d0acdac01 feat(ux): onboard users, expose the chat's settings, name failed posts
`/start` was "Hello!" and `/help` was the bare command list teloxide can
render — no argument syntax, no caption placeholders, no mention that
links only work in private chats. Both now carry that guidance, and the
bot's profile description / short description are set at startup so a
shared link says what the bot does.

`/settings` reports what this chat is configured to do (forward channel,
edit-before-forward, per-site formats, saved templates) to anyone in the
chat — `/bot_dict` is a raw admin-only dump. Templates can be removed
(`/remove_template`, listing the live names on a typo) and the prompt's
keyboard folds 3 per row with a cap: Telegram rejects a keyboard over 100
buttons outright, which would silently drop the whole prompt.

Inline results hand URLs to Telegram, which fetches them without any
site headers — pixiv's pximg.net answers 403 to that, so those items are
skipped instead of shipped broken. `needs_media_headers` answers that
question from the same per-site rule the downloader uses.

Dead-letter and retry notices name the failing post and the cause
(`failure_text`), since "Task failed after retries: task failed after 2
retries" said neither which link it was nor what happened.
2026-09-20 15:57:45 +08:00
YoursFunny d6707133cc feat(ux): answer every link, name fetch failures, keep the chat action alive
Four ways a user could get silence are closed: a registered-but-disabled
site (pixiv without a token) now answers instead of being dropped as an
unsupported link, `/test` on such a link replies instead of doing nothing,
a supported link posted in a group gets a one-line hint (channels stay
silent), and fetch failures name their cause — gone / withheld / source
risk control / site disabled / source down — instead of one generic
sentence. `FetchError::Disabled` carries the "matched but switched off"
answer, which `find_site` used to fold into `Ok(None)`.

A withheld tweet no longer degrades to "no media": without
`TWITTER_AUTH_TOKEN` it stays `Sensitive` so the reply says the media is
age-restricted, and a failed authenticated fallback propagates its own
class instead of masquerading as an empty post (`empty_fetched` is gone).

Long jobs stop looking stalled: `run_with_chat_action` re-sends the chat
action every 4s while the pipeline is pending and the hint switches from
typing to send-photo/video once the media kinds are known. Media groups
go from 9 to Telegram's 10.

`/set_format` rejects unknown `{…}` placeholders (a typo used to be
published verbatim in every caption) and resets with `-`. The
edit-before-forward prompt states its TTL and that Confirm is required,
gains a Skip button, and is rewritten in place to "expired" by the sweep
— an edit, never a new message, so a background timer cannot wake a chat.
2026-09-20 15:48:46 +08:00
17 changed files with 1242 additions and 128 deletions
+20 -14
View File
@@ -4,7 +4,7 @@
Telegram bot (teloxide) that turns post links from X/Twitter, Pixiv, Bluesky, Misskey (misskey.io), and Bilibili dynamics into media messages (images, video, GIF) with the post's title, author, and tags. It supports batch media splitting, retry with persistence, inline queries, forward-channel rebinding with caption templates, and Pixiv ugoira→MP4 transcoding. README is in Chinese; user-facing bot strings are in English. The project is a Rust port of a Python predecessor (see `queue.rs` comments referencing `utils/task_queue.py`). Telegram bot (teloxide) that turns post links from X/Twitter, Pixiv, Bluesky, Misskey (misskey.io), and Bilibili dynamics into media messages (images, video, GIF) with the post's title, author, and tags. It supports batch media splitting, retry with persistence, inline queries, forward-channel rebinding with caption templates, and Pixiv ugoira→MP4 transcoding. README is in Chinese; user-facing bot strings are in English. The project is a Rust port of a Python predecessor (see `queue.rs` comments referencing `utils/task_queue.py`).
Two-crate Cargo workspace (both v1.7.0, edition 2024, resolver 3): Two-crate Cargo workspace (both v1.8.0, edition 2024, resolver 3):
- **`crates/x-media`** — library that fetches and normalizes media from the four sites. Pure, no Telegram knowledge. - **`crates/x-media`** — library that fetches and normalizes media from the four sites. Pure, no Telegram knowledge.
- **`crates/xmedia-bot`** — the bot binary: teloxide dispatcher, SQLite-backed chat state, persistent task queue. - **`crates/xmedia-bot`** — the bot binary: teloxide dispatcher, SQLite-backed chat state, persistent task queue.
@@ -18,24 +18,30 @@ Telegram update → Dispatcher (polling or axum webhook) → dptree branches
└─ callback_query → "forward" (copy to channel) / "template|<name>" (apply caption template) └─ callback_query → "forward" (copy to channel) / "template|<name>" (apply caption template)
``` ```
Message flow: `message_handler` extracts URLs (from `url`/`text_link` entities, text + caption, deduped) → `x_media::site::fetch(url)``Fetched` → builds a `Task``send::send_media_sequence` (media groups ≤ 9, caption on first item) or `send::send_animation`. On Telegram URL-fetch failure or size error (`send_batch_via_upload`): download via `x_media::site::download_media` to a temp file (≤ 10 MiB), sniff magic bytes (`sniff_ext`), upload via multipart; oversized items fall back to `fallback_url`. On failure: `enqueue_retry` persists resume-state `Task` into the SQLite queue → workers lease (120 s lock TTL) → retry with exponential backoff (≤ 30 s, `MAX_RETRIES = 2`) → dead-letter → `notify_failure`. Success → `post_send_actions`: edit-before-forward prompt with inline buttons, or `copy_messages` to the bound forward channel. Message flow: `message_handler` extracts URLs (from `url`/`text_link` entities, text + caption, deduped) → `x_media::site::fetch(url)``Fetched` → builds a `Task``send::send_media_sequence` (media groups ≤ 10, caption on first item) or `send::send_animation`. On Telegram URL-fetch failure or size error (`send_batch_via_upload`): download via `x_media::site::download_media` to a temp file (≤ 10 MiB), sniff magic bytes (`sniff_ext`), upload via multipart; oversized items fall back to `fallback_url`. On failure: `enqueue_retry` persists resume-state `Task` into the SQLite queue → workers lease (120 s lock TTL) → retry with exponential backoff (≤ 30 s, `MAX_RETRIES = 2`) → dead-letter → `notify_failure`. Success → `post_send_actions`: edit-before-forward prompt with inline buttons, or `copy_messages` to the bound forward channel.
Debug command: `/debug <url>` runs the same `x_media::site::fetch` and replies with `debug_report` (`handlers/commands.rs`) — site id, normalized cache key, source URL, title/author/tags, sensitive flag, caption and the media list — nothing is sent, cached or forwarded; the report is capped at 4000 chars and sent with HTML parse mode: raw fields are escaped, and the caption is wrapped in a `<blockquote>` so it renders exactly like the sent media caption (escaped text and links included). Debug command: `/debug <url>` runs the same `x_media::site::fetch` and replies with `debug_report` (`handlers/commands.rs`) — site id, normalized cache key, source URL, title/author/tags, sensitive flag, caption and the media list — nothing is sent, cached or forwarded; the report is capped at 4000 chars and sent with HTML parse mode: raw fields are escaped, and the caption is wrapped in a `<blockquote>` so it renders exactly like the sent media caption (escaped text and links included). The caption it shows is `preview_caption`'s: the chat's per-site format override plus the long-post quoting, i.e. exactly what the send paths produce — showing the raw built-in caption made `/set_format` look like a no-op, and the `/set_format` success reply points users at `/debug` to preview.
The `/test <url>` command runs the ordinary link pipeline (`urls::url_media`) with `PostSend::Suppressed`: the media is sent and cached like any other link, but the chat's `forward_channel_id`/`edit_before_forward` are ignored, so a test never forwards to the channel and never opens the edit prompt (retries and dead-letter notifications behave as usual). Both commands use a custom `parse_arg_remainder` parser (whole remainder, trimmed) because teloxide's built-in `split` parser takes exactly one space-separated token. User-facing failure text is a function of the error class, never one generic sentence: `urls::fetch_error_message` maps `FetchError::NotFound` (post gone), `Sensitive` (withheld, needs `TWITTER_AUTH_TOKEN`), `Blocked` (source risk control), `Disabled { site }` (a registered site switched off — pixiv without a token, the one case `fetch` answers `Err` instead of `Ok(None)`) and `Transient`/`Http` (source down) apart. The same distinction drives the group hint: a supported link posted in a group (not a channel) gets one `GROUP_LINK_HINT` reply, because the link pipeline is private-chat only.
The `x-media` library: `site::fetch(url)` dispatches through the `SITES` registry (per-site `impl Site`, in order twitter → bsky → misskey → pixiv → bilibili) and returns `Ok(None)` for unmatched URLs. `Fetched { source_url, caption, title, content, media: Vec<Media>, sensitive, site_id, … }` (title and content are split per platform: a pixiv artwork's title and description, a bilibili headline and body, and text-only posts whose text is all `content`); `caption_with(format)` substitutes `{url} {author} {author_url} {title} {content} {tags}`. The `/test <url>` command runs the ordinary link pipeline (`urls::url_media`) with `PostSend::Suppressed`: the media is sent and cached like any other link, but the chat's `forward_channel_id`/`edit_before_forward` are ignored, so a test never forwards to the channel and never opens the edit prompt (retries and dead-letter notifications behave as usual). `/test`, `/debug`, `/set_format` and `/clear_cache` use the custom `parse_arg_remainder` parser (whole remainder, trimmed) because teloxide's built-in `split` parser takes exactly one space-separated token per field: `/set_format <site> <format>` never parsed with it (and `/clear_cache` without an argument did not either), and a command that fails to parse falls through to the URL flow in silence. `commands::tests::every_documented_invocation_parses` pins every documented form against exactly that.
The inline path (`handlers/inline.rs`) hands media URLs straight to Telegram, which fetches them itself and cannot send site-specific headers — so `x_media::site::needs_media_headers(url)` (true exactly where a site's `media_headers` is non-empty, i.e. pixiv's pximg.net) marks the media that must be skipped instead of shipped broken; locally produced media (ugoira MP4, bsky remux) fails `Url::parse` and is skipped the same way. Inline results are therefore URL-only by construction.
`url_media` is a thin wrapper over `url_media_inner`: `run_with_chat_action` sends the chat action, then re-sends it every `ACTION_REFRESH` (4 s) while the pipeline future is pending, because Telegram drops an action after ~5 s and a fetch (ugoira encode, HLS remux) plus an upload routinely outlasts that. The pipeline flips the shared `ActionHint` from `Typing` to `UploadPhoto`/`UploadVideo` once the media kinds are known. The `select!` is `biased` on the pipeline branch so a finished pipeline never emits a stray action.
The `x-media` library: `site::fetch(url)` dispatches through the `SITES` registry (per-site `impl Site`, in order twitter → bsky → misskey → pixiv → bilibili) and returns `Ok(None)` for unmatched URLs (`Err(FetchError::Disabled { site })` when the URL matches a registered site whose `enabled()` is false — see `disabled_site`). `Fetched { source_url, caption, title, content, media: Vec<Media>, sensitive, site_id, … }` (title and content are split per platform: a pixiv artwork's title and description, a bilibili headline and body, and text-only posts whose text is all `content`); `caption_with(format)` substitutes `{url} {author} {author_url} {title} {content} {tags}`.
## Key Directories ## Key Directories
| Path | Purpose | | Path | Purpose |
|---|---| |---|---|
| `crates/x-media/src/` | Fetch library. `site/mod.rs` = dispatcher + `Fetched`/`FetchError`/`download_media`/`media_size`; `media.rs` = `Media` enum; `examples/fetch.rs` = end-to-end usage sample | | `crates/x-media/src/` | Fetch library. `site/mod.rs` = dispatcher + `Fetched`/`FetchError`/`download_media`/`media_size`; `media.rs` = `Media` enum; `examples/fetch.rs` = end-to-end usage sample |
| `crates/x-media/src/site/<twitter\|pixiv\|bsky\|misskey\|bilibili>/` | One directory per site: `mod.rs` (re-exports), `interface.rs` (PATTERN, `enabled()`, `fetch_from_url()`, `cache_key`/`is_retryable`/`media_headers`, unit struct `<Name>Site` implementing `site::Site`, `From<SiteStruct> for Fetched`), `model.rs` (serde DTOs). Pixiv adds `api.rs` (auth + transport); twitter adds `auth.rs` (logged-in GraphQL `TweetDetail` fallback for NSFW tweets, gated on `TWITTER_AUTH_TOKEN`). Misskey targets misskey.io only (`POST /api/notes/show`, 400+`NO_SUCH_NOTE` → NotFound). Bilibili fetches dynamics (images/animated images only — an attached video degrades to its cover, and its title stands in for the post text, which AV dynamics do not have) from `/x/polymer/web-dynamic/v1/detail` sent with `features=itemOpusStyle` (without that flag the legacy serialization drops an image/text post's body and headline entirely — `desc` comes back `null`; the adapter still parses the legacy `major.draw`/`desc`/`archive` shapes as a fallback). No WBI signature is involved; device cookies `buvid3`/`buvid4` are fetched automatically from `/x/frontend/finger/spi` because bilibili's `-352` risk control starts rejecting plain requests, `BILIBILI_COOKIE` is the escalation when an IP stays blocked; `b23.tv` short links are deliberately unmatched. Twitter's `from_syndication_json` HTML-decodes the API text — syndication and GraphQL `full_text` both arrive pre-escaped (`&gt;` `&lt;` `&amp;` `&#39;`) — so the stored text is raw and the caption escap… | `crates/x-media/src/site/<twitter\|pixiv\|bsky\|misskey\|bilibili>/` | One directory per site: `mod.rs` (re-exports), `interface.rs` (PATTERN, `enabled()`, `fetch_from_url()`, `cache_key`/`is_retryable`/`media_headers`, unit struct `<Name>Site` implementing `site::Site`, `From<SiteStruct> for Fetched`), `model.rs` (serde DTOs). Pixiv adds `api.rs` (auth + transport); twitter adds `auth.rs` (logged-in GraphQL `TweetDetail` fallback for NSFW tweets, gated on `TWITTER_AUTH_TOKEN`; without the token a withheld tweet stays `FetchError::Sensitive` and the bot reports it as age-restricted instead of "no media"). Misskey targets misskey.io only (`POST /api/notes/show`, 400+`NO_SUCH_NOTE` → NotFound). Bilibili fetches dynamics (images/animated images only — an attached video degrades to its cover, and its title stands in for the post text, which AV dynamics do not have) from `/x/polymer/web-dynamic/v1/detail` sent with `features=itemOpusStyle` (without that flag the legacy serialization drops an image/text post's body and headline entirely — `desc` comes back `null`; the adapter still parses the legacy `major.draw`/`desc`/`archive` shapes as a fallback). No WBI signature is involved; device cookies `buvid3`/`buvid4` are fetched automatically from `/x/frontend/finger/spi` because bilibili's `-352` risk control starts rejecting plain requests, `BILIBILI_COOKIE` is the escalation when an IP stays blocked; `b23.tv` short links are deliberately unmatched. Twitter's `from_syndication_json` HTML-decodes the API text — syndication and GraphQL `full_text` both arrive pre-escaped (`&gt;` `&lt;` `&amp;` `&#39;`) — so the stored text is raw and the caption escap…
| `crates/xmedia-bot/src/main.rs` | Entry point: env/log init, command registration (`register_commands`), shared `send::BOT` force-init, queue worker start, site login validation (`site::validate_all`), 300 s edit-expiry sweep, dptree handler tree, webhook vs polling dispatch | | `crates/xmedia-bot/src/main.rs` | Entry point: env/log init, command registration (`register_commands``setMyCommands` plus the profile description texts), shared `send::BOT` force-init, queue worker start, site login validation (`site::validate_all`), 300 s edit-expiry sweep (expired prompts are rewritten in place to `EDIT_PROMPT_EXPIRED_TEXT` with an empty keyboard — an edit, never a new message, so a background timer cannot wake a chat), dptree handler tree, webhook vs polling dispatch |
| `crates/xmedia-bot/src/config.rs` | Manual env parsing into `Config` | | `crates/xmedia-bot/src/config.rs` | Manual env parsing into `Config` |
| `crates/xmedia-bot/src/db.rs` | `DbPool`: one shared SQLite connection pool (`POOL_SIZE = 4`, WAL, busy_timeout) for all three tables over `$DATA_DIR/task_queue.db` (default `data/`) — the three stores share it; `open_store` creates file + schema, `with_conn` runs all rusqlite I/O in `spawn_blocking` | | `crates/xmedia-bot/src/db.rs` | `DbPool`: one shared SQLite connection pool (`POOL_SIZE = 4`, WAL, busy_timeout) for all three tables over `$DATA_DIR/task_queue.db` (default `data/`) — the three stores share it; `open_store` creates file + schema, `with_conn` runs all rusqlite I/O in `spawn_blocking` |
| `crates/xmedia-bot/src/handlers/` | Handler modules: `mod.rs` (message entry point, `reply`, `log_key`), `commands.rs` (teloxide `BotCommands` enum + command executor, incl. `/test <url>` (send-only) / `/debug <url>` (parse-only) and the admin-only `/bot_dict` state dump), `urls.rs` (URL extraction + bounded job channel (256) drained by `URL_WORKERS = 8` workers (`start_url_workers`) — backpressure instead of unbounded spawns; teloxide's per-chat workers are sequential — batch-forwards need concurrency), `inline.rs`/`callback.rs` (inline queries / edit-before-forward buttons), `statics.rs` (global statics) | | `crates/xmedia-bot/src/handlers/` | Handler modules: `mod.rs` (message entry point, `reply`, `log_key`, the group-only `GROUP_LINK_HINT` for a supported link posted outside a private chat), `commands.rs` (teloxide `BotCommands` enum + command executor, incl. `/test <url>` (send-only) / `/debug <url>` (parse-only) and the admin-only `/bot_dict` state dump; `/set_format` rejects unknown `{…}` placeholders and resets with `-`), `urls.rs` (URL extraction + bounded job channel (256) drained by `URL_WORKERS = 8` workers (`start_url_workers`) — backpressure instead of unbounded spawns; teloxide's per-chat workers are sequential — batch-forwards need concurrency), `inline.rs`/`callback.rs` (inline queries / edit-before-forward buttons, incl. `skip`), `statics.rs` (global statics) |
| `crates/xmedia-bot/src/state.rs` | `ChatStore`: parking_lot `Mutex<HashMap>` cache + SQLite write-through (`chat_state` table) | | `crates/xmedia-bot/src/state.rs` | `ChatStore`: parking_lot `Mutex<HashMap>` cache + SQLite write-through (`chat_state` table) |
| `crates/xmedia-bot/src/link_cache.rs` | `LinkCache`: SQLite-backed cache (`link_cache` table) of successfully sent posts — raw caption fields + Telegram `file_id`s; repeat links re-send locally (no fetch/upload), TTL + prune, invalidated on permanent send failure | | `crates/xmedia-bot/src/link_cache.rs` | `LinkCache`: SQLite-backed cache (`link_cache` table) of successfully sent posts — raw caption fields + Telegram `file_id`s; repeat links re-send locally (no fetch/upload), TTL + prune, invalidated on permanent send failure |
| `crates/xmedia-bot/src/queue.rs` | `PersistentTaskQueue`: SQLite-backed queue (`tasks` table), `QUEUE_WORKERS = 4` concurrent workers (lease via `BEGIN IMMEDIATE` + `locked_until` TTL), retry→dead-letter, `notify_one` worker wakeup plus a separate `Notify` for the 30 s lease-expiry sweep (a shared one let the sweep steal the workers' wakeup permit), `busy_timeout` on all connections | | `crates/xmedia-bot/src/queue.rs` | `PersistentTaskQueue`: SQLite-backed queue (`tasks` table), `QUEUE_WORKERS = 4` concurrent workers (lease via `BEGIN IMMEDIATE` + `locked_until` TTL), retry→dead-letter, `notify_one` worker wakeup plus a separate `Notify` for the 30 s lease-expiry sweep (a shared one let the sweep steal the workers' wakeup permit), `busy_timeout` on all connections |
@@ -60,7 +66,7 @@ Docker: `docker build -t tgxmb .` then `docker run --rm -d --name tgxmb --env-fi
## Code Conventions & Common Patterns ## Code Conventions & Common Patterns
- **Errors via `thiserror` derive** (no anyhow): the public, stringified errors — `FetchError` (`Http`/`Json`/`Pixiv`/`Site`/`NotFound`/`Blocked`) and `PixivError` — derive `thiserror::Error` with `#[from]` conversions; `Display`/`source()` come from the derive. The internal control-flow enums — `QueueError` (`Retryable { delay_seconds, payload }` / `Permanent`), `SendError` (Retryable/Permanent), `Classification`, `FallbackError` — carry no `Display` and are handled by direct variant matching. New errors should follow the same split: stringified/public errors derive `thiserror`, internal flow enums stay plain. - **Errors via `thiserror` derive** (no anyhow): the public, stringified errors — `FetchError` (`Http`/`Json`/`Pixiv`/`Site`/`NotFound`/`Blocked`/`Disabled`/`Sensitive`/`TooLarge`/`Transient`/`Io`) and `PixivError` — derive `thiserror::Error` with `#[from]` conversions; `Display`/`source()` come from the derive. The internal control-flow enums — `QueueError` (`Retryable { delay_seconds, payload }` / `Permanent`), `SendError` (Retryable/Permanent), `Classification`, `FallbackError` — carry no `Display` and are handled by direct variant matching. New errors should follow the same split: stringified/public errors derive `thiserror`, internal flow enums stay plain.
- **Global state via `std::sync::LazyLock` statics**, not DI: `CONFIG`, `CHAT_STORE`, `TASK_QUEUE` in `handlers/statics.rs`; shared reqwest `CLIENT` in `x-media/src/site/mod.rs`. `Bot` is passed/cloned into handlers; queue workers share the process-wide `send::BOT` (`LazyLock<Bot>`, force-initialized in `main` so a missing token fails at startup). - **Global state via `std::sync::LazyLock` statics**, not DI: `CONFIG`, `CHAT_STORE`, `TASK_QUEUE` in `handlers/statics.rs`; shared reqwest `CLIENT` in `x-media/src/site/mod.rs`. `Bot` is passed/cloned into handlers; queue workers share the process-wide `send::BOT` (`LazyLock<Bot>`, force-initialized in `main` so a missing token fails at startup).
- **Async**: tokio multi-thread runtime (`#[tokio::main]` default). All rusqlite I/O inside `tokio::task::spawn_blocking`. Long loops use `tokio::select!` with `tokio::sync::{watch, Notify}` stop/wake channels. No streams. - **Async**: tokio multi-thread runtime (`#[tokio::main]` default). All rusqlite I/O inside `tokio::task::spawn_blocking`. Long loops use `tokio::select!` with `tokio::sync::{watch, Notify}` stop/wake channels. No streams.
- **Blocking sync primitives**: `parking_lot::Mutex` for hot caches, `tokio::sync::Mutex` for async-shared state (pixiv token cache), `AtomicBool` for feature gates. - **Blocking sync primitives**: `parking_lot::Mutex` for hot caches, `tokio::sync::Mutex` for async-shared state (pixiv token cache), `AtomicBool` for feature gates.
@@ -75,10 +81,10 @@ Docker: `docker build -t tgxmb .` then `docker run --rm -d --name tgxmb --env-fi
| File | Why it matters | | File | Why it matters |
|---|---| |---|---|
| `crates/xmedia-bot/src/main.rs` | Startup sequence, webhook vs polling, graceful shutdown (SIGINT via teloxide ctrlc / SIGTERM via `stop_token` for docker, → sweep stop → admin msg → queue stop) | | `crates/xmedia-bot/src/main.rs` | Startup sequence, webhook vs polling, graceful shutdown (SIGINT via teloxide ctrlc / SIGTERM via `stop_token` for docker, → sweep stop → admin msg → queue stop) |
| `crates/xmedia-bot/src/handlers/` | `statics.rs` = `CHAT_STORE`/`TASK_QUEUE`/`CONFIG` singletons (open `$DATA_DIR/task_queue.db`, default `data/` **relative to CWD**, dir auto-created); `commands.rs` = command dispatch (incl. `/test <url>` send-only, `/debug <url>` parse-only, and the admin-only `/bot_dict` state dump); `urls.rs` = URL extraction + the per-URL pipeline (`url_media` takes a `PostSend` mode: chat settings vs `/test`'s suppressed actions); `inline.rs` = debounced inline queries; `callback.rs` = edit-before-forward buttons (dptree entry + testable `handle_callback` core) | | `crates/xmedia-bot/src/handlers/` | `statics.rs` = `CHAT_STORE`/`TASK_QUEUE`/`CONFIG` singletons (open `$DATA_DIR/task_queue.db`, default `data/` **relative to CWD**, dir auto-created); `commands.rs` = command dispatch (incl. `/test <url>` send-only, `/debug <url>` parse-only, the read-only `/settings` every chat member can read — unlike the admin-only `/bot_dict` raw dump — and template removal; `/start`/`/help` carry the guidance teloxide's `descriptions()` cannot render, and `/set_format` rejects unknown `{…}` placeholders, resetting with `-`); `urls.rs` = URL extraction + the per-URL pipeline (`url_media` takes a `PostSend` mode: chat settings vs `/test`'s suppressed actions); `inline.rs` = debounced inline queries (hotlink-protected and local media skipped); `callback.rs` = edit-before-forward buttons (dptree entry + testable `handle_callback` core, incl. `skip`) |
| `crates/xmedia-bot/src/send/` | `mod.rs`: constants `MAX_MEDIA_GROUP = 9`; `classify_request_error`; the senders. `upload.rs`: download-and-reupload fallback triggered only by Telegram API errors (`is_media_fetch_failure` / `is_size_error`). `post_send.rs`: settlement (`settle_task`), cache write, post-send actions, queue handlers. `input_media.rs`: payload → `InputMedia` | | `crates/xmedia-bot/src/send/` | `mod.rs`: constants `MAX_MEDIA_GROUP = 10`; `classify_request_error`; the senders. `upload.rs`: download-and-reupload fallback triggered only by Telegram API errors (`is_media_fetch_failure` / `is_size_error`). `post_send.rs`: settlement (`settle_task`), cache write, post-send actions (dead-letter text via `failure_text`: post key + cause, since the raw error alone does not say which link died), queue handlers. `input_media.rs`: payload → `InputMedia` |
| `crates/xmedia-bot/src/photo.rs` | Pure-Rust photo processing (no ffmpeg): `png` (image-png) decode/encode + `zune-jpeg` decode + `fast_image_resize` Lanczos3 downscale + `jpeg-encoder`. Photos over Telegram's limits (width + height > 10000 px → `PHOTO_INVALID_DIMENSIONS`; bytes > 10 MiB) are decoded, downscaled keeping the format, PNG bit depth > 24 (RGBA 32-bit / 16-bit per channel) reduced to 24-bit RGB with alpha flattened white (≤24-bit untouched, never upconverted), and transcoded to JPEG only if still over the cap; memory budget guarded, otherwise the item's smaller fallback URL | | `crates/xmedia-bot/src/photo.rs` | Pure-Rust photo processing (no ffmpeg): `png` (image-png) decode/encode + `zune-jpeg` decode + `fast_image_resize` Lanczos3 downscale + `jpeg-encoder`. Photos over Telegram's limits (width + height > 10000 px → `PHOTO_INVALID_DIMENSIONS`; bytes > 10 MiB) are decoded, downscaled keeping the format, PNG bit depth > 24 (RGBA 32-bit / 16-bit per channel) reduced to 24-bit RGB with alpha flattened white (≤24-bit untouched, never upconverted), and transcoded to JPEG only if still over the cap; memory budget guarded, otherwise the item's smaller fallback URL |
| `crates/x-media/src/site/mod.rs` | Dispatcher, `Fetched`/`FetchError`, shared `CLIENT`, `download_media` (adds `Referer: https://www.pixiv.net/` for `pximg.net` hotlink protection) | | `crates/x-media/src/site/mod.rs` | Dispatcher, `Fetched`/`FetchError`, shared `CLIENT`, `download_media` (adds `Referer: https://www.pixiv.net/` for `pximg.net` hotlink protection), `needs_media_headers` (the same per-site rule, asked by the inline path to skip what Telegram cannot fetch) |
| `crates/x-media/src/site/pixiv/api.rs` | OAuth token exchange (hardcoded app client id/secret), access-token cache, ugoira zip→MP4 via ffmpeg in `spawn_blocking` | | `crates/x-media/src/site/pixiv/api.rs` | OAuth token exchange (hardcoded app client id/secret), access-token cache, ugoira zip→MP4 via ffmpeg in `spawn_blocking` |
| `Dockerfile` | Multi-stage: cached dep layer via stub sources + `touch *.rs` mtime bump (cargo's freshness is mtime-based and `cargo clean -p` removes 0 files — the touch is what forces the real sources to rebuild while deps stay cached), static ffmpeg from ffmpeg.martin-riedl.de (`FFMPEG_URL` arg, optional `FFMPEG_SHA256` checksum, `unzip -t` integrity check), `debian:bookworm-slim` runtime, entrypoint. Runtime ships **no libssl/libcrypto/CA bundle** — rustls webpki-roots handles all TLS, and the static ffmpeg only processes local files (downloads go through reqwest) | | `Dockerfile` | Multi-stage: cached dep layer via stub sources + `touch *.rs` mtime bump (cargo's freshness is mtime-based and `cargo clean -p` removes 0 files — the touch is what forces the real sources to rebuild while deps stay cached), static ffmpeg from ffmpeg.martin-riedl.de (`FFMPEG_URL` arg, optional `FFMPEG_SHA256` checksum, `unzip -t` integrity check), `debian:bookworm-slim` runtime, entrypoint. Runtime ships **no libssl/libcrypto/CA bundle** — rustls webpki-roots handles all TLS, and the static ffmpeg only processes local files (downloads go through reqwest) |
| `docker-entrypoint.sh` | Privilege drop: `useradd` with `LOCAL_USER_ID` (default 9001) + `setpriv` (no gosu on bookworm-slim) | | `docker-entrypoint.sh` | Privilege drop: `useradd` with `LOCAL_USER_ID` (default 9001) + `setpriv` (no gosu on bookworm-slim) |
@@ -99,9 +105,9 @@ Docker: `docker build -t tgxmb .` then `docker run --rm -d --name tgxmb --env-fi
## Testing & QA ## Testing & QA
- **~180 tests, all inline `#[cfg(test)] mod tests`** — no `tests/` integration directories. Framework: built-in Rust test + `#[tokio::test]` (dev-deps only in `x-media`: tokio macros/rt-multi-thread, dotenv). - **~200 tests, all inline `#[cfg(test)] mod tests`** — no `tests/` integration directories. Framework: built-in Rust test + `#[tokio::test]` (dev-deps only in `x-media`: tokio macros/rt-multi-thread, dotenv).
- No mocking framework anywhere (no mockito/wiremock/mockall). Conventions: pure-function units (regex parsing, serde round-trips, chunking, retry math) tested synchronously; async tests use real dependencies — file-backed SQLite via `tempfile` (`queue.rs::new_queue()` helper), live network fetches. - No mocking framework anywhere (no mockito/wiremock/mockall). Conventions: pure-function units (regex parsing, serde round-trips, chunking, retry math) tested synchronously; async tests use real dependencies — file-backed SQLite via `tempfile` (`queue.rs::new_queue()` helper), live network fetches.
- Live-network tests exist in `site/twitter/interface.rs` (5), `site/bsky/interface.rs` (2), `site/misskey/interface.rs` (1), `site/bilibili/interface.rs` (4), `site/pixiv/api.rs` (1); `photo.rs` adds one `#[ignore = "heavy: …"]` test. `site/mod.rs` also has a **token-gated but not `#[ignore]`d** pixiv download test (`download_media_pixiv_original_with_referer`): it hits `i.pximg.net` whenever `PIXIV_REFRESH_TOKEN` is set, so a local `cargo test --workspace` is not fully offline and can flake on a pixiv CDN body timeout. Test gating convention (enforced by `.github/workflows/ci.yml`): pure unit tests always run; live-network tests carry `#[ignore = "live network: ..."]` (run via `cargo test --workspace -- --ignored live`); token-gated pixiv tests early-return when `PIXIV_REFRESH_TOKEN` is absent **or empty** (an unset GitHub secret arrives as `""``is_err()` alone would run them tokenless and fail), and the bilibili live tests early-return when the API answers risk control (`-352`, which bilibili applies per IP by request volume). Run the full offline suite with `cargo test --workspace`. - Live-network tests exist in `site/twitter/interface.rs` (5), `site/bsky/interface.rs` (2), `site/misskey/interface.rs` (1), `site/bilibili/interface.rs` (4), `site/pixiv/api.rs` (1); `photo.rs` adds one `#[ignore = "heavy: …"]` test. `site/mod.rs` also has a **token-gated but not `#[ignore]`d** pixiv download test (`download_media_pixiv_original_with_referer`): it hits `i.pximg.net` whenever `PIXIV_REFRESH_TOKEN` is set, so a local `cargo test --workspace` is not fully offline and can flake on a pixiv CDN body timeout. `disabled_site_is_reported_not_ignored` (same file) is gated the other way round: it asserts `fetch` answers `FetchError::Disabled { site: "pixiv" }` for a pixiv link and early-returns when `PIXIV_REFRESH_TOKEN` **is** set (the site is then enabled). Test gating convention (enforced by `.github/workflows/ci.yml`): pure unit tests always run; live-network tests carry `#[ignore = "live network: ..."]` (run via `cargo test --workspace -- --ignored live`); token-gated pixiv tests early-return when `PIXIV_REFRESH_TOKEN` is absent **or empty** (an unset GitHub secret arrives as `""``is_err()` alone would run them tokenless and fail), and the bilibili live tests early-return when the API answers risk control (`-352`, which bilibili applies per IP by request volume). Run the full offline suite with `cargo test --workspace`.
- Fixtures are inline `serde_json::json!` builder fns (`fixture()`, `thread_json()`, `illust_json()`), not files. The shared `CLIENT` sets `pool_max_idle_per_host(0)` under `#[cfg(test)]` to avoid cross-runtime `DispatchGone`. - Fixtures are inline `serde_json::json!` builder fns (`fixture()`, `thread_json()`, `illust_json()`), not files. The shared `CLIENT` sets `pool_max_idle_per_host(0)` under `#[cfg(test)]` to avoid cross-runtime `DispatchGone`.
- **CI** — `.github/workflows/ci.yml` (actions pinned to commit SHAs, `--locked` on every cargo invocation, `concurrency` cancels superseded runs, `RUST_BACKTRACE=1`) runs `cargo fmt --check` + `cargo clippy --workspace --all-targets --locked -- -D warnings` + `cargo test --workspace --locked` + a release-profile `cargo build --release --locked` + an `actions-rust-lang/audit` dependency-vulnerability gate (offline, no secrets, on every push/PR) and a `live` job (schedule/manual/tag only, `-p x-media` since every network/secret-gated test lives there, `continue-on-error`) for the `#[ignore]`d live + token tests. `.github/workflows/docker.yml` builds and pushes the image on master/tag and runs a **build-only check on pull requests touching the build inputs** (`Dockerfile`, entrypoint, manifests, `.dockerignore`); a release tag must match both crate versions or the build stops, and `FFMPEG_URL`/`FFMPEG_SHA256` are taken from repository variables when set (a release can pin an exact ffmpeg build). `.github/dependabot.yml` keeps crates, the pinned actions and the Docker base images current. - **CI** — `.github/workflows/ci.yml` (actions pinned to commit SHAs, `--locked` on every cargo invocation, `concurrency` cancels superseded runs, `RUST_BACKTRACE=1`) runs `cargo fmt --check` + `cargo clippy --workspace --all-targets --locked -- -D warnings` + `cargo test --workspace --locked` + a release-profile `cargo build --release --locked` + an `actions-rust-lang/audit` dependency-vulnerability gate (offline, no secrets, on every push/PR) and a `live` job (schedule/manual/tag only, `-p x-media` since every network/secret-gated test lives there, `continue-on-error`) for the `#[ignore]`d live + token tests. `.github/workflows/docker.yml` builds and pushes the image on master/tag and runs a **build-only check on pull requests touching the build inputs** (`Dockerfile`, entrypoint, manifests, `.dockerignore`); a release tag must match both crate versions or the build stops, and `FFMPEG_URL`/`FFMPEG_SHA256` are taken from repository variables when set (a release can pin an exact ffmpeg build). `.github/dependabot.yml` keeps crates, the pinned actions and the Docker base images current.
- Untested and hard to test without a mock seam: `main.rs`, `config.rs`, `db.rs`, `handlers/statics.rs`, `media_sender.rs` (holds the `MockSender` itself); in `x-media`: `media.rs`, `lib.rs`, all `model.rs`. The `commands.rs` *executor* needs a real `Bot` (only its pure report builder is tested). Everything else — `handlers/{mod,callback,inline,urls}.rs`, `send/*`, `ctx.rs`, `state.rs`, `queue.rs`, `link_cache.rs`, `rate_limit.rs` — is driven through `TestStores`/`ctx::test_support` and the scripted `MockSender`. - Untested and hard to test without a mock seam: `main.rs`, `config.rs`, `db.rs`, `handlers/statics.rs`, `media_sender.rs` (holds the `MockSender` itself); in `x-media`: `media.rs`, `lib.rs`, all `model.rs`. The `commands.rs` *executor* needs a real `Bot` (only its pure report builder is tested). Everything else — `handlers/{mod,callback,inline,urls}.rs`, `send/*`, `ctx.rs`, `state.rs`, `queue.rs`, `link_cache.rs`, `rate_limit.rs` — is driven through `TestStores`/`ctx::test_support` and the scripted `MockSender`.
Generated
+2 -2
View File
@@ -2818,7 +2818,7 @@ checksum = "3ad82d2a33cdc9674dc7465672f271e096168fcdbe0f799d9e6db8c5892679dc"
[[package]] [[package]]
name = "x-media" name = "x-media"
version = "1.7.0" version = "1.8.0"
dependencies = [ dependencies = [
"bytes", "bytes",
"dotenv", "dotenv",
@@ -2838,7 +2838,7 @@ dependencies = [
[[package]] [[package]]
name = "xmedia-bot" name = "xmedia-bot"
version = "1.7.0" version = "1.8.0"
dependencies = [ dependencies = [
"bytes", "bytes",
"dotenv", "dotenv",
+16 -11
View File
@@ -4,12 +4,15 @@ A Telegram bot that turns post links from X / Twitter, Pixiv, Bluesky, Misskey (
## Features ## Features
- Sending a link in a private chat fetches and sends the images, videos and GIFs automatically; oversized media is split into batches - Sending a link in a private chat fetches and sends the images, videos and GIFs automatically; oversized media is split into batches (10 items per group)
- Text-only posts report "no media"; unsupported links are silently ignored - Text-only posts report "no media"; unsupported links are silently ignored. Fetch failures name the reason (post gone / content withheld / source risk control / site not enabled)
- Long posts (text ≥ `CAPTION_QUOTE_TEXT_CHARS`, default 200) show **the text part** of their caption inside a collapsible blockquote, with the link and author line left outside it - Long posts (text ≥ `CAPTION_QUOTE_TEXT_CHARS`, default 200) show **the text part** of their caption inside a collapsible blockquote, with the link and author line left outside it
- Inline queries (`@bot <link>`) - Inline queries (`@bot <link>`) — except Pixiv images and locally transcoded animations, which Telegram cannot fetch (no Referer) and would show broken, so they are skipped; a supported link posted in a group gets a one-line hint to use the private chat or inline mode (channels stay silent)
- Bind a forward channel for automatic forwarding; edit the caption before forwarding and apply custom templates - `/start` explains the supported sites and how to use it; `/help` lists the commands plus argument syntax, the caption placeholders and the private-chat rule; the bot's profile description texts are set at startup
- Failed sends are retried automatically with persistence; the user is notified after retries are exhausted - `/settings` shows this chat's configuration (forward channel, edit-before-forward, per-site caption formats, saved templates); templates are added with `/set_template` and removed with `/remove_template`
- Bind a forward channel for automatic forwarding; edit the caption before forwarding and apply custom templates (the prompt carries Confirm / Skip buttons, states its expiry, and is marked expired in place once it lapses)
- Failed sends are retried automatically with persistence; the notice names which link failed, how long the retry waits, or the final cause
- The chat action stays on screen for the whole fetch, so long jobs (ugoira transcode, large uploads) do not look stalled
- Pixiv ugoira animations are transcoded to MP4; Bluesky videos are remuxed (HLS stream → MP4) - Pixiv ugoira animations are transcoded to MP4; Bluesky videos are remuxed (HLS stream → MP4)
- Photos exceeding Telegram's size/dimension limits are compressed automatically (original format kept, JPEG fallback only when needed) - Photos exceeding Telegram's size/dimension limits are compressed automatically (original format kept, JPEG fallback only when needed)
- Link-result cache: after a successful send the Telegram file ids and caption fields are cached locally, so a repeated link is re-sent from local state — no source-site request, no media file stored (expiry controlled by `LINK_CACHE_TTL_SECONDS`, default 7 days) - Link-result cache: after a successful send the Telegram file ids and caption fields are cached locally, so a repeated link is re-sent from local state — no source-site request, no media file stored (expiry controlled by `LINK_CACHE_TTL_SECONDS`, default 7 days)
@@ -33,7 +36,7 @@ docker run --rm -d --name tgxmb --env-file .env -v ./data:/app/data tgxmb
Environment variables: `TELOXIDE_TOKEN` (required), `PIXIV_REFRESH_TOKEN`, `BOT_ADMIN`, `EDIT_MESSAGE_TTL_SECONDS`, `LINK_CACHE_TTL_SECONDS`, `RUST_LOG`, `TELOXIDE_PROXY`, `WEBHOOK*`, `TWITTER_AUTH_TOKEN` (optional), `BILIBILI_COOKIE` (optional). Environment variables: `TELOXIDE_TOKEN` (required), `PIXIV_REFRESH_TOKEN`, `BOT_ADMIN`, `EDIT_MESSAGE_TTL_SECONDS`, `LINK_CACHE_TTL_SECONDS`, `RUST_LOG`, `TELOXIDE_PROXY`, `WEBHOOK*`, `TWITTER_AUTH_TOKEN` (optional), `BILIBILI_COOKIE` (optional).
NSFW tweets: the public syndication endpoint does not return sensitive content. Setting `TWITTER_AUTH_TOKEN` (the `auth_token` cookie value of a logged-in x.com session) lets the bot fetch NSFW media in the logged-in state only when it hits a withheld tweet; without it, the bot reports no media. NSFW tweets: the public syndication endpoint does not return sensitive content. Setting `TWITTER_AUTH_TOKEN` (the `auth_token` cookie value of a logged-in x.com session) lets the bot fetch NSFW media in the logged-in state only when it hits a withheld tweet; without it the bot answers that the post's media is withheld and needs `TWITTER_AUTH_TOKEN`.
Bilibili dynamics are fetched anonymously by default (no login; the bot fetches bilibili's anonymous `buvid3`/`buvid4` device cookies itself to raise the success rate). If the server's egress IP gets hard-flagged by bilibili (persistent `risk control (-352)` log lines or HTTP 412), set `BILIBILI_COOKIE` (the whole cookie string from a logged-in browser, e.g. `SESSDATA=…; bili_jct=…`) to restore access. Only a dynamic's images and animations are sent; an attached video degrades to its cover image. Bilibili dynamics are fetched anonymously by default (no login; the bot fetches bilibili's anonymous `buvid3`/`buvid4` device cookies itself to raise the success rate). If the server's egress IP gets hard-flagged by bilibili (persistent `risk control (-352)` log lines or HTTP 412), set `BILIBILI_COOKIE` (the whole cookie string from a logged-in browser, e.g. `SESSDATA=…; bili_jct=…`) to restore access. Only a dynamic's images and animations are sent; an attached video degrades to its cover image.
@@ -83,10 +86,10 @@ Telegram only accepts ports 443/80/88/8443.
| Variable | Description | | Variable | Description |
|---|---| |---|---|
| `TELOXIDE_TOKEN` | Bot token (required) | | `TELOXIDE_TOKEN` | Bot token (required) |
| `PIXIV_REFRESH_TOKEN` | Pixiv refresh token; Pixiv is disabled without it | | `PIXIV_REFRESH_TOKEN` | Pixiv refresh token; Pixiv is disabled without it (a pixiv link then gets an explicit "site not enabled" reply instead of silence) |
| `BILIBILI_COOKIE` | Optional bilibili cookie string (`SESSDATA=…; bili_jct=…`); only needed when the egress IP stays risk-controlled (device cookies are fetched automatically) | | `BILIBILI_COOKIE` | Optional bilibili cookie string (`SESSDATA=…; bili_jct=…`); only needed when the egress IP stays risk-controlled (device cookies are fetched automatically) |
| `BOT_ADMIN` | Admin chat IDs, comma-separated; receives start/stop notifications | | `BOT_ADMIN` | Admin chat IDs, comma-separated; receives start/stop notifications |
| `EDIT_MESSAGE_TTL_SECONDS` | Edit-before-forward record expiry in seconds, default 86400 | | `EDIT_MESSAGE_TTL_SECONDS` | Edit-before-forward record expiry in seconds, default 86400; once lapsed the prompt is rewritten in place to "expired — nothing was forwarded" (no extra message) |
| `LINK_CACHE_TTL_SECONDS` | Link-result cache expiry in seconds, default 604800 (7 days) | | `LINK_CACHE_TTL_SECONDS` | Link-result cache expiry in seconds, default 604800 (7 days) |
| `CAPTION_QUOTE_TEXT_CHARS` | **The text part** of the caption (the joined `{title}` + `{content}`) is wrapped in a collapsible blockquote once it reaches this many characters, default 200; `0` disables | | `CAPTION_QUOTE_TEXT_CHARS` | **The text part** of the caption (the joined `{title}` + `{content}`) is wrapped in a collapsible blockquote once it reaches this many characters, default 200; `0` disables |
| `DATA_DIR` | Data directory (where the SQLite `task_queue.db` lives), default `data` (relative to the working directory, created automatically) | | `DATA_DIR` | Data directory (where the SQLite `task_queue.db` lives), default `data` (relative to the working directory, created automatically) |
@@ -114,15 +117,17 @@ Telegram only accepts ports 443/80/88/8443.
| `/help` | List all commands and usage (this command table) | | `/help` | List all commands and usage (this command table) |
| `/set_forward_channel <channel>` | Set the forward channel: `@channel` or channel ID; media messages are forwarded to it automatically afterwards | | `/set_forward_channel <channel>` | Set the forward channel: `@channel` or channel ID; media messages are forwarded to it automatically afterwards |
| `/remove_forward_channel` | Remove the forward channel | | `/remove_forward_channel` | Remove the forward channel |
| `/edit_before_forward` | Toggle "edit before forward": when enabled, the bot posts a prompt after forwarding; replying to it edits the first forwarded message's caption (or taps a template button to apply one) | | `/edit_before_forward` | Toggle "edit before forward": when enabled, the bot posts a prompt after forwarding; replying to it edits the first forwarded message's caption (or tapping a template button applies one), then `↩️ Confirm` forwards and `🛑 Skip` drops this forward; the prompt states its expiry and is marked expired in place when it lapses (nothing is forwarded) |
| `/set_template <name>` | Reply to a message containing `[]` to save it as a named template; `[]` is replaced by the original post link when forwarding (used with "edit before forward") | | `/set_template <name>` | Reply to a message containing `[]` to save it as a named template; `[]` is replaced by the original post link when forwarding (used with "edit before forward") |
| `/set_format <site> <format>` | Customize the caption format for one site. Sites: `twitter` / `bsky` / `pixiv` / `misskey` / `bilibili`. Placeholders: `{url}` `{author}` `{author_url}` `{title}` `{content}` `{tags}` | | `/remove_template <name>` | Remove a template (names are listed by `/settings`; the prompt's keyboard shows at most 60) |
| `/settings` | Show this chat's configuration: forward channel, edit-before-forward, per-site caption formats, saved templates |
| `/set_format <site> <format>` | Customize the caption format for one site. Sites: `twitter` / `bsky` / `pixiv` / `misskey` / `bilibili`. Placeholders: `{url}` `{author}` `{author_url}` `{title}` `{content}` `{tags}`; unknown placeholders are rejected with the list of valid ones, and `-` restores the site's built-in format (preview with `/debug <link>`) |
| `/clear_cache [link]` | Clear the link cache (admin only); with a link only that entry, otherwise everything | | `/clear_cache [link]` | Clear the link cache (admin only); with a link only that entry, otherwise everything |
| `/bot_dict` | Show the current chat state (debugging; admin only) | | `/bot_dict` | Show the current chat state (debugging; admin only) |
| `/test <link>` | Parse a link and send its media; no channel forward, no edit-before-forward prompt (send only) | | `/test <link>` | Parse a link and send its media; no channel forward, no edit-before-forward prompt (send only) |
| `/debug <link>` | Debug: parse a link and report the parse result only (site, title, author, tags, media list) — no media is sent | | `/debug <link>` | Debug: parse a link and report the parse result only (site, title, author, tags, media list) — no media is sent |
Link processing works only in private chats; commands work in any chat. Link processing works only in private chats; commands work in any chat. A supported link posted in a group gets a one-line hint to use the private chat or inline mode; channels stay silent.
## Notes ## Notes
+16 -11
View File
@@ -4,12 +4,15 @@ Telegram 机器人,将 X / Twitter、Pixiv、Bluesky、Misskey (misskey.io)、
## 功能 ## 功能
- 私聊发送链接后自动抓取并发送图片、视频与 GIF,超量图片自动分批 - 私聊发送链接后自动抓取并发送图片、视频与 GIF,超量图片自动分批(每批 10 张)
- 纯文字帖提示无媒体;不支持的链接静默忽略 - 纯文字帖提示无媒体;不支持的链接静默忽略。抓取失败会按原因分别提示(帖子已删除 / 内容受限 / 源站风控 / 站点未启用)
- 长帖(正文 ≥ `CAPTION_QUOTE_TEXT_CHARS`,默认 200)的**正文部分**用可折叠引用块展示,链接与作者行留在引用块外 - 长帖(正文 ≥ `CAPTION_QUOTE_TEXT_CHARS`,默认 200)的**正文部分**用可折叠引用块展示,链接与作者行留在引用块外
- 支持内联查询(`@机器人 <链接>` - 支持内联查询(`@机器人 <链接>`;Pixiv 图片与本地转码的动图不支持内联 —— Telegram 取图时无法携带 Referer,会显示破图,因此跳过);在群聊里发链接会提示改用私聊或内联查询(频道内保持静默
- 可绑定转发频道自动转发;支持转发前编辑 caption 与自定义模板 - `/start` 说明支持的站点与用法,`/help` 列出命令、参数格式、caption 占位符与私聊限制;bot 资料页(description / short description)启动时一并设置
- 发送失败自动重试并持久化,重试耗尽后通知用户 - `/settings` 查看本聊天配置(转发频道、转发前编辑开关、各站点 caption 格式、模板列表);模板可用 `/set_template` 增、`/remove_template`
- 可绑定转发频道自动转发;支持转发前编辑 caption 与自定义模板(提示消息带 Confirm / Skip 按钮并写明过期时间,过期后就地标记为已过期)
- 发送失败自动重试并持久化,重试耗尽后通知用户;提示会写明是哪条链接、重试等待多久、或最终失败的原因
- 抓取期间持续显示"正在输入 / 正在发送"状态,长任务(ugoira 转码、大图上传)不会看起来卡死
- Pixiv ugoira 动图自动转码为 MP4Bluesky 视频自动转码(HLS 流 → MP4) - Pixiv ugoira 动图自动转码为 MP4Bluesky 视频自动转码(HLS 流 → MP4)
- 超过 Telegram 尺寸/大小限制的图片自动压缩(保持原格式,必要时转 JPEG) - 超过 Telegram 尺寸/大小限制的图片自动压缩(保持原格式,必要时转 JPEG)
- 链接结果本地缓存:成功发送后缓存 Telegram file id 与 caption 等,再次收到相同链接直接本地重发,不再请求源站、不保存媒体文件(`LINK_CACHE_TTL_SECONDS` 控制过期,默认 7 天) - 链接结果本地缓存:成功发送后缓存 Telegram file id 与 caption 等,再次收到相同链接直接本地重发,不再请求源站、不保存媒体文件(`LINK_CACHE_TTL_SECONDS` 控制过期,默认 7 天)
@@ -33,7 +36,7 @@ docker run --rm -d --name tgxmb --env-file .env -v ./data:/app/data tgxmb
环境变量:`TELOXIDE_TOKEN`(必填)、`PIXIV_REFRESH_TOKEN``BOT_ADMIN``EDIT_MESSAGE_TTL_SECONDS``LINK_CACHE_TTL_SECONDS``RUST_LOG``TELOXIDE_PROXY``WEBHOOK*``TWITTER_AUTH_TOKEN`(可选)、`BILIBILI_COOKIE`(可选)。 环境变量:`TELOXIDE_TOKEN`(必填)、`PIXIV_REFRESH_TOKEN``BOT_ADMIN``EDIT_MESSAGE_TTL_SECONDS``LINK_CACHE_TTL_SECONDS``RUST_LOG``TELOXIDE_PROXY``WEBHOOK*``TWITTER_AUTH_TOKEN`(可选)、`BILIBILI_COOKIE`(可选)。
NSFW 推文:公开的 syndication 接口不返回敏感内容。设置 `TWITTER_AUTH_TOKEN`(登录 x.com 后浏览器 Cookie 里的 `auth_token` 值)后,bot 会仅在遇到 NSFW 推文时以登录态获取媒体;未设置则提示无媒体 NSFW 推文:公开的 syndication 接口不返回敏感内容。设置 `TWITTER_AUTH_TOKEN`(登录 x.com 后浏览器 Cookie 里的 `auth_token` 值)后,bot 会仅在遇到 NSFW 推文时以登录态获取媒体;未设置则回复该推文内容受限(需要配置 `TWITTER_AUTH_TOKEN`
Bilibili 动态默认匿名抓取(无需登录,bot 会自动从 B 站的匿名指纹接口取 `buvid3`/`buvid4` 设备 cookie 以提高成功率)。若服务器出口 IP 被 B 站重度风控(日志里的 `risk control (-352)` 或 HTTP 412,且持续出现),设置 `BILIBILI_COOKIE`(登录后浏览器里整条 Cookie 串,如 `SESSDATA=…; bili_jct=…`)可恢复访问。当前只发送动态里的图片与动图,动态内嵌视频发送其封面。 Bilibili 动态默认匿名抓取(无需登录,bot 会自动从 B 站的匿名指纹接口取 `buvid3`/`buvid4` 设备 cookie 以提高成功率)。若服务器出口 IP 被 B 站重度风控(日志里的 `risk control (-352)` 或 HTTP 412,且持续出现),设置 `BILIBILI_COOKIE`(登录后浏览器里整条 Cookie 串,如 `SESSDATA=…; bili_jct=…`)可恢复访问。当前只发送动态里的图片与动图,动态内嵌视频发送其封面。
@@ -83,10 +86,10 @@ Telegram 只接受 443/80/88/8443 端口。
| 变量 | 说明 | | 变量 | 说明 |
|---|---| |---|---|
| `TELOXIDE_TOKEN` | Bot token(必填) | | `TELOXIDE_TOKEN` | Bot token(必填) |
| `PIXIV_REFRESH_TOKEN` | Pixiv 刷新令牌;未设置则禁用 Pixiv | | `PIXIV_REFRESH_TOKEN` | Pixiv 刷新令牌;未设置则禁用 Pixiv(此时收到 pixiv 链接会明确回复「站点未启用」,不会静默忽略) |
| `BILIBILI_COOKIE` | 可选的 B 站 Cookie 串(`SESSDATA=…; bili_jct=…`),仅在出口 IP 被持续风控时才需要(设备 cookie 由 bot 自动获取) | | `BILIBILI_COOKIE` | 可选的 B 站 Cookie 串(`SESSDATA=…; bili_jct=…`),仅在出口 IP 被持续风控时才需要(设备 cookie 由 bot 自动获取) |
| `BOT_ADMIN` | 管理员聊天 ID,逗号分隔;接收启动/停止通知 | | `BOT_ADMIN` | 管理员聊天 ID,逗号分隔;接收启动/停止通知 |
| `EDIT_MESSAGE_TTL_SECONDS` | 转发前编辑记录过期秒数,默认 86400 | | `EDIT_MESSAGE_TTL_SECONDS` | 转发前编辑记录过期秒数,默认 86400;过期后提示消息会被就地改写为「已过期,未转发」(不额外发消息打扰) |
| `LINK_CACHE_TTL_SECONDS` | 链接结果缓存过期秒数,默认 604800(7 天) | | `LINK_CACHE_TTL_SECONDS` | 链接结果缓存过期秒数,默认 604800(7 天) |
| `CAPTION_QUOTE_TEXT_CHARS` | 正文(`{title}` + `{content}` 合计)达到该长度(字符)时,caption 的**正文部分**用可折叠引用块包裹,默认 200;`0` 关闭 | | `CAPTION_QUOTE_TEXT_CHARS` | 正文(`{title}` + `{content}` 合计)达到该长度(字符)时,caption 的**正文部分**用可折叠引用块包裹,默认 200;`0` 关闭 |
| `DATA_DIR` | 数据目录(SQLite 数据库 `task_queue.db` 所在目录),默认 `data`(相对工作目录,会自动创建) | | `DATA_DIR` | 数据目录(SQLite 数据库 `task_queue.db` 所在目录),默认 `data`(相对工作目录,会自动创建) |
@@ -114,15 +117,17 @@ Telegram 只接受 443/80/88/8443 端口。
| `/help` | 查看全部命令及用法(即本文档的命令表) | | `/help` | 查看全部命令及用法(即本文档的命令表) |
| `/set_forward_channel <频道>` | 设置转发频道,参数为 `@频道名` 或频道 ID;设置后发送的媒体消息会自动转发到该频道 | | `/set_forward_channel <频道>` | 设置转发频道,参数为 `@频道名` 或频道 ID;设置后发送的媒体消息会自动转发到该频道 |
| `/remove_forward_channel` | 取消转发频道 | | `/remove_forward_channel` | 取消转发频道 |
| `/edit_before_forward` | 开关「转发前编辑」:开启后,转发成功后 bot 会发一条提示消息,回复它可修改第一条转发消息的 caption(或点击模板按钮套用模板) | | `/edit_before_forward` | 开关「转发前编辑」:开启后,转发成功后 bot 会发一条提示消息,回复它可修改第一条转发消息的 caption(或点击模板按钮套用模板),再点 `↩️ Confirm` 才会真正转发,`🛑 Skip` 放弃本次转发;提示消息写明过期时间,过期后原地标记为已过期且不会转发 |
| `/set_template <名称>` | 回复一条含 `[]` 的消息,将其保存为命名模板;转发时 `[]` 会被替换为原帖链接(配合「转发前编辑」使用) | | `/set_template <名称>` | 回复一条含 `[]` 的消息,将其保存为命名模板;转发时 `[]` 会被替换为原帖链接(配合「转发前编辑」使用) |
| `/set_format <站点> <格式>` | 自定义某站点的 caption 格式。站点:`twitter` / `bsky` / `pixiv` / `misskey` / `bilibili`。占位符:`{url}` `{author}` `{author_url}` `{title}` `{content}` `{tags}` | | `/remove_template <名称>` | 删除某个模板(名称见 `/settings`;提示消息的模板按钮最多显示 60 个) |
| `/settings` | 查看本聊天配置:转发频道、转发前编辑开关、各站点 caption 格式、模板列表 |
| `/set_format <站点> <格式>` | 自定义某站点的 caption 格式。站点:`twitter` / `bsky` / `pixiv` / `misskey` / `bilibili`。占位符:`{url}` `{author}` `{author_url}` `{title}` `{content}` `{tags}`;未识别的占位符会被拒绝并列出可用项,格式填 `-` 恢复站点默认格式(可用 `/debug <链接>` 预览效果) |
| `/clear_cache [链接]` | 清空链接缓存(仅管理员);带链接只清该条,否则清空全部 | | `/clear_cache [链接]` | 清空链接缓存(仅管理员);带链接只清该条,否则清空全部 |
| `/bot_dict` | 查看当前聊天状态(调试用;仅管理员) | | `/bot_dict` | 查看当前聊天状态(调试用;仅管理员) |
| `/test <链接>` | 解析链接并发送媒体;不转发到频道、不弹转发前编辑提示(仅发送) | | `/test <链接>` | 解析链接并发送媒体;不转发到频道、不弹转发前编辑提示(仅发送) |
| `/debug <链接>` | 调试:只解析链接并返回解析结果(站点、标题、作者、标签、媒体列表),不发送任何媒体 | | `/debug <链接>` | 调试:只解析链接并返回解析结果(站点、标题、作者、标签、媒体列表),不发送任何媒体 |
链接处理仅限私聊;命令在任意聊天可用。 链接处理仅限私聊;命令在任意聊天可用。在群聊里发受支持的链接会回复一条提示(改用私聊或内联查询),频道内保持静默。
## 备注 ## 备注
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "x-media" name = "x-media"
version = "1.7.0" version = "1.8.0"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+79 -2
View File
@@ -247,6 +247,12 @@ pub enum FetchError {
NotFound, NotFound,
#[error("blocked")] #[error("blocked")]
Blocked, Blocked,
/// The URL matches a registered site that is disabled right now (pixiv
/// without `PIXIV_REFRESH_TOKEN`, or after a failed login). Distinct from
/// `Ok(None)` — an unsupported link — so the bot can tell the user why
/// the link was not handled instead of silently ignoring it.
#[error("{site} support is disabled")]
Disabled { site: &'static str },
/// The post exists but its content is withheld (twitter NSFW / /// The post exists but its content is withheld (twitter NSFW /
/// age-restricted tweets come back as an empty `{}` from syndication). /// age-restricted tweets come back as an empty `{}` from syndication).
#[error("content withheld (sensitive)")] #[error("content withheld (sensitive)")]
@@ -385,6 +391,17 @@ fn find_site(url: &str) -> Option<&'static dyn Site> {
.map(|site| site.as_ref()) .map(|site| site.as_ref())
} }
/// The site whose pattern matches `url` but which is disabled right now.
/// `None` when no site matches the URL at all, or when the matching site is
/// enabled. Lets the dispatcher tell "unsupported link" (silently ignored)
/// apart from "this bot has that site switched off" (reported to the user).
fn disabled_site(url: &str) -> Option<&'static str> {
SITES
.iter()
.find(|site| !site.enabled() && site.pattern().is_match(url))
.map(|site| site.id())
}
/// Every supported site id, in dispatch order. The bot's SetFormat whitelist /// Every supported site id, in dispatch order. The bot's SetFormat whitelist
/// derives from this list. /// derives from this list.
pub fn site_ids() -> Vec<&'static str> { pub fn site_ids() -> Vec<&'static str> {
@@ -408,7 +425,9 @@ pub async fn validate_all() -> Vec<(&'static str, String)> {
} }
/// Fetches a post from its URL. Returns `Ok(None)` when no site pattern /// Fetches a post from its URL. Returns `Ok(None)` when no site pattern
/// matches (unsupported links are silently ignored by the bot). /// matches (unsupported links are silently ignored by the bot) and
/// [`FetchError::Disabled`] when the URL belongs to a registered site that is
/// switched off right now — the two are different answers for the user.
/// ///
/// Transient failures are retried: 3 total attempts with 1s then 2s delays. /// Transient failures are retried: 3 total attempts with 1s then 2s delays.
/// What counts as transient is the matched site's own policy (`is_retryable` /// What counts as transient is the matched site's own policy (`is_retryable`
@@ -432,7 +451,13 @@ const MAX_FETCH_ATTEMPTS: u32 = 3;
async fn fetch_with_attempts(url: &str, attempts: u32) -> Result<Option<Fetched>, FetchError> { async fn fetch_with_attempts(url: &str, attempts: u32) -> Result<Option<Fetched>, FetchError> {
let Some(site) = find_site(url) else { let Some(site) = find_site(url) else {
return Ok(None); // A registered-but-disabled site (pixiv without a token) is not an
// unsupported link: report it, so the bot answers the user instead of
// ignoring the message.
return match disabled_site(url) {
Some(site) => Err(FetchError::Disabled { site }),
None => Ok(None),
};
}; };
for attempt in 0..attempts.max(1) { for attempt in 0..attempts.max(1) {
match site.fetch_from_url(url).await { match site.fetch_from_url(url).await {
@@ -458,6 +483,15 @@ async fn fetch_with_attempts(url: &str, attempts: u32) -> Result<Option<Fetched>
unreachable!("retry loop always returns") unreachable!("retry loop always returns")
} }
/// Whether fetching `url` requires site-specific headers (pixiv's `Referer`
/// for `pximg.net` hotlink protection, see [`Site::media_headers`]). Telegram's
/// own fetch of a media URL sends none of them, so a URL that needs them fails
/// there — callers that hand a URL to Telegram (inline query results) must
/// skip such media instead of shipping a broken item.
pub fn needs_media_headers(url: &str) -> bool {
SITES.iter().any(|site| site.media_headers(url).is_some())
}
/// Applies every site's media-header rule to a download request (pixiv's /// Applies every site's media-header rule to a download request (pixiv's
/// `Referer` for pximg.net hotlink protection). Sites contribute via their /// `Referer` for pximg.net hotlink protection). Sites contribute via their
/// `media_headers(url)` — the central download code carries no per-site logic. /// `media_headers(url)` — the central download code carries no per-site logic.
@@ -708,6 +742,49 @@ mod tests {
assert!(matches!(result, Ok(None)), "got {result:?}"); assert!(matches!(result, Ok(None)), "got {result:?}");
} }
#[test]
fn media_headers_are_reported_only_where_telegram_would_fail() {
// pixiv's CDN needs a Referer, which only the bot can send: an inline
// result pointing at it renders broken, so callers skip it.
assert!(needs_media_headers(
"https://i.pximg.net/img-original/img/2024/01/01/00/00/00/1_p0.jpg"
));
// The rest serve direct requests (verified per site in their modules).
for url in [
"https://pbs.twimg.com/media/1.jpg",
"https://cdn.bsky.app/img/1.jpg",
"https://media.misskeyusercontent.jp/io/1.webp",
"https://i0.hdslb.com/bfs/1.jpg",
] {
assert!(!needs_media_headers(url), "{url}");
}
}
#[tokio::test]
async fn disabled_site_is_reported_not_ignored() {
// pixiv is the only token-gated site; with PIXIV_REFRESH_TOKEN set it
// is enabled and this link would hit the network, so skip then.
if std::env::var("PIXIV_REFRESH_TOKEN")
.ok()
.filter(|s| !s.is_empty())
.is_some()
{
eprintln!("skipping: PIXIV_REFRESH_TOKEN is set");
return;
}
let result = fetch("https://www.pixiv.net/artworks/1").await;
assert!(
matches!(result, Err(FetchError::Disabled { site: "pixiv" })),
"got {result:?}"
);
// The cache key still resolves: the bot keys the reply and the link
// cache off it even when the site is off.
assert_eq!(
cache_key("https://www.pixiv.net/artworks/1"),
Some("pixiv:1".into())
);
}
#[tokio::test] #[tokio::test]
async fn download_media_pixiv_original_with_referer() { async fn download_media_pixiv_original_with_referer() {
// Proves the Referer header is attached for i.pximg.net: a header-less // Proves the Referer header is attached for i.pximg.net: a header-less
+9 -29
View File
@@ -43,27 +43,25 @@ pub async fn fetch_from_url(url: &str) -> Result<Fetched, FetchError> {
match fetch(id).await { match fetch(id).await {
Ok(tweet) => Ok(tweet.into()), Ok(tweet) => Ok(tweet.into()),
// Syndication withholds NSFW/age-restricted tweets (empty `{}`). // Syndication withholds NSFW/age-restricted tweets (empty `{}`).
// Retry as the logged-in user when TWITTER_AUTH_TOKEN is set; // Retry as the logged-in user when TWITTER_AUTH_TOKEN is set; without
// otherwise degrade to an empty result (the bot replies // the token the withholding is reported as `Sensitive`, so the bot can
// "No media found"). // answer "age-restricted / needs TWITTER_AUTH_TOKEN" instead of the
// misleading "No media found".
Err(FetchError::Sensitive) => { Err(FetchError::Sensitive) => {
if super::auth::enabled() { if super::auth::enabled() {
match super::auth::fetch(id).await { match super::auth::fetch(id).await {
Ok(tweet) => Ok(tweet.into()), Ok(tweet) => Ok(tweet.into()),
// The tweet is genuinely gone (deleted / suspended / // Deleted/suspended (tombstoned) and unexpected fallback
// tombstoned): report it instead of degrading to an // failures keep their own class: the bot reports what
// empty result ("No media found"). Only unexpected // actually happened rather than "No media found".
// fallback failures (network, parse) keep the NSFW
// placeholder.
Err(FetchError::NotFound) => Err(FetchError::NotFound),
Err(e) => { Err(e) => {
log::warn!("twitter auth fallback failed for {id}: {e}"); log::warn!("twitter auth fallback failed for {id}: {e}");
Ok(empty_fetched(url)) Err(e)
} }
} }
} else { } else {
log::debug!("tweet {id} is sensitive; set TWITTER_AUTH_TOKEN to fetch NSFW media"); log::debug!("tweet {id} is sensitive; set TWITTER_AUTH_TOKEN to fetch NSFW media");
Ok(empty_fetched(url)) Err(FetchError::Sensitive)
} }
} }
Err(e) => Err(e), Err(e) => Err(e),
@@ -90,24 +88,6 @@ pub fn media_headers(_url: &str) -> Option<Vec<(&'static str, String)>> {
None None
} }
/// A Fetched with no media for withheld tweets: the bot replies
/// "No media found" and moves on instead of erroring.
fn empty_fetched(url: &str) -> Fetched {
Fetched {
source_url: url.to_string(),
// The raw user-supplied URL goes into an HTML caption; escape it so
// crafted links cannot break the parse (Telegram 400).
caption: encode_text(url).into_owned(),
title: String::new(),
content: String::new(),
media: vec![],
sensitive: true,
site_id: "twitter",
render_data: None,
_keep_alive: None,
}
}
/// Fetches a tweet from the syndication endpoint. Deleted/blocked tweets /// Fetches a tweet from the syndication endpoint. Deleted/blocked tweets
/// surface as `FetchError::NotFound`; withheld content (empty tombstone, /// surface as `FetchError::NotFound`; withheld content (empty tombstone,
/// age-restricted) as `FetchError::Sensitive`. /// age-restricted) as `FetchError::Sensitive`.
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "xmedia-bot" name = "xmedia-bot"
version = "1.7.0" version = "1.8.0"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
@@ -14,6 +14,8 @@ use teloxide::types::{CallbackQuery, CallbackQueryId, MessageId};
/// The `"forward"` button's data. /// The `"forward"` button's data.
const FORWARD: &str = "forward"; const FORWARD: &str = "forward";
/// The `"skip"` button's data: drop the prompt without forwarding.
const SKIP: &str = "skip";
/// Prefix of a template button's data: `"template|<name>"`. /// Prefix of a template button's data: `"template|<name>"`.
const TEMPLATE_PREFIX: &str = "template|"; const TEMPLATE_PREFIX: &str = "template|";
@@ -70,6 +72,29 @@ async fn handle_callback(
} }
log::info!("callback from {chat_id} on prompt {prompt_message_id}: {data}"); log::info!("callback from {chat_id} on prompt {prompt_message_id}: {data}");
if data == SKIP {
// Skip works with or without a forward channel: it is the explicit
// "do not forward this" answer, and it drops the record so the forward
// can never happen later.
log::info!("edit-before-forward prompt {prompt_message_id} skipped");
ctx.chat_store
.update(chat_id, |data| {
data.edit_message.remove(&prompt_message_id);
})
.await;
let _ = ctx
.sender
.delete_message(ChatId(chat_id), MessageId(prompt_message_id as i32))
.await;
let _ = ctx
.sender
.answer_callback_query(
callback_query_id,
Some("Skipped — nothing was forwarded.".to_string()),
)
.await;
return;
}
if data == FORWARD { if data == FORWARD {
match chat_data.forward_channel_id { match chat_data.forward_channel_id {
Some(channel_id) => { Some(channel_id) => {
@@ -244,6 +269,31 @@ mod tests {
); );
} }
#[tokio::test]
async fn skip_drops_the_prompt_without_forwarding() {
// "skip" needs no forward channel and no scripted outcomes: it deletes
// the prompt and drops the record, so no forward can ever happen.
let sender = MockSender::scripted(vec![], api_error);
let stores = TestStores::new();
let ctx = stores.ctx(&sender);
seed_prompt(&ctx, crate::db::unix_now()).await;
handle_callback(&ctx, callback_id(), 1, PROMPT_ID, "skip").await;
assert_eq!(
sender.calls(),
vec!["delete_message", "answer_callback_query"]
);
assert_eq!(
sender.answers(),
vec![Some("Skipped — nothing was forwarded.".to_string())]
);
assert!(
ctx.chat_store.get(1).await.edit_message.is_empty(),
"a skipped prompt must drop its record"
);
}
#[tokio::test] #[tokio::test]
async fn forward_without_a_channel_is_reported() { async fn forward_without_a_channel_is_reported() {
let sender = MockSender::scripted(vec![], api_error); let sender = MockSender::scripted(vec![], api_error);
+536 -9
View File
@@ -4,6 +4,7 @@
use super::urls::{PostSend, url_media}; use super::urls::{PostSend, url_media};
use super::{CHAT_STORE, CONFIG, LINK_CACHE, log_key, reply, reply_html}; use super::{CHAT_STORE, CONFIG, LINK_CACHE, log_key, reply, reply_html};
use crate::ctx::AppContext; use crate::ctx::AppContext;
use crate::state::ChatData;
use teloxide::RequestError; use teloxide::RequestError;
use teloxide::prelude::*; use teloxide::prelude::*;
use teloxide::types::{ChatId, Message, Recipient}; use teloxide::types::{ChatId, Message, Recipient};
@@ -33,13 +34,20 @@ pub(crate) enum Command {
parse_with = "split" parse_with = "split"
)] )]
SetTemplate(String), SetTemplate(String),
#[command(description = "Remove a saved template", parse_with = "split")]
RemoveTemplate(String),
#[command(description = "Show this chat's settings")]
Settings,
#[command(description = "Show chat state (debug; admin only)")] #[command(description = "Show chat state (debug; admin only)")]
BotDict, BotDict,
#[command(description = "Set site caption format", parse_with = "split")] #[command(
description = "Set site caption format (- to reset)",
parse_with = parse_arg_remainder
)]
SetFormat(String), SetFormat(String),
#[command( #[command(
description = "Clear link cache (admin; optional URL, else all)", description = "Clear link cache (admin; optional URL, else all)",
parse_with = "split" parse_with = parse_arg_remainder
)] )]
ClearCache(String), ClearCache(String),
#[command( #[command(
@@ -62,6 +70,118 @@ fn parse_arg_remainder(s: String) -> Result<(String,), ParseError> {
Ok((s.trim().to_string(),)) Ok((s.trim().to_string(),))
} }
/// Placeholders `/set_format` accepts, mirroring what
/// `x_media::site::caption_from_fields` substitutes.
const FORMAT_PLACEHOLDERS: [&str; 6] = ["url", "author", "author_url", "title", "content", "tags"];
/// `/start`'s welcome: what the bot is for, where links work, where to look
/// next. The old "Hello!" left a first-time user with nothing.
const START_TEXT: &str = "\
Send me a post link and I'll send back its images, videos and GIFs with the title, author and tags.
Supported: X/Twitter, Pixiv, Bluesky, Misskey (misskey.io), Bilibili.
In a private chat just paste the link. In a group, use inline mode (type @, pick me, then the link).
/help lists every command.";
/// Appended to `/help`'s command list: argument syntax, caption
/// placeholders and the private-chat rule — none of which teloxide's
/// `descriptions()` renders (it prints `/command — description` only).
const HELP_FOOTER: &str = "\
Arguments
/set_forward_channel <@channel or channel id>
/set_template <name> reply to a message containing [] to save it
/remove_template <name> see /settings for the saved names
/set_format <site> <format> '-' restores the built-in format
/test <link> / /debug <link>
Caption placeholders (for /set_format)
{url} {author} {author_url} {title} {content} {tags}
A template's [] is replaced by the post link when forwarding.
Links are handled in private chats only; in a group use inline mode.";
/// Cap on template names echoed by `/settings`: a chat with hundreds of
/// templates must not produce a message Telegram rejects for length.
const MAX_SETTINGS_TEMPLATE_NAMES: usize = 30;
/// Sorted template names: the order `/settings`, `/remove_template` and the
/// prompt's buttons all show.
fn sorted_template_names(data: &ChatData) -> Vec<String> {
let mut names: Vec<String> = data.template.keys().cloned().collect();
names.sort();
names
}
/// `/settings`: what this chat is configured to do, readable by anyone in it
/// (unlike `/bot_dict`, which dumps the raw state and is admin-only).
fn settings_text(data: &ChatData) -> String {
let mut lines = Vec::new();
match data.forward_channel_id {
Some(id) => lines.push(format!("Forward channel: {id}")),
None => lines.push(
"Forward channel: not set (use /set_forward_channel <@channel or id>)".to_string(),
),
}
lines.push(format!(
"Edit before forward: {}",
if data.edit_before_forward {
"on"
} else {
"off"
}
));
let mut formats: Vec<String> = data
.message_format
.iter()
.map(|(site, format)| format!("{site} => {format}"))
.collect();
formats.sort();
lines.push(if formats.is_empty() {
"Caption formats: built-in for every site".to_string()
} else {
format!("Caption formats:\n {}", formats.join("\n "))
});
let names = sorted_template_names(data);
lines.push(match names.len() {
0 => "Templates: none".to_string(),
n => format!(
"Templates ({n}): {}{}",
names
.iter()
.take(MAX_SETTINGS_TEMPLATE_NAMES)
.cloned()
.collect::<Vec<_>>()
.join(", "),
if n > MAX_SETTINGS_TEMPLATE_NAMES {
format!(", +{} more", n - MAX_SETTINGS_TEMPLATE_NAMES)
} else {
String::new()
}
),
});
lines.join("\n")
}
/// The first `{…}` token in a caption format that is not a known placeholder
/// (`None` when all of them are). The renderer replaces exact keys only, so an
/// unknown token would be published verbatim in every caption of that site —
/// caught here instead.
fn unknown_placeholder(format: &str) -> Option<&str> {
let mut rest = format;
while let Some(open) = rest.find('{') {
let after = &rest[open + 1..];
// An unclosed `{` is not a placeholder token at all.
let close = after.find('}')?;
let token = &after[..close];
if !FORMAT_PLACEHOLDERS.contains(&token) {
return Some(token);
}
rest = &after[close + 1..];
}
None
}
enum SetForwardChannelError { enum SetForwardChannelError {
EmptyParameter, EmptyParameter,
NotChannel, NotChannel,
@@ -140,11 +260,17 @@ pub(crate) async fn execute_command(
) -> Result<(), RequestError> { ) -> Result<(), RequestError> {
match command { match command {
Command::Start => { Command::Start => {
bot.send_message(message.chat.id, "Hello!").await?; bot.send_message(message.chat.id, START_TEXT).await?;
} }
Command::Help => { Command::Help => {
bot.send_message(message.chat.id, Command::descriptions().to_string()) // The command list plus the parts teloxide's `descriptions()`
.await?; // cannot show: argument syntax, caption placeholders, and where a
// link actually works.
bot.send_message(
message.chat.id,
format!("{}\n\n{}", Command::descriptions(), HELP_FOOTER),
)
.await?;
} }
Command::SetForwardChannel(channel) => { Command::SetForwardChannel(channel) => {
let result = match set_forward_channel_handler(bot, message, channel).await { let result = match set_forward_channel_handler(bot, message, channel).await {
@@ -232,6 +358,41 @@ pub(crate) async fn execute_command(
}; };
reply(bot, message.chat.id.0, message.id, text).await?; reply(bot, message.chat.id.0, message.id, text).await?;
} }
Command::RemoveTemplate(name) => {
let chat_id = message.chat.id.0;
let name = name.trim().to_string();
if name.is_empty() {
reply(
bot,
chat_id,
message.id,
"Usage: /remove_template <name> (see /settings for the saved names)",
)
.await?;
return Ok(());
}
let removed = CHAT_STORE
.update(chat_id, |data| data.template.remove(&name).is_some())
.await;
let text = if removed {
format!("Template '{name}' removed.")
} else {
// Name the live templates: a typo would otherwise look like a
// successful delete.
let names = sorted_template_names(&CHAT_STORE.get(chat_id).await);
if names.is_empty() {
format!("No template named '{name}'. None are saved yet.")
} else {
format!("No template named '{name}'. Saved: {}", names.join(", "))
}
};
reply(bot, chat_id, message.id, text).await?;
}
Command::Settings => {
let chat_id = message.chat.id.0;
let data = CHAT_STORE.get(chat_id).await;
reply(bot, chat_id, message.id, settings_text(&data)).await?;
}
Command::BotDict => { Command::BotDict => {
// Debug dump of the chat's persisted state: admin only (it echoes // Debug dump of the chat's persisted state: admin only (it echoes
// forward-channel ids and templates to whoever asks). // forward-channel ids and templates to whoever asks).
@@ -284,12 +445,56 @@ pub(crate) async fn execute_command(
.await?; .await?;
return Ok(()); return Ok(());
} }
// `-` resets to the site's built-in caption: without it a chat that
// set a format once could never get back to the default (the
// built-in format string is not something a user can retype).
if format == "-" {
CHAT_STORE
.update(chat_id, |data| {
data.message_format.remove(site);
})
.await;
reply(
bot,
message.chat.id.0,
message.id,
"Format reset to the built-in one.",
)
.await?;
return Ok(());
}
// A typo like {titel} would otherwise be rendered literally into
// every caption of that site (the renderer only substitutes the
// exact keys), which is invisible until a post arrives.
if let Some(token) = unknown_placeholder(&format) {
reply(
bot,
message.chat.id.0,
message.id,
format!(
"Unknown placeholder {{{token}}}. Available: {}",
FORMAT_PLACEHOLDERS
.iter()
.map(|name| format!("{{{name}}}"))
.collect::<Vec<_>>()
.join(" ")
),
)
.await?;
return Ok(());
}
CHAT_STORE CHAT_STORE
.update(chat_id, |data| { .update(chat_id, |data| {
data.message_format.insert(site.to_string(), format); data.message_format.insert(site.to_string(), format);
}) })
.await; .await;
reply(bot, message.chat.id.0, message.id, "Format set.").await?; reply(
bot,
message.chat.id.0,
message.id,
"Format set. Use /debug <link> to preview the caption.",
)
.await?;
} }
Command::ClearCache(arg) => { Command::ClearCache(arg) => {
let sender_id = message let sender_id = message
@@ -414,6 +619,24 @@ pub(crate) async fn execute_command(
.await?; .await?;
} }
Ok(Some(fetched)) => { Ok(Some(fetched)) => {
// The preview must show what a link would actually send:
// the chat's per-site format override plus the long-post
// quoting. Rendering the raw built-in caption here made
// `/set_format` look like it did nothing.
let format = CHAT_STORE
.get(message.chat.id.0)
.await
.message_format
.get(fetched.site_name())
.cloned()
.unwrap_or_default();
let caption = preview_caption(
&format,
&fetched.caption,
&fetched.source_url,
fetched.render_fields(),
CONFIG.caption_quote_text_chars,
);
let report = debug_report( let report = debug_report(
url, url,
fetched.site_name(), fetched.site_name(),
@@ -422,7 +645,7 @@ pub(crate) async fn execute_command(
&fetched.content, &fetched.content,
fetched.render_fields(), fetched.render_fields(),
fetched.sensitive, fetched.sensitive,
&fetched.caption, &caption,
&fetched.media, &fetched.media,
); );
// HTML report: the caption renders inside a <blockquote> // HTML report: the caption renders inside a <blockquote>
@@ -440,12 +663,32 @@ fn plural(n: usize) -> &'static str {
if n == 1 { "y" } else { "ies" } if n == 1 { "y" } else { "ies" }
} }
/// Bot profile texts (Bot API `setMyDescription` / `setMyShortDescription`):
/// shown on the bot's profile page and in the share sheet. Without them a
/// shared link says nothing about what the bot does.
const BOT_DESCRIPTION: &str = "\
Send a post link from X/Twitter, Pixiv, Bluesky, Misskey (misskey.io) or Bilibili and get its images, videos and GIFs back with the title, author and tags.
Links are handled in private chats; a group can use inline mode. /help lists every command.";
const BOT_SHORT_DESCRIPTION: &str =
"Post links (X, Pixiv, Bluesky, Misskey, Bilibili) -> media messages";
/// Registers the bot's command list with Telegram so clients show it in the /// Registers the bot's command list with Telegram so clients show it in the
/// `/` menu (Bot API `setMyCommands`). /// `/` menu (Bot API `setMyCommands`), plus its profile description texts.
pub async fn register_commands(bot: &Bot) -> Result<(), RequestError> { pub async fn register_commands(bot: &Bot) -> Result<(), RequestError> {
let commands = Command::bot_commands(); let commands = Command::bot_commands();
bot.set_my_commands(commands.clone()).await?; bot.set_my_commands(commands.clone()).await?;
log::info!("registered {} commands", commands.len()); log::info!("registered {} commands", commands.len());
// Profile texts are cosmetic: a failure (rare) must not abort startup.
if let Err(e) = bot.set_my_description().description(BOT_DESCRIPTION).await {
log::warn!("failed to set the bot description: {e}");
}
if let Err(e) = bot
.set_my_short_description()
.short_description(BOT_SHORT_DESCRIPTION)
.await
{
log::warn!("failed to set the bot short description: {e}");
}
Ok(()) Ok(())
} }
@@ -457,6 +700,31 @@ const MAX_DEBUG_REPORT_CHARS: usize = 4000;
/// message, so it must stay under Telegram's 4096-char limit. /// message, so it must stay under Telegram's 4096-char limit.
const MAX_DEBUG_DUMP_CHARS: usize = 3500; const MAX_DEBUG_DUMP_CHARS: usize = 3500;
/// The caption a link would actually send for this chat: the per-site format
/// override (empty = the site's built-in caption) and, on a long post, the
/// same text quoting the send paths apply. `/debug` shows this so the preview
/// cannot drift from what the send paths produce.
fn preview_caption(
format: &str,
built_in: &str,
url: &str,
fields: Option<(&str, &str, &str, &str, &str)>,
quote_chars: usize,
) -> String {
let caption = match fields {
// Same call the send paths make through `Fetched::caption_with`: an
// empty format falls back to the built-in caption.
Some((author, author_url, title, content, tags)) => x_media::site::caption_from_fields(
format, built_in, url, author, author_url, title, content, tags,
),
None => x_media::site::truncate_caption(built_in),
};
let text = fields
.map(|(_, _, title, content, _)| x_media::site::compose_text(title, content))
.unwrap_or_default();
crate::send::quote_long_caption(&caption, &text, quote_chars).into_owned()
}
/// Builds the HTML report for the `/debug` command: what the parser produced /// Builds the HTML report for the `/debug` command: what the parser produced
/// for a link (site, canonical URL, title/author/tags, caption and the media /// for a link (site, canonical URL, title/author/tags, caption and the media
/// list) — no media is sent and nothing is cached or forwarded. Sent with /// list) — no media is sent and nothing is cached or forwarded. Sent with
@@ -539,7 +807,9 @@ fn debug_report(
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::{MAX_DEBUG_REPORT_CHARS, debug_report}; use super::{
MAX_DEBUG_REPORT_CHARS, debug_report, preview_caption, settings_text, unknown_placeholder,
};
use x_media::media::Media; use x_media::media::Media;
#[test] #[test]
@@ -657,4 +927,261 @@ mod tests {
assert!(report.chars().count() <= MAX_DEBUG_REPORT_CHARS, "{report}"); assert!(report.chars().count() <= MAX_DEBUG_REPORT_CHARS, "{report}");
assert!(report.ends_with('…'), "{report}"); assert!(report.ends_with('…'), "{report}");
} }
#[test]
fn settings_text_reports_the_chat_configuration() {
use crate::state::ChatData;
// A fresh chat: the defaults must be spelled out, including how to set
// the channel (an empty field is not a status).
let empty = settings_text(&ChatData::default());
assert!(empty.contains("Forward channel: not set"), "{empty}");
assert!(empty.contains("/set_forward_channel"), "{empty}");
assert!(empty.contains("Edit before forward: off"), "{empty}");
assert!(empty.contains("built-in for every site"), "{empty}");
assert!(empty.contains("Templates: none"), "{empty}");
let configured = ChatData {
forward_channel_id: Some(-100123),
edit_before_forward: true,
template: [("b", "[]"), ("a", "[]")]
.into_iter()
.map(|(k, v)| (k.to_string(), v.to_string()))
.collect(),
message_format: [("twitter", "{author}: {content}")]
.into_iter()
.map(|(k, v)| (k.to_string(), v.to_string()))
.collect(),
..ChatData::default()
};
let text = settings_text(&configured);
assert!(text.contains("Forward channel: -100123"), "{text}");
assert!(text.contains("Edit before forward: on"), "{text}");
assert!(text.contains("twitter => {author}: {content}"), "{text}");
// Sorted, so the same chat always reports the same thing.
assert!(text.contains("Templates (2): a, b"), "{text}");
}
#[test]
fn help_and_start_cover_what_the_command_list_cannot() {
// The placeholders the renderer substitutes must be the ones the help
// lists: a stale list is worse than none.
for placeholder in super::FORMAT_PLACEHOLDERS {
assert!(
super::HELP_FOOTER.contains(&format!("{{{placeholder}}}")),
"help does not document {{{placeholder}}}"
);
}
// The private-chat rule and the template placeholder semantics are the
// two things users got wrong most often.
assert!(super::HELP_FOOTER.contains("private chats only"));
assert!(super::HELP_FOOTER.contains("[]"));
assert!(super::START_TEXT.contains("inline mode"));
assert!(super::START_TEXT.contains("/help"));
// Both must stay inside Telegram's message limit.
assert!(super::HELP_FOOTER.chars().count() < 2000);
assert!(super::START_TEXT.chars().count() < 2000);
}
#[test]
fn every_command_is_registered_and_parses() {
use teloxide::utils::command::BotCommands;
use super::Command;
let registered: Vec<String> = Command::bot_commands()
.into_iter()
.map(|command| command.command.trim_start_matches('/').to_string())
.collect();
for expected in [
"start",
"help",
"settings",
"set_forward_channel",
"remove_template",
"set_format",
"test",
"debug",
] {
assert!(
registered.iter().any(|name| name == expected),
"{expected} missing from {registered:?}"
);
}
// Telegram caps a command description at 256 chars.
for command in Command::bot_commands() {
assert!(
command.description.chars().count() <= 256,
"{}: description too long",
command.command
);
}
// A command with a `String` argument must parse with its whole
// argument: without `parse_with`, teloxide's default parser rejects
// `/remove_template x` and the command silently falls through to the
// URL flow.
assert!(matches!(
Command::parse("/settings", ""),
Ok(Command::Settings)
));
match Command::parse("/remove_template tpl", "") {
Ok(Command::RemoveTemplate(name)) => assert_eq!(name, "tpl"),
Ok(_) => panic!("/remove_template parsed as another command"),
Err(e) => panic!("parse error: {e}"),
}
}
#[test]
fn every_documented_invocation_parses() {
use teloxide::utils::command::BotCommands;
use super::Command;
// The README's forms, verbatim. teloxide's `split` parser accepts
// EXACTLY one token per `String` field, so a command documented with
// two arguments (or an optional one) silently stops parsing — and a
// command that does not parse falls through to the URL flow in
// silence.
type Check = fn(&Command) -> bool;
let cases: Vec<(&str, Check)> = vec![
("/start", |c| matches!(c, Command::Start)),
("/help", |c| matches!(c, Command::Help)),
("/settings", |c| matches!(c, Command::Settings)),
("/edit_before_forward", |c| {
matches!(c, Command::EditBeforeForward)
}),
("/remove_forward_channel", |c| {
matches!(c, Command::RemoveForwardChannel)
}),
("/bot_dict", |c| matches!(c, Command::BotDict)),
(
"/set_forward_channel @a_channel",
|c| matches!(c, Command::SetForwardChannel(a) if a == "@a_channel"),
),
(
"/set_template tpl",
|c| matches!(c, Command::SetTemplate(a) if a == "tpl"),
),
(
"/remove_template tpl",
|c| matches!(c, Command::RemoveTemplate(a) if a == "tpl"),
),
(
"/set_format twitter {author}: {title}",
|c| matches!(c, Command::SetFormat(a) if a == "twitter {author}: {title}"),
),
(
"/set_format twitter -",
|c| matches!(c, Command::SetFormat(a) if a == "twitter -"),
),
// Documented as "clear everything" when called without a link.
(
"/clear_cache",
|c| matches!(c, Command::ClearCache(a) if a.is_empty()),
),
(
"/clear_cache https://x.com/u/status/1",
|c| matches!(c, Command::ClearCache(a) if a == "https://x.com/u/status/1"),
),
(
"/test https://x.com/u/status/1",
|c| matches!(c, Command::Test(a) if a == "https://x.com/u/status/1"),
),
(
"/debug https://x.com/u/status/1",
|c| matches!(c, Command::Debug(a) if a == "https://x.com/u/status/1"),
),
];
for (text, ok) in cases {
match Command::parse(text, "") {
Ok(parsed) => assert!(ok(&parsed), "{text} parsed as the wrong variant"),
Err(e) => panic!("{text} did not parse: {e}"),
}
}
}
#[test]
fn preview_caption_applies_the_chat_format_and_the_long_post_quote() {
let fields = Some((
"Author",
"https://x.com/u",
"Pinned title",
"Pinned body",
"#tag",
));
// No format override → the site's built-in caption, untouched.
assert_eq!(
preview_caption(
"",
"built-in caption",
"https://x.com/u/status/1",
fields,
200
),
"built-in caption"
);
// The bug this pins: `/debug` used to print the built-in caption even
// with a format set, so `/set_format` looked like it did nothing.
let formatted = preview_caption(
"{author} · {title}",
"built-in caption",
"https://x.com/u/status/1",
fields,
200,
);
assert_eq!(formatted, "Author · Pinned title");
// `{url}` comes from the canonical post URL, as in the send paths.
assert_eq!(
preview_caption(
"{url} {title}",
"built-in",
"https://x.com/u/status/1",
fields,
200
),
"https://x.com/u/status/1 Pinned title"
);
// A long post's text is quoted exactly like the send paths quote it.
let long = "".repeat(300);
let fields = Some(("Author", "https://x.com/u", "", long.as_str(), ""));
let quoted = preview_caption(
"",
"https://x.com/u/status/1\n<a href=\"https://x.com/u\">Author</a>: 正…",
"https://x.com/u/status/1",
fields,
200,
);
assert!(quoted.contains("<blockquote expandable>"), "{quoted}");
// Without render fields (a site that does not expose them) the
// built-in caption is all there is.
assert_eq!(
preview_caption("", "built-in", "https://x.com/u/status/1", None, 200),
"built-in"
);
}
#[test]
fn unknown_placeholder_finds_typos_only() {
assert_eq!(unknown_placeholder("{author} — {title}"), None);
// Every key the renderer substitutes must pass, in any combination.
assert_eq!(
unknown_placeholder("{url}{author}{author_url}{title}{content}{tags}"),
None
);
// Plain text and braces Telegram renders literally are not tokens.
assert_eq!(unknown_placeholder("no placeholders here"), None);
assert_eq!(unknown_placeholder("{unclosed"), None);
assert_eq!(unknown_placeholder("{titel}"), Some("titel"));
assert_eq!(unknown_placeholder("{title} {Content}"), Some("Content"));
// A typo after a valid token is still found.
assert_eq!(unknown_placeholder("{url} {tag}"), Some("tag"));
}
} }
+9
View File
@@ -146,6 +146,15 @@ async fn answer_inline_query(bot: Bot, query: InlineQuery) -> Result<bool, Reque
); );
for (i, media) in fetched.media.iter().enumerate() { for (i, media) in fetched.media.iter().enumerate() {
let id = format!("{i}"); let id = format!("{i}");
// Telegram fetches an inline result's URL itself and cannot
// send site-specific headers, so hotlink-protected media
// (pixiv's pximg.net) would render as a broken file there.
// Locally produced media (ugoira MP4, bsky remux) is a local
// path and does not parse as a URL at all — same skip.
if x_media::site::needs_media_headers(media.url()) {
log::debug!("inline: skipping hotlink-protected media {id}");
continue;
}
let Some(url) = url::Url::parse(media.url()).ok() else { let Some(url) = url::Url::parse(media.url()).ok() else {
continue; continue;
}; };
+64 -1
View File
@@ -23,7 +23,9 @@ use crate::media_sender::MediaSender;
use commands::{Command, execute_command}; use commands::{Command, execute_command};
use teloxide::RequestError; use teloxide::RequestError;
use teloxide::prelude::*; use teloxide::prelude::*;
use teloxide::types::{ChatId, ChatKind, Message, MessageId, ParseMode, ReplyParameters}; use teloxide::types::{
ChatId, ChatKind, Message, MessageId, ParseMode, PublicChatKind, ReplyParameters,
};
use teloxide::utils::command::BotCommands; use teloxide::utils::command::BotCommands;
use urls::{URL_JOBS, extract_urls}; use urls::{URL_JOBS, extract_urls};
@@ -175,10 +177,38 @@ pub async fn message_handler(bot: Bot, message: Message) -> Result<(), RequestEr
break; break;
} }
} }
} else if is_group(&message.chat.kind)
&& extract_urls(&message)
.iter()
.any(|url| x_media::site::cache_key(url).is_some())
{
// A supported link in a group used to be dropped in silence, which
// reads as a broken bot (the command menu is registered globally, so
// the expectation is there). Unsupported links stay ignored; the hint
// names the two paths that do work. Channels are excluded — the reply
// would be posted into the channel itself.
let _ = reply(&bot, message.chat.id.0, message.id, GROUP_LINK_HINT).await;
} }
respond(()) respond(())
} }
/// Answer for a link posted where the pipeline does not run (a group): links
/// are private-chat only, inline mode is the group path.
const GROUP_LINK_HINT: &str =
"Links are handled in private chat only — send me this link there, or use inline mode here.";
/// Groups and supergroups, as opposed to private chats and channels.
fn is_group(kind: &ChatKind) -> bool {
matches!(
kind,
ChatKind::Public(chat)
if matches!(
chat.kind,
PublicChatKind::Group | PublicChatKind::Supergroup(_)
)
)
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -271,4 +301,37 @@ mod tests {
assert!(!edit_message_handler(&ctx, 1, PROMPT_ID, "hello").await); assert!(!edit_message_handler(&ctx, 1, PROMPT_ID, "hello").await);
assert!(sender.calls().is_empty()); assert!(sender.calls().is_empty());
} }
#[test]
fn the_link_hint_is_for_groups_only() {
use teloxide::types::{ChatPrivate, ChatPublic, PublicChatChannel, PublicChatSupergroup};
let group = ChatKind::Public(ChatPublic {
title: None,
kind: PublicChatKind::Group,
});
let supergroup = ChatKind::Public(ChatPublic {
title: None,
kind: PublicChatKind::Supergroup(PublicChatSupergroup {
username: None,
is_forum: false,
}),
});
// A channel must stay silent: the hint reply would be posted into the
// channel itself.
let channel = ChatKind::Public(ChatPublic {
title: None,
kind: PublicChatKind::Channel(PublicChatChannel { username: None }),
});
let private = ChatKind::Private(ChatPrivate {
username: None,
first_name: None,
last_name: None,
});
assert!(is_group(&group));
assert!(is_group(&supergroup));
assert!(!is_group(&channel));
assert!(!is_group(&private));
}
} }
+233 -14
View File
@@ -4,9 +4,11 @@
use super::{log_key, reply}; use super::{log_key, reply};
use crate::ctx::{AppContext, CONTEXT}; use crate::ctx::{AppContext, CONTEXT};
use crate::link_cache::{CachedMediaKind, CachedPost}; use crate::link_cache::{CachedMediaKind, CachedPost};
use crate::media_sender::MediaSender;
use crate::send::{self, MediaItemPayload, Task}; use crate::send::{self, MediaItemPayload, Task};
use crate::state::ChatData; use crate::state::ChatData;
use std::collections::HashSet; use std::collections::HashSet;
use std::future::Future;
use std::sync::LazyLock; use std::sync::LazyLock;
use teloxide::types::{ChatAction, ChatId, Message, MessageEntityKind, MessageId}; use teloxide::types::{ChatAction, ChatId, Message, MessageEntityKind, MessageId};
use x_media::media::Media; use x_media::media::Media;
@@ -190,11 +192,16 @@ async fn dispatch_send(
log_key(url) log_key(url)
); );
send::enqueue_retry(ctx.task_queue, *task, delay_seconds).await; send::enqueue_retry(ctx.task_queue, *task, delay_seconds).await;
// Name the post and the wait: "queued for retry" alone left the
// user guessing which link it was and how long the wait is.
let _ = reply( let _ = reply(
ctx.sender, ctx.sender,
chat_id, chat_id,
reply_to, reply_to,
"Send failed. Task queued for retry.", format!(
"Send failed for {} — retrying in {delay_seconds:.0}s.",
log_key(url)
),
) )
.await; .await;
} }
@@ -285,6 +292,11 @@ fn build_send_task(
/// workers pass [`PostSend::FromChat`], the `/test` command /// workers pass [`PostSend::FromChat`], the `/test` command
/// [`PostSend::Suppressed`]. Everything else (cache write, retry enqueue, /// [`PostSend::Suppressed`]. Everything else (cache write, retry enqueue,
/// dead-letter notification) is identical. /// dead-letter notification) is identical.
///
/// Wraps [`url_media_inner`] with the chat-action keep-alive: Telegram expires
/// an action indicator after ~5s, while a fetch (ugoira encode, HLS remux) plus
/// a download-and-reupload fallback routinely takes longer — without the
/// refresh the chat shows nothing and the bot reads as stalled.
pub(crate) async fn url_media( pub(crate) async fn url_media(
ctx: &AppContext<'_>, ctx: &AppContext<'_>,
chat_id: i64, chat_id: i64,
@@ -292,14 +304,136 @@ pub(crate) async fn url_media(
url: &str, url: &str,
post_send: PostSend, post_send: PostSend,
) { ) {
let reply_to = MessageId(reply_to_message_id as i32); // Shared with the pipeline: once the media types are known the indicator
if let Err(e) = ctx // switches from "typing" to "sending photo/video".
.sender let hint = parking_lot::Mutex::new(ActionHint::Typing);
.send_chat_action(ChatId(chat_id), ChatAction::Typing) run_with_chat_action(
.await ctx.sender,
{ chat_id,
&hint,
url_media_inner(ctx, chat_id, reply_to_message_id, url, post_send, &hint),
)
.await;
}
/// Runs `pipeline` while keeping the chat's action indicator alive: Telegram
/// expires an action after ~5s, while a fetch (ugoira encode, HLS remux) plus a
/// download-and-reupload fallback routinely takes longer. The pipeline updates
/// `hint` when it knows what it is sending.
async fn run_with_chat_action<F: Future<Output = ()>>(
sender: &dyn MediaSender,
chat_id: i64,
hint: &parking_lot::Mutex<ActionHint>,
pipeline: F,
) {
// The guard is released before the await: a parking_lot guard held across
// it makes the future !Send, and the URL workers spawn these.
let action = hint.lock().action();
if let Err(e) = sender.send_chat_action(ChatId(chat_id), action).await {
log::error!("send_chat_action failed: {e}"); log::error!("send_chat_action failed: {e}");
} }
tokio::pin!(pipeline);
loop {
tokio::select! {
// `biased` polls the pipeline first, so a finished pipeline returns
// without ever arming the refresh timer (no stray actions).
biased;
() = &mut pipeline => return,
() = tokio::time::sleep(ACTION_REFRESH) => {
let action = hint.lock().action();
if let Err(e) = sender.send_chat_action(ChatId(chat_id), action).await {
log::error!("send_chat_action failed: {e}");
}
}
}
}
}
/// How often the chat-action indicator is refreshed while a pipeline runs.
/// Telegram's indicator lasts ~5s; refreshing slightly inside that keeps it
/// on-screen continuously.
const ACTION_REFRESH: std::time::Duration = std::time::Duration::from_secs(4);
/// What the chat action should say. Unknown before the fetch, so the pipeline
/// starts with `Typing` and switches as soon as the media types are known.
#[derive(Clone, Copy)]
enum ActionHint {
Typing,
Photo,
Video,
}
impl ActionHint {
/// Photos make Telegram label the send "sending photo"; video/animation
/// only payloads get "sending video". A mixed post takes the photo label
/// (the group's first item is always a photo, see `photos_first`).
fn for_items(items: &[MediaItemPayload]) -> Self {
if items
.iter()
.any(|item| matches!(item, MediaItemPayload::Photo { .. }))
{
Self::Photo
} else {
Self::Video
}
}
fn action(self) -> ChatAction {
match self {
Self::Typing => ChatAction::Typing,
Self::Photo => ChatAction::UploadPhoto,
Self::Video => ChatAction::UploadVideo,
}
}
}
/// User-facing text for a failed fetch. The [`FetchError`] class is what tells
/// the user whether the post is gone, withheld or the source is refusing
/// requests; a single generic sentence threw that away.
fn fetch_error_message(err: &x_media::site::FetchError) -> String {
use x_media::site::FetchError;
match err {
FetchError::NotFound => "Post not found (deleted, private or unavailable).".to_string(),
FetchError::Sensitive => concat!(
"This post's media is withheld (age-restricted). ",
"The bot owner must set TWITTER_AUTH_TOKEN to fetch it."
)
.to_string(),
FetchError::Blocked => {
"The source site refused the request (risk control). Try again later.".to_string()
}
FetchError::Disabled { site } => {
format!("{} support is disabled on this bot.", site_title(site))
}
FetchError::Transient(_) | FetchError::Http(_) => {
"The source site is unavailable right now (tried 3 times). Try again later.".to_string()
}
// Parse/shape surprises, pixiv auth details, oversized media: nothing
// actionable for the user beyond "this did not work".
_ => "Failed to fetch media from this link.".to_string(),
}
}
/// Site ids are lowercase ASCII (`pixiv`); user-facing text capitalizes the
/// first letter.
fn site_title(site: &str) -> String {
let mut chars = site.chars();
match chars.next() {
Some(first) => first.to_uppercase().collect::<String>() + chars.as_str(),
None => String::new(),
}
}
#[allow(clippy::too_many_arguments)]
async fn url_media_inner(
ctx: &AppContext<'_>,
chat_id: i64,
reply_to_message_id: i64,
url: &str,
post_send: PostSend,
hint: &parking_lot::Mutex<ActionHint>,
) {
let reply_to = MessageId(reply_to_message_id as i32);
// Link cache: a post sent before is re-sent from Telegram file ids — // Link cache: a post sent before is re-sent from Telegram file ids —
// no source-site request, no download, no upload. Keyed by the // no source-site request, no download, no upload. Keyed by the
@@ -355,6 +489,9 @@ pub(crate) async fn url_media(
}, },
}) })
.collect(); .collect();
// The indicator switches to "sending photo/video" once the kinds are
// known; `items` is moved into the task below.
*hint.lock() = ActionHint::for_items(&items);
let task = build_send_task( let task = build_send_task(
&chat_data, &chat_data,
chat_id, chat_id,
@@ -378,13 +515,7 @@ pub(crate) async fn url_media(
// Retries exhausted: notify the user (Rust-only requirement 3). // Retries exhausted: notify the user (Rust-only requirement 3).
Err(e) => { Err(e) => {
log::error!("fetch {url}: {e}"); log::error!("fetch {url}: {e}");
let _ = reply( let _ = reply(ctx.sender, chat_id, reply_to, fetch_error_message(&e)).await;
ctx.sender,
chat_id,
reply_to,
"Failed to fetch media from this link.",
)
.await;
} }
Ok(Some(mut fetched)) => { Ok(Some(mut fetched)) => {
if fetched.media.is_empty() { if fetched.media.is_empty() {
@@ -426,6 +557,9 @@ pub(crate) async fn url_media(
.iter() .iter()
.map(|media| media_to_payload(media, fetched.sensitive)) .map(|media| media_to_payload(media, fetched.sensitive))
.collect(); .collect();
// The indicator switches to "sending photo/video" once the kinds
// are known; `items` is moved into the task below.
*hint.lock() = ActionHint::for_items(&items);
let task = build_send_task( let task = build_send_task(
&chat_data, &chat_data,
chat_id, chat_id,
@@ -695,4 +829,89 @@ mod tests {
// Dead-letter notification still reaches the chat that asked. // Dead-letter notification still reaches the chat that asked.
assert_eq!(notify_chat_id, Some(1)); assert_eq!(notify_chat_id, Some(1));
} }
#[test]
fn fetch_errors_map_to_distinct_user_messages() {
use x_media::site::FetchError;
let disabled = fetch_error_message(&FetchError::Disabled { site: "pixiv" });
assert_eq!(disabled, "Pixiv support is disabled on this bot.");
assert_eq!(
fetch_error_message(&FetchError::NotFound),
"Post not found (deleted, private or unavailable)."
);
let sensitive = fetch_error_message(&FetchError::Sensitive);
assert!(sensitive.contains("TWITTER_AUTH_TOKEN"), "{sensitive}");
let blocked = fetch_error_message(&FetchError::Blocked);
assert!(blocked.contains("refused"), "{blocked}");
// Each class that has something to say must differ from the generic
// fallback — one generic sentence for everything is what this fixes.
let generic = fetch_error_message(&FetchError::TooLarge);
for text in [disabled, sensitive, blocked] {
assert_ne!(text, generic);
}
}
#[test]
fn action_hint_follows_the_media_kind() {
use MediaItemPayload::{Animation, Photo, Video};
let photo = || Photo {
media: "https://p/1.jpg".into(),
has_spoiler: false,
fallback_url: None,
file_id: false,
};
let video = || Video {
media: "https://v/1.mp4".into(),
has_spoiler: false,
thumbnail: None,
fallback_url: None,
file_id: false,
};
// Unknown before the fetch: the pipeline starts on "typing".
assert!(matches!(ActionHint::Typing.action(), ChatAction::Typing));
assert!(matches!(
ActionHint::for_items(&[photo()]).action(),
ChatAction::UploadPhoto
));
assert!(matches!(
ActionHint::for_items(&[
video(),
Animation {
media: "https://v/2.mp4".into(),
has_spoiler: false,
file_id: false,
}
])
.action(),
ChatAction::UploadVideo
));
// A mixed post takes the photo label: `photos_first` always leads with
// a photo, which is what Telegram shows.
assert!(matches!(
ActionHint::for_items(&[video(), photo()]).action(),
ChatAction::UploadPhoto
));
}
#[tokio::test(start_paused = true)]
async fn a_long_pipeline_keeps_the_chat_action_alive() {
let sender = MockSender::scripted(vec![], permanent_error);
let hint = parking_lot::Mutex::new(ActionHint::Typing);
// Three refresh windows of work: Telegram would have dropped the
// indicator twice without the keep-alive.
let pipeline = async { tokio::time::sleep(ACTION_REFRESH * 3).await };
run_with_chat_action(&sender, 1, &hint, pipeline).await;
let actions = sender
.calls()
.iter()
.filter(|call| **call == "send_chat_action")
.count();
assert_eq!(actions, 3, "expected the initial action plus two refreshes");
}
} }
+10 -4
View File
@@ -2,7 +2,7 @@ use dotenv::dotenv;
use teloxide::dptree::endpoint; use teloxide::dptree::endpoint;
use teloxide::prelude::*; use teloxide::prelude::*;
use teloxide::stop::StopToken; use teloxide::stop::StopToken;
use teloxide::types::{ChatId, InputFile, MessageId}; use teloxide::types::{ChatId, InlineKeyboardMarkup, InputFile, MessageId};
use teloxide::update_listeners::{self, UpdateListener, webhooks}; use teloxide::update_listeners::{self, UpdateListener, webhooks};
use tokio::sync::watch; use tokio::sync::watch;
use x_media::site; use x_media::site;
@@ -119,13 +119,19 @@ async fn main() {
log::debug!("rate limiter: dropped {idle_limiters} idle bucket(s)"); log::debug!("rate limiter: dropped {idle_limiters} idle bucket(s)");
} }
for (chat_id, prompt_message_id) in removed { for (chat_id, prompt_message_id) in removed {
// If the prompt was already deleted, this fails with a // Rewritten in place, not announced: the sweep is a
// 400 "message to edit not found" — log and ignore. // background timer, and a fresh message would wake the chat
// up to a full TTL later about a prompt the user already
// walked away from. The edit drops the buttons too. If the
// prompt was already deleted this fails with a 400
// "message to edit not found" — log and ignore.
if let Err(e) = bot if let Err(e) = bot
.edit_message_reply_markup( .edit_message_text(
ChatId(chat_id), ChatId(chat_id),
MessageId(prompt_message_id as i32), MessageId(prompt_message_id as i32),
send::EDIT_PROMPT_EXPIRED_TEXT,
) )
.reply_markup(InlineKeyboardMarkup::default())
.await .await
{ {
log::info!("edit-expiry sweep: prompt message gone: {e}"); log::info!("edit-expiry sweep: prompt message gone: {e}");
+10 -2
View File
@@ -342,8 +342,16 @@ impl QueueWorker {
payload, payload,
}) => { }) => {
if row.attempts as u32 >= MAX_RETRIES { if row.attempts as u32 >= MAX_RETRIES {
let message = format!("task failed after {MAX_RETRIES} retries"); // The queue keeps only the payload, not the last error, so
log::error!("dead-lettering {}: {message}", row.id); // the cause of an exhausted retry is just that: exhausted.
// (The dead-letter message is read by the user, so it must
// not restate its own wrapper — see `failure_text`.)
let message = "retries exhausted".to_string();
log::error!(
"dead-lettering {}: {message} after {} attempt(s)",
row.id,
row.attempts + 1
);
self.delete_row(&row.id).await; self.delete_row(&row.id).await;
(self.dead_letter)(payload, message).await; (self.dead_letter)(payload, message).await;
} else { } else {
+96 -8
View File
@@ -28,8 +28,8 @@ use upload::{FallbackError, PreparedItem, prepare_upload_item, send_batch_via_up
// The crate-facing API of this module lives in its submodules; re-export the // The crate-facing API of this module lives in its submodules; re-export the
// parts other modules use so call sites stay `send::x`. // parts other modules use so call sites stay `send::x`.
pub(crate) use post_send::{ pub(crate) use post_send::{
KEEP_ALIVE, Settled, dead_letter_notify, enqueue_retry, handle_task, post_send_actions, EDIT_PROMPT_EXPIRED_TEXT, KEEP_ALIVE, Settled, dead_letter_notify, enqueue_retry, handle_task,
settle_task, post_send_actions, settle_task,
}; };
/// One process-wide Bot for queue workers. Building a fresh Bot (and its HTTP /// One process-wide Bot for queue workers. Building a fresh Bot (and its HTTP
@@ -230,7 +230,9 @@ fn collect_file_ids(messages: &[Message], batch: &[MediaItemPayload], out: &mut
} }
} }
pub const MAX_MEDIA_GROUP: usize = 9; /// Telegram's `sendMediaGroup` accepts 210 items per group; 10 (not the older
/// 9) means a 10-image post arrives as one album instead of two messages.
pub const MAX_MEDIA_GROUP: usize = 10;
/// Splits media into batches of at most [`MAX_MEDIA_GROUP`] items, moving the /// Splits media into batches of at most [`MAX_MEDIA_GROUP`] items, moving the
/// items out (no per-item clone). /// items out (no per-item clone).
@@ -728,13 +730,14 @@ mod tests {
fn chunk_media_items_sizes() { fn chunk_media_items_sizes() {
assert_eq!(chunk_media_items::<i32>(vec![]), Vec::<Vec<i32>>::new()); assert_eq!(chunk_media_items::<i32>(vec![]), Vec::<Vec<i32>>::new());
assert_eq!(chunk_media_items((0..9).collect()).len(), 1); assert_eq!(chunk_media_items((0..9).collect()).len(), 1);
assert_eq!(chunk_media_items((0..10).collect()).len(), 2); assert_eq!(chunk_media_items((0..10).collect()).len(), 1);
assert_eq!(chunk_media_items((0..11).collect()).len(), 2);
assert_eq!(chunk_media_items((0..25).collect()).len(), 3); assert_eq!(chunk_media_items((0..25).collect()).len(), 3);
assert_eq!(chunk_media_items((0..25).collect())[2].len(), 7); assert_eq!(chunk_media_items((0..25).collect())[2].len(), 5);
assert!( assert!(
chunk_media_items((0..25).collect()) chunk_media_items((0..25).collect())
.iter() .iter()
.all(|c| c.len() <= 9) .all(|c| c.len() <= MAX_MEDIA_GROUP)
); );
} }
@@ -807,7 +810,88 @@ mod tests {
.flatten() .flatten()
.map(|button| button.text.clone()) .map(|button| button.text.clone())
.collect(); .collect();
assert_eq!(labels, ["a", "b", "m", "q", "y", "z", "↩️ Confirm"]); assert_eq!(
labels,
["a", "b", "m", "q", "y", "z", "↩️ Confirm", "🛑 Skip"]
);
}
#[test]
fn edit_markup_folds_and_caps_the_template_buttons() {
// Telegram rejects a keyboard over 100 buttons, which would drop the
// whole prompt; the cap keeps it well under that.
let templates: HashMap<String, String> = (0..200)
.map(|i| (format!("t{i:03}"), "[]".to_string()))
.collect();
let keyboard = build_edit_markup(&templates);
let buttons: usize = keyboard.inline_keyboard.iter().map(Vec::len).sum();
assert!(
buttons <= 100,
"a keyboard Telegram rejects would lose the prompt: {buttons}"
);
assert_eq!(
buttons,
super::post_send::MAX_TEMPLATE_BUTTONS + 2,
"the cap plus the confirm/skip pair"
);
// Names are folded, not one per row.
assert_eq!(keyboard.inline_keyboard[0].len(), 3);
assert_eq!(keyboard.inline_keyboard.last().unwrap().len(), 2);
assert_eq!(super::post_send::hidden_template_count(&templates), 140);
// Under the cap nothing is hidden and every name gets a button.
let few: HashMap<String, String> = (0..4)
.map(|i| (format!("t{i}"), "[]".to_string()))
.collect();
assert_eq!(super::post_send::hidden_template_count(&few), 0);
assert_eq!(
build_edit_markup(&few)
.inline_keyboard
.iter()
.map(Vec::len)
.sum::<usize>(),
6
);
}
#[test]
fn failure_text_names_the_post_and_the_cause() {
// A send failure names the post (the cache key) and the cause, so the
// user knows which of their links died.
let task = sequence_task("https://x.com/u/status/1");
let text = super::post_send::failure_text(Some(&task), "retries exhausted");
assert!(text.contains("twitter:1"), "{text}");
assert!(text.contains("retries exhausted"), "{text}");
// A channel-forward failure has no source URL: it must not claim a
// post failed.
let forward = Task::ForwardMessages {
from_chat_id: 1,
to_chat_id: 2,
message_ids: vec![1],
notify_chat_id: None,
notify_message_id: None,
};
let text = super::post_send::failure_text(Some(&forward), "chat not found");
assert!(text.starts_with("Forward failed permanently"), "{text}");
assert!(text.contains("chat not found"), "{text}");
}
#[test]
fn edit_prompt_text_states_the_ttl_and_the_confirm_requirement() {
use std::time::Duration;
let text = super::post_send::edit_prompt_text(Duration::from_secs(24 * 3600));
assert!(text.contains("Expires in 24h"), "{text}");
assert!(text.contains("Confirm"), "{text}");
// The wording of the whole point: no Confirm, no forward.
assert!(text.contains("Nothing is forwarded"), "{text}");
// Sub-hour TTLs must not render "0h".
assert!(
super::post_send::edit_prompt_text(Duration::from_secs(90)).contains("Expires in 1m")
);
assert!(
super::post_send::edit_prompt_text(Duration::from_secs(30)).contains("Expires in 30s")
);
} }
#[test] #[test]
@@ -1311,7 +1395,11 @@ mod tests {
post_send_actions(&ctx, &task, vec![10, 11]).await; post_send_actions(&ctx, &task, vec![10, 11]).await;
assert_eq!(sender.calls(), vec!["send_message"]); assert_eq!(sender.calls(), vec!["send_message"]);
assert_eq!(sender.messages(), vec!["Reply to edit message."]); // The prompt explains the Confirm requirement and the TTL (see the
// pure `edit_prompt_text` test for the exact wording).
let prompt_text = sender.messages().first().cloned().unwrap_or_default();
assert!(prompt_text.contains("Expires in"), "{prompt_text}");
assert!(prompt_text.contains("Confirm"), "{prompt_text}");
// The prompt's own message id keys the record the reply will edit. // The prompt's own message id keys the record the reply will edit.
let data = stores.chat_store().get(1).await; let data = stores.chat_store().get(1).await;
let record = data let record = data
+90 -19
View File
@@ -103,26 +103,76 @@ pub(crate) fn release_keep_alive(task: &Task) {
}); });
} }
/// One button per template name (column layout), then the confirm button. /// The edit-before-forward prompt's text. It names both controls and the TTL,
/// Sorted by name: the templates live in a `HashMap`, so an unsorted walk /// because the buttons alone left users waiting for a forward that never came
/// would reshuffle the buttons between prompts. /// (nothing is forwarded until Confirm).
pub(super) fn edit_prompt_text(ttl: std::time::Duration) -> String {
format!(
"Reply to edit the caption, or tap a template, then ↩️ Confirm to forward. \
Expires in {}. Nothing is forwarded until you confirm.",
coarsest_unit(ttl)
)
}
/// Text the prompt is rewritten to once its record expires. The sweep edits
/// the prompt in place (see `main`): announcing the expiry with a new message
/// would wake the chat up to a full TTL later about a prompt nobody is
/// waiting on.
pub(crate) const EDIT_PROMPT_EXPIRED_TEXT: &str = "⌛ Expired — nothing was forwarded.";
/// `24h` / `90m` / `45s`: the coarsest whole unit, so the prompt stays short.
fn coarsest_unit(ttl: std::time::Duration) -> String {
let secs = ttl.as_secs();
if secs >= 3600 {
format!("{}h", secs / 3600)
} else if secs >= 60 {
format!("{}m", secs / 60)
} else {
format!("{secs}s")
}
}
/// Templates per keyboard row. Telegram rejects a keyboard with more than 100
/// buttons *outright*, which would silently drop the whole prompt, so the
/// names are folded and capped rather than listed one per row.
pub(super) const TEMPLATE_BUTTONS_PER_ROW: usize = 3;
/// Hard cap on template buttons; the prompt text names the ones not shown.
pub(super) const MAX_TEMPLATE_BUTTONS: usize = 60;
/// Template buttons ([`TEMPLATE_BUTTONS_PER_ROW`] per row, at most
/// [`MAX_TEMPLATE_BUTTONS`]), then the confirm/skip pair. Sorted by name: the
/// templates live in a `HashMap`, so an unsorted walk would reshuffle the
/// buttons between prompts.
pub(super) fn build_edit_markup(templates: &HashMap<String, String>) -> InlineKeyboardMarkup { pub(super) fn build_edit_markup(templates: &HashMap<String, String>) -> InlineKeyboardMarkup {
let mut names: Vec<&String> = templates.keys().collect(); let mut names: Vec<&String> = templates.keys().collect();
names.sort(); names.sort();
let mut rows = Vec::with_capacity(names.len() + 1); let shown = names.len().min(MAX_TEMPLATE_BUTTONS);
for name in names { let mut rows = Vec::with_capacity(shown / TEMPLATE_BUTTONS_PER_ROW + 2);
rows.push(vec![InlineKeyboardButton::callback( for chunk in names[..shown].chunks(TEMPLATE_BUTTONS_PER_ROW) {
name.clone(), rows.push(
format!("template|{name}"), chunk
)]); .iter()
.map(|name| {
InlineKeyboardButton::callback(name.as_str(), format!("template|{name}"))
})
.collect(),
);
} }
rows.push(vec![InlineKeyboardButton::callback( // Skip exists because the prompt holds the forward hostage until Confirm:
"↩️ Confirm", // without it the only escape was deleting the message and waiting out the
"forward", // TTL for a forward that then never happens.
)]); rows.push(vec![
InlineKeyboardButton::callback("↩️ Confirm", "forward"),
InlineKeyboardButton::callback("🛑 Skip", "skip"),
]);
InlineKeyboardMarkup::new(rows) InlineKeyboardMarkup::new(rows)
} }
/// How many templates the markup could not fit, for the prompt text.
pub(super) fn hidden_template_count(templates: &HashMap<String, String>) -> usize {
templates.len().saturating_sub(MAX_TEMPLATE_BUTTONS)
}
/// Notifies a chat about a dead-lettered task (skips when `notify_chat_id` is /// Notifies a chat about a dead-lettered task (skips when `notify_chat_id` is
/// absent). /// absent).
pub(super) async fn notify_failure( pub(super) async fn notify_failure(
@@ -185,12 +235,21 @@ pub(crate) async fn post_send_actions(ctx: &AppContext<'_>, task: &Task, message
}; };
if edit_before_forward { if edit_before_forward {
let keyboard = build_edit_markup(&ctx.chat_store.get(chat_id).await.template); let templates = ctx.chat_store.get(chat_id).await.template;
let keyboard = build_edit_markup(&templates);
let mut text = edit_prompt_text(ctx.config.edit_message_ttl);
let hidden = hidden_template_count(&templates);
if hidden > 0 {
// The keyboard is capped; say so instead of silently hiding them.
text.push_str(&format!(
"\n({hidden} more templates not shown — /remove_template to prune.)"
));
}
let prompt = ctx let prompt = ctx
.sender .sender
.send_message( .send_message(
ChatId(chat_id), ChatId(chat_id),
"Reply to edit message.".to_string(), text,
Some(MessageId(reply_to as i32)), Some(MessageId(reply_to as i32)),
Some(keyboard), Some(keyboard),
) )
@@ -247,7 +306,7 @@ pub(crate) async fn post_send_actions(ctx: &AppContext<'_>, task: &Task, message
ctx.sender, ctx.sender,
notify_chat_id, notify_chat_id,
notify_message_id, notify_message_id,
&format!("Task failed after retries: {message}"), &failure_text(None, &message),
) )
.await; .await;
} }
@@ -341,6 +400,17 @@ async fn send_media_or_animation(ctx: &AppContext<'_>, task: &Task) -> Result<Ve
} }
} }
/// User-facing text for a task that will never run again: which link died and
/// why. The raw error alone left the user guessing which post it was about.
pub(super) fn failure_text(task: Option<&Task>, message: &str) -> String {
match task.and_then(|task| task.source_url()).map(log_key) {
Some(key) => format!("Send failed permanently for {key}: {message}"),
// `ForwardMessages` carries no source URL: that failure is about the
// channel copy, not about a post.
None => format!("Forward failed permanently: {message}"),
}
}
/// Dead-letter callback wired to the queue in main: settles the task and /// Dead-letter callback wired to the queue in main: settles the task and
/// notifies its chat. /// notifies its chat.
pub(crate) async fn dead_letter_notify( pub(crate) async fn dead_letter_notify(
@@ -351,8 +421,9 @@ pub(crate) async fn dead_letter_notify(
// A dead-lettered task never runs again, and the queue dead-letters retry // A dead-lettered task never runs again, and the queue dead-letters retry
// exhaustion itself (the handler is not called again), so this is the only // exhaustion itself (the handler is not called again), so this is the only
// place that sees the final payload. // place that sees the final payload.
if let Ok(task) = serde_json::from_value::<Task>(payload.clone()) { let task = serde_json::from_value::<Task>(payload.clone()).ok();
settle_task(ctx, &task, Settled::Failed).await; if let Some(task) = &task {
settle_task(ctx, task, Settled::Failed).await;
} }
let notify_chat_id = payload.get("notify_chat_id").and_then(|v| v.as_i64()); let notify_chat_id = payload.get("notify_chat_id").and_then(|v| v.as_i64());
let notify_message_id = payload.get("notify_message_id").and_then(|v| v.as_i64()); let notify_message_id = payload.get("notify_message_id").and_then(|v| v.as_i64());
@@ -360,7 +431,7 @@ pub(crate) async fn dead_letter_notify(
ctx.sender, ctx.sender,
notify_chat_id, notify_chat_id,
notify_message_id, notify_message_id,
&format!("Task failed after retries: {message}"), &failure_text(task.as_ref(), &message),
) )
.await; .await;
} }