mirror of
https://github.com/TheFunny/TelegramTwitterMediaBot.git
synced 2026-09-23 23:32:05 +00:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9e873131d4
|
||
|
|
5b77d14497
|
||
|
|
d4c36feb9a
|
||
|
|
5d0acdac01
|
||
|
|
d6707133cc
|
@@ -4,7 +4,7 @@
|
|||||||
|
|
||||||
Telegram bot (teloxide) that turns post links from X/Twitter, Pixiv, Bluesky, Misskey (misskey.io), and Bilibili dynamics into media messages (images, video, GIF) with the post's title, author, and tags. It supports batch media splitting, retry with persistence, inline queries, forward-channel rebinding with caption templates, and Pixiv ugoira→MP4 transcoding. README is in Chinese; user-facing bot strings are in English. The project is a Rust port of a Python predecessor (see `queue.rs` comments referencing `utils/task_queue.py`).
|
Telegram bot (teloxide) that turns post links from X/Twitter, Pixiv, Bluesky, Misskey (misskey.io), and Bilibili dynamics into media messages (images, video, GIF) with the post's title, author, and tags. It supports batch media splitting, retry with persistence, inline queries, forward-channel rebinding with caption templates, and Pixiv ugoira→MP4 transcoding. README is in Chinese; user-facing bot strings are in English. The project is a Rust port of a Python predecessor (see `queue.rs` comments referencing `utils/task_queue.py`).
|
||||||
|
|
||||||
Two-crate Cargo workspace (both v1.7.0, edition 2024, resolver 3):
|
Two-crate Cargo workspace (both v1.8.0, edition 2024, resolver 3):
|
||||||
|
|
||||||
- **`crates/x-media`** — library that fetches and normalizes media from the four sites. Pure, no Telegram knowledge.
|
- **`crates/x-media`** — library that fetches and normalizes media from the four sites. Pure, no Telegram knowledge.
|
||||||
- **`crates/xmedia-bot`** — the bot binary: teloxide dispatcher, SQLite-backed chat state, persistent task queue.
|
- **`crates/xmedia-bot`** — the bot binary: teloxide dispatcher, SQLite-backed chat state, persistent task queue.
|
||||||
@@ -18,24 +18,30 @@ Telegram update → Dispatcher (polling or axum webhook) → dptree branches
|
|||||||
└─ callback_query → "forward" (copy to channel) / "template|<name>" (apply caption template)
|
└─ callback_query → "forward" (copy to channel) / "template|<name>" (apply caption template)
|
||||||
```
|
```
|
||||||
|
|
||||||
Message flow: `message_handler` extracts URLs (from `url`/`text_link` entities, text + caption, deduped) → `x_media::site::fetch(url)` → `Fetched` → builds a `Task` → `send::send_media_sequence` (media groups ≤ 9, caption on first item) or `send::send_animation`. On Telegram URL-fetch failure or size error (`send_batch_via_upload`): download via `x_media::site::download_media` to a temp file (≤ 10 MiB), sniff magic bytes (`sniff_ext`), upload via multipart; oversized items fall back to `fallback_url`. On failure: `enqueue_retry` persists resume-state `Task` into the SQLite queue → workers lease (120 s lock TTL) → retry with exponential backoff (≤ 30 s, `MAX_RETRIES = 2`) → dead-letter → `notify_failure`. Success → `post_send_actions`: edit-before-forward prompt with inline buttons, or `copy_messages` to the bound forward channel.
|
Message flow: `message_handler` extracts URLs (from `url`/`text_link` entities, text + caption, deduped) → `x_media::site::fetch(url)` → `Fetched` → builds a `Task` → `send::send_media_sequence` (media groups ≤ 10, caption on first item) or `send::send_animation`. On Telegram URL-fetch failure or size error (`send_batch_via_upload`): download via `x_media::site::download_media` to a temp file (≤ 10 MiB), sniff magic bytes (`sniff_ext`), upload via multipart; oversized items fall back to `fallback_url`. On failure: `enqueue_retry` persists resume-state `Task` into the SQLite queue → workers lease (120 s lock TTL) → retry with exponential backoff (≤ 30 s, `MAX_RETRIES = 2`) → dead-letter → `notify_failure`. Success → `post_send_actions`: edit-before-forward prompt with inline buttons, or `copy_messages` to the bound forward channel.
|
||||||
|
|
||||||
Debug command: `/debug <url>` runs the same `x_media::site::fetch` and replies with `debug_report` (`handlers/commands.rs`) — site id, normalized cache key, source URL, title/author/tags, sensitive flag, caption and the media list — nothing is sent, cached or forwarded; the report is capped at 4000 chars and sent with HTML parse mode: raw fields are escaped, and the caption is wrapped in a `<blockquote>` so it renders exactly like the sent media caption (escaped text and links included).
|
Debug command: `/debug <url>` runs the same `x_media::site::fetch` and replies with `debug_report` (`handlers/commands.rs`) — site id, normalized cache key, source URL, title/author/tags, sensitive flag, caption and the media list — nothing is sent, cached or forwarded; the report is capped at 4000 chars and sent with HTML parse mode: raw fields are escaped, and the caption is wrapped in a `<blockquote>` so it renders exactly like the sent media caption (escaped text and links included). The caption it shows is `preview_caption`'s: the chat's per-site format override plus the long-post quoting, i.e. exactly what the send paths produce — showing the raw built-in caption made `/set_format` look like a no-op, and the `/set_format` success reply points users at `/debug` to preview.
|
||||||
|
|
||||||
The `/test <url>` command runs the ordinary link pipeline (`urls::url_media`) with `PostSend::Suppressed`: the media is sent and cached like any other link, but the chat's `forward_channel_id`/`edit_before_forward` are ignored, so a test never forwards to the channel and never opens the edit prompt (retries and dead-letter notifications behave as usual). Both commands use a custom `parse_arg_remainder` parser (whole remainder, trimmed) because teloxide's built-in `split` parser takes exactly one space-separated token.
|
User-facing failure text is a function of the error class, never one generic sentence: `urls::fetch_error_message` maps `FetchError::NotFound` (post gone), `Sensitive` (withheld, needs `TWITTER_AUTH_TOKEN`), `Blocked` (source risk control), `Disabled { site }` (a registered site switched off — pixiv without a token, the one case `fetch` answers `Err` instead of `Ok(None)`) and `Transient`/`Http` (source down) apart. The same distinction drives the group hint: a supported link posted in a group (not a channel) gets one `GROUP_LINK_HINT` reply, because the link pipeline is private-chat only.
|
||||||
|
|
||||||
The `x-media` library: `site::fetch(url)` dispatches through the `SITES` registry (per-site `impl Site`, in order twitter → bsky → misskey → pixiv → bilibili) and returns `Ok(None)` for unmatched URLs. `Fetched { source_url, caption, title, content, media: Vec<Media>, sensitive, site_id, … }` (title and content are split per platform: a pixiv artwork's title and description, a bilibili headline and body, and text-only posts whose text is all `content`); `caption_with(format)` substitutes `{url} {author} {author_url} {title} {content} {tags}`.
|
The `/test <url>` command runs the ordinary link pipeline (`urls::url_media`) with `PostSend::Suppressed`: the media is sent and cached like any other link, but the chat's `forward_channel_id`/`edit_before_forward` are ignored, so a test never forwards to the channel and never opens the edit prompt (retries and dead-letter notifications behave as usual). `/test`, `/debug`, `/set_format` and `/clear_cache` use the custom `parse_arg_remainder` parser (whole remainder, trimmed) because teloxide's built-in `split` parser takes exactly one space-separated token per field: `/set_format <site> <format>` never parsed with it (and `/clear_cache` without an argument did not either), and a command that fails to parse falls through to the URL flow in silence. `commands::tests::every_documented_invocation_parses` pins every documented form against exactly that.
|
||||||
|
|
||||||
|
The inline path (`handlers/inline.rs`) hands media URLs straight to Telegram, which fetches them itself and cannot send site-specific headers — so `x_media::site::needs_media_headers(url)` (true exactly where a site's `media_headers` is non-empty, i.e. pixiv's pximg.net) marks the media that must be skipped instead of shipped broken; locally produced media (ugoira MP4, bsky remux) fails `Url::parse` and is skipped the same way. Inline results are therefore URL-only by construction.
|
||||||
|
|
||||||
|
`url_media` is a thin wrapper over `url_media_inner`: `run_with_chat_action` sends the chat action, then re-sends it every `ACTION_REFRESH` (4 s) while the pipeline future is pending, because Telegram drops an action after ~5 s and a fetch (ugoira encode, HLS remux) plus an upload routinely outlasts that. The pipeline flips the shared `ActionHint` from `Typing` to `UploadPhoto`/`UploadVideo` once the media kinds are known. The `select!` is `biased` on the pipeline branch so a finished pipeline never emits a stray action.
|
||||||
|
|
||||||
|
The `x-media` library: `site::fetch(url)` dispatches through the `SITES` registry (per-site `impl Site`, in order twitter → bsky → misskey → pixiv → bilibili) and returns `Ok(None)` for unmatched URLs (`Err(FetchError::Disabled { site })` when the URL matches a registered site whose `enabled()` is false — see `disabled_site`). `Fetched { source_url, caption, title, content, media: Vec<Media>, sensitive, site_id, … }` (title and content are split per platform: a pixiv artwork's title and description, a bilibili headline and body, and text-only posts whose text is all `content`); `caption_with(format)` substitutes `{url} {author} {author_url} {title} {content} {tags}`.
|
||||||
|
|
||||||
## Key Directories
|
## Key Directories
|
||||||
|
|
||||||
| Path | Purpose |
|
| Path | Purpose |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `crates/x-media/src/` | Fetch library. `site/mod.rs` = dispatcher + `Fetched`/`FetchError`/`download_media`/`media_size`; `media.rs` = `Media` enum; `examples/fetch.rs` = end-to-end usage sample |
|
| `crates/x-media/src/` | Fetch library. `site/mod.rs` = dispatcher + `Fetched`/`FetchError`/`download_media`/`media_size`; `media.rs` = `Media` enum; `examples/fetch.rs` = end-to-end usage sample |
|
||||||
| `crates/x-media/src/site/<twitter\|pixiv\|bsky\|misskey\|bilibili>/` | One directory per site: `mod.rs` (re-exports), `interface.rs` (PATTERN, `enabled()`, `fetch_from_url()`, `cache_key`/`is_retryable`/`media_headers`, unit struct `<Name>Site` implementing `site::Site`, `From<SiteStruct> for Fetched`), `model.rs` (serde DTOs). Pixiv adds `api.rs` (auth + transport); twitter adds `auth.rs` (logged-in GraphQL `TweetDetail` fallback for NSFW tweets, gated on `TWITTER_AUTH_TOKEN`). Misskey targets misskey.io only (`POST /api/notes/show`, 400+`NO_SUCH_NOTE` → NotFound). Bilibili fetches dynamics (images/animated images only — an attached video degrades to its cover, and its title stands in for the post text, which AV dynamics do not have) from `/x/polymer/web-dynamic/v1/detail` sent with `features=itemOpusStyle` (without that flag the legacy serialization drops an image/text post's body and headline entirely — `desc` comes back `null`; the adapter still parses the legacy `major.draw`/`desc`/`archive` shapes as a fallback). No WBI signature is involved; device cookies `buvid3`/`buvid4` are fetched automatically from `/x/frontend/finger/spi` because bilibili's `-352` risk control starts rejecting plain requests, `BILIBILI_COOKIE` is the escalation when an IP stays blocked; `b23.tv` short links are deliberately unmatched. Twitter's `from_syndication_json` HTML-decodes the API text — syndication and GraphQL `full_text` both arrive pre-escaped (`>` `<` `&` `'`) — so the stored text is raw and the caption escap…
|
| `crates/x-media/src/site/<twitter\|pixiv\|bsky\|misskey\|bilibili>/` | One directory per site: `mod.rs` (re-exports), `interface.rs` (PATTERN, `enabled()`, `fetch_from_url()`, `cache_key`/`is_retryable`/`media_headers`, unit struct `<Name>Site` implementing `site::Site`, `From<SiteStruct> for Fetched`), `model.rs` (serde DTOs). Pixiv adds `api.rs` (auth + transport); twitter adds `auth.rs` (logged-in GraphQL `TweetDetail` fallback for NSFW tweets, gated on `TWITTER_AUTH_TOKEN`; without the token a withheld tweet stays `FetchError::Sensitive` and the bot reports it as age-restricted instead of "no media"). Misskey targets misskey.io only (`POST /api/notes/show`, 400+`NO_SUCH_NOTE` → NotFound). Bilibili fetches dynamics (images/animated images only — an attached video degrades to its cover, and its title stands in for the post text, which AV dynamics do not have) from `/x/polymer/web-dynamic/v1/detail` sent with `features=itemOpusStyle` (without that flag the legacy serialization drops an image/text post's body and headline entirely — `desc` comes back `null`; the adapter still parses the legacy `major.draw`/`desc`/`archive` shapes as a fallback). No WBI signature is involved; device cookies `buvid3`/`buvid4` are fetched automatically from `/x/frontend/finger/spi` because bilibili's `-352` risk control starts rejecting plain requests, `BILIBILI_COOKIE` is the escalation when an IP stays blocked; `b23.tv` short links are deliberately unmatched. Twitter's `from_syndication_json` HTML-decodes the API text — syndication and GraphQL `full_text` both arrive pre-escaped (`>` `<` `&` `'`) — so the stored text is raw and the caption escap…
|
||||||
| `crates/xmedia-bot/src/main.rs` | Entry point: env/log init, command registration (`register_commands`), shared `send::BOT` force-init, queue worker start, site login validation (`site::validate_all`), 300 s edit-expiry sweep, dptree handler tree, webhook vs polling dispatch |
|
| `crates/xmedia-bot/src/main.rs` | Entry point: env/log init, command registration (`register_commands` — `setMyCommands` plus the profile description texts), shared `send::BOT` force-init, queue worker start, site login validation (`site::validate_all`), 300 s edit-expiry sweep (expired prompts are rewritten in place to `EDIT_PROMPT_EXPIRED_TEXT` with an empty keyboard — an edit, never a new message, so a background timer cannot wake a chat), dptree handler tree, webhook vs polling dispatch |
|
||||||
| `crates/xmedia-bot/src/config.rs` | Manual env parsing into `Config` |
|
| `crates/xmedia-bot/src/config.rs` | Manual env parsing into `Config` |
|
||||||
| `crates/xmedia-bot/src/db.rs` | `DbPool`: one shared SQLite connection pool (`POOL_SIZE = 4`, WAL, busy_timeout) for all three tables over `$DATA_DIR/task_queue.db` (default `data/`) — the three stores share it; `open_store` creates file + schema, `with_conn` runs all rusqlite I/O in `spawn_blocking` |
|
| `crates/xmedia-bot/src/db.rs` | `DbPool`: one shared SQLite connection pool (`POOL_SIZE = 4`, WAL, busy_timeout) for all three tables over `$DATA_DIR/task_queue.db` (default `data/`) — the three stores share it; `open_store` creates file + schema, `with_conn` runs all rusqlite I/O in `spawn_blocking` |
|
||||||
| `crates/xmedia-bot/src/handlers/` | Handler modules: `mod.rs` (message entry point, `reply`, `log_key`), `commands.rs` (teloxide `BotCommands` enum + command executor, incl. `/test <url>` (send-only) / `/debug <url>` (parse-only) and the admin-only `/bot_dict` state dump), `urls.rs` (URL extraction + bounded job channel (256) drained by `URL_WORKERS = 8` workers (`start_url_workers`) — backpressure instead of unbounded spawns; teloxide's per-chat workers are sequential — batch-forwards need concurrency), `inline.rs`/`callback.rs` (inline queries / edit-before-forward buttons), `statics.rs` (global statics) |
|
| `crates/xmedia-bot/src/handlers/` | Handler modules: `mod.rs` (message entry point, `reply`, `log_key`, the group-only `GROUP_LINK_HINT` for a supported link posted outside a private chat), `commands.rs` (teloxide `BotCommands` enum + command executor, incl. `/test <url>` (send-only) / `/debug <url>` (parse-only) and the admin-only `/bot_dict` state dump; `/set_format` rejects unknown `{…}` placeholders and resets with `-`), `urls.rs` (URL extraction + bounded job channel (256) drained by `URL_WORKERS = 8` workers (`start_url_workers`) — backpressure instead of unbounded spawns; teloxide's per-chat workers are sequential — batch-forwards need concurrency), `inline.rs`/`callback.rs` (inline queries / edit-before-forward buttons, incl. `skip`), `statics.rs` (global statics) |
|
||||||
| `crates/xmedia-bot/src/state.rs` | `ChatStore`: parking_lot `Mutex<HashMap>` cache + SQLite write-through (`chat_state` table) |
|
| `crates/xmedia-bot/src/state.rs` | `ChatStore`: parking_lot `Mutex<HashMap>` cache + SQLite write-through (`chat_state` table) |
|
||||||
| `crates/xmedia-bot/src/link_cache.rs` | `LinkCache`: SQLite-backed cache (`link_cache` table) of successfully sent posts — raw caption fields + Telegram `file_id`s; repeat links re-send locally (no fetch/upload), TTL + prune, invalidated on permanent send failure |
|
| `crates/xmedia-bot/src/link_cache.rs` | `LinkCache`: SQLite-backed cache (`link_cache` table) of successfully sent posts — raw caption fields + Telegram `file_id`s; repeat links re-send locally (no fetch/upload), TTL + prune, invalidated on permanent send failure |
|
||||||
| `crates/xmedia-bot/src/queue.rs` | `PersistentTaskQueue`: SQLite-backed queue (`tasks` table), `QUEUE_WORKERS = 4` concurrent workers (lease via `BEGIN IMMEDIATE` + `locked_until` TTL), retry→dead-letter, `notify_one` worker wakeup plus a separate `Notify` for the 30 s lease-expiry sweep (a shared one let the sweep steal the workers' wakeup permit), `busy_timeout` on all connections |
|
| `crates/xmedia-bot/src/queue.rs` | `PersistentTaskQueue`: SQLite-backed queue (`tasks` table), `QUEUE_WORKERS = 4` concurrent workers (lease via `BEGIN IMMEDIATE` + `locked_until` TTL), retry→dead-letter, `notify_one` worker wakeup plus a separate `Notify` for the 30 s lease-expiry sweep (a shared one let the sweep steal the workers' wakeup permit), `busy_timeout` on all connections |
|
||||||
@@ -60,7 +66,7 @@ Docker: `docker build -t tgxmb .` then `docker run --rm -d --name tgxmb --env-fi
|
|||||||
|
|
||||||
## Code Conventions & Common Patterns
|
## Code Conventions & Common Patterns
|
||||||
|
|
||||||
- **Errors via `thiserror` derive** (no anyhow): the public, stringified errors — `FetchError` (`Http`/`Json`/`Pixiv`/`Site`/`NotFound`/`Blocked`) and `PixivError` — derive `thiserror::Error` with `#[from]` conversions; `Display`/`source()` come from the derive. The internal control-flow enums — `QueueError` (`Retryable { delay_seconds, payload }` / `Permanent`), `SendError` (Retryable/Permanent), `Classification`, `FallbackError` — carry no `Display` and are handled by direct variant matching. New errors should follow the same split: stringified/public errors derive `thiserror`, internal flow enums stay plain.
|
- **Errors via `thiserror` derive** (no anyhow): the public, stringified errors — `FetchError` (`Http`/`Json`/`Pixiv`/`Site`/`NotFound`/`Blocked`/`Disabled`/`Sensitive`/`TooLarge`/`Transient`/`Io`) and `PixivError` — derive `thiserror::Error` with `#[from]` conversions; `Display`/`source()` come from the derive. The internal control-flow enums — `QueueError` (`Retryable { delay_seconds, payload }` / `Permanent`), `SendError` (Retryable/Permanent), `Classification`, `FallbackError` — carry no `Display` and are handled by direct variant matching. New errors should follow the same split: stringified/public errors derive `thiserror`, internal flow enums stay plain.
|
||||||
- **Global state via `std::sync::LazyLock` statics**, not DI: `CONFIG`, `CHAT_STORE`, `TASK_QUEUE` in `handlers/statics.rs`; shared reqwest `CLIENT` in `x-media/src/site/mod.rs`. `Bot` is passed/cloned into handlers; queue workers share the process-wide `send::BOT` (`LazyLock<Bot>`, force-initialized in `main` so a missing token fails at startup).
|
- **Global state via `std::sync::LazyLock` statics**, not DI: `CONFIG`, `CHAT_STORE`, `TASK_QUEUE` in `handlers/statics.rs`; shared reqwest `CLIENT` in `x-media/src/site/mod.rs`. `Bot` is passed/cloned into handlers; queue workers share the process-wide `send::BOT` (`LazyLock<Bot>`, force-initialized in `main` so a missing token fails at startup).
|
||||||
- **Async**: tokio multi-thread runtime (`#[tokio::main]` default). All rusqlite I/O inside `tokio::task::spawn_blocking`. Long loops use `tokio::select!` with `tokio::sync::{watch, Notify}` stop/wake channels. No streams.
|
- **Async**: tokio multi-thread runtime (`#[tokio::main]` default). All rusqlite I/O inside `tokio::task::spawn_blocking`. Long loops use `tokio::select!` with `tokio::sync::{watch, Notify}` stop/wake channels. No streams.
|
||||||
- **Blocking sync primitives**: `parking_lot::Mutex` for hot caches, `tokio::sync::Mutex` for async-shared state (pixiv token cache), `AtomicBool` for feature gates.
|
- **Blocking sync primitives**: `parking_lot::Mutex` for hot caches, `tokio::sync::Mutex` for async-shared state (pixiv token cache), `AtomicBool` for feature gates.
|
||||||
@@ -75,10 +81,10 @@ Docker: `docker build -t tgxmb .` then `docker run --rm -d --name tgxmb --env-fi
|
|||||||
| File | Why it matters |
|
| File | Why it matters |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `crates/xmedia-bot/src/main.rs` | Startup sequence, webhook vs polling, graceful shutdown (SIGINT via teloxide ctrlc / SIGTERM via `stop_token` for docker, → sweep stop → admin msg → queue stop) |
|
| `crates/xmedia-bot/src/main.rs` | Startup sequence, webhook vs polling, graceful shutdown (SIGINT via teloxide ctrlc / SIGTERM via `stop_token` for docker, → sweep stop → admin msg → queue stop) |
|
||||||
| `crates/xmedia-bot/src/handlers/` | `statics.rs` = `CHAT_STORE`/`TASK_QUEUE`/`CONFIG` singletons (open `$DATA_DIR/task_queue.db`, default `data/` **relative to CWD**, dir auto-created); `commands.rs` = command dispatch (incl. `/test <url>` send-only, `/debug <url>` parse-only, and the admin-only `/bot_dict` state dump); `urls.rs` = URL extraction + the per-URL pipeline (`url_media` takes a `PostSend` mode: chat settings vs `/test`'s suppressed actions); `inline.rs` = debounced inline queries; `callback.rs` = edit-before-forward buttons (dptree entry + testable `handle_callback` core) |
|
| `crates/xmedia-bot/src/handlers/` | `statics.rs` = `CHAT_STORE`/`TASK_QUEUE`/`CONFIG` singletons (open `$DATA_DIR/task_queue.db`, default `data/` **relative to CWD**, dir auto-created); `commands.rs` = command dispatch (incl. `/test <url>` send-only, `/debug <url>` parse-only, the read-only `/settings` every chat member can read — unlike the admin-only `/bot_dict` raw dump — and template removal; `/start`/`/help` carry the guidance teloxide's `descriptions()` cannot render, and `/set_format` rejects unknown `{…}` placeholders, resetting with `-`); `urls.rs` = URL extraction + the per-URL pipeline (`url_media` takes a `PostSend` mode: chat settings vs `/test`'s suppressed actions); `inline.rs` = debounced inline queries (hotlink-protected and local media skipped); `callback.rs` = edit-before-forward buttons (dptree entry + testable `handle_callback` core, incl. `skip`) |
|
||||||
| `crates/xmedia-bot/src/send/` | `mod.rs`: constants `MAX_MEDIA_GROUP = 9`; `classify_request_error`; the senders. `upload.rs`: download-and-reupload fallback triggered only by Telegram API errors (`is_media_fetch_failure` / `is_size_error`). `post_send.rs`: settlement (`settle_task`), cache write, post-send actions, queue handlers. `input_media.rs`: payload → `InputMedia` |
|
| `crates/xmedia-bot/src/send/` | `mod.rs`: constants `MAX_MEDIA_GROUP = 10`; `classify_request_error`; the senders. `upload.rs`: download-and-reupload fallback triggered only by Telegram API errors (`is_media_fetch_failure` / `is_size_error`). `post_send.rs`: settlement (`settle_task`), cache write, post-send actions (dead-letter text via `failure_text`: post key + cause, since the raw error alone does not say which link died), queue handlers. `input_media.rs`: payload → `InputMedia` |
|
||||||
| `crates/xmedia-bot/src/photo.rs` | Pure-Rust photo processing (no ffmpeg): `png` (image-png) decode/encode + `zune-jpeg` decode + `fast_image_resize` Lanczos3 downscale + `jpeg-encoder`. Photos over Telegram's limits (width + height > 10000 px → `PHOTO_INVALID_DIMENSIONS`; bytes > 10 MiB) are decoded, downscaled keeping the format, PNG bit depth > 24 (RGBA 32-bit / 16-bit per channel) reduced to 24-bit RGB with alpha flattened white (≤24-bit untouched, never upconverted), and transcoded to JPEG only if still over the cap; memory budget guarded, otherwise the item's smaller fallback URL |
|
| `crates/xmedia-bot/src/photo.rs` | Pure-Rust photo processing (no ffmpeg): `png` (image-png) decode/encode + `zune-jpeg` decode + `fast_image_resize` Lanczos3 downscale + `jpeg-encoder`. Photos over Telegram's limits (width + height > 10000 px → `PHOTO_INVALID_DIMENSIONS`; bytes > 10 MiB) are decoded, downscaled keeping the format, PNG bit depth > 24 (RGBA 32-bit / 16-bit per channel) reduced to 24-bit RGB with alpha flattened white (≤24-bit untouched, never upconverted), and transcoded to JPEG only if still over the cap; memory budget guarded, otherwise the item's smaller fallback URL |
|
||||||
| `crates/x-media/src/site/mod.rs` | Dispatcher, `Fetched`/`FetchError`, shared `CLIENT`, `download_media` (adds `Referer: https://www.pixiv.net/` for `pximg.net` hotlink protection) |
|
| `crates/x-media/src/site/mod.rs` | Dispatcher, `Fetched`/`FetchError`, shared `CLIENT`, `download_media` (adds `Referer: https://www.pixiv.net/` for `pximg.net` hotlink protection), `needs_media_headers` (the same per-site rule, asked by the inline path to skip what Telegram cannot fetch) |
|
||||||
| `crates/x-media/src/site/pixiv/api.rs` | OAuth token exchange (hardcoded app client id/secret), access-token cache, ugoira zip→MP4 via ffmpeg in `spawn_blocking` |
|
| `crates/x-media/src/site/pixiv/api.rs` | OAuth token exchange (hardcoded app client id/secret), access-token cache, ugoira zip→MP4 via ffmpeg in `spawn_blocking` |
|
||||||
| `Dockerfile` | Multi-stage: cached dep layer via stub sources + `touch *.rs` mtime bump (cargo's freshness is mtime-based and `cargo clean -p` removes 0 files — the touch is what forces the real sources to rebuild while deps stay cached), static ffmpeg from ffmpeg.martin-riedl.de (`FFMPEG_URL` arg, optional `FFMPEG_SHA256` checksum, `unzip -t` integrity check), `debian:bookworm-slim` runtime, entrypoint. Runtime ships **no libssl/libcrypto/CA bundle** — rustls webpki-roots handles all TLS, and the static ffmpeg only processes local files (downloads go through reqwest) |
|
| `Dockerfile` | Multi-stage: cached dep layer via stub sources + `touch *.rs` mtime bump (cargo's freshness is mtime-based and `cargo clean -p` removes 0 files — the touch is what forces the real sources to rebuild while deps stay cached), static ffmpeg from ffmpeg.martin-riedl.de (`FFMPEG_URL` arg, optional `FFMPEG_SHA256` checksum, `unzip -t` integrity check), `debian:bookworm-slim` runtime, entrypoint. Runtime ships **no libssl/libcrypto/CA bundle** — rustls webpki-roots handles all TLS, and the static ffmpeg only processes local files (downloads go through reqwest) |
|
||||||
| `docker-entrypoint.sh` | Privilege drop: `useradd` with `LOCAL_USER_ID` (default 9001) + `setpriv` (no gosu on bookworm-slim) |
|
| `docker-entrypoint.sh` | Privilege drop: `useradd` with `LOCAL_USER_ID` (default 9001) + `setpriv` (no gosu on bookworm-slim) |
|
||||||
@@ -99,9 +105,9 @@ Docker: `docker build -t tgxmb .` then `docker run --rm -d --name tgxmb --env-fi
|
|||||||
|
|
||||||
## Testing & QA
|
## Testing & QA
|
||||||
|
|
||||||
- **~180 tests, all inline `#[cfg(test)] mod tests`** — no `tests/` integration directories. Framework: built-in Rust test + `#[tokio::test]` (dev-deps only in `x-media`: tokio macros/rt-multi-thread, dotenv).
|
- **~200 tests, all inline `#[cfg(test)] mod tests`** — no `tests/` integration directories. Framework: built-in Rust test + `#[tokio::test]` (dev-deps only in `x-media`: tokio macros/rt-multi-thread, dotenv).
|
||||||
- No mocking framework anywhere (no mockito/wiremock/mockall). Conventions: pure-function units (regex parsing, serde round-trips, chunking, retry math) tested synchronously; async tests use real dependencies — file-backed SQLite via `tempfile` (`queue.rs::new_queue()` helper), live network fetches.
|
- No mocking framework anywhere (no mockito/wiremock/mockall). Conventions: pure-function units (regex parsing, serde round-trips, chunking, retry math) tested synchronously; async tests use real dependencies — file-backed SQLite via `tempfile` (`queue.rs::new_queue()` helper), live network fetches.
|
||||||
- Live-network tests exist in `site/twitter/interface.rs` (5), `site/bsky/interface.rs` (2), `site/misskey/interface.rs` (1), `site/bilibili/interface.rs` (4), `site/pixiv/api.rs` (1); `photo.rs` adds one `#[ignore = "heavy: …"]` test. `site/mod.rs` also has a **token-gated but not `#[ignore]`d** pixiv download test (`download_media_pixiv_original_with_referer`): it hits `i.pximg.net` whenever `PIXIV_REFRESH_TOKEN` is set, so a local `cargo test --workspace` is not fully offline and can flake on a pixiv CDN body timeout. Test gating convention (enforced by `.github/workflows/ci.yml`): pure unit tests always run; live-network tests carry `#[ignore = "live network: ..."]` (run via `cargo test --workspace -- --ignored live`); token-gated pixiv tests early-return when `PIXIV_REFRESH_TOKEN` is absent **or empty** (an unset GitHub secret arrives as `""` — `is_err()` alone would run them tokenless and fail), and the bilibili live tests early-return when the API answers risk control (`-352`, which bilibili applies per IP by request volume). Run the full offline suite with `cargo test --workspace`.
|
- Live-network tests exist in `site/twitter/interface.rs` (5), `site/bsky/interface.rs` (2), `site/misskey/interface.rs` (1), `site/bilibili/interface.rs` (4), `site/pixiv/api.rs` (1); `photo.rs` adds one `#[ignore = "heavy: …"]` test. `site/mod.rs` also has a **token-gated but not `#[ignore]`d** pixiv download test (`download_media_pixiv_original_with_referer`): it hits `i.pximg.net` whenever `PIXIV_REFRESH_TOKEN` is set, so a local `cargo test --workspace` is not fully offline and can flake on a pixiv CDN body timeout. `disabled_site_is_reported_not_ignored` (same file) is gated the other way round: it asserts `fetch` answers `FetchError::Disabled { site: "pixiv" }` for a pixiv link and early-returns when `PIXIV_REFRESH_TOKEN` **is** set (the site is then enabled). Test gating convention (enforced by `.github/workflows/ci.yml`): pure unit tests always run; live-network tests carry `#[ignore = "live network: ..."]` (run via `cargo test --workspace -- --ignored live`); token-gated pixiv tests early-return when `PIXIV_REFRESH_TOKEN` is absent **or empty** (an unset GitHub secret arrives as `""` — `is_err()` alone would run them tokenless and fail), and the bilibili live tests early-return when the API answers risk control (`-352`, which bilibili applies per IP by request volume). Run the full offline suite with `cargo test --workspace`.
|
||||||
- Fixtures are inline `serde_json::json!` builder fns (`fixture()`, `thread_json()`, `illust_json()`), not files. The shared `CLIENT` sets `pool_max_idle_per_host(0)` under `#[cfg(test)]` to avoid cross-runtime `DispatchGone`.
|
- Fixtures are inline `serde_json::json!` builder fns (`fixture()`, `thread_json()`, `illust_json()`), not files. The shared `CLIENT` sets `pool_max_idle_per_host(0)` under `#[cfg(test)]` to avoid cross-runtime `DispatchGone`.
|
||||||
- **CI** — `.github/workflows/ci.yml` (actions pinned to commit SHAs, `--locked` on every cargo invocation, `concurrency` cancels superseded runs, `RUST_BACKTRACE=1`) runs `cargo fmt --check` + `cargo clippy --workspace --all-targets --locked -- -D warnings` + `cargo test --workspace --locked` + a release-profile `cargo build --release --locked` + an `actions-rust-lang/audit` dependency-vulnerability gate (offline, no secrets, on every push/PR) and a `live` job (schedule/manual/tag only, `-p x-media` since every network/secret-gated test lives there, `continue-on-error`) for the `#[ignore]`d live + token tests. `.github/workflows/docker.yml` builds and pushes the image on master/tag and runs a **build-only check on pull requests touching the build inputs** (`Dockerfile`, entrypoint, manifests, `.dockerignore`); a release tag must match both crate versions or the build stops, and `FFMPEG_URL`/`FFMPEG_SHA256` are taken from repository variables when set (a release can pin an exact ffmpeg build). `.github/dependabot.yml` keeps crates, the pinned actions and the Docker base images current.
|
- **CI** — `.github/workflows/ci.yml` (actions pinned to commit SHAs, `--locked` on every cargo invocation, `concurrency` cancels superseded runs, `RUST_BACKTRACE=1`) runs `cargo fmt --check` + `cargo clippy --workspace --all-targets --locked -- -D warnings` + `cargo test --workspace --locked` + a release-profile `cargo build --release --locked` + an `actions-rust-lang/audit` dependency-vulnerability gate (offline, no secrets, on every push/PR) and a `live` job (schedule/manual/tag only, `-p x-media` since every network/secret-gated test lives there, `continue-on-error`) for the `#[ignore]`d live + token tests. `.github/workflows/docker.yml` builds and pushes the image on master/tag and runs a **build-only check on pull requests touching the build inputs** (`Dockerfile`, entrypoint, manifests, `.dockerignore`); a release tag must match both crate versions or the build stops, and `FFMPEG_URL`/`FFMPEG_SHA256` are taken from repository variables when set (a release can pin an exact ffmpeg build). `.github/dependabot.yml` keeps crates, the pinned actions and the Docker base images current.
|
||||||
- Untested and hard to test without a mock seam: `main.rs`, `config.rs`, `db.rs`, `handlers/statics.rs`, `media_sender.rs` (holds the `MockSender` itself); in `x-media`: `media.rs`, `lib.rs`, all `model.rs`. The `commands.rs` *executor* needs a real `Bot` (only its pure report builder is tested). Everything else — `handlers/{mod,callback,inline,urls}.rs`, `send/*`, `ctx.rs`, `state.rs`, `queue.rs`, `link_cache.rs`, `rate_limit.rs` — is driven through `TestStores`/`ctx::test_support` and the scripted `MockSender`.
|
- Untested and hard to test without a mock seam: `main.rs`, `config.rs`, `db.rs`, `handlers/statics.rs`, `media_sender.rs` (holds the `MockSender` itself); in `x-media`: `media.rs`, `lib.rs`, all `model.rs`. The `commands.rs` *executor* needs a real `Bot` (only its pure report builder is tested). Everything else — `handlers/{mod,callback,inline,urls}.rs`, `send/*`, `ctx.rs`, `state.rs`, `queue.rs`, `link_cache.rs`, `rate_limit.rs` — is driven through `TestStores`/`ctx::test_support` and the scripted `MockSender`.
|
||||||
|
|||||||
Generated
+2
-2
@@ -2818,7 +2818,7 @@ checksum = "3ad82d2a33cdc9674dc7465672f271e096168fcdbe0f799d9e6db8c5892679dc"
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "x-media"
|
name = "x-media"
|
||||||
version = "1.7.0"
|
version = "1.8.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"bytes",
|
"bytes",
|
||||||
"dotenv",
|
"dotenv",
|
||||||
@@ -2838,7 +2838,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "xmedia-bot"
|
name = "xmedia-bot"
|
||||||
version = "1.7.0"
|
version = "1.8.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"bytes",
|
"bytes",
|
||||||
"dotenv",
|
"dotenv",
|
||||||
|
|||||||
+16
-11
@@ -4,12 +4,15 @@ A Telegram bot that turns post links from X / Twitter, Pixiv, Bluesky, Misskey (
|
|||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
- Sending a link in a private chat fetches and sends the images, videos and GIFs automatically; oversized media is split into batches
|
- Sending a link in a private chat fetches and sends the images, videos and GIFs automatically; oversized media is split into batches (10 items per group)
|
||||||
- Text-only posts report "no media"; unsupported links are silently ignored
|
- Text-only posts report "no media"; unsupported links are silently ignored. Fetch failures name the reason (post gone / content withheld / source risk control / site not enabled)
|
||||||
- Long posts (text ≥ `CAPTION_QUOTE_TEXT_CHARS`, default 200) show **the text part** of their caption inside a collapsible blockquote, with the link and author line left outside it
|
- Long posts (text ≥ `CAPTION_QUOTE_TEXT_CHARS`, default 200) show **the text part** of their caption inside a collapsible blockquote, with the link and author line left outside it
|
||||||
- Inline queries (`@bot <link>`)
|
- Inline queries (`@bot <link>`) — except Pixiv images and locally transcoded animations, which Telegram cannot fetch (no Referer) and would show broken, so they are skipped; a supported link posted in a group gets a one-line hint to use the private chat or inline mode (channels stay silent)
|
||||||
- Bind a forward channel for automatic forwarding; edit the caption before forwarding and apply custom templates
|
- `/start` explains the supported sites and how to use it; `/help` lists the commands plus argument syntax, the caption placeholders and the private-chat rule; the bot's profile description texts are set at startup
|
||||||
- Failed sends are retried automatically with persistence; the user is notified after retries are exhausted
|
- `/settings` shows this chat's configuration (forward channel, edit-before-forward, per-site caption formats, saved templates); templates are added with `/set_template` and removed with `/remove_template`
|
||||||
|
- Bind a forward channel for automatic forwarding; edit the caption before forwarding and apply custom templates (the prompt carries Confirm / Skip buttons, states its expiry, and is marked expired in place once it lapses)
|
||||||
|
- Failed sends are retried automatically with persistence; the notice names which link failed, how long the retry waits, or the final cause
|
||||||
|
- The chat action stays on screen for the whole fetch, so long jobs (ugoira transcode, large uploads) do not look stalled
|
||||||
- Pixiv ugoira animations are transcoded to MP4; Bluesky videos are remuxed (HLS stream → MP4)
|
- Pixiv ugoira animations are transcoded to MP4; Bluesky videos are remuxed (HLS stream → MP4)
|
||||||
- Photos exceeding Telegram's size/dimension limits are compressed automatically (original format kept, JPEG fallback only when needed)
|
- Photos exceeding Telegram's size/dimension limits are compressed automatically (original format kept, JPEG fallback only when needed)
|
||||||
- Link-result cache: after a successful send the Telegram file ids and caption fields are cached locally, so a repeated link is re-sent from local state — no source-site request, no media file stored (expiry controlled by `LINK_CACHE_TTL_SECONDS`, default 7 days)
|
- Link-result cache: after a successful send the Telegram file ids and caption fields are cached locally, so a repeated link is re-sent from local state — no source-site request, no media file stored (expiry controlled by `LINK_CACHE_TTL_SECONDS`, default 7 days)
|
||||||
@@ -33,7 +36,7 @@ docker run --rm -d --name tgxmb --env-file .env -v ./data:/app/data tgxmb
|
|||||||
|
|
||||||
Environment variables: `TELOXIDE_TOKEN` (required), `PIXIV_REFRESH_TOKEN`, `BOT_ADMIN`, `EDIT_MESSAGE_TTL_SECONDS`, `LINK_CACHE_TTL_SECONDS`, `RUST_LOG`, `TELOXIDE_PROXY`, `WEBHOOK*`, `TWITTER_AUTH_TOKEN` (optional), `BILIBILI_COOKIE` (optional).
|
Environment variables: `TELOXIDE_TOKEN` (required), `PIXIV_REFRESH_TOKEN`, `BOT_ADMIN`, `EDIT_MESSAGE_TTL_SECONDS`, `LINK_CACHE_TTL_SECONDS`, `RUST_LOG`, `TELOXIDE_PROXY`, `WEBHOOK*`, `TWITTER_AUTH_TOKEN` (optional), `BILIBILI_COOKIE` (optional).
|
||||||
|
|
||||||
NSFW tweets: the public syndication endpoint does not return sensitive content. Setting `TWITTER_AUTH_TOKEN` (the `auth_token` cookie value of a logged-in x.com session) lets the bot fetch NSFW media in the logged-in state only when it hits a withheld tweet; without it, the bot reports no media.
|
NSFW tweets: the public syndication endpoint does not return sensitive content. Setting `TWITTER_AUTH_TOKEN` (the `auth_token` cookie value of a logged-in x.com session) lets the bot fetch NSFW media in the logged-in state only when it hits a withheld tweet; without it the bot answers that the post's media is withheld and needs `TWITTER_AUTH_TOKEN`.
|
||||||
|
|
||||||
Bilibili dynamics are fetched anonymously by default (no login; the bot fetches bilibili's anonymous `buvid3`/`buvid4` device cookies itself to raise the success rate). If the server's egress IP gets hard-flagged by bilibili (persistent `risk control (-352)` log lines or HTTP 412), set `BILIBILI_COOKIE` (the whole cookie string from a logged-in browser, e.g. `SESSDATA=…; bili_jct=…`) to restore access. Only a dynamic's images and animations are sent; an attached video degrades to its cover image.
|
Bilibili dynamics are fetched anonymously by default (no login; the bot fetches bilibili's anonymous `buvid3`/`buvid4` device cookies itself to raise the success rate). If the server's egress IP gets hard-flagged by bilibili (persistent `risk control (-352)` log lines or HTTP 412), set `BILIBILI_COOKIE` (the whole cookie string from a logged-in browser, e.g. `SESSDATA=…; bili_jct=…`) to restore access. Only a dynamic's images and animations are sent; an attached video degrades to its cover image.
|
||||||
|
|
||||||
@@ -83,10 +86,10 @@ Telegram only accepts ports 443/80/88/8443.
|
|||||||
| Variable | Description |
|
| Variable | Description |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `TELOXIDE_TOKEN` | Bot token (required) |
|
| `TELOXIDE_TOKEN` | Bot token (required) |
|
||||||
| `PIXIV_REFRESH_TOKEN` | Pixiv refresh token; Pixiv is disabled without it |
|
| `PIXIV_REFRESH_TOKEN` | Pixiv refresh token; Pixiv is disabled without it (a pixiv link then gets an explicit "site not enabled" reply instead of silence) |
|
||||||
| `BILIBILI_COOKIE` | Optional bilibili cookie string (`SESSDATA=…; bili_jct=…`); only needed when the egress IP stays risk-controlled (device cookies are fetched automatically) |
|
| `BILIBILI_COOKIE` | Optional bilibili cookie string (`SESSDATA=…; bili_jct=…`); only needed when the egress IP stays risk-controlled (device cookies are fetched automatically) |
|
||||||
| `BOT_ADMIN` | Admin chat IDs, comma-separated; receives start/stop notifications |
|
| `BOT_ADMIN` | Admin chat IDs, comma-separated; receives start/stop notifications |
|
||||||
| `EDIT_MESSAGE_TTL_SECONDS` | Edit-before-forward record expiry in seconds, default 86400 |
|
| `EDIT_MESSAGE_TTL_SECONDS` | Edit-before-forward record expiry in seconds, default 86400; once lapsed the prompt is rewritten in place to "expired — nothing was forwarded" (no extra message) |
|
||||||
| `LINK_CACHE_TTL_SECONDS` | Link-result cache expiry in seconds, default 604800 (7 days) |
|
| `LINK_CACHE_TTL_SECONDS` | Link-result cache expiry in seconds, default 604800 (7 days) |
|
||||||
| `CAPTION_QUOTE_TEXT_CHARS` | **The text part** of the caption (the joined `{title}` + `{content}`) is wrapped in a collapsible blockquote once it reaches this many characters, default 200; `0` disables |
|
| `CAPTION_QUOTE_TEXT_CHARS` | **The text part** of the caption (the joined `{title}` + `{content}`) is wrapped in a collapsible blockquote once it reaches this many characters, default 200; `0` disables |
|
||||||
| `DATA_DIR` | Data directory (where the SQLite `task_queue.db` lives), default `data` (relative to the working directory, created automatically) |
|
| `DATA_DIR` | Data directory (where the SQLite `task_queue.db` lives), default `data` (relative to the working directory, created automatically) |
|
||||||
@@ -114,15 +117,17 @@ Telegram only accepts ports 443/80/88/8443.
|
|||||||
| `/help` | List all commands and usage (this command table) |
|
| `/help` | List all commands and usage (this command table) |
|
||||||
| `/set_forward_channel <channel>` | Set the forward channel: `@channel` or channel ID; media messages are forwarded to it automatically afterwards |
|
| `/set_forward_channel <channel>` | Set the forward channel: `@channel` or channel ID; media messages are forwarded to it automatically afterwards |
|
||||||
| `/remove_forward_channel` | Remove the forward channel |
|
| `/remove_forward_channel` | Remove the forward channel |
|
||||||
| `/edit_before_forward` | Toggle "edit before forward": when enabled, the bot posts a prompt after forwarding; replying to it edits the first forwarded message's caption (or taps a template button to apply one) |
|
| `/edit_before_forward` | Toggle "edit before forward": when enabled, the bot posts a prompt after forwarding; replying to it edits the first forwarded message's caption (or tapping a template button applies one), then `↩️ Confirm` forwards and `🛑 Skip` drops this forward; the prompt states its expiry and is marked expired in place when it lapses (nothing is forwarded) |
|
||||||
| `/set_template <name>` | Reply to a message containing `[]` to save it as a named template; `[]` is replaced by the original post link when forwarding (used with "edit before forward") |
|
| `/set_template <name>` | Reply to a message containing `[]` to save it as a named template; `[]` is replaced by the original post link when forwarding (used with "edit before forward") |
|
||||||
| `/set_format <site> <format>` | Customize the caption format for one site. Sites: `twitter` / `bsky` / `pixiv` / `misskey` / `bilibili`. Placeholders: `{url}` `{author}` `{author_url}` `{title}` `{content}` `{tags}` |
|
| `/remove_template <name>` | Remove a template (names are listed by `/settings`; the prompt's keyboard shows at most 60) |
|
||||||
|
| `/settings` | Show this chat's configuration: forward channel, edit-before-forward, per-site caption formats, saved templates |
|
||||||
|
| `/set_format <site> <format>` | Customize the caption format for one site. Sites: `twitter` / `bsky` / `pixiv` / `misskey` / `bilibili`. Placeholders: `{url}` `{author}` `{author_url}` `{title}` `{content}` `{tags}`; unknown placeholders are rejected with the list of valid ones, and `-` restores the site's built-in format (preview with `/debug <link>`) |
|
||||||
| `/clear_cache [link]` | Clear the link cache (admin only); with a link only that entry, otherwise everything |
|
| `/clear_cache [link]` | Clear the link cache (admin only); with a link only that entry, otherwise everything |
|
||||||
| `/bot_dict` | Show the current chat state (debugging; admin only) |
|
| `/bot_dict` | Show the current chat state (debugging; admin only) |
|
||||||
| `/test <link>` | Parse a link and send its media; no channel forward, no edit-before-forward prompt (send only) |
|
| `/test <link>` | Parse a link and send its media; no channel forward, no edit-before-forward prompt (send only) |
|
||||||
| `/debug <link>` | Debug: parse a link and report the parse result only (site, title, author, tags, media list) — no media is sent |
|
| `/debug <link>` | Debug: parse a link and report the parse result only (site, title, author, tags, media list) — no media is sent |
|
||||||
|
|
||||||
Link processing works only in private chats; commands work in any chat.
|
Link processing works only in private chats; commands work in any chat. A supported link posted in a group gets a one-line hint to use the private chat or inline mode; channels stay silent.
|
||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
|
|
||||||
|
|||||||
@@ -4,12 +4,15 @@ Telegram 机器人,将 X / Twitter、Pixiv、Bluesky、Misskey (misskey.io)、
|
|||||||
|
|
||||||
## 功能
|
## 功能
|
||||||
|
|
||||||
- 私聊发送链接后自动抓取并发送图片、视频与 GIF,超量图片自动分批
|
- 私聊发送链接后自动抓取并发送图片、视频与 GIF,超量图片自动分批(每批 10 张)
|
||||||
- 纯文字帖提示无媒体;不支持的链接静默忽略
|
- 纯文字帖提示无媒体;不支持的链接静默忽略。抓取失败会按原因分别提示(帖子已删除 / 内容受限 / 源站风控 / 站点未启用)
|
||||||
- 长帖(正文 ≥ `CAPTION_QUOTE_TEXT_CHARS`,默认 200)的**正文部分**用可折叠引用块展示,链接与作者行留在引用块外
|
- 长帖(正文 ≥ `CAPTION_QUOTE_TEXT_CHARS`,默认 200)的**正文部分**用可折叠引用块展示,链接与作者行留在引用块外
|
||||||
- 支持内联查询(`@机器人 <链接>`)
|
- 支持内联查询(`@机器人 <链接>`;Pixiv 图片与本地转码的动图不支持内联 —— Telegram 取图时无法携带 Referer,会显示破图,因此跳过);在群聊里发链接会提示改用私聊或内联查询(频道内保持静默)
|
||||||
- 可绑定转发频道自动转发;支持转发前编辑 caption 与自定义模板
|
- `/start` 说明支持的站点与用法,`/help` 列出命令、参数格式、caption 占位符与私聊限制;bot 资料页(description / short description)启动时一并设置
|
||||||
- 发送失败自动重试并持久化,重试耗尽后通知用户
|
- `/settings` 查看本聊天配置(转发频道、转发前编辑开关、各站点 caption 格式、模板列表);模板可用 `/set_template` 增、`/remove_template` 删
|
||||||
|
- 可绑定转发频道自动转发;支持转发前编辑 caption 与自定义模板(提示消息带 Confirm / Skip 按钮并写明过期时间,过期后就地标记为已过期)
|
||||||
|
- 发送失败自动重试并持久化,重试耗尽后通知用户;提示会写明是哪条链接、重试等待多久、或最终失败的原因
|
||||||
|
- 抓取期间持续显示"正在输入 / 正在发送"状态,长任务(ugoira 转码、大图上传)不会看起来卡死
|
||||||
- Pixiv ugoira 动图自动转码为 MP4;Bluesky 视频自动转码(HLS 流 → MP4)
|
- Pixiv ugoira 动图自动转码为 MP4;Bluesky 视频自动转码(HLS 流 → MP4)
|
||||||
- 超过 Telegram 尺寸/大小限制的图片自动压缩(保持原格式,必要时转 JPEG)
|
- 超过 Telegram 尺寸/大小限制的图片自动压缩(保持原格式,必要时转 JPEG)
|
||||||
- 链接结果本地缓存:成功发送后缓存 Telegram file id 与 caption 等,再次收到相同链接直接本地重发,不再请求源站、不保存媒体文件(`LINK_CACHE_TTL_SECONDS` 控制过期,默认 7 天)
|
- 链接结果本地缓存:成功发送后缓存 Telegram file id 与 caption 等,再次收到相同链接直接本地重发,不再请求源站、不保存媒体文件(`LINK_CACHE_TTL_SECONDS` 控制过期,默认 7 天)
|
||||||
@@ -33,7 +36,7 @@ docker run --rm -d --name tgxmb --env-file .env -v ./data:/app/data tgxmb
|
|||||||
|
|
||||||
环境变量:`TELOXIDE_TOKEN`(必填)、`PIXIV_REFRESH_TOKEN`、`BOT_ADMIN`、`EDIT_MESSAGE_TTL_SECONDS`、`LINK_CACHE_TTL_SECONDS`、`RUST_LOG`、`TELOXIDE_PROXY`、`WEBHOOK*`、`TWITTER_AUTH_TOKEN`(可选)、`BILIBILI_COOKIE`(可选)。
|
环境变量:`TELOXIDE_TOKEN`(必填)、`PIXIV_REFRESH_TOKEN`、`BOT_ADMIN`、`EDIT_MESSAGE_TTL_SECONDS`、`LINK_CACHE_TTL_SECONDS`、`RUST_LOG`、`TELOXIDE_PROXY`、`WEBHOOK*`、`TWITTER_AUTH_TOKEN`(可选)、`BILIBILI_COOKIE`(可选)。
|
||||||
|
|
||||||
NSFW 推文:公开的 syndication 接口不返回敏感内容。设置 `TWITTER_AUTH_TOKEN`(登录 x.com 后浏览器 Cookie 里的 `auth_token` 值)后,bot 会仅在遇到 NSFW 推文时以登录态获取媒体;未设置则提示无媒体。
|
NSFW 推文:公开的 syndication 接口不返回敏感内容。设置 `TWITTER_AUTH_TOKEN`(登录 x.com 后浏览器 Cookie 里的 `auth_token` 值)后,bot 会仅在遇到 NSFW 推文时以登录态获取媒体;未设置则回复该推文内容受限(需要配置 `TWITTER_AUTH_TOKEN`)。
|
||||||
|
|
||||||
Bilibili 动态默认匿名抓取(无需登录,bot 会自动从 B 站的匿名指纹接口取 `buvid3`/`buvid4` 设备 cookie 以提高成功率)。若服务器出口 IP 被 B 站重度风控(日志里的 `risk control (-352)` 或 HTTP 412,且持续出现),设置 `BILIBILI_COOKIE`(登录后浏览器里整条 Cookie 串,如 `SESSDATA=…; bili_jct=…`)可恢复访问。当前只发送动态里的图片与动图,动态内嵌视频发送其封面。
|
Bilibili 动态默认匿名抓取(无需登录,bot 会自动从 B 站的匿名指纹接口取 `buvid3`/`buvid4` 设备 cookie 以提高成功率)。若服务器出口 IP 被 B 站重度风控(日志里的 `risk control (-352)` 或 HTTP 412,且持续出现),设置 `BILIBILI_COOKIE`(登录后浏览器里整条 Cookie 串,如 `SESSDATA=…; bili_jct=…`)可恢复访问。当前只发送动态里的图片与动图,动态内嵌视频发送其封面。
|
||||||
|
|
||||||
@@ -83,10 +86,10 @@ Telegram 只接受 443/80/88/8443 端口。
|
|||||||
| 变量 | 说明 |
|
| 变量 | 说明 |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `TELOXIDE_TOKEN` | Bot token(必填) |
|
| `TELOXIDE_TOKEN` | Bot token(必填) |
|
||||||
| `PIXIV_REFRESH_TOKEN` | Pixiv 刷新令牌;未设置则禁用 Pixiv |
|
| `PIXIV_REFRESH_TOKEN` | Pixiv 刷新令牌;未设置则禁用 Pixiv(此时收到 pixiv 链接会明确回复「站点未启用」,不会静默忽略) |
|
||||||
| `BILIBILI_COOKIE` | 可选的 B 站 Cookie 串(`SESSDATA=…; bili_jct=…`),仅在出口 IP 被持续风控时才需要(设备 cookie 由 bot 自动获取) |
|
| `BILIBILI_COOKIE` | 可选的 B 站 Cookie 串(`SESSDATA=…; bili_jct=…`),仅在出口 IP 被持续风控时才需要(设备 cookie 由 bot 自动获取) |
|
||||||
| `BOT_ADMIN` | 管理员聊天 ID,逗号分隔;接收启动/停止通知 |
|
| `BOT_ADMIN` | 管理员聊天 ID,逗号分隔;接收启动/停止通知 |
|
||||||
| `EDIT_MESSAGE_TTL_SECONDS` | 转发前编辑记录过期秒数,默认 86400 |
|
| `EDIT_MESSAGE_TTL_SECONDS` | 转发前编辑记录过期秒数,默认 86400;过期后提示消息会被就地改写为「已过期,未转发」(不额外发消息打扰) |
|
||||||
| `LINK_CACHE_TTL_SECONDS` | 链接结果缓存过期秒数,默认 604800(7 天) |
|
| `LINK_CACHE_TTL_SECONDS` | 链接结果缓存过期秒数,默认 604800(7 天) |
|
||||||
| `CAPTION_QUOTE_TEXT_CHARS` | 正文(`{title}` + `{content}` 合计)达到该长度(字符)时,caption 的**正文部分**用可折叠引用块包裹,默认 200;`0` 关闭 |
|
| `CAPTION_QUOTE_TEXT_CHARS` | 正文(`{title}` + `{content}` 合计)达到该长度(字符)时,caption 的**正文部分**用可折叠引用块包裹,默认 200;`0` 关闭 |
|
||||||
| `DATA_DIR` | 数据目录(SQLite 数据库 `task_queue.db` 所在目录),默认 `data`(相对工作目录,会自动创建) |
|
| `DATA_DIR` | 数据目录(SQLite 数据库 `task_queue.db` 所在目录),默认 `data`(相对工作目录,会自动创建) |
|
||||||
@@ -114,15 +117,17 @@ Telegram 只接受 443/80/88/8443 端口。
|
|||||||
| `/help` | 查看全部命令及用法(即本文档的命令表) |
|
| `/help` | 查看全部命令及用法(即本文档的命令表) |
|
||||||
| `/set_forward_channel <频道>` | 设置转发频道,参数为 `@频道名` 或频道 ID;设置后发送的媒体消息会自动转发到该频道 |
|
| `/set_forward_channel <频道>` | 设置转发频道,参数为 `@频道名` 或频道 ID;设置后发送的媒体消息会自动转发到该频道 |
|
||||||
| `/remove_forward_channel` | 取消转发频道 |
|
| `/remove_forward_channel` | 取消转发频道 |
|
||||||
| `/edit_before_forward` | 开关「转发前编辑」:开启后,转发成功后 bot 会发一条提示消息,回复它可修改第一条转发消息的 caption(或点击模板按钮套用模板) |
|
| `/edit_before_forward` | 开关「转发前编辑」:开启后,转发成功后 bot 会发一条提示消息,回复它可修改第一条转发消息的 caption(或点击模板按钮套用模板),再点 `↩️ Confirm` 才会真正转发,`🛑 Skip` 放弃本次转发;提示消息写明过期时间,过期后原地标记为已过期且不会转发 |
|
||||||
| `/set_template <名称>` | 回复一条含 `[]` 的消息,将其保存为命名模板;转发时 `[]` 会被替换为原帖链接(配合「转发前编辑」使用) |
|
| `/set_template <名称>` | 回复一条含 `[]` 的消息,将其保存为命名模板;转发时 `[]` 会被替换为原帖链接(配合「转发前编辑」使用) |
|
||||||
| `/set_format <站点> <格式>` | 自定义某站点的 caption 格式。站点:`twitter` / `bsky` / `pixiv` / `misskey` / `bilibili`。占位符:`{url}` `{author}` `{author_url}` `{title}` `{content}` `{tags}` |
|
| `/remove_template <名称>` | 删除某个模板(名称见 `/settings`;提示消息的模板按钮最多显示 60 个) |
|
||||||
|
| `/settings` | 查看本聊天配置:转发频道、转发前编辑开关、各站点 caption 格式、模板列表 |
|
||||||
|
| `/set_format <站点> <格式>` | 自定义某站点的 caption 格式。站点:`twitter` / `bsky` / `pixiv` / `misskey` / `bilibili`。占位符:`{url}` `{author}` `{author_url}` `{title}` `{content}` `{tags}`;未识别的占位符会被拒绝并列出可用项,格式填 `-` 恢复站点默认格式(可用 `/debug <链接>` 预览效果) |
|
||||||
| `/clear_cache [链接]` | 清空链接缓存(仅管理员);带链接只清该条,否则清空全部 |
|
| `/clear_cache [链接]` | 清空链接缓存(仅管理员);带链接只清该条,否则清空全部 |
|
||||||
| `/bot_dict` | 查看当前聊天状态(调试用;仅管理员) |
|
| `/bot_dict` | 查看当前聊天状态(调试用;仅管理员) |
|
||||||
| `/test <链接>` | 解析链接并发送媒体;不转发到频道、不弹转发前编辑提示(仅发送) |
|
| `/test <链接>` | 解析链接并发送媒体;不转发到频道、不弹转发前编辑提示(仅发送) |
|
||||||
| `/debug <链接>` | 调试:只解析链接并返回解析结果(站点、标题、作者、标签、媒体列表),不发送任何媒体 |
|
| `/debug <链接>` | 调试:只解析链接并返回解析结果(站点、标题、作者、标签、媒体列表),不发送任何媒体 |
|
||||||
|
|
||||||
链接处理仅限私聊;命令在任意聊天可用。
|
链接处理仅限私聊;命令在任意聊天可用。在群聊里发受支持的链接会回复一条提示(改用私聊或内联查询),频道内保持静默。
|
||||||
|
|
||||||
## 备注
|
## 备注
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
[package]
|
[package]
|
||||||
name = "x-media"
|
name = "x-media"
|
||||||
version = "1.7.0"
|
version = "1.8.0"
|
||||||
edition = "2024"
|
edition = "2024"
|
||||||
|
|
||||||
[dependencies]
|
[dependencies]
|
||||||
|
|||||||
@@ -247,6 +247,12 @@ pub enum FetchError {
|
|||||||
NotFound,
|
NotFound,
|
||||||
#[error("blocked")]
|
#[error("blocked")]
|
||||||
Blocked,
|
Blocked,
|
||||||
|
/// The URL matches a registered site that is disabled right now (pixiv
|
||||||
|
/// without `PIXIV_REFRESH_TOKEN`, or after a failed login). Distinct from
|
||||||
|
/// `Ok(None)` — an unsupported link — so the bot can tell the user why
|
||||||
|
/// the link was not handled instead of silently ignoring it.
|
||||||
|
#[error("{site} support is disabled")]
|
||||||
|
Disabled { site: &'static str },
|
||||||
/// The post exists but its content is withheld (twitter NSFW /
|
/// The post exists but its content is withheld (twitter NSFW /
|
||||||
/// age-restricted tweets come back as an empty `{}` from syndication).
|
/// age-restricted tweets come back as an empty `{}` from syndication).
|
||||||
#[error("content withheld (sensitive)")]
|
#[error("content withheld (sensitive)")]
|
||||||
@@ -385,6 +391,17 @@ fn find_site(url: &str) -> Option<&'static dyn Site> {
|
|||||||
.map(|site| site.as_ref())
|
.map(|site| site.as_ref())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The site whose pattern matches `url` but which is disabled right now.
|
||||||
|
/// `None` when no site matches the URL at all, or when the matching site is
|
||||||
|
/// enabled. Lets the dispatcher tell "unsupported link" (silently ignored)
|
||||||
|
/// apart from "this bot has that site switched off" (reported to the user).
|
||||||
|
fn disabled_site(url: &str) -> Option<&'static str> {
|
||||||
|
SITES
|
||||||
|
.iter()
|
||||||
|
.find(|site| !site.enabled() && site.pattern().is_match(url))
|
||||||
|
.map(|site| site.id())
|
||||||
|
}
|
||||||
|
|
||||||
/// Every supported site id, in dispatch order. The bot's SetFormat whitelist
|
/// Every supported site id, in dispatch order. The bot's SetFormat whitelist
|
||||||
/// derives from this list.
|
/// derives from this list.
|
||||||
pub fn site_ids() -> Vec<&'static str> {
|
pub fn site_ids() -> Vec<&'static str> {
|
||||||
@@ -408,7 +425,9 @@ pub async fn validate_all() -> Vec<(&'static str, String)> {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// Fetches a post from its URL. Returns `Ok(None)` when no site pattern
|
/// Fetches a post from its URL. Returns `Ok(None)` when no site pattern
|
||||||
/// matches (unsupported links are silently ignored by the bot).
|
/// matches (unsupported links are silently ignored by the bot) and
|
||||||
|
/// [`FetchError::Disabled`] when the URL belongs to a registered site that is
|
||||||
|
/// switched off right now — the two are different answers for the user.
|
||||||
///
|
///
|
||||||
/// Transient failures are retried: 3 total attempts with 1s then 2s delays.
|
/// Transient failures are retried: 3 total attempts with 1s then 2s delays.
|
||||||
/// What counts as transient is the matched site's own policy (`is_retryable`
|
/// What counts as transient is the matched site's own policy (`is_retryable`
|
||||||
@@ -432,7 +451,13 @@ const MAX_FETCH_ATTEMPTS: u32 = 3;
|
|||||||
|
|
||||||
async fn fetch_with_attempts(url: &str, attempts: u32) -> Result<Option<Fetched>, FetchError> {
|
async fn fetch_with_attempts(url: &str, attempts: u32) -> Result<Option<Fetched>, FetchError> {
|
||||||
let Some(site) = find_site(url) else {
|
let Some(site) = find_site(url) else {
|
||||||
return Ok(None);
|
// A registered-but-disabled site (pixiv without a token) is not an
|
||||||
|
// unsupported link: report it, so the bot answers the user instead of
|
||||||
|
// ignoring the message.
|
||||||
|
return match disabled_site(url) {
|
||||||
|
Some(site) => Err(FetchError::Disabled { site }),
|
||||||
|
None => Ok(None),
|
||||||
|
};
|
||||||
};
|
};
|
||||||
for attempt in 0..attempts.max(1) {
|
for attempt in 0..attempts.max(1) {
|
||||||
match site.fetch_from_url(url).await {
|
match site.fetch_from_url(url).await {
|
||||||
@@ -458,6 +483,15 @@ async fn fetch_with_attempts(url: &str, attempts: u32) -> Result<Option<Fetched>
|
|||||||
unreachable!("retry loop always returns")
|
unreachable!("retry loop always returns")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Whether fetching `url` requires site-specific headers (pixiv's `Referer`
|
||||||
|
/// for `pximg.net` hotlink protection, see [`Site::media_headers`]). Telegram's
|
||||||
|
/// own fetch of a media URL sends none of them, so a URL that needs them fails
|
||||||
|
/// there — callers that hand a URL to Telegram (inline query results) must
|
||||||
|
/// skip such media instead of shipping a broken item.
|
||||||
|
pub fn needs_media_headers(url: &str) -> bool {
|
||||||
|
SITES.iter().any(|site| site.media_headers(url).is_some())
|
||||||
|
}
|
||||||
|
|
||||||
/// Applies every site's media-header rule to a download request (pixiv's
|
/// Applies every site's media-header rule to a download request (pixiv's
|
||||||
/// `Referer` for pximg.net hotlink protection). Sites contribute via their
|
/// `Referer` for pximg.net hotlink protection). Sites contribute via their
|
||||||
/// `media_headers(url)` — the central download code carries no per-site logic.
|
/// `media_headers(url)` — the central download code carries no per-site logic.
|
||||||
@@ -708,6 +742,49 @@ mod tests {
|
|||||||
assert!(matches!(result, Ok(None)), "got {result:?}");
|
assert!(matches!(result, Ok(None)), "got {result:?}");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn media_headers_are_reported_only_where_telegram_would_fail() {
|
||||||
|
// pixiv's CDN needs a Referer, which only the bot can send: an inline
|
||||||
|
// result pointing at it renders broken, so callers skip it.
|
||||||
|
assert!(needs_media_headers(
|
||||||
|
"https://i.pximg.net/img-original/img/2024/01/01/00/00/00/1_p0.jpg"
|
||||||
|
));
|
||||||
|
// The rest serve direct requests (verified per site in their modules).
|
||||||
|
for url in [
|
||||||
|
"https://pbs.twimg.com/media/1.jpg",
|
||||||
|
"https://cdn.bsky.app/img/1.jpg",
|
||||||
|
"https://media.misskeyusercontent.jp/io/1.webp",
|
||||||
|
"https://i0.hdslb.com/bfs/1.jpg",
|
||||||
|
] {
|
||||||
|
assert!(!needs_media_headers(url), "{url}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn disabled_site_is_reported_not_ignored() {
|
||||||
|
// pixiv is the only token-gated site; with PIXIV_REFRESH_TOKEN set it
|
||||||
|
// is enabled and this link would hit the network, so skip then.
|
||||||
|
if std::env::var("PIXIV_REFRESH_TOKEN")
|
||||||
|
.ok()
|
||||||
|
.filter(|s| !s.is_empty())
|
||||||
|
.is_some()
|
||||||
|
{
|
||||||
|
eprintln!("skipping: PIXIV_REFRESH_TOKEN is set");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let result = fetch("https://www.pixiv.net/artworks/1").await;
|
||||||
|
assert!(
|
||||||
|
matches!(result, Err(FetchError::Disabled { site: "pixiv" })),
|
||||||
|
"got {result:?}"
|
||||||
|
);
|
||||||
|
// The cache key still resolves: the bot keys the reply and the link
|
||||||
|
// cache off it even when the site is off.
|
||||||
|
assert_eq!(
|
||||||
|
cache_key("https://www.pixiv.net/artworks/1"),
|
||||||
|
Some("pixiv:1".into())
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
async fn download_media_pixiv_original_with_referer() {
|
async fn download_media_pixiv_original_with_referer() {
|
||||||
// Proves the Referer header is attached for i.pximg.net: a header-less
|
// Proves the Referer header is attached for i.pximg.net: a header-less
|
||||||
|
|||||||
@@ -43,27 +43,25 @@ pub async fn fetch_from_url(url: &str) -> Result<Fetched, FetchError> {
|
|||||||
match fetch(id).await {
|
match fetch(id).await {
|
||||||
Ok(tweet) => Ok(tweet.into()),
|
Ok(tweet) => Ok(tweet.into()),
|
||||||
// Syndication withholds NSFW/age-restricted tweets (empty `{}`).
|
// Syndication withholds NSFW/age-restricted tweets (empty `{}`).
|
||||||
// Retry as the logged-in user when TWITTER_AUTH_TOKEN is set;
|
// Retry as the logged-in user when TWITTER_AUTH_TOKEN is set; without
|
||||||
// otherwise degrade to an empty result (the bot replies
|
// the token the withholding is reported as `Sensitive`, so the bot can
|
||||||
// "No media found").
|
// answer "age-restricted / needs TWITTER_AUTH_TOKEN" instead of the
|
||||||
|
// misleading "No media found".
|
||||||
Err(FetchError::Sensitive) => {
|
Err(FetchError::Sensitive) => {
|
||||||
if super::auth::enabled() {
|
if super::auth::enabled() {
|
||||||
match super::auth::fetch(id).await {
|
match super::auth::fetch(id).await {
|
||||||
Ok(tweet) => Ok(tweet.into()),
|
Ok(tweet) => Ok(tweet.into()),
|
||||||
// The tweet is genuinely gone (deleted / suspended /
|
// Deleted/suspended (tombstoned) and unexpected fallback
|
||||||
// tombstoned): report it instead of degrading to an
|
// failures keep their own class: the bot reports what
|
||||||
// empty result ("No media found"). Only unexpected
|
// actually happened rather than "No media found".
|
||||||
// fallback failures (network, parse) keep the NSFW
|
|
||||||
// placeholder.
|
|
||||||
Err(FetchError::NotFound) => Err(FetchError::NotFound),
|
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
log::warn!("twitter auth fallback failed for {id}: {e}");
|
log::warn!("twitter auth fallback failed for {id}: {e}");
|
||||||
Ok(empty_fetched(url))
|
Err(e)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
log::debug!("tweet {id} is sensitive; set TWITTER_AUTH_TOKEN to fetch NSFW media");
|
log::debug!("tweet {id} is sensitive; set TWITTER_AUTH_TOKEN to fetch NSFW media");
|
||||||
Ok(empty_fetched(url))
|
Err(FetchError::Sensitive)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
Err(e) => Err(e),
|
Err(e) => Err(e),
|
||||||
@@ -90,24 +88,6 @@ pub fn media_headers(_url: &str) -> Option<Vec<(&'static str, String)>> {
|
|||||||
None
|
None
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A Fetched with no media for withheld tweets: the bot replies
|
|
||||||
/// "No media found" and moves on instead of erroring.
|
|
||||||
fn empty_fetched(url: &str) -> Fetched {
|
|
||||||
Fetched {
|
|
||||||
source_url: url.to_string(),
|
|
||||||
// The raw user-supplied URL goes into an HTML caption; escape it so
|
|
||||||
// crafted links cannot break the parse (Telegram 400).
|
|
||||||
caption: encode_text(url).into_owned(),
|
|
||||||
title: String::new(),
|
|
||||||
content: String::new(),
|
|
||||||
media: vec![],
|
|
||||||
sensitive: true,
|
|
||||||
site_id: "twitter",
|
|
||||||
render_data: None,
|
|
||||||
_keep_alive: None,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Fetches a tweet from the syndication endpoint. Deleted/blocked tweets
|
/// Fetches a tweet from the syndication endpoint. Deleted/blocked tweets
|
||||||
/// surface as `FetchError::NotFound`; withheld content (empty tombstone,
|
/// surface as `FetchError::NotFound`; withheld content (empty tombstone,
|
||||||
/// age-restricted) as `FetchError::Sensitive`.
|
/// age-restricted) as `FetchError::Sensitive`.
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
[package]
|
[package]
|
||||||
name = "xmedia-bot"
|
name = "xmedia-bot"
|
||||||
version = "1.7.0"
|
version = "1.8.0"
|
||||||
edition = "2024"
|
edition = "2024"
|
||||||
|
|
||||||
[dependencies]
|
[dependencies]
|
||||||
|
|||||||
@@ -14,6 +14,8 @@ use teloxide::types::{CallbackQuery, CallbackQueryId, MessageId};
|
|||||||
|
|
||||||
/// The `"forward"` button's data.
|
/// The `"forward"` button's data.
|
||||||
const FORWARD: &str = "forward";
|
const FORWARD: &str = "forward";
|
||||||
|
/// The `"skip"` button's data: drop the prompt without forwarding.
|
||||||
|
const SKIP: &str = "skip";
|
||||||
/// Prefix of a template button's data: `"template|<name>"`.
|
/// Prefix of a template button's data: `"template|<name>"`.
|
||||||
const TEMPLATE_PREFIX: &str = "template|";
|
const TEMPLATE_PREFIX: &str = "template|";
|
||||||
|
|
||||||
@@ -70,6 +72,29 @@ async fn handle_callback(
|
|||||||
}
|
}
|
||||||
|
|
||||||
log::info!("callback from {chat_id} on prompt {prompt_message_id}: {data}");
|
log::info!("callback from {chat_id} on prompt {prompt_message_id}: {data}");
|
||||||
|
if data == SKIP {
|
||||||
|
// Skip works with or without a forward channel: it is the explicit
|
||||||
|
// "do not forward this" answer, and it drops the record so the forward
|
||||||
|
// can never happen later.
|
||||||
|
log::info!("edit-before-forward prompt {prompt_message_id} skipped");
|
||||||
|
ctx.chat_store
|
||||||
|
.update(chat_id, |data| {
|
||||||
|
data.edit_message.remove(&prompt_message_id);
|
||||||
|
})
|
||||||
|
.await;
|
||||||
|
let _ = ctx
|
||||||
|
.sender
|
||||||
|
.delete_message(ChatId(chat_id), MessageId(prompt_message_id as i32))
|
||||||
|
.await;
|
||||||
|
let _ = ctx
|
||||||
|
.sender
|
||||||
|
.answer_callback_query(
|
||||||
|
callback_query_id,
|
||||||
|
Some("Skipped — nothing was forwarded.".to_string()),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
return;
|
||||||
|
}
|
||||||
if data == FORWARD {
|
if data == FORWARD {
|
||||||
match chat_data.forward_channel_id {
|
match chat_data.forward_channel_id {
|
||||||
Some(channel_id) => {
|
Some(channel_id) => {
|
||||||
@@ -244,6 +269,31 @@ mod tests {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn skip_drops_the_prompt_without_forwarding() {
|
||||||
|
// "skip" needs no forward channel and no scripted outcomes: it deletes
|
||||||
|
// the prompt and drops the record, so no forward can ever happen.
|
||||||
|
let sender = MockSender::scripted(vec![], api_error);
|
||||||
|
let stores = TestStores::new();
|
||||||
|
let ctx = stores.ctx(&sender);
|
||||||
|
seed_prompt(&ctx, crate::db::unix_now()).await;
|
||||||
|
|
||||||
|
handle_callback(&ctx, callback_id(), 1, PROMPT_ID, "skip").await;
|
||||||
|
|
||||||
|
assert_eq!(
|
||||||
|
sender.calls(),
|
||||||
|
vec!["delete_message", "answer_callback_query"]
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
sender.answers(),
|
||||||
|
vec![Some("Skipped — nothing was forwarded.".to_string())]
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
ctx.chat_store.get(1).await.edit_message.is_empty(),
|
||||||
|
"a skipped prompt must drop its record"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
async fn forward_without_a_channel_is_reported() {
|
async fn forward_without_a_channel_is_reported() {
|
||||||
let sender = MockSender::scripted(vec![], api_error);
|
let sender = MockSender::scripted(vec![], api_error);
|
||||||
|
|||||||
@@ -4,6 +4,7 @@
|
|||||||
use super::urls::{PostSend, url_media};
|
use super::urls::{PostSend, url_media};
|
||||||
use super::{CHAT_STORE, CONFIG, LINK_CACHE, log_key, reply, reply_html};
|
use super::{CHAT_STORE, CONFIG, LINK_CACHE, log_key, reply, reply_html};
|
||||||
use crate::ctx::AppContext;
|
use crate::ctx::AppContext;
|
||||||
|
use crate::state::ChatData;
|
||||||
use teloxide::RequestError;
|
use teloxide::RequestError;
|
||||||
use teloxide::prelude::*;
|
use teloxide::prelude::*;
|
||||||
use teloxide::types::{ChatId, Message, Recipient};
|
use teloxide::types::{ChatId, Message, Recipient};
|
||||||
@@ -33,13 +34,20 @@ pub(crate) enum Command {
|
|||||||
parse_with = "split"
|
parse_with = "split"
|
||||||
)]
|
)]
|
||||||
SetTemplate(String),
|
SetTemplate(String),
|
||||||
|
#[command(description = "Remove a saved template", parse_with = "split")]
|
||||||
|
RemoveTemplate(String),
|
||||||
|
#[command(description = "Show this chat's settings")]
|
||||||
|
Settings,
|
||||||
#[command(description = "Show chat state (debug; admin only)")]
|
#[command(description = "Show chat state (debug; admin only)")]
|
||||||
BotDict,
|
BotDict,
|
||||||
#[command(description = "Set site caption format", parse_with = "split")]
|
#[command(
|
||||||
|
description = "Set site caption format (- to reset)",
|
||||||
|
parse_with = parse_arg_remainder
|
||||||
|
)]
|
||||||
SetFormat(String),
|
SetFormat(String),
|
||||||
#[command(
|
#[command(
|
||||||
description = "Clear link cache (admin; optional URL, else all)",
|
description = "Clear link cache (admin; optional URL, else all)",
|
||||||
parse_with = "split"
|
parse_with = parse_arg_remainder
|
||||||
)]
|
)]
|
||||||
ClearCache(String),
|
ClearCache(String),
|
||||||
#[command(
|
#[command(
|
||||||
@@ -62,6 +70,118 @@ fn parse_arg_remainder(s: String) -> Result<(String,), ParseError> {
|
|||||||
Ok((s.trim().to_string(),))
|
Ok((s.trim().to_string(),))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Placeholders `/set_format` accepts, mirroring what
|
||||||
|
/// `x_media::site::caption_from_fields` substitutes.
|
||||||
|
const FORMAT_PLACEHOLDERS: [&str; 6] = ["url", "author", "author_url", "title", "content", "tags"];
|
||||||
|
|
||||||
|
/// `/start`'s welcome: what the bot is for, where links work, where to look
|
||||||
|
/// next. The old "Hello!" left a first-time user with nothing.
|
||||||
|
const START_TEXT: &str = "\
|
||||||
|
Send me a post link and I'll send back its images, videos and GIFs with the title, author and tags.
|
||||||
|
|
||||||
|
Supported: X/Twitter, Pixiv, Bluesky, Misskey (misskey.io), Bilibili.
|
||||||
|
In a private chat just paste the link. In a group, use inline mode (type @, pick me, then the link).
|
||||||
|
|
||||||
|
/help lists every command.";
|
||||||
|
|
||||||
|
/// Appended to `/help`'s command list: argument syntax, caption
|
||||||
|
/// placeholders and the private-chat rule — none of which teloxide's
|
||||||
|
/// `descriptions()` renders (it prints `/command — description` only).
|
||||||
|
const HELP_FOOTER: &str = "\
|
||||||
|
Arguments
|
||||||
|
/set_forward_channel <@channel or channel id>
|
||||||
|
/set_template <name> — reply to a message containing [] to save it
|
||||||
|
/remove_template <name> — see /settings for the saved names
|
||||||
|
/set_format <site> <format> — '-' restores the built-in format
|
||||||
|
/test <link> / /debug <link>
|
||||||
|
|
||||||
|
Caption placeholders (for /set_format)
|
||||||
|
{url} {author} {author_url} {title} {content} {tags}
|
||||||
|
A template's [] is replaced by the post link when forwarding.
|
||||||
|
|
||||||
|
Links are handled in private chats only; in a group use inline mode.";
|
||||||
|
|
||||||
|
/// Cap on template names echoed by `/settings`: a chat with hundreds of
|
||||||
|
/// templates must not produce a message Telegram rejects for length.
|
||||||
|
const MAX_SETTINGS_TEMPLATE_NAMES: usize = 30;
|
||||||
|
|
||||||
|
/// Sorted template names: the order `/settings`, `/remove_template` and the
|
||||||
|
/// prompt's buttons all show.
|
||||||
|
fn sorted_template_names(data: &ChatData) -> Vec<String> {
|
||||||
|
let mut names: Vec<String> = data.template.keys().cloned().collect();
|
||||||
|
names.sort();
|
||||||
|
names
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `/settings`: what this chat is configured to do, readable by anyone in it
|
||||||
|
/// (unlike `/bot_dict`, which dumps the raw state and is admin-only).
|
||||||
|
fn settings_text(data: &ChatData) -> String {
|
||||||
|
let mut lines = Vec::new();
|
||||||
|
match data.forward_channel_id {
|
||||||
|
Some(id) => lines.push(format!("Forward channel: {id}")),
|
||||||
|
None => lines.push(
|
||||||
|
"Forward channel: not set (use /set_forward_channel <@channel or id>)".to_string(),
|
||||||
|
),
|
||||||
|
}
|
||||||
|
lines.push(format!(
|
||||||
|
"Edit before forward: {}",
|
||||||
|
if data.edit_before_forward {
|
||||||
|
"on"
|
||||||
|
} else {
|
||||||
|
"off"
|
||||||
|
}
|
||||||
|
));
|
||||||
|
let mut formats: Vec<String> = data
|
||||||
|
.message_format
|
||||||
|
.iter()
|
||||||
|
.map(|(site, format)| format!("{site} => {format}"))
|
||||||
|
.collect();
|
||||||
|
formats.sort();
|
||||||
|
lines.push(if formats.is_empty() {
|
||||||
|
"Caption formats: built-in for every site".to_string()
|
||||||
|
} else {
|
||||||
|
format!("Caption formats:\n {}", formats.join("\n "))
|
||||||
|
});
|
||||||
|
let names = sorted_template_names(data);
|
||||||
|
lines.push(match names.len() {
|
||||||
|
0 => "Templates: none".to_string(),
|
||||||
|
n => format!(
|
||||||
|
"Templates ({n}): {}{}",
|
||||||
|
names
|
||||||
|
.iter()
|
||||||
|
.take(MAX_SETTINGS_TEMPLATE_NAMES)
|
||||||
|
.cloned()
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
.join(", "),
|
||||||
|
if n > MAX_SETTINGS_TEMPLATE_NAMES {
|
||||||
|
format!(", +{} more", n - MAX_SETTINGS_TEMPLATE_NAMES)
|
||||||
|
} else {
|
||||||
|
String::new()
|
||||||
|
}
|
||||||
|
),
|
||||||
|
});
|
||||||
|
lines.join("\n")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The first `{…}` token in a caption format that is not a known placeholder
|
||||||
|
/// (`None` when all of them are). The renderer replaces exact keys only, so an
|
||||||
|
/// unknown token would be published verbatim in every caption of that site —
|
||||||
|
/// caught here instead.
|
||||||
|
fn unknown_placeholder(format: &str) -> Option<&str> {
|
||||||
|
let mut rest = format;
|
||||||
|
while let Some(open) = rest.find('{') {
|
||||||
|
let after = &rest[open + 1..];
|
||||||
|
// An unclosed `{` is not a placeholder token at all.
|
||||||
|
let close = after.find('}')?;
|
||||||
|
let token = &after[..close];
|
||||||
|
if !FORMAT_PLACEHOLDERS.contains(&token) {
|
||||||
|
return Some(token);
|
||||||
|
}
|
||||||
|
rest = &after[close + 1..];
|
||||||
|
}
|
||||||
|
None
|
||||||
|
}
|
||||||
|
|
||||||
enum SetForwardChannelError {
|
enum SetForwardChannelError {
|
||||||
EmptyParameter,
|
EmptyParameter,
|
||||||
NotChannel,
|
NotChannel,
|
||||||
@@ -140,11 +260,17 @@ pub(crate) async fn execute_command(
|
|||||||
) -> Result<(), RequestError> {
|
) -> Result<(), RequestError> {
|
||||||
match command {
|
match command {
|
||||||
Command::Start => {
|
Command::Start => {
|
||||||
bot.send_message(message.chat.id, "Hello!").await?;
|
bot.send_message(message.chat.id, START_TEXT).await?;
|
||||||
}
|
}
|
||||||
Command::Help => {
|
Command::Help => {
|
||||||
bot.send_message(message.chat.id, Command::descriptions().to_string())
|
// The command list plus the parts teloxide's `descriptions()`
|
||||||
.await?;
|
// cannot show: argument syntax, caption placeholders, and where a
|
||||||
|
// link actually works.
|
||||||
|
bot.send_message(
|
||||||
|
message.chat.id,
|
||||||
|
format!("{}\n\n{}", Command::descriptions(), HELP_FOOTER),
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
}
|
}
|
||||||
Command::SetForwardChannel(channel) => {
|
Command::SetForwardChannel(channel) => {
|
||||||
let result = match set_forward_channel_handler(bot, message, channel).await {
|
let result = match set_forward_channel_handler(bot, message, channel).await {
|
||||||
@@ -232,6 +358,41 @@ pub(crate) async fn execute_command(
|
|||||||
};
|
};
|
||||||
reply(bot, message.chat.id.0, message.id, text).await?;
|
reply(bot, message.chat.id.0, message.id, text).await?;
|
||||||
}
|
}
|
||||||
|
Command::RemoveTemplate(name) => {
|
||||||
|
let chat_id = message.chat.id.0;
|
||||||
|
let name = name.trim().to_string();
|
||||||
|
if name.is_empty() {
|
||||||
|
reply(
|
||||||
|
bot,
|
||||||
|
chat_id,
|
||||||
|
message.id,
|
||||||
|
"Usage: /remove_template <name> (see /settings for the saved names)",
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
let removed = CHAT_STORE
|
||||||
|
.update(chat_id, |data| data.template.remove(&name).is_some())
|
||||||
|
.await;
|
||||||
|
let text = if removed {
|
||||||
|
format!("Template '{name}' removed.")
|
||||||
|
} else {
|
||||||
|
// Name the live templates: a typo would otherwise look like a
|
||||||
|
// successful delete.
|
||||||
|
let names = sorted_template_names(&CHAT_STORE.get(chat_id).await);
|
||||||
|
if names.is_empty() {
|
||||||
|
format!("No template named '{name}'. None are saved yet.")
|
||||||
|
} else {
|
||||||
|
format!("No template named '{name}'. Saved: {}", names.join(", "))
|
||||||
|
}
|
||||||
|
};
|
||||||
|
reply(bot, chat_id, message.id, text).await?;
|
||||||
|
}
|
||||||
|
Command::Settings => {
|
||||||
|
let chat_id = message.chat.id.0;
|
||||||
|
let data = CHAT_STORE.get(chat_id).await;
|
||||||
|
reply(bot, chat_id, message.id, settings_text(&data)).await?;
|
||||||
|
}
|
||||||
Command::BotDict => {
|
Command::BotDict => {
|
||||||
// Debug dump of the chat's persisted state: admin only (it echoes
|
// Debug dump of the chat's persisted state: admin only (it echoes
|
||||||
// forward-channel ids and templates to whoever asks).
|
// forward-channel ids and templates to whoever asks).
|
||||||
@@ -284,12 +445,56 @@ pub(crate) async fn execute_command(
|
|||||||
.await?;
|
.await?;
|
||||||
return Ok(());
|
return Ok(());
|
||||||
}
|
}
|
||||||
|
// `-` resets to the site's built-in caption: without it a chat that
|
||||||
|
// set a format once could never get back to the default (the
|
||||||
|
// built-in format string is not something a user can retype).
|
||||||
|
if format == "-" {
|
||||||
|
CHAT_STORE
|
||||||
|
.update(chat_id, |data| {
|
||||||
|
data.message_format.remove(site);
|
||||||
|
})
|
||||||
|
.await;
|
||||||
|
reply(
|
||||||
|
bot,
|
||||||
|
message.chat.id.0,
|
||||||
|
message.id,
|
||||||
|
"Format reset to the built-in one.",
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
// A typo like {titel} would otherwise be rendered literally into
|
||||||
|
// every caption of that site (the renderer only substitutes the
|
||||||
|
// exact keys), which is invisible until a post arrives.
|
||||||
|
if let Some(token) = unknown_placeholder(&format) {
|
||||||
|
reply(
|
||||||
|
bot,
|
||||||
|
message.chat.id.0,
|
||||||
|
message.id,
|
||||||
|
format!(
|
||||||
|
"Unknown placeholder {{{token}}}. Available: {}",
|
||||||
|
FORMAT_PLACEHOLDERS
|
||||||
|
.iter()
|
||||||
|
.map(|name| format!("{{{name}}}"))
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
.join(" ")
|
||||||
|
),
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
CHAT_STORE
|
CHAT_STORE
|
||||||
.update(chat_id, |data| {
|
.update(chat_id, |data| {
|
||||||
data.message_format.insert(site.to_string(), format);
|
data.message_format.insert(site.to_string(), format);
|
||||||
})
|
})
|
||||||
.await;
|
.await;
|
||||||
reply(bot, message.chat.id.0, message.id, "Format set.").await?;
|
reply(
|
||||||
|
bot,
|
||||||
|
message.chat.id.0,
|
||||||
|
message.id,
|
||||||
|
"Format set. Use /debug <link> to preview the caption.",
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
}
|
}
|
||||||
Command::ClearCache(arg) => {
|
Command::ClearCache(arg) => {
|
||||||
let sender_id = message
|
let sender_id = message
|
||||||
@@ -414,6 +619,24 @@ pub(crate) async fn execute_command(
|
|||||||
.await?;
|
.await?;
|
||||||
}
|
}
|
||||||
Ok(Some(fetched)) => {
|
Ok(Some(fetched)) => {
|
||||||
|
// The preview must show what a link would actually send:
|
||||||
|
// the chat's per-site format override plus the long-post
|
||||||
|
// quoting. Rendering the raw built-in caption here made
|
||||||
|
// `/set_format` look like it did nothing.
|
||||||
|
let format = CHAT_STORE
|
||||||
|
.get(message.chat.id.0)
|
||||||
|
.await
|
||||||
|
.message_format
|
||||||
|
.get(fetched.site_name())
|
||||||
|
.cloned()
|
||||||
|
.unwrap_or_default();
|
||||||
|
let caption = preview_caption(
|
||||||
|
&format,
|
||||||
|
&fetched.caption,
|
||||||
|
&fetched.source_url,
|
||||||
|
fetched.render_fields(),
|
||||||
|
CONFIG.caption_quote_text_chars,
|
||||||
|
);
|
||||||
let report = debug_report(
|
let report = debug_report(
|
||||||
url,
|
url,
|
||||||
fetched.site_name(),
|
fetched.site_name(),
|
||||||
@@ -422,7 +645,7 @@ pub(crate) async fn execute_command(
|
|||||||
&fetched.content,
|
&fetched.content,
|
||||||
fetched.render_fields(),
|
fetched.render_fields(),
|
||||||
fetched.sensitive,
|
fetched.sensitive,
|
||||||
&fetched.caption,
|
&caption,
|
||||||
&fetched.media,
|
&fetched.media,
|
||||||
);
|
);
|
||||||
// HTML report: the caption renders inside a <blockquote>
|
// HTML report: the caption renders inside a <blockquote>
|
||||||
@@ -440,12 +663,32 @@ fn plural(n: usize) -> &'static str {
|
|||||||
if n == 1 { "y" } else { "ies" }
|
if n == 1 { "y" } else { "ies" }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Bot profile texts (Bot API `setMyDescription` / `setMyShortDescription`):
|
||||||
|
/// shown on the bot's profile page and in the share sheet. Without them a
|
||||||
|
/// shared link says nothing about what the bot does.
|
||||||
|
const BOT_DESCRIPTION: &str = "\
|
||||||
|
Send a post link from X/Twitter, Pixiv, Bluesky, Misskey (misskey.io) or Bilibili and get its images, videos and GIFs back with the title, author and tags.
|
||||||
|
Links are handled in private chats; a group can use inline mode. /help lists every command.";
|
||||||
|
const BOT_SHORT_DESCRIPTION: &str =
|
||||||
|
"Post links (X, Pixiv, Bluesky, Misskey, Bilibili) -> media messages";
|
||||||
|
|
||||||
/// Registers the bot's command list with Telegram so clients show it in the
|
/// Registers the bot's command list with Telegram so clients show it in the
|
||||||
/// `/` menu (Bot API `setMyCommands`).
|
/// `/` menu (Bot API `setMyCommands`), plus its profile description texts.
|
||||||
pub async fn register_commands(bot: &Bot) -> Result<(), RequestError> {
|
pub async fn register_commands(bot: &Bot) -> Result<(), RequestError> {
|
||||||
let commands = Command::bot_commands();
|
let commands = Command::bot_commands();
|
||||||
bot.set_my_commands(commands.clone()).await?;
|
bot.set_my_commands(commands.clone()).await?;
|
||||||
log::info!("registered {} commands", commands.len());
|
log::info!("registered {} commands", commands.len());
|
||||||
|
// Profile texts are cosmetic: a failure (rare) must not abort startup.
|
||||||
|
if let Err(e) = bot.set_my_description().description(BOT_DESCRIPTION).await {
|
||||||
|
log::warn!("failed to set the bot description: {e}");
|
||||||
|
}
|
||||||
|
if let Err(e) = bot
|
||||||
|
.set_my_short_description()
|
||||||
|
.short_description(BOT_SHORT_DESCRIPTION)
|
||||||
|
.await
|
||||||
|
{
|
||||||
|
log::warn!("failed to set the bot short description: {e}");
|
||||||
|
}
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -457,6 +700,31 @@ const MAX_DEBUG_REPORT_CHARS: usize = 4000;
|
|||||||
/// message, so it must stay under Telegram's 4096-char limit.
|
/// message, so it must stay under Telegram's 4096-char limit.
|
||||||
const MAX_DEBUG_DUMP_CHARS: usize = 3500;
|
const MAX_DEBUG_DUMP_CHARS: usize = 3500;
|
||||||
|
|
||||||
|
/// The caption a link would actually send for this chat: the per-site format
|
||||||
|
/// override (empty = the site's built-in caption) and, on a long post, the
|
||||||
|
/// same text quoting the send paths apply. `/debug` shows this so the preview
|
||||||
|
/// cannot drift from what the send paths produce.
|
||||||
|
fn preview_caption(
|
||||||
|
format: &str,
|
||||||
|
built_in: &str,
|
||||||
|
url: &str,
|
||||||
|
fields: Option<(&str, &str, &str, &str, &str)>,
|
||||||
|
quote_chars: usize,
|
||||||
|
) -> String {
|
||||||
|
let caption = match fields {
|
||||||
|
// Same call the send paths make through `Fetched::caption_with`: an
|
||||||
|
// empty format falls back to the built-in caption.
|
||||||
|
Some((author, author_url, title, content, tags)) => x_media::site::caption_from_fields(
|
||||||
|
format, built_in, url, author, author_url, title, content, tags,
|
||||||
|
),
|
||||||
|
None => x_media::site::truncate_caption(built_in),
|
||||||
|
};
|
||||||
|
let text = fields
|
||||||
|
.map(|(_, _, title, content, _)| x_media::site::compose_text(title, content))
|
||||||
|
.unwrap_or_default();
|
||||||
|
crate::send::quote_long_caption(&caption, &text, quote_chars).into_owned()
|
||||||
|
}
|
||||||
|
|
||||||
/// Builds the HTML report for the `/debug` command: what the parser produced
|
/// Builds the HTML report for the `/debug` command: what the parser produced
|
||||||
/// for a link (site, canonical URL, title/author/tags, caption and the media
|
/// for a link (site, canonical URL, title/author/tags, caption and the media
|
||||||
/// list) — no media is sent and nothing is cached or forwarded. Sent with
|
/// list) — no media is sent and nothing is cached or forwarded. Sent with
|
||||||
@@ -539,7 +807,9 @@ fn debug_report(
|
|||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::{MAX_DEBUG_REPORT_CHARS, debug_report};
|
use super::{
|
||||||
|
MAX_DEBUG_REPORT_CHARS, debug_report, preview_caption, settings_text, unknown_placeholder,
|
||||||
|
};
|
||||||
use x_media::media::Media;
|
use x_media::media::Media;
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
@@ -657,4 +927,261 @@ mod tests {
|
|||||||
assert!(report.chars().count() <= MAX_DEBUG_REPORT_CHARS, "{report}");
|
assert!(report.chars().count() <= MAX_DEBUG_REPORT_CHARS, "{report}");
|
||||||
assert!(report.ends_with('…'), "{report}");
|
assert!(report.ends_with('…'), "{report}");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn settings_text_reports_the_chat_configuration() {
|
||||||
|
use crate::state::ChatData;
|
||||||
|
|
||||||
|
// A fresh chat: the defaults must be spelled out, including how to set
|
||||||
|
// the channel (an empty field is not a status).
|
||||||
|
let empty = settings_text(&ChatData::default());
|
||||||
|
assert!(empty.contains("Forward channel: not set"), "{empty}");
|
||||||
|
assert!(empty.contains("/set_forward_channel"), "{empty}");
|
||||||
|
assert!(empty.contains("Edit before forward: off"), "{empty}");
|
||||||
|
assert!(empty.contains("built-in for every site"), "{empty}");
|
||||||
|
assert!(empty.contains("Templates: none"), "{empty}");
|
||||||
|
|
||||||
|
let configured = ChatData {
|
||||||
|
forward_channel_id: Some(-100123),
|
||||||
|
edit_before_forward: true,
|
||||||
|
template: [("b", "[]"), ("a", "[]")]
|
||||||
|
.into_iter()
|
||||||
|
.map(|(k, v)| (k.to_string(), v.to_string()))
|
||||||
|
.collect(),
|
||||||
|
message_format: [("twitter", "{author}: {content}")]
|
||||||
|
.into_iter()
|
||||||
|
.map(|(k, v)| (k.to_string(), v.to_string()))
|
||||||
|
.collect(),
|
||||||
|
..ChatData::default()
|
||||||
|
};
|
||||||
|
let text = settings_text(&configured);
|
||||||
|
assert!(text.contains("Forward channel: -100123"), "{text}");
|
||||||
|
assert!(text.contains("Edit before forward: on"), "{text}");
|
||||||
|
assert!(text.contains("twitter => {author}: {content}"), "{text}");
|
||||||
|
// Sorted, so the same chat always reports the same thing.
|
||||||
|
assert!(text.contains("Templates (2): a, b"), "{text}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn help_and_start_cover_what_the_command_list_cannot() {
|
||||||
|
// The placeholders the renderer substitutes must be the ones the help
|
||||||
|
// lists: a stale list is worse than none.
|
||||||
|
for placeholder in super::FORMAT_PLACEHOLDERS {
|
||||||
|
assert!(
|
||||||
|
super::HELP_FOOTER.contains(&format!("{{{placeholder}}}")),
|
||||||
|
"help does not document {{{placeholder}}}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
// The private-chat rule and the template placeholder semantics are the
|
||||||
|
// two things users got wrong most often.
|
||||||
|
assert!(super::HELP_FOOTER.contains("private chats only"));
|
||||||
|
assert!(super::HELP_FOOTER.contains("[]"));
|
||||||
|
assert!(super::START_TEXT.contains("inline mode"));
|
||||||
|
assert!(super::START_TEXT.contains("/help"));
|
||||||
|
// Both must stay inside Telegram's message limit.
|
||||||
|
assert!(super::HELP_FOOTER.chars().count() < 2000);
|
||||||
|
assert!(super::START_TEXT.chars().count() < 2000);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn every_command_is_registered_and_parses() {
|
||||||
|
use teloxide::utils::command::BotCommands;
|
||||||
|
|
||||||
|
use super::Command;
|
||||||
|
|
||||||
|
let registered: Vec<String> = Command::bot_commands()
|
||||||
|
.into_iter()
|
||||||
|
.map(|command| command.command.trim_start_matches('/').to_string())
|
||||||
|
.collect();
|
||||||
|
for expected in [
|
||||||
|
"start",
|
||||||
|
"help",
|
||||||
|
"settings",
|
||||||
|
"set_forward_channel",
|
||||||
|
"remove_template",
|
||||||
|
"set_format",
|
||||||
|
"test",
|
||||||
|
"debug",
|
||||||
|
] {
|
||||||
|
assert!(
|
||||||
|
registered.iter().any(|name| name == expected),
|
||||||
|
"{expected} missing from {registered:?}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
// Telegram caps a command description at 256 chars.
|
||||||
|
for command in Command::bot_commands() {
|
||||||
|
assert!(
|
||||||
|
command.description.chars().count() <= 256,
|
||||||
|
"{}: description too long",
|
||||||
|
command.command
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// A command with a `String` argument must parse with its whole
|
||||||
|
// argument: without `parse_with`, teloxide's default parser rejects
|
||||||
|
// `/remove_template x` and the command silently falls through to the
|
||||||
|
// URL flow.
|
||||||
|
assert!(matches!(
|
||||||
|
Command::parse("/settings", ""),
|
||||||
|
Ok(Command::Settings)
|
||||||
|
));
|
||||||
|
match Command::parse("/remove_template tpl", "") {
|
||||||
|
Ok(Command::RemoveTemplate(name)) => assert_eq!(name, "tpl"),
|
||||||
|
Ok(_) => panic!("/remove_template parsed as another command"),
|
||||||
|
Err(e) => panic!("parse error: {e}"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn every_documented_invocation_parses() {
|
||||||
|
use teloxide::utils::command::BotCommands;
|
||||||
|
|
||||||
|
use super::Command;
|
||||||
|
|
||||||
|
// The README's forms, verbatim. teloxide's `split` parser accepts
|
||||||
|
// EXACTLY one token per `String` field, so a command documented with
|
||||||
|
// two arguments (or an optional one) silently stops parsing — and a
|
||||||
|
// command that does not parse falls through to the URL flow in
|
||||||
|
// silence.
|
||||||
|
type Check = fn(&Command) -> bool;
|
||||||
|
let cases: Vec<(&str, Check)> = vec![
|
||||||
|
("/start", |c| matches!(c, Command::Start)),
|
||||||
|
("/help", |c| matches!(c, Command::Help)),
|
||||||
|
("/settings", |c| matches!(c, Command::Settings)),
|
||||||
|
("/edit_before_forward", |c| {
|
||||||
|
matches!(c, Command::EditBeforeForward)
|
||||||
|
}),
|
||||||
|
("/remove_forward_channel", |c| {
|
||||||
|
matches!(c, Command::RemoveForwardChannel)
|
||||||
|
}),
|
||||||
|
("/bot_dict", |c| matches!(c, Command::BotDict)),
|
||||||
|
(
|
||||||
|
"/set_forward_channel @a_channel",
|
||||||
|
|c| matches!(c, Command::SetForwardChannel(a) if a == "@a_channel"),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"/set_template tpl",
|
||||||
|
|c| matches!(c, Command::SetTemplate(a) if a == "tpl"),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"/remove_template tpl",
|
||||||
|
|c| matches!(c, Command::RemoveTemplate(a) if a == "tpl"),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"/set_format twitter {author}: {title}",
|
||||||
|
|c| matches!(c, Command::SetFormat(a) if a == "twitter {author}: {title}"),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"/set_format twitter -",
|
||||||
|
|c| matches!(c, Command::SetFormat(a) if a == "twitter -"),
|
||||||
|
),
|
||||||
|
// Documented as "clear everything" when called without a link.
|
||||||
|
(
|
||||||
|
"/clear_cache",
|
||||||
|
|c| matches!(c, Command::ClearCache(a) if a.is_empty()),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"/clear_cache https://x.com/u/status/1",
|
||||||
|
|c| matches!(c, Command::ClearCache(a) if a == "https://x.com/u/status/1"),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"/test https://x.com/u/status/1",
|
||||||
|
|c| matches!(c, Command::Test(a) if a == "https://x.com/u/status/1"),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"/debug https://x.com/u/status/1",
|
||||||
|
|c| matches!(c, Command::Debug(a) if a == "https://x.com/u/status/1"),
|
||||||
|
),
|
||||||
|
];
|
||||||
|
|
||||||
|
for (text, ok) in cases {
|
||||||
|
match Command::parse(text, "") {
|
||||||
|
Ok(parsed) => assert!(ok(&parsed), "{text} parsed as the wrong variant"),
|
||||||
|
Err(e) => panic!("{text} did not parse: {e}"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn preview_caption_applies_the_chat_format_and_the_long_post_quote() {
|
||||||
|
let fields = Some((
|
||||||
|
"Author",
|
||||||
|
"https://x.com/u",
|
||||||
|
"Pinned title",
|
||||||
|
"Pinned body",
|
||||||
|
"#tag",
|
||||||
|
));
|
||||||
|
|
||||||
|
// No format override → the site's built-in caption, untouched.
|
||||||
|
assert_eq!(
|
||||||
|
preview_caption(
|
||||||
|
"",
|
||||||
|
"built-in caption",
|
||||||
|
"https://x.com/u/status/1",
|
||||||
|
fields,
|
||||||
|
200
|
||||||
|
),
|
||||||
|
"built-in caption"
|
||||||
|
);
|
||||||
|
|
||||||
|
// The bug this pins: `/debug` used to print the built-in caption even
|
||||||
|
// with a format set, so `/set_format` looked like it did nothing.
|
||||||
|
let formatted = preview_caption(
|
||||||
|
"{author} · {title}",
|
||||||
|
"built-in caption",
|
||||||
|
"https://x.com/u/status/1",
|
||||||
|
fields,
|
||||||
|
200,
|
||||||
|
);
|
||||||
|
assert_eq!(formatted, "Author · Pinned title");
|
||||||
|
|
||||||
|
// `{url}` comes from the canonical post URL, as in the send paths.
|
||||||
|
assert_eq!(
|
||||||
|
preview_caption(
|
||||||
|
"{url} {title}",
|
||||||
|
"built-in",
|
||||||
|
"https://x.com/u/status/1",
|
||||||
|
fields,
|
||||||
|
200
|
||||||
|
),
|
||||||
|
"https://x.com/u/status/1 Pinned title"
|
||||||
|
);
|
||||||
|
|
||||||
|
// A long post's text is quoted exactly like the send paths quote it.
|
||||||
|
let long = "正".repeat(300);
|
||||||
|
let fields = Some(("Author", "https://x.com/u", "", long.as_str(), ""));
|
||||||
|
let quoted = preview_caption(
|
||||||
|
"",
|
||||||
|
"https://x.com/u/status/1\n<a href=\"https://x.com/u\">Author</a>: 正…",
|
||||||
|
"https://x.com/u/status/1",
|
||||||
|
fields,
|
||||||
|
200,
|
||||||
|
);
|
||||||
|
assert!(quoted.contains("<blockquote expandable>"), "{quoted}");
|
||||||
|
|
||||||
|
// Without render fields (a site that does not expose them) the
|
||||||
|
// built-in caption is all there is.
|
||||||
|
assert_eq!(
|
||||||
|
preview_caption("", "built-in", "https://x.com/u/status/1", None, 200),
|
||||||
|
"built-in"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn unknown_placeholder_finds_typos_only() {
|
||||||
|
assert_eq!(unknown_placeholder("{author} — {title}"), None);
|
||||||
|
// Every key the renderer substitutes must pass, in any combination.
|
||||||
|
assert_eq!(
|
||||||
|
unknown_placeholder("{url}{author}{author_url}{title}{content}{tags}"),
|
||||||
|
None
|
||||||
|
);
|
||||||
|
// Plain text and braces Telegram renders literally are not tokens.
|
||||||
|
assert_eq!(unknown_placeholder("no placeholders here"), None);
|
||||||
|
assert_eq!(unknown_placeholder("{unclosed"), None);
|
||||||
|
|
||||||
|
assert_eq!(unknown_placeholder("{titel}"), Some("titel"));
|
||||||
|
assert_eq!(unknown_placeholder("{title} {Content}"), Some("Content"));
|
||||||
|
// A typo after a valid token is still found.
|
||||||
|
assert_eq!(unknown_placeholder("{url} {tag}"), Some("tag"));
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -146,6 +146,15 @@ async fn answer_inline_query(bot: Bot, query: InlineQuery) -> Result<bool, Reque
|
|||||||
);
|
);
|
||||||
for (i, media) in fetched.media.iter().enumerate() {
|
for (i, media) in fetched.media.iter().enumerate() {
|
||||||
let id = format!("{i}");
|
let id = format!("{i}");
|
||||||
|
// Telegram fetches an inline result's URL itself and cannot
|
||||||
|
// send site-specific headers, so hotlink-protected media
|
||||||
|
// (pixiv's pximg.net) would render as a broken file there.
|
||||||
|
// Locally produced media (ugoira MP4, bsky remux) is a local
|
||||||
|
// path and does not parse as a URL at all — same skip.
|
||||||
|
if x_media::site::needs_media_headers(media.url()) {
|
||||||
|
log::debug!("inline: skipping hotlink-protected media {id}");
|
||||||
|
continue;
|
||||||
|
}
|
||||||
let Some(url) = url::Url::parse(media.url()).ok() else {
|
let Some(url) = url::Url::parse(media.url()).ok() else {
|
||||||
continue;
|
continue;
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -23,7 +23,9 @@ use crate::media_sender::MediaSender;
|
|||||||
use commands::{Command, execute_command};
|
use commands::{Command, execute_command};
|
||||||
use teloxide::RequestError;
|
use teloxide::RequestError;
|
||||||
use teloxide::prelude::*;
|
use teloxide::prelude::*;
|
||||||
use teloxide::types::{ChatId, ChatKind, Message, MessageId, ParseMode, ReplyParameters};
|
use teloxide::types::{
|
||||||
|
ChatId, ChatKind, Message, MessageId, ParseMode, PublicChatKind, ReplyParameters,
|
||||||
|
};
|
||||||
use teloxide::utils::command::BotCommands;
|
use teloxide::utils::command::BotCommands;
|
||||||
use urls::{URL_JOBS, extract_urls};
|
use urls::{URL_JOBS, extract_urls};
|
||||||
|
|
||||||
@@ -175,10 +177,38 @@ pub async fn message_handler(bot: Bot, message: Message) -> Result<(), RequestEr
|
|||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
} else if is_group(&message.chat.kind)
|
||||||
|
&& extract_urls(&message)
|
||||||
|
.iter()
|
||||||
|
.any(|url| x_media::site::cache_key(url).is_some())
|
||||||
|
{
|
||||||
|
// A supported link in a group used to be dropped in silence, which
|
||||||
|
// reads as a broken bot (the command menu is registered globally, so
|
||||||
|
// the expectation is there). Unsupported links stay ignored; the hint
|
||||||
|
// names the two paths that do work. Channels are excluded — the reply
|
||||||
|
// would be posted into the channel itself.
|
||||||
|
let _ = reply(&bot, message.chat.id.0, message.id, GROUP_LINK_HINT).await;
|
||||||
}
|
}
|
||||||
respond(())
|
respond(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Answer for a link posted where the pipeline does not run (a group): links
|
||||||
|
/// are private-chat only, inline mode is the group path.
|
||||||
|
const GROUP_LINK_HINT: &str =
|
||||||
|
"Links are handled in private chat only — send me this link there, or use inline mode here.";
|
||||||
|
|
||||||
|
/// Groups and supergroups, as opposed to private chats and channels.
|
||||||
|
fn is_group(kind: &ChatKind) -> bool {
|
||||||
|
matches!(
|
||||||
|
kind,
|
||||||
|
ChatKind::Public(chat)
|
||||||
|
if matches!(
|
||||||
|
chat.kind,
|
||||||
|
PublicChatKind::Group | PublicChatKind::Supergroup(_)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
@@ -271,4 +301,37 @@ mod tests {
|
|||||||
assert!(!edit_message_handler(&ctx, 1, PROMPT_ID, "hello").await);
|
assert!(!edit_message_handler(&ctx, 1, PROMPT_ID, "hello").await);
|
||||||
assert!(sender.calls().is_empty());
|
assert!(sender.calls().is_empty());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_link_hint_is_for_groups_only() {
|
||||||
|
use teloxide::types::{ChatPrivate, ChatPublic, PublicChatChannel, PublicChatSupergroup};
|
||||||
|
|
||||||
|
let group = ChatKind::Public(ChatPublic {
|
||||||
|
title: None,
|
||||||
|
kind: PublicChatKind::Group,
|
||||||
|
});
|
||||||
|
let supergroup = ChatKind::Public(ChatPublic {
|
||||||
|
title: None,
|
||||||
|
kind: PublicChatKind::Supergroup(PublicChatSupergroup {
|
||||||
|
username: None,
|
||||||
|
is_forum: false,
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
// A channel must stay silent: the hint reply would be posted into the
|
||||||
|
// channel itself.
|
||||||
|
let channel = ChatKind::Public(ChatPublic {
|
||||||
|
title: None,
|
||||||
|
kind: PublicChatKind::Channel(PublicChatChannel { username: None }),
|
||||||
|
});
|
||||||
|
let private = ChatKind::Private(ChatPrivate {
|
||||||
|
username: None,
|
||||||
|
first_name: None,
|
||||||
|
last_name: None,
|
||||||
|
});
|
||||||
|
|
||||||
|
assert!(is_group(&group));
|
||||||
|
assert!(is_group(&supergroup));
|
||||||
|
assert!(!is_group(&channel));
|
||||||
|
assert!(!is_group(&private));
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -4,9 +4,11 @@
|
|||||||
use super::{log_key, reply};
|
use super::{log_key, reply};
|
||||||
use crate::ctx::{AppContext, CONTEXT};
|
use crate::ctx::{AppContext, CONTEXT};
|
||||||
use crate::link_cache::{CachedMediaKind, CachedPost};
|
use crate::link_cache::{CachedMediaKind, CachedPost};
|
||||||
|
use crate::media_sender::MediaSender;
|
||||||
use crate::send::{self, MediaItemPayload, Task};
|
use crate::send::{self, MediaItemPayload, Task};
|
||||||
use crate::state::ChatData;
|
use crate::state::ChatData;
|
||||||
use std::collections::HashSet;
|
use std::collections::HashSet;
|
||||||
|
use std::future::Future;
|
||||||
use std::sync::LazyLock;
|
use std::sync::LazyLock;
|
||||||
use teloxide::types::{ChatAction, ChatId, Message, MessageEntityKind, MessageId};
|
use teloxide::types::{ChatAction, ChatId, Message, MessageEntityKind, MessageId};
|
||||||
use x_media::media::Media;
|
use x_media::media::Media;
|
||||||
@@ -190,11 +192,16 @@ async fn dispatch_send(
|
|||||||
log_key(url)
|
log_key(url)
|
||||||
);
|
);
|
||||||
send::enqueue_retry(ctx.task_queue, *task, delay_seconds).await;
|
send::enqueue_retry(ctx.task_queue, *task, delay_seconds).await;
|
||||||
|
// Name the post and the wait: "queued for retry" alone left the
|
||||||
|
// user guessing which link it was and how long the wait is.
|
||||||
let _ = reply(
|
let _ = reply(
|
||||||
ctx.sender,
|
ctx.sender,
|
||||||
chat_id,
|
chat_id,
|
||||||
reply_to,
|
reply_to,
|
||||||
"Send failed. Task queued for retry.",
|
format!(
|
||||||
|
"Send failed for {} — retrying in {delay_seconds:.0}s.",
|
||||||
|
log_key(url)
|
||||||
|
),
|
||||||
)
|
)
|
||||||
.await;
|
.await;
|
||||||
}
|
}
|
||||||
@@ -285,6 +292,11 @@ fn build_send_task(
|
|||||||
/// workers pass [`PostSend::FromChat`], the `/test` command
|
/// workers pass [`PostSend::FromChat`], the `/test` command
|
||||||
/// [`PostSend::Suppressed`]. Everything else (cache write, retry enqueue,
|
/// [`PostSend::Suppressed`]. Everything else (cache write, retry enqueue,
|
||||||
/// dead-letter notification) is identical.
|
/// dead-letter notification) is identical.
|
||||||
|
///
|
||||||
|
/// Wraps [`url_media_inner`] with the chat-action keep-alive: Telegram expires
|
||||||
|
/// an action indicator after ~5s, while a fetch (ugoira encode, HLS remux) plus
|
||||||
|
/// a download-and-reupload fallback routinely takes longer — without the
|
||||||
|
/// refresh the chat shows nothing and the bot reads as stalled.
|
||||||
pub(crate) async fn url_media(
|
pub(crate) async fn url_media(
|
||||||
ctx: &AppContext<'_>,
|
ctx: &AppContext<'_>,
|
||||||
chat_id: i64,
|
chat_id: i64,
|
||||||
@@ -292,14 +304,136 @@ pub(crate) async fn url_media(
|
|||||||
url: &str,
|
url: &str,
|
||||||
post_send: PostSend,
|
post_send: PostSend,
|
||||||
) {
|
) {
|
||||||
let reply_to = MessageId(reply_to_message_id as i32);
|
// Shared with the pipeline: once the media types are known the indicator
|
||||||
if let Err(e) = ctx
|
// switches from "typing" to "sending photo/video".
|
||||||
.sender
|
let hint = parking_lot::Mutex::new(ActionHint::Typing);
|
||||||
.send_chat_action(ChatId(chat_id), ChatAction::Typing)
|
run_with_chat_action(
|
||||||
.await
|
ctx.sender,
|
||||||
{
|
chat_id,
|
||||||
|
&hint,
|
||||||
|
url_media_inner(ctx, chat_id, reply_to_message_id, url, post_send, &hint),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Runs `pipeline` while keeping the chat's action indicator alive: Telegram
|
||||||
|
/// expires an action after ~5s, while a fetch (ugoira encode, HLS remux) plus a
|
||||||
|
/// download-and-reupload fallback routinely takes longer. The pipeline updates
|
||||||
|
/// `hint` when it knows what it is sending.
|
||||||
|
async fn run_with_chat_action<F: Future<Output = ()>>(
|
||||||
|
sender: &dyn MediaSender,
|
||||||
|
chat_id: i64,
|
||||||
|
hint: &parking_lot::Mutex<ActionHint>,
|
||||||
|
pipeline: F,
|
||||||
|
) {
|
||||||
|
// The guard is released before the await: a parking_lot guard held across
|
||||||
|
// it makes the future !Send, and the URL workers spawn these.
|
||||||
|
let action = hint.lock().action();
|
||||||
|
if let Err(e) = sender.send_chat_action(ChatId(chat_id), action).await {
|
||||||
log::error!("send_chat_action failed: {e}");
|
log::error!("send_chat_action failed: {e}");
|
||||||
}
|
}
|
||||||
|
tokio::pin!(pipeline);
|
||||||
|
loop {
|
||||||
|
tokio::select! {
|
||||||
|
// `biased` polls the pipeline first, so a finished pipeline returns
|
||||||
|
// without ever arming the refresh timer (no stray actions).
|
||||||
|
biased;
|
||||||
|
() = &mut pipeline => return,
|
||||||
|
() = tokio::time::sleep(ACTION_REFRESH) => {
|
||||||
|
let action = hint.lock().action();
|
||||||
|
if let Err(e) = sender.send_chat_action(ChatId(chat_id), action).await {
|
||||||
|
log::error!("send_chat_action failed: {e}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// How often the chat-action indicator is refreshed while a pipeline runs.
|
||||||
|
/// Telegram's indicator lasts ~5s; refreshing slightly inside that keeps it
|
||||||
|
/// on-screen continuously.
|
||||||
|
const ACTION_REFRESH: std::time::Duration = std::time::Duration::from_secs(4);
|
||||||
|
|
||||||
|
/// What the chat action should say. Unknown before the fetch, so the pipeline
|
||||||
|
/// starts with `Typing` and switches as soon as the media types are known.
|
||||||
|
#[derive(Clone, Copy)]
|
||||||
|
enum ActionHint {
|
||||||
|
Typing,
|
||||||
|
Photo,
|
||||||
|
Video,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl ActionHint {
|
||||||
|
/// Photos make Telegram label the send "sending photo"; video/animation
|
||||||
|
/// only payloads get "sending video". A mixed post takes the photo label
|
||||||
|
/// (the group's first item is always a photo, see `photos_first`).
|
||||||
|
fn for_items(items: &[MediaItemPayload]) -> Self {
|
||||||
|
if items
|
||||||
|
.iter()
|
||||||
|
.any(|item| matches!(item, MediaItemPayload::Photo { .. }))
|
||||||
|
{
|
||||||
|
Self::Photo
|
||||||
|
} else {
|
||||||
|
Self::Video
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn action(self) -> ChatAction {
|
||||||
|
match self {
|
||||||
|
Self::Typing => ChatAction::Typing,
|
||||||
|
Self::Photo => ChatAction::UploadPhoto,
|
||||||
|
Self::Video => ChatAction::UploadVideo,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// User-facing text for a failed fetch. The [`FetchError`] class is what tells
|
||||||
|
/// the user whether the post is gone, withheld or the source is refusing
|
||||||
|
/// requests; a single generic sentence threw that away.
|
||||||
|
fn fetch_error_message(err: &x_media::site::FetchError) -> String {
|
||||||
|
use x_media::site::FetchError;
|
||||||
|
match err {
|
||||||
|
FetchError::NotFound => "Post not found (deleted, private or unavailable).".to_string(),
|
||||||
|
FetchError::Sensitive => concat!(
|
||||||
|
"This post's media is withheld (age-restricted). ",
|
||||||
|
"The bot owner must set TWITTER_AUTH_TOKEN to fetch it."
|
||||||
|
)
|
||||||
|
.to_string(),
|
||||||
|
FetchError::Blocked => {
|
||||||
|
"The source site refused the request (risk control). Try again later.".to_string()
|
||||||
|
}
|
||||||
|
FetchError::Disabled { site } => {
|
||||||
|
format!("{} support is disabled on this bot.", site_title(site))
|
||||||
|
}
|
||||||
|
FetchError::Transient(_) | FetchError::Http(_) => {
|
||||||
|
"The source site is unavailable right now (tried 3 times). Try again later.".to_string()
|
||||||
|
}
|
||||||
|
// Parse/shape surprises, pixiv auth details, oversized media: nothing
|
||||||
|
// actionable for the user beyond "this did not work".
|
||||||
|
_ => "Failed to fetch media from this link.".to_string(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Site ids are lowercase ASCII (`pixiv`); user-facing text capitalizes the
|
||||||
|
/// first letter.
|
||||||
|
fn site_title(site: &str) -> String {
|
||||||
|
let mut chars = site.chars();
|
||||||
|
match chars.next() {
|
||||||
|
Some(first) => first.to_uppercase().collect::<String>() + chars.as_str(),
|
||||||
|
None => String::new(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[allow(clippy::too_many_arguments)]
|
||||||
|
async fn url_media_inner(
|
||||||
|
ctx: &AppContext<'_>,
|
||||||
|
chat_id: i64,
|
||||||
|
reply_to_message_id: i64,
|
||||||
|
url: &str,
|
||||||
|
post_send: PostSend,
|
||||||
|
hint: &parking_lot::Mutex<ActionHint>,
|
||||||
|
) {
|
||||||
|
let reply_to = MessageId(reply_to_message_id as i32);
|
||||||
|
|
||||||
// Link cache: a post sent before is re-sent from Telegram file ids —
|
// Link cache: a post sent before is re-sent from Telegram file ids —
|
||||||
// no source-site request, no download, no upload. Keyed by the
|
// no source-site request, no download, no upload. Keyed by the
|
||||||
@@ -355,6 +489,9 @@ pub(crate) async fn url_media(
|
|||||||
},
|
},
|
||||||
})
|
})
|
||||||
.collect();
|
.collect();
|
||||||
|
// The indicator switches to "sending photo/video" once the kinds are
|
||||||
|
// known; `items` is moved into the task below.
|
||||||
|
*hint.lock() = ActionHint::for_items(&items);
|
||||||
let task = build_send_task(
|
let task = build_send_task(
|
||||||
&chat_data,
|
&chat_data,
|
||||||
chat_id,
|
chat_id,
|
||||||
@@ -378,13 +515,7 @@ pub(crate) async fn url_media(
|
|||||||
// Retries exhausted: notify the user (Rust-only requirement 3).
|
// Retries exhausted: notify the user (Rust-only requirement 3).
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
log::error!("fetch {url}: {e}");
|
log::error!("fetch {url}: {e}");
|
||||||
let _ = reply(
|
let _ = reply(ctx.sender, chat_id, reply_to, fetch_error_message(&e)).await;
|
||||||
ctx.sender,
|
|
||||||
chat_id,
|
|
||||||
reply_to,
|
|
||||||
"Failed to fetch media from this link.",
|
|
||||||
)
|
|
||||||
.await;
|
|
||||||
}
|
}
|
||||||
Ok(Some(mut fetched)) => {
|
Ok(Some(mut fetched)) => {
|
||||||
if fetched.media.is_empty() {
|
if fetched.media.is_empty() {
|
||||||
@@ -426,6 +557,9 @@ pub(crate) async fn url_media(
|
|||||||
.iter()
|
.iter()
|
||||||
.map(|media| media_to_payload(media, fetched.sensitive))
|
.map(|media| media_to_payload(media, fetched.sensitive))
|
||||||
.collect();
|
.collect();
|
||||||
|
// The indicator switches to "sending photo/video" once the kinds
|
||||||
|
// are known; `items` is moved into the task below.
|
||||||
|
*hint.lock() = ActionHint::for_items(&items);
|
||||||
let task = build_send_task(
|
let task = build_send_task(
|
||||||
&chat_data,
|
&chat_data,
|
||||||
chat_id,
|
chat_id,
|
||||||
@@ -695,4 +829,89 @@ mod tests {
|
|||||||
// Dead-letter notification still reaches the chat that asked.
|
// Dead-letter notification still reaches the chat that asked.
|
||||||
assert_eq!(notify_chat_id, Some(1));
|
assert_eq!(notify_chat_id, Some(1));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn fetch_errors_map_to_distinct_user_messages() {
|
||||||
|
use x_media::site::FetchError;
|
||||||
|
|
||||||
|
let disabled = fetch_error_message(&FetchError::Disabled { site: "pixiv" });
|
||||||
|
assert_eq!(disabled, "Pixiv support is disabled on this bot.");
|
||||||
|
assert_eq!(
|
||||||
|
fetch_error_message(&FetchError::NotFound),
|
||||||
|
"Post not found (deleted, private or unavailable)."
|
||||||
|
);
|
||||||
|
let sensitive = fetch_error_message(&FetchError::Sensitive);
|
||||||
|
assert!(sensitive.contains("TWITTER_AUTH_TOKEN"), "{sensitive}");
|
||||||
|
let blocked = fetch_error_message(&FetchError::Blocked);
|
||||||
|
assert!(blocked.contains("refused"), "{blocked}");
|
||||||
|
|
||||||
|
// Each class that has something to say must differ from the generic
|
||||||
|
// fallback — one generic sentence for everything is what this fixes.
|
||||||
|
let generic = fetch_error_message(&FetchError::TooLarge);
|
||||||
|
for text in [disabled, sensitive, blocked] {
|
||||||
|
assert_ne!(text, generic);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn action_hint_follows_the_media_kind() {
|
||||||
|
use MediaItemPayload::{Animation, Photo, Video};
|
||||||
|
|
||||||
|
let photo = || Photo {
|
||||||
|
media: "https://p/1.jpg".into(),
|
||||||
|
has_spoiler: false,
|
||||||
|
fallback_url: None,
|
||||||
|
file_id: false,
|
||||||
|
};
|
||||||
|
let video = || Video {
|
||||||
|
media: "https://v/1.mp4".into(),
|
||||||
|
has_spoiler: false,
|
||||||
|
thumbnail: None,
|
||||||
|
fallback_url: None,
|
||||||
|
file_id: false,
|
||||||
|
};
|
||||||
|
|
||||||
|
// Unknown before the fetch: the pipeline starts on "typing".
|
||||||
|
assert!(matches!(ActionHint::Typing.action(), ChatAction::Typing));
|
||||||
|
assert!(matches!(
|
||||||
|
ActionHint::for_items(&[photo()]).action(),
|
||||||
|
ChatAction::UploadPhoto
|
||||||
|
));
|
||||||
|
assert!(matches!(
|
||||||
|
ActionHint::for_items(&[
|
||||||
|
video(),
|
||||||
|
Animation {
|
||||||
|
media: "https://v/2.mp4".into(),
|
||||||
|
has_spoiler: false,
|
||||||
|
file_id: false,
|
||||||
|
}
|
||||||
|
])
|
||||||
|
.action(),
|
||||||
|
ChatAction::UploadVideo
|
||||||
|
));
|
||||||
|
// A mixed post takes the photo label: `photos_first` always leads with
|
||||||
|
// a photo, which is what Telegram shows.
|
||||||
|
assert!(matches!(
|
||||||
|
ActionHint::for_items(&[video(), photo()]).action(),
|
||||||
|
ChatAction::UploadPhoto
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test(start_paused = true)]
|
||||||
|
async fn a_long_pipeline_keeps_the_chat_action_alive() {
|
||||||
|
let sender = MockSender::scripted(vec![], permanent_error);
|
||||||
|
let hint = parking_lot::Mutex::new(ActionHint::Typing);
|
||||||
|
// Three refresh windows of work: Telegram would have dropped the
|
||||||
|
// indicator twice without the keep-alive.
|
||||||
|
let pipeline = async { tokio::time::sleep(ACTION_REFRESH * 3).await };
|
||||||
|
|
||||||
|
run_with_chat_action(&sender, 1, &hint, pipeline).await;
|
||||||
|
|
||||||
|
let actions = sender
|
||||||
|
.calls()
|
||||||
|
.iter()
|
||||||
|
.filter(|call| **call == "send_chat_action")
|
||||||
|
.count();
|
||||||
|
assert_eq!(actions, 3, "expected the initial action plus two refreshes");
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,7 +2,7 @@ use dotenv::dotenv;
|
|||||||
use teloxide::dptree::endpoint;
|
use teloxide::dptree::endpoint;
|
||||||
use teloxide::prelude::*;
|
use teloxide::prelude::*;
|
||||||
use teloxide::stop::StopToken;
|
use teloxide::stop::StopToken;
|
||||||
use teloxide::types::{ChatId, InputFile, MessageId};
|
use teloxide::types::{ChatId, InlineKeyboardMarkup, InputFile, MessageId};
|
||||||
use teloxide::update_listeners::{self, UpdateListener, webhooks};
|
use teloxide::update_listeners::{self, UpdateListener, webhooks};
|
||||||
use tokio::sync::watch;
|
use tokio::sync::watch;
|
||||||
use x_media::site;
|
use x_media::site;
|
||||||
@@ -119,13 +119,19 @@ async fn main() {
|
|||||||
log::debug!("rate limiter: dropped {idle_limiters} idle bucket(s)");
|
log::debug!("rate limiter: dropped {idle_limiters} idle bucket(s)");
|
||||||
}
|
}
|
||||||
for (chat_id, prompt_message_id) in removed {
|
for (chat_id, prompt_message_id) in removed {
|
||||||
// If the prompt was already deleted, this fails with a
|
// Rewritten in place, not announced: the sweep is a
|
||||||
// 400 "message to edit not found" — log and ignore.
|
// background timer, and a fresh message would wake the chat
|
||||||
|
// up to a full TTL later about a prompt the user already
|
||||||
|
// walked away from. The edit drops the buttons too. If the
|
||||||
|
// prompt was already deleted this fails with a 400
|
||||||
|
// "message to edit not found" — log and ignore.
|
||||||
if let Err(e) = bot
|
if let Err(e) = bot
|
||||||
.edit_message_reply_markup(
|
.edit_message_text(
|
||||||
ChatId(chat_id),
|
ChatId(chat_id),
|
||||||
MessageId(prompt_message_id as i32),
|
MessageId(prompt_message_id as i32),
|
||||||
|
send::EDIT_PROMPT_EXPIRED_TEXT,
|
||||||
)
|
)
|
||||||
|
.reply_markup(InlineKeyboardMarkup::default())
|
||||||
.await
|
.await
|
||||||
{
|
{
|
||||||
log::info!("edit-expiry sweep: prompt message gone: {e}");
|
log::info!("edit-expiry sweep: prompt message gone: {e}");
|
||||||
|
|||||||
@@ -342,8 +342,16 @@ impl QueueWorker {
|
|||||||
payload,
|
payload,
|
||||||
}) => {
|
}) => {
|
||||||
if row.attempts as u32 >= MAX_RETRIES {
|
if row.attempts as u32 >= MAX_RETRIES {
|
||||||
let message = format!("task failed after {MAX_RETRIES} retries");
|
// The queue keeps only the payload, not the last error, so
|
||||||
log::error!("dead-lettering {}: {message}", row.id);
|
// the cause of an exhausted retry is just that: exhausted.
|
||||||
|
// (The dead-letter message is read by the user, so it must
|
||||||
|
// not restate its own wrapper — see `failure_text`.)
|
||||||
|
let message = "retries exhausted".to_string();
|
||||||
|
log::error!(
|
||||||
|
"dead-lettering {}: {message} after {} attempt(s)",
|
||||||
|
row.id,
|
||||||
|
row.attempts + 1
|
||||||
|
);
|
||||||
self.delete_row(&row.id).await;
|
self.delete_row(&row.id).await;
|
||||||
(self.dead_letter)(payload, message).await;
|
(self.dead_letter)(payload, message).await;
|
||||||
} else {
|
} else {
|
||||||
|
|||||||
@@ -28,8 +28,8 @@ use upload::{FallbackError, PreparedItem, prepare_upload_item, send_batch_via_up
|
|||||||
// The crate-facing API of this module lives in its submodules; re-export the
|
// The crate-facing API of this module lives in its submodules; re-export the
|
||||||
// parts other modules use so call sites stay `send::x`.
|
// parts other modules use so call sites stay `send::x`.
|
||||||
pub(crate) use post_send::{
|
pub(crate) use post_send::{
|
||||||
KEEP_ALIVE, Settled, dead_letter_notify, enqueue_retry, handle_task, post_send_actions,
|
EDIT_PROMPT_EXPIRED_TEXT, KEEP_ALIVE, Settled, dead_letter_notify, enqueue_retry, handle_task,
|
||||||
settle_task,
|
post_send_actions, settle_task,
|
||||||
};
|
};
|
||||||
|
|
||||||
/// One process-wide Bot for queue workers. Building a fresh Bot (and its HTTP
|
/// One process-wide Bot for queue workers. Building a fresh Bot (and its HTTP
|
||||||
@@ -230,7 +230,9 @@ fn collect_file_ids(messages: &[Message], batch: &[MediaItemPayload], out: &mut
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
pub const MAX_MEDIA_GROUP: usize = 9;
|
/// Telegram's `sendMediaGroup` accepts 2–10 items per group; 10 (not the older
|
||||||
|
/// 9) means a 10-image post arrives as one album instead of two messages.
|
||||||
|
pub const MAX_MEDIA_GROUP: usize = 10;
|
||||||
|
|
||||||
/// Splits media into batches of at most [`MAX_MEDIA_GROUP`] items, moving the
|
/// Splits media into batches of at most [`MAX_MEDIA_GROUP`] items, moving the
|
||||||
/// items out (no per-item clone).
|
/// items out (no per-item clone).
|
||||||
@@ -728,13 +730,14 @@ mod tests {
|
|||||||
fn chunk_media_items_sizes() {
|
fn chunk_media_items_sizes() {
|
||||||
assert_eq!(chunk_media_items::<i32>(vec![]), Vec::<Vec<i32>>::new());
|
assert_eq!(chunk_media_items::<i32>(vec![]), Vec::<Vec<i32>>::new());
|
||||||
assert_eq!(chunk_media_items((0..9).collect()).len(), 1);
|
assert_eq!(chunk_media_items((0..9).collect()).len(), 1);
|
||||||
assert_eq!(chunk_media_items((0..10).collect()).len(), 2);
|
assert_eq!(chunk_media_items((0..10).collect()).len(), 1);
|
||||||
|
assert_eq!(chunk_media_items((0..11).collect()).len(), 2);
|
||||||
assert_eq!(chunk_media_items((0..25).collect()).len(), 3);
|
assert_eq!(chunk_media_items((0..25).collect()).len(), 3);
|
||||||
assert_eq!(chunk_media_items((0..25).collect())[2].len(), 7);
|
assert_eq!(chunk_media_items((0..25).collect())[2].len(), 5);
|
||||||
assert!(
|
assert!(
|
||||||
chunk_media_items((0..25).collect())
|
chunk_media_items((0..25).collect())
|
||||||
.iter()
|
.iter()
|
||||||
.all(|c| c.len() <= 9)
|
.all(|c| c.len() <= MAX_MEDIA_GROUP)
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -807,7 +810,88 @@ mod tests {
|
|||||||
.flatten()
|
.flatten()
|
||||||
.map(|button| button.text.clone())
|
.map(|button| button.text.clone())
|
||||||
.collect();
|
.collect();
|
||||||
assert_eq!(labels, ["a", "b", "m", "q", "y", "z", "↩️ Confirm"]);
|
assert_eq!(
|
||||||
|
labels,
|
||||||
|
["a", "b", "m", "q", "y", "z", "↩️ Confirm", "🛑 Skip"]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn edit_markup_folds_and_caps_the_template_buttons() {
|
||||||
|
// Telegram rejects a keyboard over 100 buttons, which would drop the
|
||||||
|
// whole prompt; the cap keeps it well under that.
|
||||||
|
let templates: HashMap<String, String> = (0..200)
|
||||||
|
.map(|i| (format!("t{i:03}"), "[]".to_string()))
|
||||||
|
.collect();
|
||||||
|
let keyboard = build_edit_markup(&templates);
|
||||||
|
let buttons: usize = keyboard.inline_keyboard.iter().map(Vec::len).sum();
|
||||||
|
assert!(
|
||||||
|
buttons <= 100,
|
||||||
|
"a keyboard Telegram rejects would lose the prompt: {buttons}"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
buttons,
|
||||||
|
super::post_send::MAX_TEMPLATE_BUTTONS + 2,
|
||||||
|
"the cap plus the confirm/skip pair"
|
||||||
|
);
|
||||||
|
// Names are folded, not one per row.
|
||||||
|
assert_eq!(keyboard.inline_keyboard[0].len(), 3);
|
||||||
|
assert_eq!(keyboard.inline_keyboard.last().unwrap().len(), 2);
|
||||||
|
assert_eq!(super::post_send::hidden_template_count(&templates), 140);
|
||||||
|
// Under the cap nothing is hidden and every name gets a button.
|
||||||
|
let few: HashMap<String, String> = (0..4)
|
||||||
|
.map(|i| (format!("t{i}"), "[]".to_string()))
|
||||||
|
.collect();
|
||||||
|
assert_eq!(super::post_send::hidden_template_count(&few), 0);
|
||||||
|
assert_eq!(
|
||||||
|
build_edit_markup(&few)
|
||||||
|
.inline_keyboard
|
||||||
|
.iter()
|
||||||
|
.map(Vec::len)
|
||||||
|
.sum::<usize>(),
|
||||||
|
6
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn failure_text_names_the_post_and_the_cause() {
|
||||||
|
// A send failure names the post (the cache key) and the cause, so the
|
||||||
|
// user knows which of their links died.
|
||||||
|
let task = sequence_task("https://x.com/u/status/1");
|
||||||
|
let text = super::post_send::failure_text(Some(&task), "retries exhausted");
|
||||||
|
assert!(text.contains("twitter:1"), "{text}");
|
||||||
|
assert!(text.contains("retries exhausted"), "{text}");
|
||||||
|
|
||||||
|
// A channel-forward failure has no source URL: it must not claim a
|
||||||
|
// post failed.
|
||||||
|
let forward = Task::ForwardMessages {
|
||||||
|
from_chat_id: 1,
|
||||||
|
to_chat_id: 2,
|
||||||
|
message_ids: vec![1],
|
||||||
|
notify_chat_id: None,
|
||||||
|
notify_message_id: None,
|
||||||
|
};
|
||||||
|
let text = super::post_send::failure_text(Some(&forward), "chat not found");
|
||||||
|
assert!(text.starts_with("Forward failed permanently"), "{text}");
|
||||||
|
assert!(text.contains("chat not found"), "{text}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn edit_prompt_text_states_the_ttl_and_the_confirm_requirement() {
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
let text = super::post_send::edit_prompt_text(Duration::from_secs(24 * 3600));
|
||||||
|
assert!(text.contains("Expires in 24h"), "{text}");
|
||||||
|
assert!(text.contains("Confirm"), "{text}");
|
||||||
|
// The wording of the whole point: no Confirm, no forward.
|
||||||
|
assert!(text.contains("Nothing is forwarded"), "{text}");
|
||||||
|
// Sub-hour TTLs must not render "0h".
|
||||||
|
assert!(
|
||||||
|
super::post_send::edit_prompt_text(Duration::from_secs(90)).contains("Expires in 1m")
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
super::post_send::edit_prompt_text(Duration::from_secs(30)).contains("Expires in 30s")
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
@@ -1311,7 +1395,11 @@ mod tests {
|
|||||||
post_send_actions(&ctx, &task, vec![10, 11]).await;
|
post_send_actions(&ctx, &task, vec![10, 11]).await;
|
||||||
|
|
||||||
assert_eq!(sender.calls(), vec!["send_message"]);
|
assert_eq!(sender.calls(), vec!["send_message"]);
|
||||||
assert_eq!(sender.messages(), vec!["Reply to edit message."]);
|
// The prompt explains the Confirm requirement and the TTL (see the
|
||||||
|
// pure `edit_prompt_text` test for the exact wording).
|
||||||
|
let prompt_text = sender.messages().first().cloned().unwrap_or_default();
|
||||||
|
assert!(prompt_text.contains("Expires in"), "{prompt_text}");
|
||||||
|
assert!(prompt_text.contains("Confirm"), "{prompt_text}");
|
||||||
// The prompt's own message id keys the record the reply will edit.
|
// The prompt's own message id keys the record the reply will edit.
|
||||||
let data = stores.chat_store().get(1).await;
|
let data = stores.chat_store().get(1).await;
|
||||||
let record = data
|
let record = data
|
||||||
|
|||||||
@@ -103,26 +103,76 @@ pub(crate) fn release_keep_alive(task: &Task) {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
/// One button per template name (column layout), then the confirm button.
|
/// The edit-before-forward prompt's text. It names both controls and the TTL,
|
||||||
/// Sorted by name: the templates live in a `HashMap`, so an unsorted walk
|
/// because the buttons alone left users waiting for a forward that never came
|
||||||
/// would reshuffle the buttons between prompts.
|
/// (nothing is forwarded until Confirm).
|
||||||
|
pub(super) fn edit_prompt_text(ttl: std::time::Duration) -> String {
|
||||||
|
format!(
|
||||||
|
"Reply to edit the caption, or tap a template, then ↩️ Confirm to forward. \
|
||||||
|
Expires in {}. Nothing is forwarded until you confirm.",
|
||||||
|
coarsest_unit(ttl)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Text the prompt is rewritten to once its record expires. The sweep edits
|
||||||
|
/// the prompt in place (see `main`): announcing the expiry with a new message
|
||||||
|
/// would wake the chat up to a full TTL later about a prompt nobody is
|
||||||
|
/// waiting on.
|
||||||
|
pub(crate) const EDIT_PROMPT_EXPIRED_TEXT: &str = "⌛ Expired — nothing was forwarded.";
|
||||||
|
|
||||||
|
/// `24h` / `90m` / `45s`: the coarsest whole unit, so the prompt stays short.
|
||||||
|
fn coarsest_unit(ttl: std::time::Duration) -> String {
|
||||||
|
let secs = ttl.as_secs();
|
||||||
|
if secs >= 3600 {
|
||||||
|
format!("{}h", secs / 3600)
|
||||||
|
} else if secs >= 60 {
|
||||||
|
format!("{}m", secs / 60)
|
||||||
|
} else {
|
||||||
|
format!("{secs}s")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Templates per keyboard row. Telegram rejects a keyboard with more than 100
|
||||||
|
/// buttons *outright*, which would silently drop the whole prompt, so the
|
||||||
|
/// names are folded and capped rather than listed one per row.
|
||||||
|
pub(super) const TEMPLATE_BUTTONS_PER_ROW: usize = 3;
|
||||||
|
/// Hard cap on template buttons; the prompt text names the ones not shown.
|
||||||
|
pub(super) const MAX_TEMPLATE_BUTTONS: usize = 60;
|
||||||
|
|
||||||
|
/// Template buttons ([`TEMPLATE_BUTTONS_PER_ROW`] per row, at most
|
||||||
|
/// [`MAX_TEMPLATE_BUTTONS`]), then the confirm/skip pair. Sorted by name: the
|
||||||
|
/// templates live in a `HashMap`, so an unsorted walk would reshuffle the
|
||||||
|
/// buttons between prompts.
|
||||||
pub(super) fn build_edit_markup(templates: &HashMap<String, String>) -> InlineKeyboardMarkup {
|
pub(super) fn build_edit_markup(templates: &HashMap<String, String>) -> InlineKeyboardMarkup {
|
||||||
let mut names: Vec<&String> = templates.keys().collect();
|
let mut names: Vec<&String> = templates.keys().collect();
|
||||||
names.sort();
|
names.sort();
|
||||||
let mut rows = Vec::with_capacity(names.len() + 1);
|
let shown = names.len().min(MAX_TEMPLATE_BUTTONS);
|
||||||
for name in names {
|
let mut rows = Vec::with_capacity(shown / TEMPLATE_BUTTONS_PER_ROW + 2);
|
||||||
rows.push(vec![InlineKeyboardButton::callback(
|
for chunk in names[..shown].chunks(TEMPLATE_BUTTONS_PER_ROW) {
|
||||||
name.clone(),
|
rows.push(
|
||||||
format!("template|{name}"),
|
chunk
|
||||||
)]);
|
.iter()
|
||||||
|
.map(|name| {
|
||||||
|
InlineKeyboardButton::callback(name.as_str(), format!("template|{name}"))
|
||||||
|
})
|
||||||
|
.collect(),
|
||||||
|
);
|
||||||
}
|
}
|
||||||
rows.push(vec![InlineKeyboardButton::callback(
|
// Skip exists because the prompt holds the forward hostage until Confirm:
|
||||||
"↩️ Confirm",
|
// without it the only escape was deleting the message and waiting out the
|
||||||
"forward",
|
// TTL for a forward that then never happens.
|
||||||
)]);
|
rows.push(vec![
|
||||||
|
InlineKeyboardButton::callback("↩️ Confirm", "forward"),
|
||||||
|
InlineKeyboardButton::callback("🛑 Skip", "skip"),
|
||||||
|
]);
|
||||||
InlineKeyboardMarkup::new(rows)
|
InlineKeyboardMarkup::new(rows)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// How many templates the markup could not fit, for the prompt text.
|
||||||
|
pub(super) fn hidden_template_count(templates: &HashMap<String, String>) -> usize {
|
||||||
|
templates.len().saturating_sub(MAX_TEMPLATE_BUTTONS)
|
||||||
|
}
|
||||||
|
|
||||||
/// Notifies a chat about a dead-lettered task (skips when `notify_chat_id` is
|
/// Notifies a chat about a dead-lettered task (skips when `notify_chat_id` is
|
||||||
/// absent).
|
/// absent).
|
||||||
pub(super) async fn notify_failure(
|
pub(super) async fn notify_failure(
|
||||||
@@ -185,12 +235,21 @@ pub(crate) async fn post_send_actions(ctx: &AppContext<'_>, task: &Task, message
|
|||||||
};
|
};
|
||||||
|
|
||||||
if edit_before_forward {
|
if edit_before_forward {
|
||||||
let keyboard = build_edit_markup(&ctx.chat_store.get(chat_id).await.template);
|
let templates = ctx.chat_store.get(chat_id).await.template;
|
||||||
|
let keyboard = build_edit_markup(&templates);
|
||||||
|
let mut text = edit_prompt_text(ctx.config.edit_message_ttl);
|
||||||
|
let hidden = hidden_template_count(&templates);
|
||||||
|
if hidden > 0 {
|
||||||
|
// The keyboard is capped; say so instead of silently hiding them.
|
||||||
|
text.push_str(&format!(
|
||||||
|
"\n({hidden} more templates not shown — /remove_template to prune.)"
|
||||||
|
));
|
||||||
|
}
|
||||||
let prompt = ctx
|
let prompt = ctx
|
||||||
.sender
|
.sender
|
||||||
.send_message(
|
.send_message(
|
||||||
ChatId(chat_id),
|
ChatId(chat_id),
|
||||||
"Reply to edit message.".to_string(),
|
text,
|
||||||
Some(MessageId(reply_to as i32)),
|
Some(MessageId(reply_to as i32)),
|
||||||
Some(keyboard),
|
Some(keyboard),
|
||||||
)
|
)
|
||||||
@@ -247,7 +306,7 @@ pub(crate) async fn post_send_actions(ctx: &AppContext<'_>, task: &Task, message
|
|||||||
ctx.sender,
|
ctx.sender,
|
||||||
notify_chat_id,
|
notify_chat_id,
|
||||||
notify_message_id,
|
notify_message_id,
|
||||||
&format!("Task failed after retries: {message}"),
|
&failure_text(None, &message),
|
||||||
)
|
)
|
||||||
.await;
|
.await;
|
||||||
}
|
}
|
||||||
@@ -341,6 +400,17 @@ async fn send_media_or_animation(ctx: &AppContext<'_>, task: &Task) -> Result<Ve
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// User-facing text for a task that will never run again: which link died and
|
||||||
|
/// why. The raw error alone left the user guessing which post it was about.
|
||||||
|
pub(super) fn failure_text(task: Option<&Task>, message: &str) -> String {
|
||||||
|
match task.and_then(|task| task.source_url()).map(log_key) {
|
||||||
|
Some(key) => format!("Send failed permanently for {key}: {message}"),
|
||||||
|
// `ForwardMessages` carries no source URL: that failure is about the
|
||||||
|
// channel copy, not about a post.
|
||||||
|
None => format!("Forward failed permanently: {message}"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Dead-letter callback wired to the queue in main: settles the task and
|
/// Dead-letter callback wired to the queue in main: settles the task and
|
||||||
/// notifies its chat.
|
/// notifies its chat.
|
||||||
pub(crate) async fn dead_letter_notify(
|
pub(crate) async fn dead_letter_notify(
|
||||||
@@ -351,8 +421,9 @@ pub(crate) async fn dead_letter_notify(
|
|||||||
// A dead-lettered task never runs again, and the queue dead-letters retry
|
// A dead-lettered task never runs again, and the queue dead-letters retry
|
||||||
// exhaustion itself (the handler is not called again), so this is the only
|
// exhaustion itself (the handler is not called again), so this is the only
|
||||||
// place that sees the final payload.
|
// place that sees the final payload.
|
||||||
if let Ok(task) = serde_json::from_value::<Task>(payload.clone()) {
|
let task = serde_json::from_value::<Task>(payload.clone()).ok();
|
||||||
settle_task(ctx, &task, Settled::Failed).await;
|
if let Some(task) = &task {
|
||||||
|
settle_task(ctx, task, Settled::Failed).await;
|
||||||
}
|
}
|
||||||
let notify_chat_id = payload.get("notify_chat_id").and_then(|v| v.as_i64());
|
let notify_chat_id = payload.get("notify_chat_id").and_then(|v| v.as_i64());
|
||||||
let notify_message_id = payload.get("notify_message_id").and_then(|v| v.as_i64());
|
let notify_message_id = payload.get("notify_message_id").and_then(|v| v.as_i64());
|
||||||
@@ -360,7 +431,7 @@ pub(crate) async fn dead_letter_notify(
|
|||||||
ctx.sender,
|
ctx.sender,
|
||||||
notify_chat_id,
|
notify_chat_id,
|
||||||
notify_message_id,
|
notify_message_id,
|
||||||
&format!("Task failed after retries: {message}"),
|
&failure_text(task.as_ref(), &message),
|
||||||
)
|
)
|
||||||
.await;
|
.await;
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user