mirror of
https://github.com/TheFunny/TelegramTwitterMediaBot.git
synced 2026-09-25 23:52:04 +00:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
fb601f4d5d
|
||
|
|
d8dd4fa91e
|
||
|
|
af96caff40
|
||
|
|
52184ba6fb
|
||
|
|
0eb4e5c78d
|
||
|
|
5c51de217a
|
||
|
|
c1f5d3ca54
|
||
|
|
1bb6968108 | ||
|
|
14b444d109
|
||
|
|
de3105d4cd | ||
|
|
c46103a23f | ||
|
|
9529f64b41
|
||
|
|
d0de17329d | ||
|
|
9af37e92b4 | ||
|
|
e68a1dbd30 | ||
|
|
b9c6d16ff0
|
@@ -80,6 +80,11 @@ jobs:
|
||||
echo "build=true" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
# No `actions/checkout` here on purpose: `docker/build-push-action` defaults
|
||||
# to the Git context (`https://github.com/<owner>/<repo>.git#<ref>`), so
|
||||
# BuildKit clones the repo itself and authenticates with the automatic
|
||||
# github.token. Adding `context: .` below without a checkout step would hand
|
||||
# BuildKit an empty workspace.
|
||||
docker:
|
||||
needs: should-build
|
||||
if: needs.should-build.outputs.build == 'true'
|
||||
|
||||
@@ -2,9 +2,9 @@
|
||||
|
||||
## Project Overview
|
||||
|
||||
Telegram bot (teloxide) that turns post links from X/Twitter, Pixiv, Bluesky, and Misskey (misskey.io) into media messages (images, video, GIF) with the post's title, author, and tags. It supports batch media splitting, retry with persistence, inline queries, forward-channel rebinding with caption templates, and Pixiv ugoira→MP4 transcoding. README is in Chinese; user-facing bot strings are in English. The project is a Rust port of a Python predecessor (see `queue.rs` comments referencing `utils/task_queue.py`).
|
||||
Telegram bot (teloxide) that turns post links from X/Twitter, Pixiv, Bluesky, Misskey (misskey.io), and Bilibili dynamics into media messages (images, video, GIF) with the post's title, author, and tags. It supports batch media splitting, retry with persistence, inline queries, forward-channel rebinding with caption templates, and Pixiv ugoira→MP4 transcoding. README is in Chinese; user-facing bot strings are in English. The project is a Rust port of a Python predecessor (see `queue.rs` comments referencing `utils/task_queue.py`).
|
||||
|
||||
Two-crate Cargo workspace (both v1.6.0, edition 2024, resolver 3):
|
||||
Two-crate Cargo workspace (both v1.7.0, edition 2024, resolver 3):
|
||||
|
||||
- **`crates/x-media`** — library that fetches and normalizes media from the four sites. Pure, no Telegram knowledge.
|
||||
- **`crates/xmedia-bot`** — the bot binary: teloxide dispatcher, SQLite-backed chat state, persistent task queue.
|
||||
@@ -24,14 +24,14 @@ Debug command: `/debug <url>` runs the same `x_media::site::fetch` and replies w
|
||||
|
||||
The `/test <url>` command runs the ordinary link pipeline (`urls::url_media`) with `PostSend::Suppressed`: the media is sent and cached like any other link, but the chat's `forward_channel_id`/`edit_before_forward` are ignored, so a test never forwards to the channel and never opens the edit prompt (retries and dead-letter notifications behave as usual). Both commands use a custom `parse_arg_remainder` parser (whole remainder, trimmed) because teloxide's built-in `split` parser takes exactly one space-separated token.
|
||||
|
||||
The `x-media` library: `site::fetch(url)` dispatches through the `SITES` registry (per-site `impl Site`, in order twitter → bsky → misskey → pixiv) and returns `Ok(None)` for unmatched URLs. `Fetched { source_url, caption, title, media: Vec<Media>, sensitive, site_id, … }`; `caption_with(format)` substitutes `{url} {author} {author_url} {title} {tags}`.
|
||||
The `x-media` library: `site::fetch(url)` dispatches through the `SITES` registry (per-site `impl Site`, in order twitter → bsky → misskey → pixiv → bilibili) and returns `Ok(None)` for unmatched URLs. `Fetched { source_url, caption, title, content, media: Vec<Media>, sensitive, site_id, … }` (title and content are split per platform: a pixiv artwork's title and description, a bilibili headline and body, and text-only posts whose text is all `content`); `caption_with(format)` substitutes `{url} {author} {author_url} {title} {content} {tags}`.
|
||||
|
||||
## Key Directories
|
||||
|
||||
| Path | Purpose |
|
||||
|---|---|
|
||||
| `crates/x-media/src/` | Fetch library. `site/mod.rs` = dispatcher + `Fetched`/`FetchError`/`download_media`/`media_size`; `media.rs` = `Media` enum; `examples/fetch.rs` = end-to-end usage sample |
|
||||
| `crates/x-media/src/site/<twitter\|pixiv\|bsky\|misskey>/` | One directory per site: `mod.rs` (re-exports), `interface.rs` (PATTERN, `enabled()`, `fetch_from_url()`, `cache_key`/`is_retryable`/`media_headers`, unit struct `<Name>Site` implementing `site::Site`, `From<SiteStruct> for Fetched`), `model.rs` (serde DTOs). Pixiv adds `api.rs` (auth + transport); twitter adds `auth.rs` (logged-in GraphQL `TweetDetail` fallback for NSFW tweets, gated on `TWITTER_AUTH_TOKEN`). Misskey targets misskey.io only (`POST /api/notes/show`, 400+`NO_SUCH_NOTE` → NotFound). Twitter's `from_syndication_json` HTML-decodes the API text — syndication and GraphQL `full_text` both arrive pre-escaped (`>` `<` `&` `'`) — so the stored text is raw and the caption escapes exactly once |
|
||||
| `crates/x-media/src/site/<twitter\|pixiv\|bsky\|misskey\|bilibili>/` | One directory per site: `mod.rs` (re-exports), `interface.rs` (PATTERN, `enabled()`, `fetch_from_url()`, `cache_key`/`is_retryable`/`media_headers`, unit struct `<Name>Site` implementing `site::Site`, `From<SiteStruct> for Fetched`), `model.rs` (serde DTOs). Pixiv adds `api.rs` (auth + transport); twitter adds `auth.rs` (logged-in GraphQL `TweetDetail` fallback for NSFW tweets, gated on `TWITTER_AUTH_TOKEN`). Misskey targets misskey.io only (`POST /api/notes/show`, 400+`NO_SUCH_NOTE` → NotFound). Bilibili fetches dynamics (images/animated images only — an attached video degrades to its cover, and its title stands in for the post text, which AV dynamics do not have) from `/x/polymer/web-dynamic/v1/detail` sent with `features=itemOpusStyle` (without that flag the legacy serialization drops an image/text post's body and headline entirely — `desc` comes back `null`; the adapter still parses the legacy `major.draw`/`desc`/`archive` shapes as a fallback). No WBI signature is involved; device cookies `buvid3`/`buvid4` are fetched automatically from `/x/frontend/finger/spi` because bilibili's `-352` risk control starts rejecting plain requests, `BILIBILI_COOKIE` is the escalation when an IP stays blocked; `b23.tv` short links are deliberately unmatched. Twitter's `from_syndication_json` HTML-decodes the API text — syndication and GraphQL `full_text` both arrive pre-escaped (`>` `<` `&` `'`) — so the stored text is raw and the caption escap…
|
||||
| `crates/xmedia-bot/src/main.rs` | Entry point: env/log init, command registration (`register_commands`), shared `send::BOT` force-init, queue worker start, site login validation (`site::validate_all`), 300 s edit-expiry sweep, dptree handler tree, webhook vs polling dispatch |
|
||||
| `crates/xmedia-bot/src/config.rs` | Manual env parsing into `Config` |
|
||||
| `crates/xmedia-bot/src/db.rs` | `DbPool`: one shared SQLite connection pool (`POOL_SIZE = 4`, WAL, busy_timeout) for all three tables over `$DATA_DIR/task_queue.db` (default `data/`) — the three stores share it; `open_store` creates file + schema, `with_conn` runs all rusqlite I/O in `spawn_blocking` |
|
||||
@@ -92,16 +92,16 @@ Docker: `docker build -t tgxmb .` then `docker run --rm -d --name tgxmb --env-fi
|
||||
- Package manager: **Cargo** (workspace with path dep `x-media` ← `xmedia-bot`). No `[workspace.package]`/shared deps — each crate lists deps independently.
|
||||
- **TLS is rustls end-to-end** (no native-tls/openssl in the tree, no libssl in the Docker runtime image): `teloxide` is declared `default-features = false` with `["webhooks-axum", "macros", "rustls", "ctrlc_handler"]` (the removed `default` also carried `native-tls` and `ctrlc_handler` — the latter must stay); x-media's reqwest is `default-features = false` with `["json", "rustls-tls"]` (webpki-roots baked in, so the image ships no CA bundle). One reqwest 0.12.28 in the lock.
|
||||
- **Versioning**: bump the version in all three places (`crates/x-media/Cargo.toml`, `crates/xmedia-bot/Cargo.toml`, `Cargo.lock`) and **keep `README.md`, `README.en.md` and `AGENTS.md` in sync with the code on every bump**, then commit (`chore: bump version to X.Y.Z`), create an annotated tag `vX.Y.Z`, and push branch + tag (the tag push triggers the Docker Hub build). The tag must equal both crate versions: `.github/workflows/docker.yml` verifies that before building, and `--locked` verifies the lock file.
|
||||
- Config is **environment-variable driven** (dotenv loads `.env`, gitignored; no `.env.example` exists). Key vars: `TELOXIDE_TOKEN` (required), `PIXIV_REFRESH_TOKEN`, `TWITTER_AUTH_TOKEN` (optional; x.com `auth_token` cookie — enables the logged-in GraphQL fallback that fetches NSFW tweets syndication withholds), `BOT_ADMIN` (comma-separated ids), `EDIT_MESSAGE_TTL_SECONDS` (default 86400), `LINK_CACHE_TTL_SECONDS` (default 604800), `DATA_DIR` (default `data`, CWD-relative; the SQLite dir, auto-created), `WEBHOOK`/`WEBHOOK_URL`/`WEBHOOK_LISTEN`/`WEBHOOK_PORT`/`WEBHOOK_CERT`/`WEBHOOK_SECRET_TOKEN` (webhook mode requires URL/listen/port, `.expect`ed; `WEBHOOK_CERT` is Telegram-facing self-signed validation only — TLS must be terminated by a reverse proxy), `RUST_LOG`, `TELOXIDE_PROXY`, `LOCAL_USER_ID` (entrypoint only).
|
||||
- Config is **environment-variable driven** (dotenv loads `.env`, gitignored; no `.env.example` exists). Key vars: `TELOXIDE_TOKEN` (required), `PIXIV_REFRESH_TOKEN`, `TWITTER_AUTH_TOKEN` (optional; x.com `auth_token` cookie — enables the logged-in GraphQL fallback that fetches NSFW tweets syndication withholds), `BILIBILI_COOKIE` (optional; whole bilibili cookie string — bilibili dynamics fetch anonymously and add their own device cookies, this only rescues an egress IP that bilibili has hard-flagged with `-352`/412), `BOT_ADMIN` (comma-separated ids), `EDIT_MESSAGE_TTL_SECONDS` (default 86400), `LINK_CACHE_TTL_SECONDS` (default 604800), `CAPTION_QUOTE_TEXT_CHARS` (default 200; a post whose text — the `title` plus `content` joined, see `site::compose_text` — reaches this length gets that text wrapped in an expandable blockquote inside its caption, the URL and author line staying outside; `0` disables it. Applied at the send boundary in `send::quote_long_caption`, which locates the text as what follows the author link, so a `/set_format` that moves `{title}`/`{content}` elsewhere and pixiv's title-inside-a-link layout opt out; `copy_messages` forwards and queued retries inherit the wrap, while the edit-before-forward rewrite stays unquoted by design), `DATA_DIR` (default `data`, CWD-relative; the SQLite dir, auto-created), `WEBHOOK`/`WEBHOOK_URL`/`WEBHOOK_LISTEN`/`WEBHOOK_PORT`/`WEBHOOK_CERT`/`WEBHOOK_SECRET_TOKEN` (webhook mode requires URL/listen/port, `.expect`ed; `WEBHOOK_CERT` is Telegram-facing self-signed validation only — TLS must be terminated by a reverse proxy), `RUST_LOG`, `TELOXIDE_PROXY`, `LOCAL_USER_ID` (entrypoint only).
|
||||
- SQLite via `rusqlite` with `bundled` feature (no system libsqlite needed). DB file `$DATA_DIR/task_queue.db` (default `data/task_queue.db`, CWD-relative — run from the workspace root, or `/app` in Docker; set `DATA_DIR` to pin state anywhere). Mount `./data` and `./cert` volumes.
|
||||
- `.gitattributes` enforces LF for `*.sh` (CRLF breaks shebangs in containers). `.gitignore`: `.env`, `data/`, `cert/`, `docker-compose.yml`, `/target`, `.idea/`.
|
||||
- Docs are in Chinese (README, AGENTS.md); user-facing bot strings are in English. Keep that split when editing user-facing strings and docs.
|
||||
|
||||
## Testing & QA
|
||||
|
||||
- **~135 tests, all inline `#[cfg(test)] mod tests`** — no `tests/` integration directories. Framework: built-in Rust test + `#[tokio::test]` (dev-deps only in `x-media`: tokio macros/rt-multi-thread, dotenv).
|
||||
- **~180 tests, all inline `#[cfg(test)] mod tests`** — no `tests/` integration directories. Framework: built-in Rust test + `#[tokio::test]` (dev-deps only in `x-media`: tokio macros/rt-multi-thread, dotenv).
|
||||
- No mocking framework anywhere (no mockito/wiremock/mockall). Conventions: pure-function units (regex parsing, serde round-trips, chunking, retry math) tested synchronously; async tests use real dependencies — file-backed SQLite via `tempfile` (`queue.rs::new_queue()` helper), live network fetches.
|
||||
- Live-network tests exist in `site/twitter/interface.rs` (5), `site/bsky/interface.rs` (2), `site/misskey/interface.rs` (1), `site/pixiv/api.rs` (1); `photo.rs` adds one `#[ignore = "heavy: …"]` test. `site/mod.rs` also has a **token-gated but not `#[ignore]`d** pixiv download test (`download_media_pixiv_original_with_referer`): it hits `i.pximg.net` whenever `PIXIV_REFRESH_TOKEN` is set, so a local `cargo test --workspace` is not fully offline and can flake on a pixiv CDN body timeout. Test gating convention (enforced by `.github/workflows/ci.yml`): pure unit tests always run; live-network tests carry `#[ignore = "live network: ..."]` (run via `cargo test --workspace -- --ignored live`); token-gated pixiv tests early-return when `PIXIV_REFRESH_TOKEN` is absent **or empty** (an unset GitHub secret arrives as `""` — `is_err()` alone would run them tokenless and fail). Run the full offline suite with `cargo test --workspace`.
|
||||
- Live-network tests exist in `site/twitter/interface.rs` (5), `site/bsky/interface.rs` (2), `site/misskey/interface.rs` (1), `site/bilibili/interface.rs` (4), `site/pixiv/api.rs` (1); `photo.rs` adds one `#[ignore = "heavy: …"]` test. `site/mod.rs` also has a **token-gated but not `#[ignore]`d** pixiv download test (`download_media_pixiv_original_with_referer`): it hits `i.pximg.net` whenever `PIXIV_REFRESH_TOKEN` is set, so a local `cargo test --workspace` is not fully offline and can flake on a pixiv CDN body timeout. Test gating convention (enforced by `.github/workflows/ci.yml`): pure unit tests always run; live-network tests carry `#[ignore = "live network: ..."]` (run via `cargo test --workspace -- --ignored live`); token-gated pixiv tests early-return when `PIXIV_REFRESH_TOKEN` is absent **or empty** (an unset GitHub secret arrives as `""` — `is_err()` alone would run them tokenless and fail), and the bilibili live tests early-return when the API answers risk control (`-352`, which bilibili applies per IP by request volume). Run the full offline suite with `cargo test --workspace`.
|
||||
- Fixtures are inline `serde_json::json!` builder fns (`fixture()`, `thread_json()`, `illust_json()`), not files. The shared `CLIENT` sets `pool_max_idle_per_host(0)` under `#[cfg(test)]` to avoid cross-runtime `DispatchGone`.
|
||||
- **CI** — `.github/workflows/ci.yml` (actions pinned to commit SHAs, `--locked` on every cargo invocation, `concurrency` cancels superseded runs, `RUST_BACKTRACE=1`) runs `cargo fmt --check` + `cargo clippy --workspace --all-targets --locked -- -D warnings` + `cargo test --workspace --locked` + a release-profile `cargo build --release --locked` + an `actions-rust-lang/audit` dependency-vulnerability gate (offline, no secrets, on every push/PR) and a `live` job (schedule/manual/tag only, `-p x-media` since every network/secret-gated test lives there, `continue-on-error`) for the `#[ignore]`d live + token tests. `.github/workflows/docker.yml` builds and pushes the image on master/tag and runs a **build-only check on pull requests touching the build inputs** (`Dockerfile`, entrypoint, manifests, `.dockerignore`); a release tag must match both crate versions or the build stops, and `FFMPEG_URL`/`FFMPEG_SHA256` are taken from repository variables when set (a release can pin an exact ffmpeg build). `.github/dependabot.yml` keeps crates, the pinned actions and the Docker base images current.
|
||||
- Untested and hard to test without a mock seam: `main.rs`, `config.rs`, `db.rs`, `handlers/statics.rs`, `media_sender.rs` (holds the `MockSender` itself); in `x-media`: `media.rs`, `lib.rs`, all `model.rs`. The `commands.rs` *executor* needs a real `Bot` (only its pure report builder is tested). Everything else — `handlers/{mod,callback,inline,urls}.rs`, `send/*`, `ctx.rs`, `state.rs`, `queue.rs`, `link_cache.rs`, `rate_limit.rs` — is driven through `TestStores`/`ctx::test_support` and the scripted `MockSender`.
|
||||
|
||||
@@ -0,0 +1,168 @@
|
||||
# Bilibili 动态支持:研究与实现记录
|
||||
|
||||
状态:已实现(`crates/x-media/src/site/bilibili/`)。本文记录上游调研、实测数据与最终设计;
|
||||
长期契约以 `AGENTS.md` 为准。
|
||||
|
||||
范围:**只发动态里的图片与动图**。动态内嵌视频不发流,降级为封面图;`b23.tv` 短链不匹配;
|
||||
视频页 / 番剧 / 直播间 / 专栏 / 音频均不支持。
|
||||
|
||||
---
|
||||
|
||||
## 1. 上游实现研究
|
||||
|
||||
### 1.1 nazurin(`nazurin/sites/bilibili/`,4 个文件 ~6 KB)
|
||||
|
||||
- 入口正则:`t\.bilibili\.com/(\d+)`、`t\.bilibili\.com/h5/dynamic/detail/(\d+)`、`bilibili\.com/opus/(\d+)`。
|
||||
- 请求:`GET https://api.bilibili.com/x/polymer/web-dynamic/v1/detail?id={id}`,仅加 `Referer: https://t.bilibili.com/{id}`。
|
||||
**无 cookie、无 WBI 签名、无 `build` 参数**。
|
||||
- 错误:`code == 4101147` → not found;`code != 0` 或缺 `data` → 报错。
|
||||
- 媒体:只取 `item.modules.module_dynamic.major.draw.items[].src`;缩略图 `src + "@518w.jpg"`;
|
||||
`size` 字段单位是 **KB**。`major` 为空或 `draw.items` 为空 → "No image found"。
|
||||
**忽略视频、转发(forward)与纯文字动态**。
|
||||
- caption:`"#" + module_author.name` + `module_dynamic.desc.text`,链接写死 `https://www.bilibili.com/opus/{id}`。
|
||||
|
||||
### 1.2 telegram-bili-feed-helper(`biliparser/provider/bilibili/`,9 个文件 ~57 KB)
|
||||
|
||||
- 9 个策略类(Video/Opus/Live/Audio/Read + Feed 基类 + Credential + api 工具):门禁正则
|
||||
`bilibili\.com|b23\.tv|BV\w{10}|av\d+`,再分流,兜底 `client.head(url)` 跟随重定向后按子串分流。
|
||||
- 动态:`GET /x/polymer/web-dynamic/desktop/v1/detail?id={id}&build=11605`(**单条,无分页**);
|
||||
客户端带桌面 UA、随机 `buvid3={uuid}infoc`;登录态用 `bilibili-api-python` 的 `Credential`
|
||||
(Redis 持久化 `SESSDATA/bili_jct/buvid3/buvid4/ac_time_value/DedeUserID`,扫码登录)。
|
||||
- **同样没有 WBI 签名 / appkey 签名**:playurl 用的是非 WBI 的 `/x/player/playurl`。
|
||||
- 媒体:`major.type` 分派 —— DRAW 取全部 `items[].src`;ARCHIVE/PGC/ARTICLE/MUSIC/COMMON/LIVE
|
||||
只取一张 `cover`;FORWARD 取原动态作者/正文并递归进 `orig` 找媒体。
|
||||
- 视频:仅独立 video 策略解析(`qn` 720P→480P→360P 试 durl,再退 DASH + ffmpeg 合并);
|
||||
**动态内嵌视频只发封面**。
|
||||
- 错误:要求 `status==200 && code==0`;风控 `-352`/`-412` 无特殊处理。
|
||||
|
||||
### 1.3 取舍
|
||||
|
||||
| 维度 | nazurin | bff | 本仓库 |
|
||||
|---|---|---|---|
|
||||
| 接口 | `v1/detail?id=` | `desktop/v1/detail?id=&build=` | `v1/detail?id=`(实测可用) |
|
||||
| 认证 | 无 | buvid3 + SESSDATA | 默认匿名;可选 `BILIBILI_COOKIE` |
|
||||
| WBI | 无 | 无 | 不实现(无需求) |
|
||||
| 图片 | `major.draw.items` | 同 + forward 递归 | 同,加 `orig` 递归、`http→https`、`.gif → Animated` |
|
||||
| 视频 | 完全忽略 | 动态内嵌视频发封面 | 发封面(不发流) |
|
||||
| 短链 | 不匹配 | 跟随重定向 | 不匹配(多数短链是视频,会让"静默忽略"变成失败提示) |
|
||||
|
||||
---
|
||||
|
||||
## 2. 实测验证(2026-09-17,真实请求)
|
||||
|
||||
| 验证项 | 结果 |
|
||||
|---|---|
|
||||
| `v1/detail?id=`(无 cookie、UA `Mozilla/5.0`、带 Referer) | `200 {"code":0}` ✅ |
|
||||
| 同上,不带 cookie 也不带 Referer | `200 {"code":0}` ✅(无强制鉴权) |
|
||||
| bff 的 `bilibili_pc/…Electron/22.3.27` UA | `code:-352` ❌ → **不要抄它的 UA** |
|
||||
| `desktop/v1/detail?build=11605` | `code:-352` ❌ |
|
||||
| `feed/space?host_mid=`(用户时间线) | 首次成功、随后 `-352`,也见过 HTTP 412 → **不碰** |
|
||||
| 不存在 / 已删除的动态 | `code:500` "Cannot read property 'only_fans' of undefined"(nazurin 的 4101147 已失效) |
|
||||
| 非数字 id | `code:-400` param parsing failed |
|
||||
| 图片 `i0.hdslb.com/bfs/new_dyn/*.jpg` | `HEAD 200 image/jpeg`,带/不带 Referer 均可;`+@518w.jpg` → 25–42 KB ✅ |
|
||||
| `t.bilibili.com/h5/dynamic/detail/<id>` | `200` ✅ |
|
||||
| `m.bilibili.com/dynamic/<id>` | `302 → t.bilibili.com/<id>` ✅ |
|
||||
| `www.bilibili.com/opus/<id>` | `200`,转发动态 `302 → t.bilibili.com/<id>` ✅ |
|
||||
| `b23.tv/BV1JTtt6JEZu` | `302 → www.bilibili.com/video/BV…`(视频) |
|
||||
| `b23.tv/<无效码>` | **HTTP 200** + `{"code":-404}` ⚠️ 短链判定不能只看状态码 |
|
||||
| `playurl`(仅调研用,未采用) | `fnval=1` 匿名给 durl:720P=9.18 MiB / 360P=2.97 MiB;`fnval=4048` 匿名 DASH 上限仅 480P |
|
||||
| `dyn_archive` 字段 | 有 `aid/bvid/cover/title/duration_text`,**没有 `cid`**(所以发流要再来一次 `view` 请求) |
|
||||
| **风控阶梯(同一 IP 连续请求后实测)** | ① 无 cookie → `-352`;② 仅 `buvid3` → 仍 `-352`;③ `buvid3`+`buvid4`(取自匿名 `/x/frontend/finger/spi`)→ **`code:0` 恢复**;④ 继续高频请求后 → 连同 buvid 一起 `-352`(此时只有登录 cookie 或换 IP) |
|
||||
| **正文位置(24 条真实动态逐条审计)** | 有正文的动态都在 `module_dynamic.desc.text`(图文/转发/纯文字,含 34–193 字样本);**AV(视频投稿)动态 `desc` 恒为 `null`**,内容在 `major.archive.title` / `.desc` 卡片里 → 已做 title 回退 |
|
||||
| **`features=itemOpusStyle` 的效果** | 同一端点带此参数后,图文帖改为 `major.opus` 形态:`pics[]`(图,key 是 `url`)、`summary.text`(正文,未截断,实测 307 字整段)、`title`(可选标题);不带参数则是 legacy `major.draw` + `desc`,而 **opus 图文帖的 `desc` 为 `null`、正文与标题完全丢失**(`opus/1248857553488576532`:legacy `desc:null`,带参数 `summary.text="[doge_金箍]黑白搭配"`)。AV / 转发帖不受该参数影响 → 适配器改为请求时带参数,并保留 legacy 形态兜底 |
|
||||
| feed 与 detail 的差异 | `feed/space` 的 item 会把 `desc.text` 挖空,**只有 detail 有正文** → 排查时不要用 feed 数据判断正文缺失 |
|
||||
| 不存在的 19 位 id | `4101105 请求数据发生错误`(提示可重试,但只出现在不可能存在的 id 上)→ 仍归入永久错误,见 `code_error` 注释 |
|
||||
|
||||
测试样本(live 测试用):
|
||||
|
||||
| 样本 | id | 期望 |
|
||||
|---|---|---|
|
||||
| 图片动态(2 图 + 话题) | `1245284537985925159` | 2 个 `Illustration`,`{tags}` = `ALin出道20周年快乐` |
|
||||
| 转发动态 | `1248982077447077907` | 媒体来自 `orig`(1 图),正文可含 `//@` |
|
||||
| 视频动态 | `1248717597691609105` | 封面 1 张 `Illustration` |
|
||||
| 纯文字动态 | `1246767523595026450` | `media` 为空 |
|
||||
|
||||
关键字段路径:
|
||||
|
||||
```
|
||||
data.item.id_str
|
||||
data.item.modules.module_author.{name,mid}
|
||||
data.item.modules.module_dynamic.desc.text
|
||||
data.item.modules.module_dynamic.topic.{id,name} # 单话题,{tags} 来源
|
||||
data.item.modules.module_dynamic.major.{draw.items[].src, archive.cover}
|
||||
data.item.orig # 转发时存在,结构与 item 相同
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. 实现
|
||||
|
||||
```
|
||||
crates/x-media/src/site/bilibili/mod.rs # re-export
|
||||
crates/x-media/src/site/bilibili/interface.rs # PATTERN / cache_key / enabled / is_retryable /
|
||||
# media_headers / BilibiliSite / fetch / code_error /
|
||||
# From<Item> for Fetched / caption / 12 单测 + 2 live
|
||||
crates/x-media/src/site/bilibili/model.rs # 纯 Deserialize DTO(全 Option)
|
||||
```
|
||||
|
||||
- **正则**(同时用于分发、抽 id、缓存键,一个正则三用):
|
||||
`^(?:https?://)?(?:www|t|m)\.bilibili\.com/(?:opus/|dynamic/|h5/dynamic/detail/)?(\d+)`
|
||||
- **缓存键**:`bilibili:<动态 id>`;`source_url` 统一 `https://www.bilibili.com/opus/{id}`。
|
||||
- **请求**:`GET /x/polymer/web-dynamic/v1/detail?id=` + `Referer: https://www.bilibili.com/`;
|
||||
`Cookie` 头按优先级取:`BILIBILI_COOKIE` → 缓存的设备 cookie(`GET /x/frontend/finger/spi` 取 `buvid3`/`buvid4`,
|
||||
进程内缓存一次;取不到就不带 cookie,仅 debug 日志)→ 无。指纹接口本身失败**不**让抓取失败。
|
||||
走共享 `CLIENT`(UA `Mozilla/5.0`,30s 超时,`TELOXIDE_PROXY` 透传)。
|
||||
- **错误映射**:`0` → 成功;`-352/-412` 与 HTTP 412 → `Transient`(可重试,队列退避;首次记一条 warn 提示
|
||||
`BILIBILI_COOKIE`);`500`/`4101147` → `NotFound`(永久);其他 code → `Site`(永久)。
|
||||
- **媒体**:
|
||||
- `major.opus.pics[]`(带 `features=itemOpusStyle` 时的图文帖形态,字段名是 `url`)→ 每张一张图;
|
||||
其次 `major.draw.items[]`(legacy,字段名 `src`)→ 同样逐张;`http://` / `//` → `https://`,非 https 开头直接丢弃。
|
||||
`.gif` → `Media::Animated`(`thumbnail_url` 留空,Telegram 自己取首帧——`@518w.jpg` 只对 jpg/webp 实测过),
|
||||
其余 → `Media::Illustration`(`thumbnail_url = url + "@518w.jpg"`,兼作超大时的降级 URL)。
|
||||
- `major.archive.cover` → 1 张 `Illustration`(视频不发流)。
|
||||
- 转发且自身无媒体 → 递归取 `orig` 的媒体;正文拼 `//@{原作者}:\n{原文}`。
|
||||
- 其他 major(PGC/ARTICLE/MUSIC/LIVE/COMMON)不建模 → 无媒体,走既有 "No media found"。
|
||||
- **正文 / title**(按信息量从多到少回退):`major.opus.title` + `major.opus.summary.text`
|
||||
→ `module_dynamic.desc.text` → `major.archive.title`。三者分别对应:图文文档(标题+正文)、
|
||||
legacy/转发帖正文、视频投稿卡片标题。开头结尾空白做 trim;整体再由既有 `truncate_caption` 截断。
|
||||
- **caption**(与 misskey 同形):`{opus 链接}\n<a href="space.bilibili.com/{mid}">{name}</a>: {正文}`;
|
||||
`RenderData` 的 `{tags}` 来自话题名;正文由既有 `truncate_caption` 截断。
|
||||
- **注册表**:`SITES` 末尾追加 → `/set_format` 白名单、链接缓存、启动校验、日志前缀全部自动生效。
|
||||
- **bot 侧仅文案**:`handlers/commands.rs` 三处站点清单字符串 + `state.rs`/`handlers/mod.rs` 注释。
|
||||
|
||||
### 与原计划的偏差(及原因)
|
||||
|
||||
| 原计划 | 实际 | 原因 |
|
||||
|---|---|---|
|
||||
| `x/web-interface/view` + `playurl` 发视频 | 不做 | 需求收窄为图片/动图;视频只发封面 |
|
||||
| `site/mod.rs` 加 `MAX_MEDIA_UPLOAD_BYTES` 常量 | 不加 | 没有视频尺寸决策就不需要该常量,避免跨 crate 耦合 |
|
||||
| `b23.tv` 短链(跟随重定向) | 不匹配 | 多数短链指向视频,匹配后会把"静默忽略"变成用户的 "Failed to fetch media" |
|
||||
| `validate()` 校验 cookie | 不做 | 匿名可用,cookie 失效不致命;校验要额外请求一个端点,收益低 |
|
||||
| `media_headers` 给 hdslb 加 Referer | 返回 `None` | 实测图片与 durl 均无需 Referer(注释里记了这条验证) |
|
||||
| 计划阶段认为设备 cookie 是 YAGNI,不实现 | **实现**(`buvid3`+`buvid4`) | 计划之后做了对照实验:同一 IP 上"无 cookie → -352、只有 buvid3 → -352、buvid3+buvid4 → code:0",说明这是对本适配器主要失败模式的直接修复,而不是冗余保险 |
|
||||
| 只用不带参数的 `v1/detail` | 加 `features=itemOpusStyle` | 用户实测反馈"有内容的动态没有 title":不带参数时 opus 图文帖返回 legacy 形态,`desc` 为 `null`,正文与标题整个丢失。带参数后同一 ID 返回 `major.opus.summary.text` / `title` / `pics`。AV / 转发帖不受影响,legacy 形态仍保留为兜底 |
|
||||
|
||||
---
|
||||
|
||||
## 4. 测试与验证
|
||||
|
||||
- 单元(13):正则匹配/拒绝/忽略短链、缓存键归一、图片映射(https 归一 + 缩略图 + `.gif → Animated`)、
|
||||
封面、转发取 `orig` 媒体与正文拼接、纯文字无媒体、caption 转义、业务 code 分类(可重试性)、URL 归一、
|
||||
设备 cookie 拼装。
|
||||
- live(3,`#[ignore = "live network: …"]`):设备 cookie 可取、图片动态 2 图、纯文字动态无媒体。
|
||||
CI 的 `live` job 已覆盖。动态接口被风控时这两条 live 测试打印 `skipping:` 并提前返回(与 pixiv 的
|
||||
token 门控同款约定),设备 cookie 那条仍会真实执行。
|
||||
- 实测命令:
|
||||
`cargo run -p x-media --example fetch -- https://www.bilibili.com/opus/1245284537985925159`
|
||||
(输出 2 张 `https://i0.hdslb.com/…jpg` + `@518w.jpg` 缩略图 + 话题 tags)。
|
||||
- 全套:`cargo fmt --check`、`cargo clippy --workspace --all-targets -- -D warnings`、`cargo test --workspace` 全绿。
|
||||
|
||||
## 5. 已知限制
|
||||
|
||||
- 风控按 IP/请求量漂移,阶梯见 §2 最后一行:轻度靠设备 cookie 自愈,重度需 `BILIBILI_COOKIE` 或换 IP。
|
||||
被拦时按**可重试**失败处理(队列退避)+ 一条 warn,不会静默丢帖。
|
||||
- 接口 schema 会漂移(`module_dynamic.major` 实测可为 `null` 而正文留在 `desc`);DTO 全 `Option`,
|
||||
未知形态降级为"无媒体",不 panic。
|
||||
- 动态内嵌视频只发封面图(与 bff 同策略),不下载流。
|
||||
- 纯文字动态复用既有 "No media found" 回复。
|
||||
- `b23.tv` 短链不被匹配(见上表)。
|
||||
Generated
+154
-221
@@ -10,25 +10,13 @@ checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa"
|
||||
|
||||
[[package]]
|
||||
name = "aes"
|
||||
version = "0.8.4"
|
||||
version = "0.9.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b169f7a6d4742236a0a00c541b845991d0ac43e546831af1249753ab4c3aa3a0"
|
||||
checksum = "35f0f96ce78e38c3dc6d8948aa8163d06385be74000f3c7a95bf1eef35d3ea32"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"cipher",
|
||||
"cpufeatures 0.2.17",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ahash"
|
||||
version = "0.8.12"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"once_cell",
|
||||
"version_check",
|
||||
"zerocopy",
|
||||
"cpubits",
|
||||
"cpufeatures",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -72,15 +60,6 @@ dependencies = [
|
||||
"object",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "arbitrary"
|
||||
version = "1.4.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c3d036a3c4ab069c7b410a2ce876bd74808d2d0888a82667669f8e783a898bf1"
|
||||
dependencies = [
|
||||
"derive_arbitrary",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "atomic-waker"
|
||||
version = "1.1.2"
|
||||
@@ -171,11 +150,12 @@ checksum = "3ded4057c258ba199e2d26386d3af3780957ecaee6c4ef4041c6b4b8b97c0b06"
|
||||
|
||||
[[package]]
|
||||
name = "block-buffer"
|
||||
version = "0.10.4"
|
||||
version = "0.12.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71"
|
||||
checksum = "d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa"
|
||||
dependencies = [
|
||||
"generic-array",
|
||||
"hybrid-array",
|
||||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -199,12 +179,6 @@ version = "1.25.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "95832e849adfb21180ccb6826a99da14e5d266ae5c2e668e1602cf234f153797"
|
||||
|
||||
[[package]]
|
||||
name = "byteorder"
|
||||
version = "1.5.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b"
|
||||
|
||||
[[package]]
|
||||
name = "bytes"
|
||||
version = "1.12.1"
|
||||
@@ -213,21 +187,11 @@ checksum = "fc652a48c352aef3ea3aed32080501cf3ef6ed5da78602a020c991775b0aff04"
|
||||
|
||||
[[package]]
|
||||
name = "bzip2"
|
||||
version = "0.5.2"
|
||||
version = "0.6.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "49ecfb22d906f800d4fe833b6282cf4dc1c298f5057ca0b5445e5c209735ca47"
|
||||
checksum = "f3a53fac24f34a81bc9954b5d6cfce0c21e18ec6959f44f56e8e90e4bb7c346c"
|
||||
dependencies = [
|
||||
"bzip2-sys",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "bzip2-sys"
|
||||
version = "0.1.13+1.0.8"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "225bff33b2141874fe80d71e07d6eec4f85c5c216453dd96388240f96e1acc14"
|
||||
dependencies = [
|
||||
"cc",
|
||||
"pkg-config",
|
||||
"libbz2-rs-sys",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -261,7 +225,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "65c35e4b699c7e15ccbe7ee35c005e4fc0a278d22238a2857e6ce2dadeda1b06"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"cpufeatures 0.3.1",
|
||||
"cpufeatures",
|
||||
"rand_core 0.10.1",
|
||||
]
|
||||
|
||||
@@ -279,14 +243,20 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "cipher"
|
||||
version = "0.4.4"
|
||||
version = "0.5.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad"
|
||||
checksum = "e8cf2a2c93cd704877c0858356ed03480ff301ee950b43f1cbe4573b088bfa6c"
|
||||
dependencies = [
|
||||
"crypto-common",
|
||||
"inout",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "cmov"
|
||||
version = "0.5.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0c9ea0ac24bc397ab3c98583a3c9ba74fa56b09a4449bbe172b9b1ddb016027a"
|
||||
|
||||
[[package]]
|
||||
name = "colored"
|
||||
version = "3.1.1"
|
||||
@@ -297,10 +267,16 @@ dependencies = [
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "constant_time_eq"
|
||||
version = "0.3.1"
|
||||
name = "const-oid"
|
||||
version = "0.10.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7c74b8349d32d297c9134b8c88677813a227df8f779daa29bfc29c183fe3dca6"
|
||||
checksum = "a6ef517f0926dd24a1582492c791b6a4818a4d94e789a334894aa15b0d12f55c"
|
||||
|
||||
[[package]]
|
||||
name = "constant_time_eq"
|
||||
version = "0.4.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3d52eff69cd5e647efe296129160853a42795992097e8af39800e1060caeea9b"
|
||||
|
||||
[[package]]
|
||||
name = "core-foundation-sys"
|
||||
@@ -309,13 +285,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b"
|
||||
|
||||
[[package]]
|
||||
name = "cpufeatures"
|
||||
version = "0.2.17"
|
||||
name = "cpubits"
|
||||
version = "0.1.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280"
|
||||
dependencies = [
|
||||
"libc",
|
||||
]
|
||||
checksum = "15b85f9c39137c3a891689859392b1bd49812121d0d61c9caf00d46ed5ce06ae"
|
||||
|
||||
[[package]]
|
||||
name = "cpufeatures"
|
||||
@@ -326,21 +299,6 @@ dependencies = [
|
||||
"libc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "crc"
|
||||
version = "3.4.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "5eb8a2a1cd12ab0d987a5d5e825195d372001a4094a0376319d5a0ad71c1ba0d"
|
||||
dependencies = [
|
||||
"crc-catalog",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "crc-catalog"
|
||||
version = "2.5.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "217698eaf96b4a3f0bc4f3662aaa55bdf913cd54d7204591faa790070c6d0853"
|
||||
|
||||
[[package]]
|
||||
name = "crc32fast"
|
||||
version = "1.5.2"
|
||||
@@ -351,19 +309,21 @@ dependencies = [
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "crossbeam-utils"
|
||||
version = "0.8.23"
|
||||
name = "crypto-common"
|
||||
version = "0.2.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "a31eee39dddec8330830986fcd7625edb5a24ec90ea038215273bbc3adb08ac6"
|
||||
checksum = "ce6e4c961d6cd6c9a86db418387425e8bdeaf05b3c8bc1411e6dca4c252f1453"
|
||||
dependencies = [
|
||||
"hybrid-array",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "crypto-common"
|
||||
version = "0.1.7"
|
||||
name = "ctutils"
|
||||
version = "0.4.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
|
||||
checksum = "7d5515a3834141de9eafb9717ad39eea8247b5674e6066c404e8c4b365d2a29e"
|
||||
dependencies = [
|
||||
"generic-array",
|
||||
"typenum",
|
||||
"cmov",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -446,17 +406,6 @@ dependencies = [
|
||||
"serde_core",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "derive_arbitrary"
|
||||
version = "1.4.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1e567bd82dcff979e4b03460c307b3cdc9e96fde3d73bed1496d2bc75d9dd62a"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 2.0.119",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "derive_more"
|
||||
version = "1.0.0"
|
||||
@@ -480,13 +429,15 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "digest"
|
||||
version = "0.10.7"
|
||||
version = "0.11.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292"
|
||||
checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2"
|
||||
dependencies = [
|
||||
"block-buffer",
|
||||
"const-oid",
|
||||
"crypto-common",
|
||||
"subtle",
|
||||
"ctutils",
|
||||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -632,6 +583,12 @@ dependencies = [
|
||||
"zlib-rs",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "foldhash"
|
||||
version = "0.2.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "77ce24cb58228fbb8aa041425bb1050850ac19177686ea6e0f41a70416f56fdb"
|
||||
|
||||
[[package]]
|
||||
name = "form_urlencoded"
|
||||
version = "1.2.2"
|
||||
@@ -729,16 +686,6 @@ dependencies = [
|
||||
"slab",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "generic-array"
|
||||
version = "0.14.7"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a"
|
||||
dependencies = [
|
||||
"typenum",
|
||||
"version_check",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "getrandom"
|
||||
version = "0.2.17"
|
||||
@@ -752,20 +699,6 @@ dependencies = [
|
||||
"wasm-bindgen",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "getrandom"
|
||||
version = "0.3.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"js-sys",
|
||||
"libc",
|
||||
"r-efi 5.3.0",
|
||||
"wasip2",
|
||||
"wasm-bindgen",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "getrandom"
|
||||
version = "0.4.3"
|
||||
@@ -775,7 +708,7 @@ dependencies = [
|
||||
"cfg-if",
|
||||
"js-sys",
|
||||
"libc",
|
||||
"r-efi 6.0.0",
|
||||
"r-efi",
|
||||
"rand_core 0.10.1",
|
||||
"wasm-bindgen",
|
||||
]
|
||||
@@ -788,11 +721,11 @@ checksum = "8a9ee70c43aaf417c914396645a0fa852624801b24ebb7ae78fe8272889ac888"
|
||||
|
||||
[[package]]
|
||||
name = "hashbrown"
|
||||
version = "0.14.5"
|
||||
version = "0.16.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1"
|
||||
checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100"
|
||||
dependencies = [
|
||||
"ahash",
|
||||
"foldhash",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -800,14 +733,17 @@ name = "hashbrown"
|
||||
version = "0.17.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
|
||||
dependencies = [
|
||||
"foldhash",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "hashlink"
|
||||
version = "0.9.1"
|
||||
version = "0.12.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6ba4ff7128dee98c7dc9794b6a411377e1404dba1c97deb8d1a55297bd25d8af"
|
||||
checksum = "a596f1b20ed2cc5ecac41a164aaebc7258057060f06c0cf7a2ba3991ee7990fb"
|
||||
dependencies = [
|
||||
"hashbrown 0.14.5",
|
||||
"hashbrown 0.17.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -830,9 +766,9 @@ checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70"
|
||||
|
||||
[[package]]
|
||||
name = "hmac"
|
||||
version = "0.12.1"
|
||||
version = "0.13.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6c49c37c09c17a53d937dfbb742eb3a961d65a994e6bcdcf37e7399d0cc8ab5e"
|
||||
checksum = "6303bc9732ae41b04cb554b844a762b4115a61bfaa81e3e83050991eeb56863f"
|
||||
dependencies = [
|
||||
"digest",
|
||||
]
|
||||
@@ -894,6 +830,15 @@ version = "2.4.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "15cdd26707701c53297e2fa6afb323d55fbc1d0810c3aec078ae3ef0424c3c15"
|
||||
|
||||
[[package]]
|
||||
name = "hybrid-array"
|
||||
version = "0.4.15"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "27f864f10dfb56725ce5ce5472bc52252c8f93a4ab86327122cebf62c5f59a17"
|
||||
dependencies = [
|
||||
"typenum",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "hyper"
|
||||
version = "1.11.1"
|
||||
@@ -1132,11 +1077,11 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "inout"
|
||||
version = "0.1.4"
|
||||
version = "0.2.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "879f10e63c20629ecabbb64a8010319738c66a5cd0c29b02d63d272b03751d01"
|
||||
checksum = "4250ce6452e92010fdf7268ccc5d14faa80bb12fc741938534c58f16804e03c7"
|
||||
dependencies = [
|
||||
"generic-array",
|
||||
"hybrid-array",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -1252,6 +1197,12 @@ dependencies = [
|
||||
"wasm-bindgen",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "libbz2-rs-sys"
|
||||
version = "0.2.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "34b357333733e8260735ba5894eb928c02ecc69c78715f01a8019e7fa7f2db4c"
|
||||
|
||||
[[package]]
|
||||
name = "libc"
|
||||
version = "0.2.189"
|
||||
@@ -1260,9 +1211,9 @@ checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2"
|
||||
|
||||
[[package]]
|
||||
name = "libsqlite3-sys"
|
||||
version = "0.30.1"
|
||||
version = "0.38.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2e99fb7a497b1e3339bc746195567ed8d3e24945ecd636e3619d20b9de9e9149"
|
||||
checksum = "f1d20bef17f513b9b3004532233187769cd072d790971f4e4da0e346eb6401e8"
|
||||
dependencies = [
|
||||
"cc",
|
||||
"pkg-config",
|
||||
@@ -1309,24 +1260,12 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4050469837a6ff301cd14c1f8f24f88549e6d548f24f64e2148eb0f72cebc51f"
|
||||
|
||||
[[package]]
|
||||
name = "lzma-rs"
|
||||
version = "0.3.0"
|
||||
name = "lzma-rust2"
|
||||
version = "0.16.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "297e814c836ae64db86b36cf2a557ba54368d03f6afcd7d947c266692f71115e"
|
||||
checksum = "ca93e534d1142d1d0dcca6d25fe302508a5dfb40b302802904577725ea0b695b"
|
||||
dependencies = [
|
||||
"byteorder",
|
||||
"crc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "lzma-sys"
|
||||
version = "0.1.20"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "5fda04ab3764e6cde78b9974eec4f779acaba7c4e84b36eca3cf77c581b85d27"
|
||||
dependencies = [
|
||||
"cc",
|
||||
"libc",
|
||||
"pkg-config",
|
||||
"sha2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -1443,9 +1382,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "pbkdf2"
|
||||
version = "0.12.2"
|
||||
version = "0.13.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f8ed6a7761f76e3b9f92dfb0a60a6a6477c61024b775147ff0973a02653abaf2"
|
||||
checksum = "112d82ceb8c5bf524d9af484d4e4970c9fd5a0cc15ba14ad93dccd28873b0629"
|
||||
dependencies = [
|
||||
"digest",
|
||||
"hmac",
|
||||
@@ -1532,6 +1471,12 @@ version = "0.2.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391"
|
||||
|
||||
[[package]]
|
||||
name = "ppmd-rust"
|
||||
version = "1.5.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "196a7c80b9a7652aba7cc070827516c2abe4ccdf53d128e1944003cf5726cff1"
|
||||
|
||||
[[package]]
|
||||
name = "ppv-lite86"
|
||||
version = "0.2.21"
|
||||
@@ -1656,12 +1601,6 @@ dependencies = [
|
||||
"proc-macro2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "r-efi"
|
||||
version = "5.3.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f"
|
||||
|
||||
[[package]]
|
||||
name = "r-efi"
|
||||
version = "6.0.0"
|
||||
@@ -1857,10 +1796,20 @@ dependencies = [
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rusqlite"
|
||||
version = "0.32.1"
|
||||
name = "rsqlite-vfs"
|
||||
version = "0.1.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7753b721174eb8ff87a9a0e799e2d7bc3749323e773db92e0984debb00019d6e"
|
||||
checksum = "c51c9ae4df8a7fba42103df5c621fa3c37eccf3a3c650879e90fc48b11cc192c"
|
||||
dependencies = [
|
||||
"hashbrown 0.16.1",
|
||||
"thiserror",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rusqlite"
|
||||
version = "0.40.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "23f2a97da3e3873c73cb2a2e71b35c40ff95e0b1eefa8d72d8499a6928c3b5b3"
|
||||
dependencies = [
|
||||
"bitflags 2.13.2",
|
||||
"fallible-iterator",
|
||||
@@ -1868,6 +1817,7 @@ dependencies = [
|
||||
"hashlink",
|
||||
"libsqlite3-sys",
|
||||
"smallvec",
|
||||
"sqlite-wasm-rs",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -2067,12 +2017,23 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "sha1"
|
||||
version = "0.10.7"
|
||||
version = "0.11.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "a978451301f4db1d02937a4ab3ccce137717b81826e79b7d49ffe3244a13c3b8"
|
||||
checksum = "aacc4cc499359472b4abe1bf11d0b12e688af9a805fa5e3016f9a386dc2d0214"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"cpufeatures 0.2.17",
|
||||
"cpufeatures",
|
||||
"digest",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sha2"
|
||||
version = "0.11.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "446ba717509524cb3f22f17ecc096f10f4822d76ab5c0b9822c5f9c284e825f4"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"cpufeatures",
|
||||
"digest",
|
||||
]
|
||||
|
||||
@@ -2120,6 +2081,18 @@ dependencies = [
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sqlite-wasm-rs"
|
||||
version = "0.5.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "dc3efc0da82635d7e1ced0053bbbfa8c7ab9645d0bf36ceb4f7127bb85315d75"
|
||||
dependencies = [
|
||||
"cc",
|
||||
"js-sys",
|
||||
"rsqlite-vfs",
|
||||
"wasm-bindgen",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "stable_deref_trait"
|
||||
version = "1.2.1"
|
||||
@@ -2328,6 +2301,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "cdb87b95ec50ddfa440816d227a17b2ccbdda963a316a727fda0fc4334f7d134"
|
||||
dependencies = [
|
||||
"deranged",
|
||||
"js-sys",
|
||||
"num-conv",
|
||||
"powerfmt",
|
||||
"serde_core",
|
||||
@@ -2502,6 +2476,12 @@ version = "0.2.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b"
|
||||
|
||||
[[package]]
|
||||
name = "typed-path"
|
||||
version = "0.12.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8e28f89b80c87b8fb0cf04ab448d5dd0dd0ade2f8891bae878de66a75a28600e"
|
||||
|
||||
[[package]]
|
||||
name = "typenum"
|
||||
version = "1.20.1"
|
||||
@@ -2568,12 +2548,6 @@ version = "0.2.15"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
|
||||
|
||||
[[package]]
|
||||
name = "version_check"
|
||||
version = "0.9.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
|
||||
|
||||
[[package]]
|
||||
name = "want"
|
||||
version = "0.3.1"
|
||||
@@ -2589,15 +2563,6 @@ version = "0.11.1+wasi-snapshot-preview1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b"
|
||||
|
||||
[[package]]
|
||||
name = "wasip2"
|
||||
version = "1.0.4+wasi-0.2.12"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b67efb37e106e55ce722a510d6b5f9c17f083e5fc79afc2badeb12cc313d9487"
|
||||
dependencies = [
|
||||
"wit-bindgen",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "wasm-bindgen"
|
||||
version = "0.2.128"
|
||||
@@ -2845,12 +2810,6 @@ version = "0.52.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec"
|
||||
|
||||
[[package]]
|
||||
name = "wit-bindgen"
|
||||
version = "0.57.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1ebf944e87a7c253233ad6766e082e3cd714b5d03812acc24c318f549614536e"
|
||||
|
||||
[[package]]
|
||||
name = "writeable"
|
||||
version = "0.6.4"
|
||||
@@ -2859,13 +2818,13 @@ checksum = "3ad82d2a33cdc9674dc7465672f271e096168fcdbe0f799d9e6db8c5892679dc"
|
||||
|
||||
[[package]]
|
||||
name = "x-media"
|
||||
version = "1.6.0"
|
||||
version = "1.7.0"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"dotenv",
|
||||
"html-escape",
|
||||
"log",
|
||||
"rand 0.8.8",
|
||||
"rand 0.10.2",
|
||||
"regex",
|
||||
"reqwest",
|
||||
"serde",
|
||||
@@ -2879,7 +2838,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "xmedia-bot"
|
||||
version = "1.6.0"
|
||||
version = "1.7.0"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"dotenv",
|
||||
@@ -2890,7 +2849,7 @@ dependencies = [
|
||||
"parking_lot",
|
||||
"png",
|
||||
"pretty_env_logger",
|
||||
"rand 0.8.8",
|
||||
"rand 0.10.2",
|
||||
"rusqlite",
|
||||
"serde",
|
||||
"serde_json",
|
||||
@@ -2902,15 +2861,6 @@ dependencies = [
|
||||
"zune-jpeg",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "xz2"
|
||||
version = "0.1.7"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "388c44dc09d76f1536602ead6d325eb532f5c122f17782bd57fb47baeeb767e2"
|
||||
dependencies = [
|
||||
"lzma-sys",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "yoke"
|
||||
version = "0.8.3"
|
||||
@@ -2980,20 +2930,6 @@ name = "zeroize"
|
||||
version = "1.9.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e13c156562582aa81c60cb29407084cdb54c4164760106ab78e6c5b0858cf64e"
|
||||
dependencies = [
|
||||
"zeroize_derive",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "zeroize_derive"
|
||||
version = "1.5.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3c50655cbb0fe3fc43170059e702f1ce5e19b84cec58dc87b037a09935c2f328"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 2.0.119",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "zerotrie"
|
||||
@@ -3030,29 +2966,26 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "zip"
|
||||
version = "2.4.2"
|
||||
version = "8.6.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "fabe6324e908f85a1c52063ce7aa26b68dcb7eb6dbc83a2d148403c9bc3eba50"
|
||||
checksum = "2d04a6b5381502aa6087c94c669499eb1602eb9c5e8198e534de571f7154809b"
|
||||
dependencies = [
|
||||
"aes",
|
||||
"arbitrary",
|
||||
"bzip2",
|
||||
"constant_time_eq",
|
||||
"crc32fast",
|
||||
"crossbeam-utils",
|
||||
"deflate64",
|
||||
"displaydoc",
|
||||
"flate2",
|
||||
"getrandom 0.3.4",
|
||||
"getrandom 0.4.3",
|
||||
"hmac",
|
||||
"indexmap 2.14.2",
|
||||
"lzma-rs",
|
||||
"lzma-rust2",
|
||||
"memchr",
|
||||
"pbkdf2",
|
||||
"ppmd-rust",
|
||||
"sha1",
|
||||
"thiserror",
|
||||
"time",
|
||||
"xz2",
|
||||
"typed-path",
|
||||
"zeroize",
|
||||
"zopfli",
|
||||
"zstd",
|
||||
|
||||
+8
-3
@@ -1,11 +1,12 @@
|
||||
# TelegramXMediaBot
|
||||
|
||||
A Telegram bot that turns post links from X / Twitter, Pixiv, Bluesky, and Misskey (misskey.io) into media messages (images, video, GIF) with the post's title, author, and tags.
|
||||
A Telegram bot that turns post links from X / Twitter, Pixiv, Bluesky, Misskey (misskey.io), and Bilibili dynamics into media messages (images, video, GIF) with the post's title, author, and tags.
|
||||
|
||||
## Features
|
||||
|
||||
- Sending a link in a private chat fetches and sends the images, videos and GIFs automatically; oversized media is split into batches
|
||||
- Text-only posts report "no media"; unsupported links are silently ignored
|
||||
- Long posts (text ≥ `CAPTION_QUOTE_TEXT_CHARS`, default 200) show **the text part** of their caption inside a collapsible blockquote, with the link and author line left outside it
|
||||
- Inline queries (`@bot <link>`)
|
||||
- Bind a forward channel for automatic forwarding; edit the caption before forwarding and apply custom templates
|
||||
- Failed sends are retried automatically with persistence; the user is notified after retries are exhausted
|
||||
@@ -30,10 +31,12 @@ docker build -t tgxmb .
|
||||
docker run --rm -d --name tgxmb --env-file .env -v ./data:/app/data tgxmb
|
||||
```
|
||||
|
||||
Environment variables: `TELOXIDE_TOKEN` (required), `PIXIV_REFRESH_TOKEN`, `BOT_ADMIN`, `EDIT_MESSAGE_TTL_SECONDS`, `LINK_CACHE_TTL_SECONDS`, `RUST_LOG`, `TELOXIDE_PROXY`, `WEBHOOK*`, `TWITTER_AUTH_TOKEN` (optional).
|
||||
Environment variables: `TELOXIDE_TOKEN` (required), `PIXIV_REFRESH_TOKEN`, `BOT_ADMIN`, `EDIT_MESSAGE_TTL_SECONDS`, `LINK_CACHE_TTL_SECONDS`, `RUST_LOG`, `TELOXIDE_PROXY`, `WEBHOOK*`, `TWITTER_AUTH_TOKEN` (optional), `BILIBILI_COOKIE` (optional).
|
||||
|
||||
NSFW tweets: the public syndication endpoint does not return sensitive content. Setting `TWITTER_AUTH_TOKEN` (the `auth_token` cookie value of a logged-in x.com session) lets the bot fetch NSFW media in the logged-in state only when it hits a withheld tweet; without it, the bot reports no media.
|
||||
|
||||
Bilibili dynamics are fetched anonymously by default (no login; the bot fetches bilibili's anonymous `buvid3`/`buvid4` device cookies itself to raise the success rate). If the server's egress IP gets hard-flagged by bilibili (persistent `risk control (-352)` log lines or HTTP 412), set `BILIBILI_COOKIE` (the whole cookie string from a logged-in browser, e.g. `SESSDATA=…; bili_jct=…`) to restore access. Only a dynamic's images and animations are sent; an attached video degrades to its cover image.
|
||||
|
||||
### Webhook deployment (needs a reverse proxy)
|
||||
|
||||
`docker-compose.yml.example` ships an [nginx-proxy](https://github.com/nginx-proxy/nginx-proxy) + [acme-companion](https://github.com/nginx-proxy/acme-companion) reverse-proxy orchestration. Pick one deployment shape:
|
||||
@@ -81,9 +84,11 @@ Telegram only accepts ports 443/80/88/8443.
|
||||
|---|---|
|
||||
| `TELOXIDE_TOKEN` | Bot token (required) |
|
||||
| `PIXIV_REFRESH_TOKEN` | Pixiv refresh token; Pixiv is disabled without it |
|
||||
| `BILIBILI_COOKIE` | Optional bilibili cookie string (`SESSDATA=…; bili_jct=…`); only needed when the egress IP stays risk-controlled (device cookies are fetched automatically) |
|
||||
| `BOT_ADMIN` | Admin chat IDs, comma-separated; receives start/stop notifications |
|
||||
| `EDIT_MESSAGE_TTL_SECONDS` | Edit-before-forward record expiry in seconds, default 86400 |
|
||||
| `LINK_CACHE_TTL_SECONDS` | Link-result cache expiry in seconds, default 604800 (7 days) |
|
||||
| `CAPTION_QUOTE_TEXT_CHARS` | **The text part** of the caption (the joined `{title}` + `{content}`) is wrapped in a collapsible blockquote once it reaches this many characters, default 200; `0` disables |
|
||||
| `DATA_DIR` | Data directory (where the SQLite `task_queue.db` lives), default `data` (relative to the working directory, created automatically) |
|
||||
| `RUST_LOG` | Log level |
|
||||
| `TELOXIDE_PROXY` | HTTP proxy (e.g. `http://127.0.0.1:10808`); applies to both the Telegram Bot API and site fetches — required on restricted networks (e.g. behind the GFW) |
|
||||
@@ -111,7 +116,7 @@ Telegram only accepts ports 443/80/88/8443.
|
||||
| `/remove_forward_channel` | Remove the forward channel |
|
||||
| `/edit_before_forward` | Toggle "edit before forward": when enabled, the bot posts a prompt after forwarding; replying to it edits the first forwarded message's caption (or taps a template button to apply one) |
|
||||
| `/set_template <name>` | Reply to a message containing `[]` to save it as a named template; `[]` is replaced by the original post link when forwarding (used with "edit before forward") |
|
||||
| `/set_format <site> <format>` | Customize the caption format for one site. Sites: `twitter` / `bsky` / `pixiv` / `misskey`. Placeholders: `{url}` `{author}` `{author_url}` `{title}` `{tags}` |
|
||||
| `/set_format <site> <format>` | Customize the caption format for one site. Sites: `twitter` / `bsky` / `pixiv` / `misskey` / `bilibili`. Placeholders: `{url}` `{author}` `{author_url}` `{title}` `{content}` `{tags}` |
|
||||
| `/clear_cache [link]` | Clear the link cache (admin only); with a link only that entry, otherwise everything |
|
||||
| `/bot_dict` | Show the current chat state (debugging; admin only) |
|
||||
| `/test <link>` | Parse a link and send its media; no channel forward, no edit-before-forward prompt (send only) |
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
# TelegramXMediaBot
|
||||
|
||||
Telegram 机器人,将 X / Twitter、Pixiv、Bluesky、Misskey (misskey.io) 的帖子链接转换为媒体消息发送,附带帖子标题、作者与标签。
|
||||
Telegram 机器人,将 X / Twitter、Pixiv、Bluesky、Misskey (misskey.io)、Bilibili 动态的帖子链接转换为媒体消息发送,附带帖子标题、作者与标签。
|
||||
|
||||
## 功能
|
||||
|
||||
- 私聊发送链接后自动抓取并发送图片、视频与 GIF,超量图片自动分批
|
||||
- 纯文字帖提示无媒体;不支持的链接静默忽略
|
||||
- 长帖(正文 ≥ `CAPTION_QUOTE_TEXT_CHARS`,默认 200)的**正文部分**用可折叠引用块展示,链接与作者行留在引用块外
|
||||
- 支持内联查询(`@机器人 <链接>`)
|
||||
- 可绑定转发频道自动转发;支持转发前编辑 caption 与自定义模板
|
||||
- 发送失败自动重试并持久化,重试耗尽后通知用户
|
||||
@@ -30,10 +31,12 @@ docker build -t tgxmb .
|
||||
docker run --rm -d --name tgxmb --env-file .env -v ./data:/app/data tgxmb
|
||||
```
|
||||
|
||||
环境变量:`TELOXIDE_TOKEN`(必填)、`PIXIV_REFRESH_TOKEN`、`BOT_ADMIN`、`EDIT_MESSAGE_TTL_SECONDS`、`LINK_CACHE_TTL_SECONDS`、`RUST_LOG`、`TELOXIDE_PROXY`、`WEBHOOK*`、`TWITTER_AUTH_TOKEN`(可选)。
|
||||
环境变量:`TELOXIDE_TOKEN`(必填)、`PIXIV_REFRESH_TOKEN`、`BOT_ADMIN`、`EDIT_MESSAGE_TTL_SECONDS`、`LINK_CACHE_TTL_SECONDS`、`RUST_LOG`、`TELOXIDE_PROXY`、`WEBHOOK*`、`TWITTER_AUTH_TOKEN`(可选)、`BILIBILI_COOKIE`(可选)。
|
||||
|
||||
NSFW 推文:公开的 syndication 接口不返回敏感内容。设置 `TWITTER_AUTH_TOKEN`(登录 x.com 后浏览器 Cookie 里的 `auth_token` 值)后,bot 会仅在遇到 NSFW 推文时以登录态获取媒体;未设置则提示无媒体。
|
||||
|
||||
Bilibili 动态默认匿名抓取(无需登录,bot 会自动从 B 站的匿名指纹接口取 `buvid3`/`buvid4` 设备 cookie 以提高成功率)。若服务器出口 IP 被 B 站重度风控(日志里的 `risk control (-352)` 或 HTTP 412,且持续出现),设置 `BILIBILI_COOKIE`(登录后浏览器里整条 Cookie 串,如 `SESSDATA=…; bili_jct=…`)可恢复访问。当前只发送动态里的图片与动图,动态内嵌视频发送其封面。
|
||||
|
||||
### Webhook 部署(需要反向代理)
|
||||
|
||||
`docker-compose.yml.example` 内置了 [nginx-proxy](https://github.com/nginx-proxy/nginx-proxy) + [acme-companion](https://github.com/nginx-proxy/acme-companion) 反向代理编排,按部署环境二选一:
|
||||
@@ -81,9 +84,11 @@ Telegram 只接受 443/80/88/8443 端口。
|
||||
|---|---|
|
||||
| `TELOXIDE_TOKEN` | Bot token(必填) |
|
||||
| `PIXIV_REFRESH_TOKEN` | Pixiv 刷新令牌;未设置则禁用 Pixiv |
|
||||
| `BILIBILI_COOKIE` | 可选的 B 站 Cookie 串(`SESSDATA=…; bili_jct=…`),仅在出口 IP 被持续风控时才需要(设备 cookie 由 bot 自动获取) |
|
||||
| `BOT_ADMIN` | 管理员聊天 ID,逗号分隔;接收启动/停止通知 |
|
||||
| `EDIT_MESSAGE_TTL_SECONDS` | 转发前编辑记录过期秒数,默认 86400 |
|
||||
| `LINK_CACHE_TTL_SECONDS` | 链接结果缓存过期秒数,默认 604800(7 天) |
|
||||
| `CAPTION_QUOTE_TEXT_CHARS` | 正文(`{title}` + `{content}` 合计)达到该长度(字符)时,caption 的**正文部分**用可折叠引用块包裹,默认 200;`0` 关闭 |
|
||||
| `DATA_DIR` | 数据目录(SQLite 数据库 `task_queue.db` 所在目录),默认 `data`(相对工作目录,会自动创建) |
|
||||
| `RUST_LOG` | 日志级别 |
|
||||
| `TELOXIDE_PROXY` | HTTP 代理(如 `http://127.0.0.1:10808`);同时作用于 Telegram Bot API 与站点抓取请求,网络受限环境(如 GFW)必需 |
|
||||
@@ -111,7 +116,7 @@ Telegram 只接受 443/80/88/8443 端口。
|
||||
| `/remove_forward_channel` | 取消转发频道 |
|
||||
| `/edit_before_forward` | 开关「转发前编辑」:开启后,转发成功后 bot 会发一条提示消息,回复它可修改第一条转发消息的 caption(或点击模板按钮套用模板) |
|
||||
| `/set_template <名称>` | 回复一条含 `[]` 的消息,将其保存为命名模板;转发时 `[]` 会被替换为原帖链接(配合「转发前编辑」使用) |
|
||||
| `/set_format <站点> <格式>` | 自定义某站点的 caption 格式。站点:`twitter` / `bsky` / `pixiv` / `misskey`。占位符:`{url}` `{author}` `{author_url}` `{title}` `{tags}` |
|
||||
| `/set_format <站点> <格式>` | 自定义某站点的 caption 格式。站点:`twitter` / `bsky` / `pixiv` / `misskey` / `bilibili`。占位符:`{url}` `{author}` `{author_url}` `{title}` `{content}` `{tags}` |
|
||||
| `/clear_cache [链接]` | 清空链接缓存(仅管理员);带链接只清该条,否则清空全部 |
|
||||
| `/bot_dict` | 查看当前聊天状态(调试用;仅管理员) |
|
||||
| `/test <链接>` | 解析链接并发送媒体;不转发到频道、不弹转发前编辑提示(仅发送) |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "x-media"
|
||||
version = "1.6.0"
|
||||
version = "1.7.0"
|
||||
edition = "2024"
|
||||
|
||||
[dependencies]
|
||||
@@ -11,10 +11,10 @@ regex = "1.12"
|
||||
html-escape = "0.2"
|
||||
url = "2.5.2"
|
||||
bytes = "1"
|
||||
zip = "2"
|
||||
zip = "8"
|
||||
tempfile = "3"
|
||||
thiserror = "2"
|
||||
rand = "0.8"
|
||||
rand = "0.10"
|
||||
log = "0.4"
|
||||
tokio = { version = "1.40", features = ["time"] }
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,6 @@
|
||||
mod interface;
|
||||
mod model;
|
||||
|
||||
pub use interface::{
|
||||
BilibiliSite, PATTERN, cache_key, enabled, fetch_from_url, is_retryable, media_headers,
|
||||
};
|
||||
@@ -0,0 +1,158 @@
|
||||
//! Serde DTOs for the Bilibili dynamic detail endpoint
|
||||
//! (`/x/polymer/web-dynamic/v1/detail`), mirroring live responses
|
||||
//! (field paths verified 2026-09-17). Every field is optional so an API
|
||||
//! shape change degrades to "no media" instead of a parse failure.
|
||||
|
||||
use serde::Deserialize;
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Detail {
|
||||
/// Business code: `0` = OK, `-352`/`-412` = risk control, `500`/`4101147`
|
||||
/// = gone.
|
||||
pub(crate) code: i64,
|
||||
#[serde(default)]
|
||||
pub(crate) message: Option<String>,
|
||||
#[serde(default)]
|
||||
pub(crate) data: Option<Data>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Data {
|
||||
#[serde(default)]
|
||||
pub(crate) item: Option<Box<Item>>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Item {
|
||||
/// The dynamic id, same numeric id as in the URL.
|
||||
#[serde(default)]
|
||||
pub(crate) id_str: String,
|
||||
#[serde(default)]
|
||||
pub(crate) modules: Option<Modules>,
|
||||
/// The quoted dynamic when this item is a forward. A forward shell often
|
||||
/// carries no media of its own — the original holds it.
|
||||
#[serde(default)]
|
||||
pub(crate) orig: Option<Box<Item>>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Modules {
|
||||
#[serde(default)]
|
||||
pub(crate) module_author: Option<Author>,
|
||||
#[serde(default)]
|
||||
pub(crate) module_dynamic: Option<Dynamic>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Author {
|
||||
#[serde(default)]
|
||||
pub(crate) name: String,
|
||||
#[serde(default)]
|
||||
pub(crate) mid: Option<i64>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Dynamic {
|
||||
#[serde(default)]
|
||||
pub(crate) desc: Option<Desc>,
|
||||
#[serde(default)]
|
||||
pub(crate) major: Option<Major>,
|
||||
/// A single topic (`{"id":…,"name":…}`), the dynamic's only tag source.
|
||||
#[serde(default)]
|
||||
pub(crate) topic: Option<Topic>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Desc {
|
||||
#[serde(default)]
|
||||
pub(crate) text: String,
|
||||
}
|
||||
|
||||
/// `major` is a tagged union: `type` (`MAJOR_TYPE_DRAW` / `_OPUS` /
|
||||
/// `_ARCHIVE` / …) plus one payload object per type. Only the three payloads
|
||||
/// this adapter reads are modeled; an unknown major simply yields no media.
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Major {
|
||||
#[serde(default)]
|
||||
pub(crate) draw: Option<Draw>,
|
||||
#[serde(default)]
|
||||
pub(crate) opus: Option<Opus>,
|
||||
#[serde(default)]
|
||||
pub(crate) archive: Option<Archive>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Draw {
|
||||
#[serde(default)]
|
||||
pub(crate) items: Vec<Pic>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Pic {
|
||||
/// `major.draw` image URL.
|
||||
#[serde(default)]
|
||||
pub(crate) src: Option<String>,
|
||||
/// `major.opus.pics` image URL — the opus shape names the field
|
||||
/// differently while carrying the same image.
|
||||
#[serde(default)]
|
||||
pub(crate) url: Option<String>,
|
||||
}
|
||||
|
||||
impl Pic {
|
||||
/// The image URL, whichever key this serialization put it under.
|
||||
pub(crate) fn url(&self) -> Option<&str> {
|
||||
self.src.as_deref().or(self.url.as_deref())
|
||||
}
|
||||
}
|
||||
|
||||
/// `major.opus`: the serialization of an image/text post the web client asks
|
||||
/// for (`features=itemOpusStyle`). It carries the parts the legacy shape drops
|
||||
/// entirely — the document title and body of an opus post, whose
|
||||
/// `module_dynamic.desc` comes back `null`.
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Opus {
|
||||
/// Document headline; often absent.
|
||||
#[serde(default)]
|
||||
pub(crate) title: Option<String>,
|
||||
/// Document body (untruncated: a 307-char sample came back whole).
|
||||
#[serde(default)]
|
||||
pub(crate) summary: Option<Desc>,
|
||||
#[serde(default)]
|
||||
pub(crate) pics: Vec<Pic>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Archive {
|
||||
/// The attached video's cover — the only image an AV dynamic has (the
|
||||
/// video itself is deliberately not resolved, see the module docs).
|
||||
#[serde(default)]
|
||||
pub(crate) cover: Option<String>,
|
||||
/// The video's title. An AV dynamic has no body of its own (`desc` comes
|
||||
/// back `null`), so this card title is the post's content.
|
||||
#[serde(default)]
|
||||
pub(crate) title: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Topic {
|
||||
#[serde(default)]
|
||||
pub(crate) name: String,
|
||||
}
|
||||
|
||||
/// Response of the anonymous fingerprint endpoint (`/x/frontend/finger/spi`),
|
||||
/// the source of the adapter's device cookies.
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Fingerprint {
|
||||
#[serde(default)]
|
||||
pub(crate) data: Option<FingerprintData>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct FingerprintData {
|
||||
/// Sent as the `buvid3` cookie.
|
||||
#[serde(default, rename = "b_3")]
|
||||
pub(crate) buvid3: String,
|
||||
/// Sent as the `buvid4` cookie.
|
||||
#[serde(default, rename = "b_4")]
|
||||
pub(crate) buvid4: String,
|
||||
}
|
||||
@@ -330,13 +330,16 @@ impl From<Post> for Fetched {
|
||||
url: url.clone(),
|
||||
author: encode_text(&post.author).into_owned(),
|
||||
author_url: author_url.clone(),
|
||||
title: encode_text(&post.text).into_owned(),
|
||||
// A post has no title: its text is all content.
|
||||
title: String::new(),
|
||||
content: encode_text(&post.text).into_owned(),
|
||||
tags: String::new(),
|
||||
});
|
||||
Fetched {
|
||||
source_url: url,
|
||||
caption: post.caption(),
|
||||
title: post.text.clone(),
|
||||
title: String::new(),
|
||||
content: post.text.clone(),
|
||||
media: post.media,
|
||||
sensitive: post.sensitive,
|
||||
site_id: "bsky",
|
||||
@@ -410,7 +413,8 @@ mod tests {
|
||||
fetched.source_url,
|
||||
"https://bsky.app/profile/user.bsky.social/post/3xxxx"
|
||||
);
|
||||
assert_eq!(fetched.title, "hello <world>");
|
||||
assert_eq!(fetched.title, "");
|
||||
assert_eq!(fetched.content, "hello <world>");
|
||||
assert_eq!(fetched.media.len(), 1);
|
||||
assert!(!fetched.sensitive);
|
||||
// display_name absent -> empty fallback
|
||||
|
||||
@@ -123,21 +123,23 @@ impl From<model::Note> for Fetched {
|
||||
let cw = content.cw.as_deref().unwrap_or_default();
|
||||
// Notes carry hashtags inline in the text (no structured tags array);
|
||||
// a CW note gets the marker prefixed so recipients see the spoiler.
|
||||
let mut title = cw.to_string();
|
||||
if !cw.is_empty() && !title.ends_with(' ') {
|
||||
title.push(' ');
|
||||
let mut text = cw.to_string();
|
||||
if !cw.is_empty() && !text.ends_with(' ') {
|
||||
text.push(' ');
|
||||
}
|
||||
title.push_str(content.text.as_deref().unwrap_or_default().trim());
|
||||
let title = title.trim().to_string();
|
||||
text.push_str(content.text.as_deref().unwrap_or_default().trim());
|
||||
let text = text.trim().to_string();
|
||||
|
||||
let caption = caption(&url, &author_url, &author, &title);
|
||||
let caption = caption(&url, &author_url, &author, &text);
|
||||
let sensitive = content.cw.is_some() || content.files.iter().any(|f| f.is_sensitive);
|
||||
let media: Vec<Media> = content.files.iter().filter_map(media_from_file).collect();
|
||||
|
||||
Fetched {
|
||||
source_url: url.clone(),
|
||||
caption,
|
||||
title: title.clone(),
|
||||
// A note has no title: its text (CW marker included) is content.
|
||||
title: String::new(),
|
||||
content: text.clone(),
|
||||
media,
|
||||
sensitive,
|
||||
site_id: "misskey",
|
||||
@@ -145,7 +147,8 @@ impl From<model::Note> for Fetched {
|
||||
url,
|
||||
author: encode_text(&author).into_owned(),
|
||||
author_url: author_url.clone(),
|
||||
title: encode_text(&title).into_owned(),
|
||||
title: String::new(),
|
||||
content: encode_text(&text).into_owned(),
|
||||
tags: String::new(),
|
||||
}),
|
||||
_keep_alive: None,
|
||||
@@ -257,7 +260,8 @@ mod tests {
|
||||
"https://misskey.io/notes/aotihl10lqrs015s"
|
||||
);
|
||||
assert_eq!(fetched.site_id, "misskey");
|
||||
assert_eq!(fetched.title, "hello");
|
||||
assert_eq!(fetched.title, "");
|
||||
assert_eq!(fetched.content, "hello");
|
||||
assert!(fetched.sensitive);
|
||||
assert_eq!(fetched.media.len(), 1);
|
||||
match &fetched.media[0] {
|
||||
@@ -308,7 +312,8 @@ mod tests {
|
||||
note["text"] = serde_json::json!("body");
|
||||
let fetched: Fetched = note_json(note).into();
|
||||
assert!(fetched.sensitive);
|
||||
assert_eq!(fetched.title, "spoiler body");
|
||||
assert_eq!(fetched.title, "");
|
||||
assert_eq!(fetched.content, "spoiler body");
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -341,7 +346,8 @@ mod tests {
|
||||
}
|
||||
});
|
||||
let fetched: Fetched = note_json(note).into();
|
||||
assert_eq!(fetched.title, "inner text");
|
||||
assert_eq!(fetched.title, "");
|
||||
assert_eq!(fetched.content, "inner text");
|
||||
assert_eq!(fetched.media.len(), 1);
|
||||
// The source URL still points at the renote shell the user posted.
|
||||
assert_eq!(
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
//! Site fetching dispatcher and unified result types.
|
||||
//!
|
||||
//! Dispatch order: twitter → bsky → misskey → pixiv. Each site module
|
||||
//! exports a `PATTERN`, `enabled()` and `fetch_from_url()`; a future site
|
||||
//! plugs in by adding one guarded entry in `SITES`.
|
||||
//! Dispatch order: twitter → bsky → misskey → pixiv → bilibili. Each site
|
||||
//! module exports a `PATTERN`, `enabled()` and `fetch_from_url()`; a future
|
||||
//! site plugs in by adding one guarded entry in `SITES`.
|
||||
|
||||
use std::future::Future;
|
||||
use std::pin::Pin;
|
||||
@@ -13,6 +13,7 @@ use std::time::Duration;
|
||||
use regex::Regex;
|
||||
use thiserror::Error;
|
||||
|
||||
pub mod bilibili;
|
||||
pub mod bsky;
|
||||
pub mod misskey;
|
||||
pub mod pixiv;
|
||||
@@ -20,22 +21,31 @@ pub mod twitter;
|
||||
|
||||
pub use pixiv::PixivError;
|
||||
|
||||
/// The result of fetching a post: canonical URL, HTML caption, raw text,
|
||||
/// media list and spoiler flag. Produced by [`fetch`].
|
||||
/// The result of fetching a post: canonical URL, HTML caption, the post's
|
||||
/// title and body, media list and spoiler flag. Produced by [`fetch`].
|
||||
#[derive(Debug)]
|
||||
pub struct Fetched {
|
||||
/// Canonical URL: `x.com/{author}/status/{id}` |
|
||||
/// `https://www.pixiv.net/artworks/{id}` |
|
||||
/// `https://bsky.app/profile/{handle}/post/{rkey}`
|
||||
/// `https://bsky.app/profile/{handle}/post/{rkey}` |
|
||||
/// `https://www.bilibili.com/opus/{id}`
|
||||
pub source_url: String,
|
||||
/// The exact HTML produced by the site's caption().
|
||||
pub caption: String,
|
||||
/// Raw post text (tweet text / bsky text / pixiv title).
|
||||
/// The post's own title, where the platform has one: a pixiv artwork's
|
||||
/// title, the headline of a bilibili opus post or the title of the video
|
||||
/// an AV dynamic attaches. Empty on the platforms whose posts are text
|
||||
/// only (x/twitter, bsky, misskey) and on bilibili posts without a
|
||||
/// headline.
|
||||
pub title: String,
|
||||
/// The post's body text, as the platform exposes it: a tweet, a bsky or
|
||||
/// misskey post, a bilibili dynamic's text, a pixiv artwork's description
|
||||
/// (HTML flattened). Empty when the post has no text at all.
|
||||
pub content: String,
|
||||
pub media: Vec<crate::media::Media>,
|
||||
/// Spoiler flag for all media of this post.
|
||||
pub sensitive: bool,
|
||||
/// Site id (`"twitter"` / `"bsky"` / `"pixiv"`): the single source of
|
||||
/// Site id (`"twitter"` / `"bsky"` / `"pixiv"` / `"bilibili"`): the single source of
|
||||
/// truth for site identity — caption-format lookup, cache-key prefix and
|
||||
/// the SetFormat whitelist all derive from it. Set by the producing site.
|
||||
pub site_id: &'static str,
|
||||
@@ -46,24 +56,38 @@ pub struct Fetched {
|
||||
pub(crate) _keep_alive: Option<tempfile::TempDir>,
|
||||
}
|
||||
|
||||
/// Values for the `{url} {author} {author_url} {title} {tags}` placeholders in
|
||||
/// user-supplied caption formats, substituted by [`caption_from_fields`] as
|
||||
/// HTML text (never as an attribute value).
|
||||
/// Values for the `{url} {author} {author_url} {title} {content} {tags}`
|
||||
/// placeholders in user-supplied caption formats, substituted by
|
||||
/// [`caption_from_fields`] as HTML text (never as an attribute value).
|
||||
///
|
||||
/// `author`, `title` and `tags` come from the site API (post text, display
|
||||
/// names) and are HTML-escaped at construction. `url` and `author_url` stay
|
||||
/// raw: they are canonical URLs the adapter builds from numeric ids and
|
||||
/// API-constrained handles/DIDs, so they carry no escapable character — the
|
||||
/// bot's `/test` report relies on that when it embeds them.
|
||||
/// `author`, `title`, `content` and `tags` come from the site API (post
|
||||
/// text, display names, descriptions) and are HTML-escaped at construction.
|
||||
/// `url` and `author_url` stay raw: they are canonical URLs the adapter
|
||||
/// builds from numeric ids and API-constrained handles/DIDs, so they carry
|
||||
/// no escapable character — the bot's `/test` report relies on that when it
|
||||
/// embeds them.
|
||||
#[derive(Debug)]
|
||||
pub(crate) struct RenderData {
|
||||
pub url: String,
|
||||
pub author: String,
|
||||
pub author_url: String,
|
||||
pub title: String,
|
||||
pub content: String,
|
||||
pub tags: String,
|
||||
}
|
||||
|
||||
/// The post's text as one string: title and content joined by a line break,
|
||||
/// each only when it is non-empty. This is what the sites' built-in captions
|
||||
/// show after the author line, and what the bot quotes when it is long.
|
||||
pub fn compose_text(title: &str, content: &str) -> String {
|
||||
match (title.is_empty(), content.is_empty()) {
|
||||
(false, false) => format!("{title}\n{content}"),
|
||||
(false, true) => title.to_string(),
|
||||
(true, false) => content.to_string(),
|
||||
(true, true) => String::new(),
|
||||
}
|
||||
}
|
||||
|
||||
impl Fetched {
|
||||
/// The site this post came from (used for per-site format overrides).
|
||||
/// A thin alias over [`Fetched::site_id`] kept for callers that read the
|
||||
@@ -87,21 +111,23 @@ impl Fetched {
|
||||
&data.author,
|
||||
&data.author_url,
|
||||
&data.title,
|
||||
&data.content,
|
||||
&data.tags,
|
||||
),
|
||||
_ => truncate_caption(&self.caption),
|
||||
}
|
||||
}
|
||||
|
||||
/// The pre-escaped placeholder values (author, author_url, title, tags)
|
||||
/// a caller needs to rebuild a caption later, e.g. for a cached post
|
||||
/// where the [`Fetched`] is no longer available.
|
||||
pub fn render_fields(&self) -> Option<(&str, &str, &str, &str)> {
|
||||
/// The pre-escaped placeholder values (author, author_url, title,
|
||||
/// content, tags) a caller needs to rebuild a caption later, e.g. for a
|
||||
/// cached post where the [`Fetched`] is no longer available.
|
||||
pub fn render_fields(&self) -> Option<(&str, &str, &str, &str, &str)> {
|
||||
self.render_data.as_ref().map(|d| {
|
||||
(
|
||||
d.author.as_str(),
|
||||
d.author_url.as_str(),
|
||||
d.title.as_str(),
|
||||
d.content.as_str(),
|
||||
d.tags.as_str(),
|
||||
)
|
||||
})
|
||||
@@ -147,6 +173,11 @@ pub fn truncate_caption(caption: &str) -> String {
|
||||
/// [`Fetched::caption_with`]. An empty format returns `built_in` unchanged.
|
||||
/// The result is truncated to [`MAX_CAPTION_CHARS`] (Telegram's caption
|
||||
/// limit for HTML parse mode).
|
||||
///
|
||||
/// One flat argument per placeholder keeps the two callers (the fresh and the
|
||||
/// cached caption path) mirroring each other; the same shape as the bot's
|
||||
/// `debug_report`.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub fn caption_from_fields(
|
||||
format: &str,
|
||||
built_in: &str,
|
||||
@@ -154,6 +185,7 @@ pub fn caption_from_fields(
|
||||
author: &str,
|
||||
author_url: &str,
|
||||
title: &str,
|
||||
content: &str,
|
||||
tags: &str,
|
||||
) -> String {
|
||||
if format.is_empty() {
|
||||
@@ -166,6 +198,7 @@ pub fn caption_from_fields(
|
||||
.replace("{author}", author)
|
||||
.replace("{author_url}", author_url)
|
||||
.replace("{title}", title)
|
||||
.replace("{content}", content)
|
||||
.replace("{tags}", tags),
|
||||
)
|
||||
}
|
||||
@@ -283,7 +316,7 @@ pub(crate) fn log_once_ffmpeg_missing() {
|
||||
}
|
||||
|
||||
/// Site adapter: one impl per supported site (twitter / bsky / misskey /
|
||||
/// pixiv), registered in `SITES`. All site-specific knowledge — URL pattern,
|
||||
/// pixiv / bilibili), registered in `SITES`. All site-specific knowledge — URL pattern,
|
||||
/// cache-key format, fetch, retry policy, media-host headers, startup
|
||||
/// validation — lives in the site module; the central dispatcher only
|
||||
/// iterates the registry.
|
||||
@@ -294,9 +327,9 @@ pub(crate) fn log_once_ffmpeg_missing() {
|
||||
/// site structs are stateless unit structs, so the boxed futures never
|
||||
/// borrow from `self` beyond the call's scope.
|
||||
pub trait Site: Send + Sync {
|
||||
/// Stable site id (`"twitter"` / `"bsky"` / `"misskey"` / `"pixiv"`):
|
||||
/// caption-format lookup, cache-key prefixes and the SetFormat whitelist
|
||||
/// derive from it.
|
||||
/// Stable site id (`"twitter"` / `"bsky"` / `"misskey"` / `"pixiv"` /
|
||||
/// `"bilibili"`): caption-format lookup, cache-key prefixes and the
|
||||
/// SetFormat whitelist derive from it.
|
||||
fn id(&self) -> &'static str;
|
||||
/// URL pattern; the dispatcher's first match wins (dispatch order).
|
||||
fn pattern(&self) -> &'static Regex;
|
||||
@@ -332,14 +365,15 @@ pub trait Site: Send + Sync {
|
||||
type SiteFuture<'a, T, E = FetchError> = Pin<Box<dyn Future<Output = Result<T, E>> + Send + 'a>>;
|
||||
|
||||
/// The one registry of supported sites, in dispatch order (twitter → bsky →
|
||||
/// misskey → pixiv). Adding a site = new module + one `Box::new(...)` entry
|
||||
/// here; the bot crate never lists sites itself.
|
||||
/// misskey → pixiv → bilibili). Adding a site = new module + one
|
||||
/// `Box::new(...)` entry here; the bot crate never lists sites itself.
|
||||
static SITES: LazyLock<Vec<Box<dyn Site>>> = LazyLock::new(|| {
|
||||
vec![
|
||||
Box::new(twitter::TwitterSite),
|
||||
Box::new(bsky::BskySite),
|
||||
Box::new(misskey::MisskeySite),
|
||||
Box::new(pixiv::PixivSite),
|
||||
Box::new(bilibili::BilibiliSite),
|
||||
]
|
||||
});
|
||||
|
||||
@@ -541,6 +575,10 @@ mod tests {
|
||||
cache_key("https://bsky.app/profile/handle.example/post/3lorem"),
|
||||
Some("bsky:handle.example/3lorem".into())
|
||||
);
|
||||
assert_eq!(
|
||||
cache_key("https://t.bilibili.com/1245284537985925159"),
|
||||
Some("bilibili:1245284537985925159".into())
|
||||
);
|
||||
assert_eq!(cache_key("https://example.com/not-a-post"), None);
|
||||
}
|
||||
|
||||
@@ -549,16 +587,21 @@ mod tests {
|
||||
assert_eq!(site_id_from_key("twitter:123"), "twitter");
|
||||
assert_eq!(site_id_from_key("pixiv:123"), "pixiv");
|
||||
assert_eq!(site_id_from_key("bsky:handle.example/3lorem"), "bsky");
|
||||
assert_eq!(site_id_from_key("bilibili:123"), "bilibili");
|
||||
assert_eq!(site_id_from_key("unknown:1"), "unknown");
|
||||
assert_eq!(site_id_from_key("no-colon"), "unknown");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn registry_lists_all_sites_in_dispatch_order() {
|
||||
assert_eq!(site_ids(), vec!["twitter", "bsky", "misskey", "pixiv"]);
|
||||
assert_eq!(
|
||||
site_ids(),
|
||||
vec!["twitter", "bsky", "misskey", "pixiv", "bilibili"]
|
||||
);
|
||||
// Enabled sites dispatch; unsupported URLs never match.
|
||||
assert!(find_site("https://x.com/u/status/1").is_some());
|
||||
assert!(find_site("https://misskey.io/notes/abc").is_some());
|
||||
assert!(find_site("https://t.bilibili.com/1245284537985925159").is_some());
|
||||
assert!(find_site("https://example.com/x").is_none());
|
||||
// Cache keys are pattern-driven, independent of the enabled() gate
|
||||
// (pixiv is disabled in tests without PIXIV_REFRESH_TOKEN).
|
||||
@@ -586,25 +629,34 @@ mod tests {
|
||||
// The format string is escaped, the field values are substituted
|
||||
// verbatim (callers pass the already-escaped render data).
|
||||
let out = caption_from_fields(
|
||||
"see {author} at {url} — {title}",
|
||||
"see {author} at {url} — {title}: {content}",
|
||||
"",
|
||||
"https://x.com/u/status/1",
|
||||
"A & B",
|
||||
"https://x.com/u",
|
||||
"hello <world>",
|
||||
"the body",
|
||||
"",
|
||||
);
|
||||
assert_eq!(
|
||||
out,
|
||||
"see A & B at https://x.com/u/status/1 — hello <world>"
|
||||
"see A & B at https://x.com/u/status/1 — hello <world>: the body"
|
||||
);
|
||||
// Empty format keeps the built-in caption untouched.
|
||||
assert_eq!(
|
||||
caption_from_fields("", "built-in", "u", "a", "au", "t", "g"),
|
||||
caption_from_fields("", "built-in", "u", "a", "au", "t", "c", "g"),
|
||||
"built-in"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn compose_text_joins_title_and_content() {
|
||||
assert_eq!(compose_text("标题", "正文"), "标题\n正文");
|
||||
assert_eq!(compose_text("标题", ""), "标题");
|
||||
assert_eq!(compose_text("", "正文"), "正文");
|
||||
assert_eq!(compose_text("", ""), "");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn truncate_caption_keeps_short_text() {
|
||||
assert_eq!(truncate_caption("short"), "short");
|
||||
|
||||
@@ -107,10 +107,60 @@ pub fn media_headers(url: &str) -> Option<Vec<(&'static str, String)>> {
|
||||
}
|
||||
}
|
||||
|
||||
/// Flattens the app API's HTML description into plain text: `<br>` (and `<p>`)
|
||||
/// become line breaks, other tags are dropped, entities decoded, the ends
|
||||
/// trimmed. A caption shows text, not markup, so the author's `<a href>` links
|
||||
/// contribute their link text only.
|
||||
fn flatten_html(raw: &str) -> String {
|
||||
let mut out = String::with_capacity(raw.len());
|
||||
let mut chars = raw.chars().peekable();
|
||||
while let Some(c) = chars.next() {
|
||||
// Only `<` followed by `/` or a letter opens a tag — a bare `<` in
|
||||
// prose ("2 < 3") is text.
|
||||
let opens_tag = c == '<'
|
||||
&& chars
|
||||
.peek()
|
||||
.is_some_and(|next| *next == '/' || next.is_ascii_alphabetic());
|
||||
if !opens_tag {
|
||||
out.push(c);
|
||||
continue;
|
||||
}
|
||||
let mut tag = String::new();
|
||||
let mut closed = false;
|
||||
for c in chars.by_ref() {
|
||||
if c == '>' {
|
||||
closed = true;
|
||||
break;
|
||||
}
|
||||
tag.push(c);
|
||||
}
|
||||
if !closed {
|
||||
// Unclosed `<…`: keep it as text rather than dropping the tail.
|
||||
out.push('<');
|
||||
out.push_str(&tag);
|
||||
break;
|
||||
}
|
||||
// `<br>`, `<br/>`, `<br />` with or without attributes, and both
|
||||
// halves of a paragraph break the line; everything else is dropped.
|
||||
let tag = tag
|
||||
.trim()
|
||||
.trim_start_matches('/')
|
||||
.trim_end_matches('/')
|
||||
.trim()
|
||||
.to_ascii_lowercase();
|
||||
if tag == "p" || tag.starts_with("br") {
|
||||
out.push('\n');
|
||||
}
|
||||
}
|
||||
html_escape::decode_html_entities(&out).trim().to_string()
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct Illustration {
|
||||
id: String,
|
||||
title: String,
|
||||
/// The artwork's description, HTML flattened to plain text.
|
||||
content: String,
|
||||
author: String,
|
||||
author_id: String,
|
||||
tags: Vec<String>,
|
||||
@@ -150,6 +200,7 @@ impl Illustration {
|
||||
pub fn from_model(model: &IllustrationModel) -> Self {
|
||||
let id = model.id.to_string();
|
||||
let title = model.title.clone();
|
||||
let content = flatten_html(&model.caption);
|
||||
let author = model.user.name.clone();
|
||||
let author_id = model.user.id.to_string();
|
||||
let mut tags: Vec<String> = model.tags.iter().map(|tag| tag.name.clone()).collect();
|
||||
@@ -193,6 +244,7 @@ impl Illustration {
|
||||
Self {
|
||||
id,
|
||||
title,
|
||||
content,
|
||||
author,
|
||||
author_id,
|
||||
tags,
|
||||
@@ -218,12 +270,14 @@ impl From<Illustration> for Fetched {
|
||||
author: encode_text(&illustration.author).into_owned(),
|
||||
author_url: author_url.clone(),
|
||||
title: encode_text(&illustration.title).into_owned(),
|
||||
content: encode_text(&illustration.content).into_owned(),
|
||||
tags: encode_text(&tags).into_owned(),
|
||||
});
|
||||
Fetched {
|
||||
source_url: url,
|
||||
caption: illustration.caption(),
|
||||
title: illustration.title.clone(),
|
||||
content: illustration.content.clone(),
|
||||
media: illustration.media,
|
||||
sensitive: illustration.nsfw,
|
||||
site_id: "pixiv",
|
||||
@@ -262,6 +316,7 @@ mod tests {
|
||||
"illust": {
|
||||
"id": 123,
|
||||
"title": "Art <title>",
|
||||
"caption": "一行说明<br />二行 <a href=\"https://x.example/\">链接</a> & 结尾",
|
||||
"type": type_,
|
||||
"image_urls": {
|
||||
"medium": "medium.jpg",
|
||||
@@ -284,6 +339,44 @@ mod tests {
|
||||
Illustration::from_model(&model)
|
||||
}
|
||||
|
||||
/// The description arrives as HTML and becomes plain-text content: breaks
|
||||
/// kept, tags dropped (links keep their text), entities decoded.
|
||||
#[test]
|
||||
fn from_json_maps_description_to_content() {
|
||||
let v = illust_json("illust", 1, None, Some("o.jpg"), vec![], 0);
|
||||
let illustration = parse(v);
|
||||
assert_eq!(illustration.content, "一行说明\n二行 链接 & 结尾");
|
||||
|
||||
let fetched: Fetched = illustration.into();
|
||||
assert_eq!(fetched.title, "Art <title>");
|
||||
assert_eq!(fetched.content, "一行说明\n二行 链接 & 结尾");
|
||||
// The built-in caption keeps its layout: the description stays out of
|
||||
// it and is available through `{content}`.
|
||||
assert!(!fetched.caption.contains("一行说明"), "{}", fetched.caption);
|
||||
assert_eq!(
|
||||
fetched.render_fields().unwrap().3,
|
||||
"一行说明\n二行 链接 & 结尾"
|
||||
);
|
||||
assert!(
|
||||
fetched
|
||||
.caption_with("{title}: {content}")
|
||||
.ends_with("一行说明\n二行 链接 & 结尾")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn flatten_html_handles_common_markup() {
|
||||
assert_eq!(flatten_html(""), "");
|
||||
assert_eq!(flatten_html("plain"), "plain");
|
||||
assert_eq!(flatten_html("a<br />b<br/>c<br>d"), "a\nb\nc\nd");
|
||||
// A paragraph break is a blank line, exactly like `<br /><br />` —
|
||||
// writing it as one newline would flatten the author's paragraphs.
|
||||
assert_eq!(flatten_html("<p>one</p><p>two</p>"), "one\n\ntwo");
|
||||
assert_eq!(flatten_html("a & b <c>"), "a & b <c>");
|
||||
// Nothing to strip: angle brackets that are not a tag survive.
|
||||
assert_eq!(flatten_html("2 < 3"), "2 < 3");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pattern_matches_all_forms() {
|
||||
let cases = [
|
||||
|
||||
@@ -6,6 +6,10 @@ use serde::Deserialize;
|
||||
pub struct IllustrationModel {
|
||||
pub id: u64,
|
||||
pub title: String,
|
||||
/// The artwork's description as the app API returns it — HTML in most
|
||||
/// works (`<br />`, `<a href>`, sometimes `<p>`), empty for many.
|
||||
#[serde(default)]
|
||||
pub caption: String,
|
||||
pub r#type: TypeModel,
|
||||
pub image_urls: ImageUrlsModel,
|
||||
pub user: UserInfoModel,
|
||||
|
||||
@@ -327,7 +327,8 @@ mod tests {
|
||||
"https://x.com/nsfw_author/status/2083868672721039569"
|
||||
);
|
||||
// The appended media short link (no URL-entity mapping) is stripped.
|
||||
assert_eq!(fetched.title, "nsfw content");
|
||||
assert_eq!(fetched.title, "");
|
||||
assert_eq!(fetched.content, "nsfw content");
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -99,6 +99,7 @@ fn empty_fetched(url: &str) -> Fetched {
|
||||
// crafted links cannot break the parse (Telegram 400).
|
||||
caption: encode_text(url).into_owned(),
|
||||
title: String::new(),
|
||||
content: String::new(),
|
||||
media: vec![],
|
||||
sensitive: true,
|
||||
site_id: "twitter",
|
||||
@@ -352,17 +353,20 @@ impl From<Tweet> for Fetched {
|
||||
fn from(tweet: Tweet) -> Self {
|
||||
let url = tweet.url();
|
||||
let author_url = tweet.author_url();
|
||||
// A tweet has no title: its text is all content.
|
||||
let render_data = Some(crate::site::RenderData {
|
||||
url: url.clone(),
|
||||
author: encode_text(&tweet.author).into_owned(),
|
||||
author_url: author_url.clone(),
|
||||
title: encode_text(&tweet.text).into_owned(),
|
||||
title: String::new(),
|
||||
content: encode_text(&tweet.text).into_owned(),
|
||||
tags: String::new(),
|
||||
});
|
||||
Fetched {
|
||||
source_url: url,
|
||||
caption: tweet.caption(),
|
||||
title: tweet.text.clone(),
|
||||
title: String::new(),
|
||||
content: tweet.text.clone(),
|
||||
media: tweet.media,
|
||||
sensitive: tweet.sensitive,
|
||||
site_id: "twitter",
|
||||
@@ -437,7 +441,8 @@ mod tests {
|
||||
assert_eq!(tweet.text, ">^ω^< & more 'quoted'");
|
||||
assert_eq!(tweet.author, "O'Brien");
|
||||
let fetched: Fetched = tweet.into();
|
||||
assert_eq!(fetched.title, ">^ω^< & more 'quoted'");
|
||||
assert_eq!(fetched.title, "");
|
||||
assert_eq!(fetched.content, ">^ω^< & more 'quoted'");
|
||||
// The caption escapes the raw text exactly once (encode_text covers
|
||||
// & < >; apostrophes stay literal — they are harmless in text).
|
||||
assert!(
|
||||
@@ -496,7 +501,8 @@ mod tests {
|
||||
fetched.source_url,
|
||||
"https://x.com/author_handle/status/861627479294746624"
|
||||
);
|
||||
assert_eq!(fetched.title, "a & b <c>");
|
||||
assert_eq!(fetched.title, "");
|
||||
assert_eq!(fetched.content, "a & b <c>");
|
||||
assert!(fetched.sensitive);
|
||||
assert_eq!(fetched.media.len(), 2);
|
||||
match &fetched.media[0] {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "xmedia-bot"
|
||||
version = "1.6.0"
|
||||
version = "1.7.0"
|
||||
edition = "2024"
|
||||
|
||||
[dependencies]
|
||||
@@ -13,8 +13,8 @@ pretty_env_logger = "0.5"
|
||||
dotenv = "0.15"
|
||||
url = "2.5.2"
|
||||
html-escape = "0.2"
|
||||
rusqlite = { version = "0.32", features = ["bundled"] }
|
||||
rand = "0.8"
|
||||
rusqlite = { version = "0.40", features = ["bundled"] }
|
||||
rand = "0.10"
|
||||
tempfile = "3"
|
||||
parking_lot = "0.12"
|
||||
bytes = "1"
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
//! Central env handling. The only other places that read env are
|
||||
//! `Bot::from_env` (TELOXIDE_TOKEN) and x-media (PIXIV_REFRESH_TOKEN).
|
||||
//! `Bot::from_env` (TELOXIDE_TOKEN) and x-media (PIXIV_REFRESH_TOKEN,
|
||||
//! TWITTER_AUTH_TOKEN, BILIBILI_COOKIE).
|
||||
|
||||
use std::env;
|
||||
use std::net::IpAddr;
|
||||
@@ -12,6 +13,10 @@ pub struct Config {
|
||||
pub edit_message_ttl: Duration,
|
||||
/// LINK_CACHE_TTL_SECONDS, default 604800 (7 days).
|
||||
pub link_cache_ttl: Duration,
|
||||
/// CAPTION_QUOTE_TEXT_CHARS, default 200: a post whose text (title plus
|
||||
/// content) is at least this many characters gets that text wrapped in an
|
||||
/// expandable blockquote inside its caption. `0` disables the wrap.
|
||||
pub caption_quote_text_chars: usize,
|
||||
// Webhook settings (moved out of main; names/defaults unchanged).
|
||||
pub webhook_enabled: bool,
|
||||
pub webhook_url: Option<url::Url>,
|
||||
@@ -57,6 +62,7 @@ impl Config {
|
||||
Duration::from_secs(parse_u64("EDIT_MESSAGE_TTL_SECONDS", 24 * 3600));
|
||||
let link_cache_ttl =
|
||||
Duration::from_secs(parse_u64("LINK_CACHE_TTL_SECONDS", 7 * 24 * 3600));
|
||||
let caption_quote_text_chars = parse_u64("CAPTION_QUOTE_TEXT_CHARS", 200) as usize;
|
||||
|
||||
let webhook_enabled = env::var("WEBHOOK")
|
||||
.is_ok_and(|v| matches!(v.to_lowercase().as_str(), "true" | "yes" | "1"));
|
||||
@@ -92,6 +98,7 @@ impl Config {
|
||||
admin_ids,
|
||||
edit_message_ttl,
|
||||
link_cache_ttl,
|
||||
caption_quote_text_chars,
|
||||
webhook_enabled,
|
||||
webhook_url,
|
||||
webhook_listen,
|
||||
|
||||
@@ -88,6 +88,12 @@ pub(crate) mod test_support {
|
||||
&self.chat_store
|
||||
}
|
||||
|
||||
/// The parsed config, mutable so a test can pin a knob (e.g. the
|
||||
/// caption-quote threshold) instead of depending on the environment.
|
||||
pub(crate) fn config_mut(&mut self) -> &mut Config {
|
||||
&mut self.config
|
||||
}
|
||||
|
||||
pub(crate) fn link_cache(&self) -> &LinkCache {
|
||||
&self.link_cache
|
||||
}
|
||||
|
||||
@@ -279,7 +279,7 @@ pub(crate) async fn execute_command(
|
||||
bot,
|
||||
message.chat.id.0,
|
||||
message.id,
|
||||
"Unknown site. Use twitter, bsky, pixiv or misskey.",
|
||||
"Unknown site. Use twitter, bsky, pixiv, misskey or bilibili.",
|
||||
)
|
||||
.await?;
|
||||
return Ok(());
|
||||
@@ -320,7 +320,7 @@ pub(crate) async fn execute_command(
|
||||
bot,
|
||||
message.chat.id.0,
|
||||
message.id,
|
||||
"Unrecognized link. Use a twitter/x, pixiv, bsky or misskey post URL.",
|
||||
"Unrecognized link. Use a twitter/x, pixiv, bsky, misskey or bilibili post URL.",
|
||||
)
|
||||
.await?;
|
||||
return Ok(());
|
||||
@@ -358,7 +358,7 @@ pub(crate) async fn execute_command(
|
||||
bot,
|
||||
message.chat.id.0,
|
||||
message.id,
|
||||
"No enabled site matches this link (twitter/x, pixiv, bsky or misskey).",
|
||||
"No enabled site matches this link (twitter/x, pixiv, bsky, misskey or bilibili).",
|
||||
)
|
||||
.await?;
|
||||
return Ok(());
|
||||
@@ -400,7 +400,7 @@ pub(crate) async fn execute_command(
|
||||
bot,
|
||||
message.chat.id.0,
|
||||
message.id,
|
||||
"No enabled site matches this link (twitter/x, pixiv, bsky or misskey).",
|
||||
"No enabled site matches this link (twitter/x, pixiv, bsky, misskey or bilibili).",
|
||||
)
|
||||
.await?;
|
||||
}
|
||||
@@ -419,6 +419,7 @@ pub(crate) async fn execute_command(
|
||||
fetched.site_name(),
|
||||
&fetched.source_url,
|
||||
&fetched.title,
|
||||
&fetched.content,
|
||||
fetched.render_fields(),
|
||||
fetched.sensitive,
|
||||
&fetched.caption,
|
||||
@@ -471,7 +472,8 @@ fn debug_report(
|
||||
site_id: &str,
|
||||
source_url: &str,
|
||||
title: &str,
|
||||
render: Option<(&str, &str, &str, &str)>,
|
||||
content: &str,
|
||||
render: Option<(&str, &str, &str, &str, &str)>,
|
||||
sensitive: bool,
|
||||
caption: &str,
|
||||
media: &[x_media::media::Media],
|
||||
@@ -491,7 +493,8 @@ fn debug_report(
|
||||
html_escape::encode_text(source_url)
|
||||
));
|
||||
lines.push(format!("title: {}", html_escape::encode_text(title)));
|
||||
if let Some((author, author_url, _title, tags)) = render {
|
||||
lines.push(format!("content: {}", html_escape::encode_text(content)));
|
||||
if let Some((author, author_url, _title, _content, tags)) = render {
|
||||
// The render fields are already pre-escaped for HTML captions; embed
|
||||
// them as-is so the report renders them exactly like the final
|
||||
// caption. `author_url` is raw and gets escaped here.
|
||||
@@ -559,7 +562,14 @@ mod tests {
|
||||
"twitter",
|
||||
"https://x.com/u/status/1",
|
||||
"My title",
|
||||
Some(("Author", "https://x.com/u", "My title", "tag1 tag2")),
|
||||
"My content",
|
||||
Some((
|
||||
"Author",
|
||||
"https://x.com/u",
|
||||
"My title",
|
||||
"My content",
|
||||
"tag1 tag2",
|
||||
)),
|
||||
false,
|
||||
"<a href=\"https://x.com/u\">Author</a> · My title",
|
||||
&media,
|
||||
@@ -567,6 +577,7 @@ mod tests {
|
||||
assert!(report.contains("site: twitter"), "{report}");
|
||||
assert!(report.contains("key: twitter:1"), "{report}");
|
||||
assert!(report.contains("title: My title"), "{report}");
|
||||
assert!(report.contains("content: My content"), "{report}");
|
||||
assert!(report.contains("author: Author"), "{report}");
|
||||
assert!(report.contains("author_url: https://x.com/u"), "{report}");
|
||||
assert!(report.contains("tags: tag1 tag2"), "{report}");
|
||||
@@ -584,7 +595,7 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn debug_report_without_render_data_and_no_media() {
|
||||
let report = debug_report("u", "pixiv", "s", "t", None, true, "c", &[]);
|
||||
let report = debug_report("u", "pixiv", "s", "t", "c", None, true, "p", &[]);
|
||||
assert!(!report.contains("author:"), "{report}");
|
||||
assert!(report.contains("sensitive: true"), "{report}");
|
||||
assert!(report.contains("media (0):"), "{report}");
|
||||
@@ -601,10 +612,12 @@ mod tests {
|
||||
"twitter",
|
||||
"https://x.com/u/status/1",
|
||||
"A & B <C>",
|
||||
"body & <more>",
|
||||
Some((
|
||||
"A & B",
|
||||
"https://x.com/u",
|
||||
"A & B <C>",
|
||||
"body & <more>",
|
||||
"#a & #b",
|
||||
)),
|
||||
false,
|
||||
@@ -640,7 +653,7 @@ mod tests {
|
||||
fallback_url: None,
|
||||
})
|
||||
.collect();
|
||||
let report = debug_report("u", "twitter", "s", "t", None, false, "c", &media);
|
||||
let report = debug_report("u", "twitter", "s", "t", "c", None, false, "p", &media);
|
||||
assert!(report.chars().count() <= MAX_DEBUG_REPORT_CHARS, "{report}");
|
||||
assert!(report.ends_with('…'), "{report}");
|
||||
}
|
||||
|
||||
@@ -127,8 +127,23 @@ async fn answer_inline_query(bot: Bot, query: InlineQuery) -> Result<bool, Reque
|
||||
Ok(Some(fetched)) => {
|
||||
let mut results: Vec<InlineQueryResult> = Vec::new();
|
||||
// Inline results have the same 1024-char caption limit as regular
|
||||
// messages; truncate once here for all items.
|
||||
// messages; truncate once here for all items, then apply the same
|
||||
// long-post quoting as the send paths. `answer_inline_query` has no
|
||||
// `AppContext` (the debounce spawns it), so the parsed config comes
|
||||
// from the process-wide static, and the text is the *escaped*
|
||||
// title/content the built-in caption embeds (the raw
|
||||
// `Fetched.title`/`content` differ whenever the post contains
|
||||
// `<`/`&`).
|
||||
let caption = x_media::site::truncate_caption(&fetched.caption);
|
||||
let text = fetched
|
||||
.render_fields()
|
||||
.map(|(_, _, title, content, _)| x_media::site::compose_text(title, content))
|
||||
.unwrap_or_default();
|
||||
let caption = crate::send::quote_long_caption(
|
||||
&caption,
|
||||
&text,
|
||||
super::CONFIG.caption_quote_text_chars,
|
||||
);
|
||||
for (i, media) in fetched.media.iter().enumerate() {
|
||||
let id = format!("{i}");
|
||||
let Some(url) = url::Url::parse(media.url()).ok() else {
|
||||
@@ -138,7 +153,7 @@ async fn answer_inline_query(bot: Bot, query: InlineQuery) -> Result<bool, Reque
|
||||
.thumbnail_url()
|
||||
.and_then(|t| url::Url::parse(t).ok())
|
||||
.unwrap_or_else(|| url.clone());
|
||||
let caption = caption.clone();
|
||||
let caption = caption.clone().into_owned();
|
||||
let result = match media {
|
||||
Media::Illustration { .. } => {
|
||||
// Inline photo results have their own (smaller) size
|
||||
|
||||
@@ -60,8 +60,8 @@ pub(crate) async fn reply_html(
|
||||
|
||||
/// Log prefix tying the whole lifecycle of one link (fetch → send → cache →
|
||||
/// forward) together: the normalized cache key (`twitter:123…`, `pixiv:123`,
|
||||
/// `bsky:handle/rkey`) instead of the raw URL, so logs stay short and do not
|
||||
/// echo full user-submitted URLs at info level.
|
||||
/// `bsky:handle/rkey`, `bilibili:123…`) instead of the raw URL, so logs stay
|
||||
/// short and do not echo full user-submitted URLs at info level.
|
||||
pub fn log_key(url: &str) -> String {
|
||||
x_media::site::cache_key(url).unwrap_or_else(|| "<unsupported>".to_string())
|
||||
}
|
||||
|
||||
@@ -327,6 +327,7 @@ pub(crate) async fn url_media(
|
||||
&cached.author,
|
||||
&cached.author_url,
|
||||
&cached.title,
|
||||
&cached.content,
|
||||
&cached.tags,
|
||||
)
|
||||
};
|
||||
@@ -406,18 +407,20 @@ pub(crate) async fn url_media(
|
||||
let caption = fetched.caption_with(&format);
|
||||
// Raw render data for the link cache; the send fills in the
|
||||
// Telegram file ids and persists the entry.
|
||||
let cache_data = fetched
|
||||
.render_fields()
|
||||
.map(|(author, author_url, title, tags)| CachedPost {
|
||||
url: fetched.source_url.clone(),
|
||||
caption: fetched.caption.clone(),
|
||||
title: title.to_string(),
|
||||
author: author.to_string(),
|
||||
author_url: author_url.to_string(),
|
||||
tags: tags.to_string(),
|
||||
sensitive: fetched.sensitive,
|
||||
media: vec![],
|
||||
});
|
||||
let cache_data =
|
||||
fetched
|
||||
.render_fields()
|
||||
.map(|(author, author_url, title, content, tags)| CachedPost {
|
||||
url: fetched.source_url.clone(),
|
||||
caption: fetched.caption.clone(),
|
||||
title: title.to_string(),
|
||||
content: content.to_string(),
|
||||
author: author.to_string(),
|
||||
author_url: author_url.to_string(),
|
||||
tags: tags.to_string(),
|
||||
sensitive: fetched.sensitive,
|
||||
media: vec![],
|
||||
});
|
||||
let items: Vec<MediaItemPayload> = fetched
|
||||
.media
|
||||
.iter()
|
||||
@@ -465,6 +468,7 @@ mod tests {
|
||||
url: "https://x.com/u/status/1".into(),
|
||||
caption: "cap".into(),
|
||||
title: "t".into(),
|
||||
content: "c".into(),
|
||||
author: "a".into(),
|
||||
author_url: "au".into(),
|
||||
tags: "".into(),
|
||||
@@ -530,6 +534,35 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
/// The caption-quote threshold matches the post's text inside the caption,
|
||||
/// so a long-text cache hit is quoted and a short-text one is not.
|
||||
#[tokio::test]
|
||||
async fn cache_hit_quotes_a_long_text_caption() {
|
||||
let mut stores = TestStores::new();
|
||||
stores.config_mut().caption_quote_text_chars = 3;
|
||||
let prefix = "https://x.com/u/status/1\n<a href=\"au\">a</a>: ";
|
||||
|
||||
for (text, expected) in [
|
||||
(
|
||||
"abc",
|
||||
format!("{prefix}<blockquote expandable>abc</blockquote>"),
|
||||
),
|
||||
("ab", format!("{prefix}ab")),
|
||||
] {
|
||||
let sender = MockSender::scripted(vec![Outcome::GroupOk], permanent_error);
|
||||
let ctx = stores.ctx(&sender);
|
||||
let mut entry = cached_photo_entry();
|
||||
entry.caption = format!("{prefix}{text}");
|
||||
entry.title = String::new();
|
||||
entry.content = text.into();
|
||||
stores.link_cache().put("twitter:1", &entry).await;
|
||||
|
||||
url_media(&ctx, 1, 2, "https://x.com/u/status/1", PostSend::FromChat).await;
|
||||
|
||||
assert_eq!(sender.captions(), vec![expected], "text {text:?}");
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn unsupported_url_is_ignored_silently() {
|
||||
let stores = TestStores::new();
|
||||
|
||||
@@ -38,6 +38,10 @@ pub struct CachedPost {
|
||||
/// override).
|
||||
pub caption: String,
|
||||
pub title: String,
|
||||
/// The post's body text. Defaulted on read: entries written before the
|
||||
/// title/content split carry it inside `title`.
|
||||
#[serde(default)]
|
||||
pub content: String,
|
||||
pub author: String,
|
||||
pub author_url: String,
|
||||
pub tags: String,
|
||||
@@ -182,6 +186,7 @@ mod tests {
|
||||
url: "https://x.com/u/status/1".into(),
|
||||
caption: "cap".into(),
|
||||
title: "t".into(),
|
||||
content: "c".into(),
|
||||
author: "a".into(),
|
||||
author_url: "au".into(),
|
||||
tags: "".into(),
|
||||
@@ -193,6 +198,49 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
/// A payload written before the title/content split has no `content`
|
||||
/// field. It must still read back — the cache deletes what it cannot
|
||||
/// parse — with its text left where it was stored (`title`) and the
|
||||
/// caption it replays untouched. No migration: a self-hosted cache entry
|
||||
/// lives one TTL, and moving the text would only reshuffle `/set_format`
|
||||
/// placeholders until it expires.
|
||||
#[tokio::test]
|
||||
async fn pre_split_entry_still_parses() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let cache = LinkCache::new(
|
||||
crate::db::open_store(dir.path().join("c.db").to_str().unwrap()).unwrap(),
|
||||
);
|
||||
let legacy = serde_json::json!({
|
||||
"url": "https://x.com/u/status/1",
|
||||
"caption": "https://x.com/u/status/1\n<a href=\"au\">a</a>: old text",
|
||||
"title": "old text",
|
||||
"author": "a",
|
||||
"author_url": "au",
|
||||
"tags": "",
|
||||
"sensitive": false,
|
||||
"media": [{"kind": "photo", "file_id": "AgAC..."}]
|
||||
});
|
||||
{
|
||||
let conn = rusqlite::Connection::open(dir.path().join("c.db")).unwrap();
|
||||
conn.execute(
|
||||
"INSERT INTO link_cache (url, payload, created_at) VALUES (?1, ?2, ?3)",
|
||||
params!["twitter:1", legacy.to_string(), now_f64()],
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
let got = cache
|
||||
.get("twitter:1", Duration::from_secs(3600))
|
||||
.await
|
||||
.expect("a pre-split payload must not be dropped");
|
||||
assert_eq!(got.title, "old text");
|
||||
assert_eq!(got.content, "");
|
||||
assert_eq!(
|
||||
got.caption,
|
||||
"https://x.com/u/status/1\n<a href=\"au\">a</a>: old text"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn put_get_roundtrip() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
|
||||
@@ -327,9 +327,20 @@ pub(crate) mod test_support {
|
||||
&self,
|
||||
_chat_id: ChatId,
|
||||
_reply_to: MessageId,
|
||||
_items: Vec<InputMedia>,
|
||||
items: Vec<InputMedia>,
|
||||
) -> BoxFuture<'_, Result<Vec<Message>, RequestError>> {
|
||||
Box::pin(async move {
|
||||
// Record the captions exactly as Telegram receives them (only
|
||||
// the first item of a group carries one), so tests can assert
|
||||
// what a recipient sees.
|
||||
self.captions
|
||||
.lock()
|
||||
.extend(items.iter().filter_map(|item| match item {
|
||||
InputMedia::Photo(photo) => photo.caption.clone(),
|
||||
InputMedia::Video(video) => video.caption.clone(),
|
||||
InputMedia::Animation(animation) => animation.caption.clone(),
|
||||
_ => None,
|
||||
}));
|
||||
match self.next("send_media_group") {
|
||||
Outcome::GroupOk => Ok(Vec::new()),
|
||||
Outcome::GroupErr => Err(self.error()),
|
||||
|
||||
@@ -17,8 +17,8 @@ use crate::link_cache::{CachedMedia, CachedMediaKind, CachedPost};
|
||||
use crate::media_sender::MediaSender;
|
||||
use input_media::{build_media_group, input_file_for, item_url};
|
||||
use post_send::{cache_animation_send, cache_sent_task};
|
||||
use rand::Rng;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::borrow::Cow;
|
||||
use std::sync::LazyLock;
|
||||
use teloxide::prelude::*;
|
||||
use teloxide::types::{ChatId, InputFile, InputMedia, MessageId};
|
||||
@@ -261,7 +261,7 @@ pub fn photos_first(items: Vec<MediaItemPayload>) -> Vec<MediaItemPayload> {
|
||||
|
||||
/// Exponential backoff with jitter, capped at 30s.
|
||||
pub fn retry_delay_seconds(attempts: u32) -> f64 {
|
||||
let jitter: f64 = rand::thread_rng().gen_range(0.2..0.8);
|
||||
let jitter: f64 = rand::random_range(0.2..0.8);
|
||||
(2f64.powi(attempts as i32) + jitter).min(30.0)
|
||||
}
|
||||
|
||||
@@ -389,6 +389,61 @@ fn updated_sequence_task(task: &Task, batch_index: usize, sent_message_ids: Vec<
|
||||
updated
|
||||
}
|
||||
|
||||
/// The caption's text tail: everything after the author link, provided it
|
||||
/// really is the post's text.
|
||||
///
|
||||
/// `text` is the *escaped* title + content the caption embeds; the caption may
|
||||
/// have been truncated inside it, in which case only its prefix is present, so
|
||||
/// the tail only has to match the text's start. `None` for a caption with
|
||||
/// another layout — pixiv's title-inside-a-link, a `/set_format` that moves
|
||||
/// `{title}`/`{content}` off the author line — which is left unquoted instead
|
||||
/// of guessing where the text begins.
|
||||
fn text_tail<'c>(caption: &'c str, text: &str) -> Option<&'c str> {
|
||||
let (_, tail) = caption.rsplit_once("</a>: ")?;
|
||||
let visible = tail.strip_suffix('\u{2026}').unwrap_or(tail);
|
||||
(!visible.is_empty() && text.starts_with(visible)).then_some(tail)
|
||||
}
|
||||
|
||||
/// The text a task's caption embeds, read from the same cache snapshot the
|
||||
/// caption came from: `title` and `content` joined the way the sites' built-in
|
||||
/// captions join them.
|
||||
fn task_text(task: &Task) -> String {
|
||||
task.cache_data()
|
||||
.map(|data| x_media::site::compose_text(&data.title, &data.content))
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
/// Wraps the post's text inside the caption in an expandable blockquote once
|
||||
/// that text is long enough that the message would otherwise be a wall of text
|
||||
/// (`threshold` is `CAPTION_QUOTE_TEXT_CHARS`; `0` disables the wrap). The URL
|
||||
/// and the author line stay outside the quote.
|
||||
///
|
||||
/// Applied at the send boundary, after the caller's `truncate_caption`:
|
||||
/// Telegram measures a caption *after entities parsing*, so the tags cost no
|
||||
/// length and a wrapped caption cannot exceed the 1024-character limit.
|
||||
/// Retries replay the task's (unwrapped) caption, so the decision is remade on
|
||||
/// every attempt — changing the threshold takes effect immediately.
|
||||
///
|
||||
/// A caption that already carries a blockquote is left as it is: the API
|
||||
/// rejects nested ones ("all other entities can't contain each other"), and a
|
||||
/// user-written `/set_format` template may contain one.
|
||||
pub(crate) fn quote_long_caption<'a>(
|
||||
caption: &'a str,
|
||||
text: &str,
|
||||
threshold: usize,
|
||||
) -> Cow<'a, str> {
|
||||
if threshold == 0 || caption.contains("<blockquote") || text.chars().count() < threshold {
|
||||
return Cow::Borrowed(caption);
|
||||
}
|
||||
let Some(tail) = text_tail(caption, text) else {
|
||||
return Cow::Borrowed(caption);
|
||||
};
|
||||
let prefix = &caption[..caption.len() - tail.len()];
|
||||
Cow::Owned(format!(
|
||||
"{prefix}<blockquote expandable>{tail}</blockquote>"
|
||||
))
|
||||
}
|
||||
|
||||
/// Sends the media batches starting at `task.batch_index`, extending
|
||||
/// `sent_message_ids`. Returns all sent message ids on full success; on
|
||||
/// failure returns a [`SendError`] whose task carries the resumed state.
|
||||
@@ -407,6 +462,10 @@ pub async fn send_media_sequence(ctx: &AppContext<'_>, task: &Task) -> Result<Ve
|
||||
};
|
||||
let chat_id = *chat_id;
|
||||
let reply_to = *reply_to_message_id;
|
||||
// A long post is quoted so the message reads as a card rather than a wall
|
||||
// of text; the text comes from the same cache snapshot as the caption.
|
||||
let text = task_text(task);
|
||||
let caption = quote_long_caption(caption, &text, ctx.config.caption_quote_text_chars);
|
||||
let mut sent = sent_message_ids.clone();
|
||||
// File ids accumulated across batches for the link cache. Only a fresh
|
||||
// (non-resumed) full send populates the cache.
|
||||
@@ -415,7 +474,7 @@ pub async fn send_media_sequence(ctx: &AppContext<'_>, task: &Task) -> Result<Ve
|
||||
for idx in *batch_index..media_batches.len() {
|
||||
let batch = &media_batches[idx];
|
||||
let caption = if idx == 0 {
|
||||
Some(caption.as_str())
|
||||
Some(caption.as_ref())
|
||||
} else {
|
||||
None
|
||||
};
|
||||
@@ -516,6 +575,10 @@ pub async fn send_animation(ctx: &AppContext<'_>, task: &Task) -> Result<Vec<i64
|
||||
};
|
||||
let chat_id = *chat_id;
|
||||
let reply_to = *reply_to_message_id;
|
||||
// Same long-post quoting as the media-group path (see
|
||||
// `quote_long_caption`); both sends below share this string.
|
||||
let text = task_text(task);
|
||||
let caption = quote_long_caption(caption, &text, ctx.config.caption_quote_text_chars);
|
||||
let (media_url, has_spoiler) = match animation {
|
||||
MediaItemPayload::Animation {
|
||||
media, has_spoiler, ..
|
||||
@@ -537,7 +600,7 @@ pub async fn send_animation(ctx: &AppContext<'_>, task: &Task) -> Result<Vec<i64
|
||||
ctx.sender,
|
||||
chat_id,
|
||||
reply_to,
|
||||
caption,
|
||||
&caption,
|
||||
has_spoiler,
|
||||
url_file,
|
||||
)
|
||||
@@ -572,7 +635,7 @@ pub async fn send_animation(ctx: &AppContext<'_>, task: &Task) -> Result<Vec<i64
|
||||
ctx.sender,
|
||||
chat_id,
|
||||
reply_to,
|
||||
caption,
|
||||
&caption,
|
||||
has_spoiler,
|
||||
animation.media,
|
||||
)
|
||||
@@ -931,10 +994,16 @@ mod tests {
|
||||
}
|
||||
|
||||
fn sequence_task(media: &str) -> Task {
|
||||
sequence_task_with(media, "cap", None)
|
||||
}
|
||||
|
||||
/// A media-group task; `text` (when given) rides in the link-cache
|
||||
/// snapshot as `content`, which is where the quote threshold reads it.
|
||||
fn sequence_task_with(media: &str, caption: &str, text: Option<&str>) -> Task {
|
||||
Task::SendMediaSequence {
|
||||
chat_id: 1,
|
||||
reply_to_message_id: 2,
|
||||
caption: "cap".into(),
|
||||
caption: caption.into(),
|
||||
media_batches: vec![vec![MediaItemPayload::Photo {
|
||||
media: media.to_string(),
|
||||
has_spoiler: false,
|
||||
@@ -948,7 +1017,99 @@ mod tests {
|
||||
forward_channel_id: None,
|
||||
notify_chat_id: Some(1),
|
||||
notify_message_id: Some(2),
|
||||
cache_data: None,
|
||||
// The snapshot splits the post's text into title/content the way a
|
||||
// real fetch does; the quote threshold joins them again.
|
||||
cache_data: text.map(|text| CachedPost {
|
||||
url: "https://x.com/u/status/1".into(),
|
||||
caption: caption.into(),
|
||||
title: String::new(),
|
||||
content: text.into(),
|
||||
author: "me".into(),
|
||||
author_url: "https://x.com/u".into(),
|
||||
tags: String::new(),
|
||||
sensitive: false,
|
||||
media: vec![],
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
/// A media-group task whose built-in caption carries `text` behind the
|
||||
/// author link — the shape the quote threshold locates the text in.
|
||||
fn sequence_task_with_text(media: &str, text: &str) -> Task {
|
||||
let caption =
|
||||
format!("https://x.com/u/status/1\n<a href=\"https://x.com/u\">me</a>: {text}");
|
||||
sequence_task_with(media, &caption, Some(text))
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn quote_long_caption_wraps_only_the_text_tail() {
|
||||
let text = "一二三四五";
|
||||
let prefix = "https://x.com/u/status/1\n<a href=\"https://x.com/u\">me</a>: ";
|
||||
let caption = format!("{prefix}{text}");
|
||||
|
||||
// Only the text goes inside the quote; the URL and author line stay
|
||||
// outside.
|
||||
assert_eq!(
|
||||
quote_long_caption(&caption, text, 5),
|
||||
format!("{prefix}<blockquote expandable>{text}</blockquote>")
|
||||
);
|
||||
// One char below the threshold, disabled, and a short text: untouched.
|
||||
assert_eq!(
|
||||
quote_long_caption(&caption, text, 6),
|
||||
format!("{prefix}{text}")
|
||||
);
|
||||
assert_eq!(quote_long_caption(&caption, text, 0), caption);
|
||||
// No author-line anchor means no text to locate — a pixiv caption
|
||||
// (title inside the link) and a `{content}`-first format stay as they
|
||||
// are rather than risking a blockquote nested in a tag.
|
||||
let pixiv =
|
||||
format!("<a href=\"https://pixiv.net/1\">{text}</a> / <a href=\"u\">me</a>\ntag");
|
||||
assert_eq!(quote_long_caption(&pixiv, text, 5), pixiv);
|
||||
let content_first = format!("{text}\nhttps://x.com/u/status/1");
|
||||
assert_eq!(quote_long_caption(&content_first, text, 5), content_first);
|
||||
// An empty body has nothing to quote.
|
||||
assert_eq!(quote_long_caption(prefix, text, 5), prefix);
|
||||
// A caption that already carries a blockquote is never nested.
|
||||
let quoted = format!("<blockquote>{caption}</blockquote>");
|
||||
assert_eq!(quote_long_caption("ed, text, 5), quoted);
|
||||
}
|
||||
|
||||
/// `truncate_caption` cuts inside the text and appends an ellipsis; the
|
||||
/// visible prefix still marks it, so the long-text case that most needs
|
||||
/// quoting is still quoted.
|
||||
#[test]
|
||||
fn quote_long_caption_wraps_a_truncated_text() {
|
||||
let text = "一二三四五六七八九十";
|
||||
let prefix = "https://x.com/u/status/1\n<a href=\"https://x.com/u\">me</a>: ";
|
||||
let caption = format!("{prefix}一二三四五…");
|
||||
assert_eq!(
|
||||
quote_long_caption(&caption, text, 5),
|
||||
format!("{prefix}<blockquote expandable>一二三四五…</blockquote>")
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn long_text_caption_reaches_telegram_quoted() {
|
||||
// The threshold is pinned here instead of read from the environment.
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let file = dir.path().join("media.jpg");
|
||||
std::fs::write(&file, b"not-a-real-jpeg").unwrap();
|
||||
let mut stores = TestStores::new();
|
||||
stores.config_mut().caption_quote_text_chars = 5;
|
||||
let prefix = "https://x.com/u/status/1\n<a href=\"https://x.com/u\">me</a>: ";
|
||||
|
||||
for (text, expected) in [
|
||||
(
|
||||
"一二三四五",
|
||||
format!("{prefix}<blockquote expandable>一二三四五</blockquote>"),
|
||||
),
|
||||
("一二三四", format!("{prefix}一二三四")),
|
||||
] {
|
||||
let sender = MockSender::scripted(vec![Outcome::GroupOk], media_fetch_error);
|
||||
let ctx = stores.ctx(&sender);
|
||||
let task = sequence_task_with_text(file.to_str().unwrap(), text);
|
||||
assert!(send_media_sequence(&ctx, &task).await.is_ok());
|
||||
assert_eq!(sender.captions(), vec![expected], "text {text:?}");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1260,6 +1421,7 @@ mod tests {
|
||||
url: "https://x.com/u/status/1".into(),
|
||||
caption: "cap".into(),
|
||||
title: "t".into(),
|
||||
content: "c".into(),
|
||||
author: "a".into(),
|
||||
author_url: "au".into(),
|
||||
tags: String::new(),
|
||||
|
||||
@@ -17,8 +17,8 @@ pub struct ChatData {
|
||||
pub edit_message: HashMap<i64, EditMessage>,
|
||||
/// name -> HTML template containing "[]"
|
||||
pub template: HashMap<String, String>,
|
||||
/// site name (twitter/bsky/misskey/pixiv) -> user-supplied caption format
|
||||
/// with {url} {author} {author_url} {title} {tags} placeholders.
|
||||
/// site name (twitter/bsky/misskey/pixiv/bilibili) -> user-supplied caption format
|
||||
/// with {url} {author} {author_url} {title} {content} {tags} placeholders.
|
||||
pub message_format: HashMap<String, String>,
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user