mirror of
https://github.com/TheFunny/TelegramTwitterMediaBot.git
synced 2026-09-23 23:32:05 +00:00
feat(sites): add bilibili dynamic support (images and animated images)
Fetch `t.bilibili.com/<id>`, `www.bilibili.com/opus/<id>`, `t.bilibili.com/h5/dynamic/detail/<id>` and `m.bilibili.com/dynamic/<id>` through the anonymous `/x/polymer/web-dynamic/v1/detail` endpoint (no cookie, no WBI signature; the site adds the device cookies `/x/frontend/finger/spi` hands out, which is what lifts bilibili's `-352` risk control). Media: the `major.draw` grid (`.gif` sources become animations, the rest photos with a downscaled `@518w.jpg` thumbnail used both as preview and as the oversized fallback), an attached video's cover, and the quoted dynamic's media for forwards. The video stream itself is not resolved; `b23.tv` short links stay unmatched (they mostly point at videos, so matching them would turn a silently ignored link into a failure reply). `-352`/`-412` map to a retryable error so the queue backs off instead of dropping the post; a removed dynamic (`500`) is permanent. Registry-driven, so no bot-side code changes beyond the site lists in the command replies; found while researching nazurin and telegram-bili-feed-helper (see BILIBILI_PLAN.md).
This commit is contained in:
@@ -2,7 +2,7 @@
|
||||
|
||||
## Project Overview
|
||||
|
||||
Telegram bot (teloxide) that turns post links from X/Twitter, Pixiv, Bluesky, and Misskey (misskey.io) into media messages (images, video, GIF) with the post's title, author, and tags. It supports batch media splitting, retry with persistence, inline queries, forward-channel rebinding with caption templates, and Pixiv ugoira→MP4 transcoding. README is in Chinese; user-facing bot strings are in English. The project is a Rust port of a Python predecessor (see `queue.rs` comments referencing `utils/task_queue.py`).
|
||||
Telegram bot (teloxide) that turns post links from X/Twitter, Pixiv, Bluesky, Misskey (misskey.io), and Bilibili dynamics into media messages (images, video, GIF) with the post's title, author, and tags. It supports batch media splitting, retry with persistence, inline queries, forward-channel rebinding with caption templates, and Pixiv ugoira→MP4 transcoding. README is in Chinese; user-facing bot strings are in English. The project is a Rust port of a Python predecessor (see `queue.rs` comments referencing `utils/task_queue.py`).
|
||||
|
||||
Two-crate Cargo workspace (both v1.6.0, edition 2024, resolver 3):
|
||||
|
||||
@@ -24,14 +24,14 @@ Debug command: `/debug <url>` runs the same `x_media::site::fetch` and replies w
|
||||
|
||||
The `/test <url>` command runs the ordinary link pipeline (`urls::url_media`) with `PostSend::Suppressed`: the media is sent and cached like any other link, but the chat's `forward_channel_id`/`edit_before_forward` are ignored, so a test never forwards to the channel and never opens the edit prompt (retries and dead-letter notifications behave as usual). Both commands use a custom `parse_arg_remainder` parser (whole remainder, trimmed) because teloxide's built-in `split` parser takes exactly one space-separated token.
|
||||
|
||||
The `x-media` library: `site::fetch(url)` dispatches through the `SITES` registry (per-site `impl Site`, in order twitter → bsky → misskey → pixiv) and returns `Ok(None)` for unmatched URLs. `Fetched { source_url, caption, title, media: Vec<Media>, sensitive, site_id, … }`; `caption_with(format)` substitutes `{url} {author} {author_url} {title} {tags}`.
|
||||
The `x-media` library: `site::fetch(url)` dispatches through the `SITES` registry (per-site `impl Site`, in order twitter → bsky → misskey → pixiv → bilibili) and returns `Ok(None)` for unmatched URLs. `Fetched { source_url, caption, title, media: Vec<Media>, sensitive, site_id, … }`; `caption_with(format)` substitutes `{url} {author} {author_url} {title} {tags}`.
|
||||
|
||||
## Key Directories
|
||||
|
||||
| Path | Purpose |
|
||||
|---|---|
|
||||
| `crates/x-media/src/` | Fetch library. `site/mod.rs` = dispatcher + `Fetched`/`FetchError`/`download_media`/`media_size`; `media.rs` = `Media` enum; `examples/fetch.rs` = end-to-end usage sample |
|
||||
| `crates/x-media/src/site/<twitter\|pixiv\|bsky\|misskey>/` | One directory per site: `mod.rs` (re-exports), `interface.rs` (PATTERN, `enabled()`, `fetch_from_url()`, `cache_key`/`is_retryable`/`media_headers`, unit struct `<Name>Site` implementing `site::Site`, `From<SiteStruct> for Fetched`), `model.rs` (serde DTOs). Pixiv adds `api.rs` (auth + transport); twitter adds `auth.rs` (logged-in GraphQL `TweetDetail` fallback for NSFW tweets, gated on `TWITTER_AUTH_TOKEN`). Misskey targets misskey.io only (`POST /api/notes/show`, 400+`NO_SUCH_NOTE` → NotFound). Twitter's `from_syndication_json` HTML-decodes the API text — syndication and GraphQL `full_text` both arrive pre-escaped (`>` `<` `&` `'`) — so the stored text is raw and the caption escapes exactly once |
|
||||
| `crates/x-media/src/site/<twitter\|pixiv\|bsky\|misskey\|bilibili>/` | One directory per site: `mod.rs` (re-exports), `interface.rs` (PATTERN, `enabled()`, `fetch_from_url()`, `cache_key`/`is_retryable`/`media_headers`, unit struct `<Name>Site` implementing `site::Site`, `From<SiteStruct> for Fetched`), `model.rs` (serde DTOs). Pixiv adds `api.rs` (auth + transport); twitter adds `auth.rs` (logged-in GraphQL `TweetDetail` fallback for NSFW tweets, gated on `TWITTER_AUTH_TOKEN`). Misskey targets misskey.io only (`POST /api/notes/show`, 400+`NO_SUCH_NOTE` → NotFound). Bilibili fetches dynamics (images/animated images only — an attached video degrades to its cover) from the anonymous `/x/polymer/web-dynamic/v1/detail` (no WBI signature; device cookies `buvid3`/`buvid4` are fetched automatically from `/x/frontend/finger/spi` because bilibili's `-352` risk control starts rejecting plain requests, `BILIBILI_COOKIE` is the escalation when an IP stays blocked; `b23.tv` short links are deliberately unmatched). Twitter's `from_syndication_json` HTML-decodes the API text — syndication and GraphQL `full_text` both arrive pre-escaped (`>` `<` `&` `'`) — so the stored text is raw and the caption escap…
|
||||
| `crates/xmedia-bot/src/main.rs` | Entry point: env/log init, command registration (`register_commands`), shared `send::BOT` force-init, queue worker start, site login validation (`site::validate_all`), 300 s edit-expiry sweep, dptree handler tree, webhook vs polling dispatch |
|
||||
| `crates/xmedia-bot/src/config.rs` | Manual env parsing into `Config` |
|
||||
| `crates/xmedia-bot/src/db.rs` | `DbPool`: one shared SQLite connection pool (`POOL_SIZE = 4`, WAL, busy_timeout) for all three tables over `$DATA_DIR/task_queue.db` (default `data/`) — the three stores share it; `open_store` creates file + schema, `with_conn` runs all rusqlite I/O in `spawn_blocking` |
|
||||
@@ -92,16 +92,16 @@ Docker: `docker build -t tgxmb .` then `docker run --rm -d --name tgxmb --env-fi
|
||||
- Package manager: **Cargo** (workspace with path dep `x-media` ← `xmedia-bot`). No `[workspace.package]`/shared deps — each crate lists deps independently.
|
||||
- **TLS is rustls end-to-end** (no native-tls/openssl in the tree, no libssl in the Docker runtime image): `teloxide` is declared `default-features = false` with `["webhooks-axum", "macros", "rustls", "ctrlc_handler"]` (the removed `default` also carried `native-tls` and `ctrlc_handler` — the latter must stay); x-media's reqwest is `default-features = false` with `["json", "rustls-tls"]` (webpki-roots baked in, so the image ships no CA bundle). One reqwest 0.12.28 in the lock.
|
||||
- **Versioning**: bump the version in all three places (`crates/x-media/Cargo.toml`, `crates/xmedia-bot/Cargo.toml`, `Cargo.lock`) and **keep `README.md`, `README.en.md` and `AGENTS.md` in sync with the code on every bump**, then commit (`chore: bump version to X.Y.Z`), create an annotated tag `vX.Y.Z`, and push branch + tag (the tag push triggers the Docker Hub build). The tag must equal both crate versions: `.github/workflows/docker.yml` verifies that before building, and `--locked` verifies the lock file.
|
||||
- Config is **environment-variable driven** (dotenv loads `.env`, gitignored; no `.env.example` exists). Key vars: `TELOXIDE_TOKEN` (required), `PIXIV_REFRESH_TOKEN`, `TWITTER_AUTH_TOKEN` (optional; x.com `auth_token` cookie — enables the logged-in GraphQL fallback that fetches NSFW tweets syndication withholds), `BOT_ADMIN` (comma-separated ids), `EDIT_MESSAGE_TTL_SECONDS` (default 86400), `LINK_CACHE_TTL_SECONDS` (default 604800), `DATA_DIR` (default `data`, CWD-relative; the SQLite dir, auto-created), `WEBHOOK`/`WEBHOOK_URL`/`WEBHOOK_LISTEN`/`WEBHOOK_PORT`/`WEBHOOK_CERT`/`WEBHOOK_SECRET_TOKEN` (webhook mode requires URL/listen/port, `.expect`ed; `WEBHOOK_CERT` is Telegram-facing self-signed validation only — TLS must be terminated by a reverse proxy), `RUST_LOG`, `TELOXIDE_PROXY`, `LOCAL_USER_ID` (entrypoint only).
|
||||
- Config is **environment-variable driven** (dotenv loads `.env`, gitignored; no `.env.example` exists). Key vars: `TELOXIDE_TOKEN` (required), `PIXIV_REFRESH_TOKEN`, `TWITTER_AUTH_TOKEN` (optional; x.com `auth_token` cookie — enables the logged-in GraphQL fallback that fetches NSFW tweets syndication withholds), `BILIBILI_COOKIE` (optional; whole bilibili cookie string — bilibili dynamics fetch anonymously and add their own device cookies, this only rescues an egress IP that bilibili has hard-flagged with `-352`/412), `BOT_ADMIN` (comma-separated ids), `EDIT_MESSAGE_TTL_SECONDS` (default 86400), `LINK_CACHE_TTL_SECONDS` (default 604800), `DATA_DIR` (default `data`, CWD-relative; the SQLite dir, auto-created), `WEBHOOK`/`WEBHOOK_URL`/`WEBHOOK_LISTEN`/`WEBHOOK_PORT`/`WEBHOOK_CERT`/`WEBHOOK_SECRET_TOKEN` (webhook mode requires URL/listen/port, `.expect`ed; `WEBHOOK_CERT` is Telegram-facing self-signed validation only — TLS must be terminated by a reverse proxy), `RUST_LOG`, `TELOXIDE_PROXY`, `LOCAL_USER_ID` (entrypoint only).
|
||||
- SQLite via `rusqlite` with `bundled` feature (no system libsqlite needed). DB file `$DATA_DIR/task_queue.db` (default `data/task_queue.db`, CWD-relative — run from the workspace root, or `/app` in Docker; set `DATA_DIR` to pin state anywhere). Mount `./data` and `./cert` volumes.
|
||||
- `.gitattributes` enforces LF for `*.sh` (CRLF breaks shebangs in containers). `.gitignore`: `.env`, `data/`, `cert/`, `docker-compose.yml`, `/target`, `.idea/`.
|
||||
- Docs are in Chinese (README, AGENTS.md); user-facing bot strings are in English. Keep that split when editing user-facing strings and docs.
|
||||
|
||||
## Testing & QA
|
||||
|
||||
- **~135 tests, all inline `#[cfg(test)] mod tests`** — no `tests/` integration directories. Framework: built-in Rust test + `#[tokio::test]` (dev-deps only in `x-media`: tokio macros/rt-multi-thread, dotenv).
|
||||
- **~150 tests, all inline `#[cfg(test)] mod tests`** — no `tests/` integration directories. Framework: built-in Rust test + `#[tokio::test]` (dev-deps only in `x-media`: tokio macros/rt-multi-thread, dotenv).
|
||||
- No mocking framework anywhere (no mockito/wiremock/mockall). Conventions: pure-function units (regex parsing, serde round-trips, chunking, retry math) tested synchronously; async tests use real dependencies — file-backed SQLite via `tempfile` (`queue.rs::new_queue()` helper), live network fetches.
|
||||
- Live-network tests exist in `site/twitter/interface.rs` (5), `site/bsky/interface.rs` (2), `site/misskey/interface.rs` (1), `site/pixiv/api.rs` (1); `photo.rs` adds one `#[ignore = "heavy: …"]` test. `site/mod.rs` also has a **token-gated but not `#[ignore]`d** pixiv download test (`download_media_pixiv_original_with_referer`): it hits `i.pximg.net` whenever `PIXIV_REFRESH_TOKEN` is set, so a local `cargo test --workspace` is not fully offline and can flake on a pixiv CDN body timeout. Test gating convention (enforced by `.github/workflows/ci.yml`): pure unit tests always run; live-network tests carry `#[ignore = "live network: ..."]` (run via `cargo test --workspace -- --ignored live`); token-gated pixiv tests early-return when `PIXIV_REFRESH_TOKEN` is absent **or empty** (an unset GitHub secret arrives as `""` — `is_err()` alone would run them tokenless and fail). Run the full offline suite with `cargo test --workspace`.
|
||||
- Live-network tests exist in `site/twitter/interface.rs` (5), `site/bsky/interface.rs` (2), `site/misskey/interface.rs` (1), `site/bilibili/interface.rs` (3), `site/pixiv/api.rs` (1); `photo.rs` adds one `#[ignore = "heavy: …"]` test. `site/mod.rs` also has a **token-gated but not `#[ignore]`d** pixiv download test (`download_media_pixiv_original_with_referer`): it hits `i.pximg.net` whenever `PIXIV_REFRESH_TOKEN` is set, so a local `cargo test --workspace` is not fully offline and can flake on a pixiv CDN body timeout. Test gating convention (enforced by `.github/workflows/ci.yml`): pure unit tests always run; live-network tests carry `#[ignore = "live network: ..."]` (run via `cargo test --workspace -- --ignored live`); token-gated pixiv tests early-return when `PIXIV_REFRESH_TOKEN` is absent **or empty** (an unset GitHub secret arrives as `""` — `is_err()` alone would run them tokenless and fail), and the bilibili live tests early-return when the API answers risk control (`-352`, which bilibili applies per IP by request volume). Run the full offline suite with `cargo test --workspace`.
|
||||
- Fixtures are inline `serde_json::json!` builder fns (`fixture()`, `thread_json()`, `illust_json()`), not files. The shared `CLIENT` sets `pool_max_idle_per_host(0)` under `#[cfg(test)]` to avoid cross-runtime `DispatchGone`.
|
||||
- **CI** — `.github/workflows/ci.yml` (actions pinned to commit SHAs, `--locked` on every cargo invocation, `concurrency` cancels superseded runs, `RUST_BACKTRACE=1`) runs `cargo fmt --check` + `cargo clippy --workspace --all-targets --locked -- -D warnings` + `cargo test --workspace --locked` + a release-profile `cargo build --release --locked` + an `actions-rust-lang/audit` dependency-vulnerability gate (offline, no secrets, on every push/PR) and a `live` job (schedule/manual/tag only, `-p x-media` since every network/secret-gated test lives there, `continue-on-error`) for the `#[ignore]`d live + token tests. `.github/workflows/docker.yml` builds and pushes the image on master/tag and runs a **build-only check on pull requests touching the build inputs** (`Dockerfile`, entrypoint, manifests, `.dockerignore`); a release tag must match both crate versions or the build stops, and `FFMPEG_URL`/`FFMPEG_SHA256` are taken from repository variables when set (a release can pin an exact ffmpeg build). `.github/dependabot.yml` keeps crates, the pinned actions and the Docker base images current.
|
||||
- Untested and hard to test without a mock seam: `main.rs`, `config.rs`, `db.rs`, `handlers/statics.rs`, `media_sender.rs` (holds the `MockSender` itself); in `x-media`: `media.rs`, `lib.rs`, all `model.rs`. The `commands.rs` *executor* needs a real `Bot` (only its pure report builder is tested). Everything else — `handlers/{mod,callback,inline,urls}.rs`, `send/*`, `ctx.rs`, `state.rs`, `queue.rs`, `link_cache.rs`, `rate_limit.rs` — is driven through `TestStores`/`ctx::test_support` and the scripted `MockSender`.
|
||||
|
||||
@@ -0,0 +1,159 @@
|
||||
# Bilibili 动态支持：研究与实现记录
|
||||
|
||||
状态：已实现（`crates/x-media/src/site/bilibili/`）。本文记录上游调研、实测数据与最终设计；
|
||||
长期契约以 `AGENTS.md` 为准。
|
||||
|
||||
范围：**只发动态里的图片与动图**。动态内嵌视频不发流，降级为封面图；`b23.tv` 短链不匹配；
|
||||
视频页 / 番剧 / 直播间 / 专栏 / 音频均不支持。
|
||||
|
||||
---
|
||||
|
||||
## 1. 上游实现研究
|
||||
|
||||
### 1.1 nazurin（`nazurin/sites/bilibili/`，4 个文件 ~6 KB）
|
||||
|
||||
- 入口正则：`t\.bilibili\.com/(\d+)`、`t\.bilibili\.com/h5/dynamic/detail/(\d+)`、`bilibili\.com/opus/(\d+)`。
|
||||
- 请求：`GET https://api.bilibili.com/x/polymer/web-dynamic/v1/detail?id={id}`，仅加 `Referer: https://t.bilibili.com/{id}`。
|
||||
**无 cookie、无 WBI 签名、无 `build` 参数**。
|
||||
- 错误：`code == 4101147` → not found；`code != 0` 或缺 `data` → 报错。
|
||||
- 媒体：只取 `item.modules.module_dynamic.major.draw.items[].src`；缩略图 `src + "@518w.jpg"`；
|
||||
`size` 字段单位是 **KB**。`major` 为空或 `draw.items` 为空 → "No image found"。
|
||||
**忽略视频、转发（forward）与纯文字动态**。
|
||||
- caption：`"#" + module_author.name` + `module_dynamic.desc.text`，链接写死 `https://www.bilibili.com/opus/{id}`。
|
||||
|
||||
### 1.2 telegram-bili-feed-helper（`biliparser/provider/bilibili/`，9 个文件 ~57 KB）
|
||||
|
||||
- 9 个策略类（Video/Opus/Live/Audio/Read + Feed 基类 + Credential + api 工具）：门禁正则
|
||||
`bilibili\.com|b23\.tv|BV\w{10}|av\d+`，再分流，兜底 `client.head(url)` 跟随重定向后按子串分流。
|
||||
- 动态：`GET /x/polymer/web-dynamic/desktop/v1/detail?id={id}&build=11605`（**单条，无分页**）；
|
||||
客户端带桌面 UA、随机 `buvid3={uuid}infoc`；登录态用 `bilibili-api-python` 的 `Credential`
|
||||
（Redis 持久化 `SESSDATA/bili_jct/buvid3/buvid4/ac_time_value/DedeUserID`，扫码登录）。
|
||||
- **同样没有 WBI 签名 / appkey 签名**：playurl 用的是非 WBI 的 `/x/player/playurl`。
|
||||
- 媒体：`major.type` 分派 —— DRAW 取全部 `items[].src`；ARCHIVE/PGC/ARTICLE/MUSIC/COMMON/LIVE
|
||||
只取一张 `cover`；FORWARD 取原动态作者/正文并递归进 `orig` 找媒体。
|
||||
- 视频：仅独立 video 策略解析（`qn` 720P→480P→360P 试 durl，再退 DASH + ffmpeg 合并）；
|
||||
**动态内嵌视频只发封面**。
|
||||
- 错误：要求 `status==200 && code==0`；风控 `-352`/`-412` 无特殊处理。
|
||||
|
||||
### 1.3 取舍
|
||||
|
||||
| 维度 | nazurin | bff | 本仓库 |
|
||||
|---|---|---|---|
|
||||
| 接口 | `v1/detail?id=` | `desktop/v1/detail?id=&build=` | `v1/detail?id=`（实测可用） |
|
||||
| 认证 | 无 | buvid3 + SESSDATA | 默认匿名；可选 `BILIBILI_COOKIE` |
|
||||
| WBI | 无 | 无 | 不实现（无需求） |
|
||||
| 图片 | `major.draw.items` | 同 + forward 递归 | 同，加 `orig` 递归、`http→https`、`.gif → Animated` |
|
||||
| 视频 | 完全忽略 | 动态内嵌视频发封面 | 发封面（不发流） |
|
||||
| 短链 | 不匹配 | 跟随重定向 | 不匹配（多数短链是视频，会让"静默忽略"变成失败提示） |
|
||||
|
||||
---
|
||||
|
||||
## 2. 实测验证（2026-09-17，真实请求）
|
||||
|
||||
| 验证项 | 结果 |
|
||||
|---|---|
|
||||
| `v1/detail?id=`（无 cookie、UA `Mozilla/5.0`、带 Referer） | `200 {"code":0}` ✅ |
|
||||
| 同上，不带 cookie 也不带 Referer | `200 {"code":0}` ✅（无强制鉴权） |
|
||||
| bff 的 `bilibili_pc/…Electron/22.3.27` UA | `code:-352` ❌ → **不要抄它的 UA** |
|
||||
| `desktop/v1/detail?build=11605` | `code:-352` ❌ |
|
||||
| `feed/space?host_mid=`（用户时间线） | 首次成功、随后 `-352`，也见过 HTTP 412 → **不碰** |
|
||||
| 不存在 / 已删除的动态 | `code:500` "Cannot read property 'only_fans' of undefined"（nazurin 的 4101147 已失效） |
|
||||
| 非数字 id | `code:-400` param parsing failed |
|
||||
| 图片 `i0.hdslb.com/bfs/new_dyn/*.jpg` | `HEAD 200 image/jpeg`，带/不带 Referer 均可；`+@518w.jpg` → 25–42 KB ✅ |
|
||||
| `t.bilibili.com/h5/dynamic/detail/<id>` | `200` ✅ |
|
||||
| `m.bilibili.com/dynamic/<id>` | `302 → t.bilibili.com/<id>` ✅ |
|
||||
| `www.bilibili.com/opus/<id>` | `200`，转发动态 `302 → t.bilibili.com/<id>` ✅ |
|
||||
| `b23.tv/BV1JTtt6JEZu` | `302 → www.bilibili.com/video/BV…`（视频） |
|
||||
| `b23.tv/<无效码>` | **HTTP 200** + `{"code":-404}` ⚠️ 短链判定不能只看状态码 |
|
||||
| `playurl`（仅调研用，未采用） | `fnval=1` 匿名给 durl：720P=9.18 MiB / 360P=2.97 MiB；`fnval=4048` 匿名 DASH 上限仅 480P |
|
||||
| `dyn_archive` 字段 | 有 `aid/bvid/cover/title/duration_text`，**没有 `cid`**（所以发流要再来一次 `view` 请求） |
|
||||
| **风控阶梯（同一 IP 连续请求后实测）** | ① 无 cookie → `-352`；② 仅 `buvid3` → 仍 `-352`；③ `buvid3`+`buvid4`（取自匿名 `/x/frontend/finger/spi`）→ **`code:0` 恢复**；④ 继续高频请求后 → 连同 buvid 一起 `-352`（此时只有登录 cookie 或换 IP） |
|
||||
|
||||
测试样本（live 测试用）：
|
||||
|
||||
| 样本 | id | 期望 |
|
||||
|---|---|---|
|
||||
| 图片动态（2 图 + 话题） | `1245284537985925159` | 2 个 `Illustration`，`{tags}` = `ALin出道20周年快乐` |
|
||||
| 转发动态 | `1248982077447077907` | 媒体来自 `orig`（1 图），正文可含 `//@` |
|
||||
| 视频动态 | `1248717597691609105` | 封面 1 张 `Illustration` |
|
||||
| 纯文字动态 | `1246767523595026450` | `media` 为空 |
|
||||
|
||||
关键字段路径：
|
||||
|
||||
```
|
||||
data.item.id_str
|
||||
data.item.modules.module_author.{name,mid}
|
||||
data.item.modules.module_dynamic.desc.text
|
||||
data.item.modules.module_dynamic.topic.{id,name} # 单话题，{tags} 来源
|
||||
data.item.modules.module_dynamic.major.{draw.items[].src, archive.cover}
|
||||
data.item.orig # 转发时存在，结构与 item 相同
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. 实现
|
||||
|
||||
```
|
||||
crates/x-media/src/site/bilibili/mod.rs # re-export
|
||||
crates/x-media/src/site/bilibili/interface.rs # PATTERN / cache_key / enabled / is_retryable /
|
||||
# media_headers / BilibiliSite / fetch / code_error /
|
||||
# From<Item> for Fetched / caption / 12 单测 + 2 live
|
||||
crates/x-media/src/site/bilibili/model.rs # 纯 Deserialize DTO（全 Option）
|
||||
```
|
||||
|
||||
- **正则**（同时用于分发、抽 id、缓存键，一个正则三用）：
|
||||
`^(?:https?://)?(?:www|t|m)\.bilibili\.com/(?:opus/|dynamic/|h5/dynamic/detail/)?(\d+)`
|
||||
- **缓存键**：`bilibili:<动态 id>`；`source_url` 统一 `https://www.bilibili.com/opus/{id}`。
|
||||
- **请求**：`GET /x/polymer/web-dynamic/v1/detail?id=` + `Referer: https://www.bilibili.com/`；
|
||||
`Cookie` 头按优先级取：`BILIBILI_COOKIE` → 缓存的设备 cookie（`GET /x/frontend/finger/spi` 取 `buvid3`/`buvid4`，
|
||||
进程内缓存一次；取不到就不带 cookie，仅 debug 日志）→ 无。指纹接口本身失败**不**让抓取失败。
|
||||
走共享 `CLIENT`（UA `Mozilla/5.0`，30s 超时，`TELOXIDE_PROXY` 透传）。
|
||||
- **错误映射**：`0` → 成功；`-352/-412` 与 HTTP 412 → `Transient`（可重试，队列退避；首次记一条 warn 提示
|
||||
`BILIBILI_COOKIE`）；`500`/`4101147` → `NotFound`（永久）；其他 code → `Site`（永久）。
|
||||
- **媒体**：
|
||||
- `major.draw.items[]` → 每张一张图（`http://` / `//` → `https://`，非 https 开头直接丢弃）；
|
||||
`.gif` → `Media::Animated`（`thumbnail_url` 留空，Telegram 自己取首帧——`@518w.jpg` 只对 jpg/webp 实测过），
|
||||
其余 → `Media::Illustration`（`thumbnail_url = url + "@518w.jpg"`，兼作超大时的降级 URL）。
|
||||
- `major.archive.cover` → 1 张 `Illustration`（视频不发流）。
|
||||
- 转发且自身无媒体 → 递归取 `orig` 的媒体；正文拼 `//@{原作者}:\n{原文}`。
|
||||
- 其他 major（PGC/ARTICLE/MUSIC/LIVE/COMMON）不建模 → 无媒体，走既有 "No media found"。
|
||||
- **caption**（与 misskey 同形）：`{opus 链接}\n<a href="space.bilibili.com/{mid}">{name}</a>: {正文}`；
|
||||
`RenderData` 的 `{tags}` 来自话题名；正文由既有 `truncate_caption` 截断。
|
||||
- **注册表**：`SITES` 末尾追加 → `/set_format` 白名单、链接缓存、启动校验、日志前缀全部自动生效。
|
||||
- **bot 侧仅文案**：`handlers/commands.rs` 三处站点清单字符串 + `state.rs`/`handlers/mod.rs` 注释。
|
||||
|
||||
### 与原计划的偏差（及原因）
|
||||
|
||||
| 原计划 | 实际 | 原因 |
|
||||
|---|---|---|
|
||||
| `x/web-interface/view` + `playurl` 发视频 | 不做 | 需求收窄为图片/动图；视频只发封面 |
|
||||
| `site/mod.rs` 加 `MAX_MEDIA_UPLOAD_BYTES` 常量 | 不加 | 没有视频尺寸决策就不需要该常量，避免跨 crate 耦合 |
|
||||
| `b23.tv` 短链（跟随重定向） | 不匹配 | 多数短链指向视频，匹配后会把"静默忽略"变成用户的 "Failed to fetch media" |
|
||||
| `validate()` 校验 cookie | 不做 | 匿名可用，cookie 失效不致命；校验要额外请求一个端点，收益低 |
|
||||
| `media_headers` 给 hdslb 加 Referer | 返回 `None` | 实测图片与 durl 均无需 Referer（注释里记了这条验证） |
|
||||
| 计划阶段认为设备 cookie 是 YAGNI，不实现 | **实现**（`buvid3`+`buvid4`） | 计划之后做了对照实验：同一 IP 上"无 cookie → -352、只有 buvid3 → -352、buvid3+buvid4 → code:0"，说明这是对本适配器主要失败模式的直接修复，而不是冗余保险 |
|
||||
|
||||
---
|
||||
|
||||
## 4. 测试与验证
|
||||
|
||||
- 单元（13）：正则匹配/拒绝/忽略短链、缓存键归一、图片映射（https 归一 + 缩略图 + `.gif → Animated`）、
|
||||
封面、转发取 `orig` 媒体与正文拼接、纯文字无媒体、caption 转义、业务 code 分类（可重试性）、URL 归一、
|
||||
设备 cookie 拼装。
|
||||
- live（3，`#[ignore = "live network: …"]`）：设备 cookie 可取、图片动态 2 图、纯文字动态无媒体。
|
||||
CI 的 `live` job 已覆盖。动态接口被风控时这两条 live 测试打印 `skipping:` 并提前返回（与 pixiv 的
|
||||
token 门控同款约定），设备 cookie 那条仍会真实执行。
|
||||
- 实测命令：
|
||||
`cargo run -p x-media --example fetch -- https://www.bilibili.com/opus/1245284537985925159`
|
||||
（输出 2 张 `https://i0.hdslb.com/…jpg` + `@518w.jpg` 缩略图 + 话题 tags）。
|
||||
- 全套：`cargo fmt --check`、`cargo clippy --workspace --all-targets -- -D warnings`、`cargo test --workspace` 全绿。
|
||||
|
||||
## 5. 已知限制
|
||||
|
||||
- 风控按 IP/请求量漂移，阶梯见 §2 最后一行：轻度靠设备 cookie 自愈，重度需 `BILIBILI_COOKIE` 或换 IP。
|
||||
被拦时按**可重试**失败处理（队列退避）+ 一条 warn，不会静默丢帖。
|
||||
- 接口 schema 会漂移（`module_dynamic.major` 实测可为 `null` 而正文留在 `desc`）；DTO 全 `Option`，
|
||||
未知形态降级为"无媒体"，不 panic。
|
||||
- 动态内嵌视频只发封面图（与 bff 同策略），不下载流。
|
||||
- 纯文字动态复用既有 "No media found" 回复。
|
||||
- `b23.tv` 短链不被匹配（见上表）。
|
||||
+6
-3
@@ -1,6 +1,6 @@
|
||||
# TelegramXMediaBot
|
||||
|
||||
A Telegram bot that turns post links from X / Twitter, Pixiv, Bluesky, and Misskey (misskey.io) into media messages (images, video, GIF) with the post's title, author, and tags.
|
||||
A Telegram bot that turns post links from X / Twitter, Pixiv, Bluesky, Misskey (misskey.io), and Bilibili dynamics into media messages (images, video, GIF) with the post's title, author, and tags.
|
||||
|
||||
## Features
|
||||
|
||||
@@ -30,10 +30,12 @@ docker build -t tgxmb .
|
||||
docker run --rm -d --name tgxmb --env-file .env -v ./data:/app/data tgxmb
|
||||
```
|
||||
|
||||
Environment variables: `TELOXIDE_TOKEN` (required), `PIXIV_REFRESH_TOKEN`, `BOT_ADMIN`, `EDIT_MESSAGE_TTL_SECONDS`, `LINK_CACHE_TTL_SECONDS`, `RUST_LOG`, `TELOXIDE_PROXY`, `WEBHOOK*`, `TWITTER_AUTH_TOKEN` (optional).
|
||||
Environment variables: `TELOXIDE_TOKEN` (required), `PIXIV_REFRESH_TOKEN`, `BOT_ADMIN`, `EDIT_MESSAGE_TTL_SECONDS`, `LINK_CACHE_TTL_SECONDS`, `RUST_LOG`, `TELOXIDE_PROXY`, `WEBHOOK*`, `TWITTER_AUTH_TOKEN` (optional), `BILIBILI_COOKIE` (optional).
|
||||
|
||||
NSFW tweets: the public syndication endpoint does not return sensitive content. Setting `TWITTER_AUTH_TOKEN` (the `auth_token` cookie value of a logged-in x.com session) lets the bot fetch NSFW media in the logged-in state only when it hits a withheld tweet; without it, the bot reports no media.
|
||||
|
||||
Bilibili dynamics are fetched anonymously by default (no login; the bot fetches bilibili's anonymous `buvid3`/`buvid4` device cookies itself to raise the success rate). If the server's egress IP gets hard-flagged by bilibili (persistent `risk control (-352)` log lines or HTTP 412), set `BILIBILI_COOKIE` (the whole cookie string from a logged-in browser, e.g. `SESSDATA=…; bili_jct=…`) to restore access. Only a dynamic's images and animations are sent; an attached video degrades to its cover image.
|
||||
|
||||
### Webhook deployment (needs a reverse proxy)
|
||||
|
||||
`docker-compose.yml.example` ships an [nginx-proxy](https://github.com/nginx-proxy/nginx-proxy) + [acme-companion](https://github.com/nginx-proxy/acme-companion) reverse-proxy orchestration. Pick one deployment shape:
|
||||
@@ -81,6 +83,7 @@ Telegram only accepts ports 443/80/88/8443.
|
||||
|---|---|
|
||||
| `TELOXIDE_TOKEN` | Bot token (required) |
|
||||
| `PIXIV_REFRESH_TOKEN` | Pixiv refresh token; Pixiv is disabled without it |
|
||||
| `BILIBILI_COOKIE` | Optional bilibili cookie string (`SESSDATA=…; bili_jct=…`); only needed when the egress IP stays risk-controlled (device cookies are fetched automatically) |
|
||||
| `BOT_ADMIN` | Admin chat IDs, comma-separated; receives start/stop notifications |
|
||||
| `EDIT_MESSAGE_TTL_SECONDS` | Edit-before-forward record expiry in seconds, default 86400 |
|
||||
| `LINK_CACHE_TTL_SECONDS` | Link-result cache expiry in seconds, default 604800 (7 days) |
|
||||
@@ -111,7 +114,7 @@ Telegram only accepts ports 443/80/88/8443.
|
||||
| `/remove_forward_channel` | Remove the forward channel |
|
||||
| `/edit_before_forward` | Toggle "edit before forward": when enabled, the bot posts a prompt after forwarding; replying to it edits the first forwarded message's caption (or taps a template button to apply one) |
|
||||
| `/set_template <name>` | Reply to a message containing `[]` to save it as a named template; `[]` is replaced by the original post link when forwarding (used with "edit before forward") |
|
||||
| `/set_format <site> <format>` | Customize the caption format for one site. Sites: `twitter` / `bsky` / `pixiv` / `misskey`. Placeholders: `{url}` `{author}` `{author_url}` `{title}` `{tags}` |
|
||||
| `/set_format <site> <format>` | Customize the caption format for one site. Sites: `twitter` / `bsky` / `pixiv` / `misskey` / `bilibili`. Placeholders: `{url}` `{author}` `{author_url}` `{title}` `{tags}` |
|
||||
| `/clear_cache [link]` | Clear the link cache (admin only); with a link only that entry, otherwise everything |
|
||||
| `/bot_dict` | Show the current chat state (debugging; admin only) |
|
||||
| `/test <link>` | Parse a link and send its media; no channel forward, no edit-before-forward prompt (send only) |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# TelegramXMediaBot
|
||||
|
||||
Telegram 机器人，将 X / Twitter、Pixiv、Bluesky、Misskey (misskey.io) 的帖子链接转换为媒体消息发送，附带帖子标题、作者与标签。
|
||||
Telegram 机器人，将 X / Twitter、Pixiv、Bluesky、Misskey (misskey.io)、Bilibili 动态的帖子链接转换为媒体消息发送，附带帖子标题、作者与标签。
|
||||
|
||||
## 功能
|
||||
|
||||
@@ -30,10 +30,12 @@ docker build -t tgxmb .
|
||||
docker run --rm -d --name tgxmb --env-file .env -v ./data:/app/data tgxmb
|
||||
```
|
||||
|
||||
环境变量：`TELOXIDE_TOKEN`（必填）、`PIXIV_REFRESH_TOKEN`、`BOT_ADMIN`、`EDIT_MESSAGE_TTL_SECONDS`、`LINK_CACHE_TTL_SECONDS`、`RUST_LOG`、`TELOXIDE_PROXY`、`WEBHOOK*`、`TWITTER_AUTH_TOKEN`（可选）。
|
||||
环境变量：`TELOXIDE_TOKEN`（必填）、`PIXIV_REFRESH_TOKEN`、`BOT_ADMIN`、`EDIT_MESSAGE_TTL_SECONDS`、`LINK_CACHE_TTL_SECONDS`、`RUST_LOG`、`TELOXIDE_PROXY`、`WEBHOOK*`、`TWITTER_AUTH_TOKEN`（可选）、`BILIBILI_COOKIE`（可选）。
|
||||
|
||||
NSFW 推文：公开的 syndication 接口不返回敏感内容。设置 `TWITTER_AUTH_TOKEN`（登录 x.com 后浏览器 Cookie 里的 `auth_token` 值）后，bot 会仅在遇到 NSFW 推文时以登录态获取媒体；未设置则提示无媒体。
|
||||
|
||||
Bilibili 动态默认匿名抓取（无需登录，bot 会自动从 B 站的匿名指纹接口取 `buvid3`/`buvid4` 设备 cookie 以提高成功率）。若服务器出口 IP 被 B 站重度风控（日志里的 `risk control (-352)` 或 HTTP 412，且持续出现），设置 `BILIBILI_COOKIE`（登录后浏览器里整条 Cookie 串，如 `SESSDATA=…; bili_jct=…`）可恢复访问。当前只发送动态里的图片与动图，动态内嵌视频发送其封面。
|
||||
|
||||
### Webhook 部署（需要反向代理）
|
||||
|
||||
`docker-compose.yml.example` 内置了 [nginx-proxy](https://github.com/nginx-proxy/nginx-proxy) + [acme-companion](https://github.com/nginx-proxy/acme-companion) 反向代理编排，按部署环境二选一：
|
||||
@@ -81,6 +83,7 @@ Telegram 只接受 443/80/88/8443 端口。
|
||||
|---|---|
|
||||
| `TELOXIDE_TOKEN` | Bot token（必填） |
|
||||
| `PIXIV_REFRESH_TOKEN` | Pixiv 刷新令牌；未设置则禁用 Pixiv |
|
||||
| `BILIBILI_COOKIE` | 可选的 B 站 Cookie 串（`SESSDATA=…; bili_jct=…`），仅在出口 IP 被持续风控时才需要（设备 cookie 由 bot 自动获取） |
|
||||
| `BOT_ADMIN` | 管理员聊天 ID，逗号分隔；接收启动/停止通知 |
|
||||
| `EDIT_MESSAGE_TTL_SECONDS` | 转发前编辑记录过期秒数，默认 86400 |
|
||||
| `LINK_CACHE_TTL_SECONDS` | 链接结果缓存过期秒数，默认 604800（7 天） |
|
||||
@@ -111,7 +114,7 @@ Telegram 只接受 443/80/88/8443 端口。
|
||||
| `/remove_forward_channel` | 取消转发频道 |
|
||||
| `/edit_before_forward` | 开关「转发前编辑」：开启后，转发成功后 bot 会发一条提示消息，回复它可修改第一条转发消息的 caption（或点击模板按钮套用模板） |
|
||||
| `/set_template <名称>` | 回复一条含 `[]` 的消息，将其保存为命名模板；转发时 `[]` 会被替换为原帖链接（配合「转发前编辑」使用） |
|
||||
| `/set_format <站点> <格式>` | 自定义某站点的 caption 格式。站点：`twitter` / `bsky` / `pixiv` / `misskey`。占位符：`{url}` `{author}` `{author_url}` `{title}` `{tags}` |
|
||||
| `/set_format <站点> <格式>` | 自定义某站点的 caption 格式。站点：`twitter` / `bsky` / `pixiv` / `misskey` / `bilibili`。占位符：`{url}` `{author}` `{author_url}` `{title}` `{tags}` |
|
||||
| `/clear_cache [链接]` | 清空链接缓存（仅管理员）；带链接只清该条，否则清空全部 |
|
||||
| `/bot_dict` | 查看当前聊天状态（调试用；仅管理员） |
|
||||
| `/test <链接>` | 解析链接并发送媒体；不转发到频道、不弹转发前编辑提示（仅发送） |
|
||||
|
||||
@@ -0,0 +1,788 @@
|
||||
//! Site adapter for Bilibili dynamics (图片 / 动图): URL pattern, API fetch and
|
||||
//! normalization into [`Fetched`] (see [`crate::site::Site`]).
|
||||
//!
|
||||
//! Scope: the *media images* of a dynamic — the `major.draw` image grid and
|
||||
//! the cover of an attached video. Animated (`.gif`) pictures become
|
||||
//! [`Media::Animated`], everything else a photo. The video stream itself is
|
||||
//! deliberately **not** resolved: `dyn_archive` carries no `cid`, so playing
|
||||
//! it would mean a second `x/web-interface/view` round trip plus
|
||||
//! `x/player/playurl` and its size/quality chasing — the cover plus the post
|
||||
//! link is what the operator asked for.
|
||||
//!
|
||||
//! `b23.tv` short links are not matched: most of them point at videos, which
|
||||
//! this adapter does not handle, and matching them would turn a silently
|
||||
//! ignored link into the bot's "Failed to fetch media" reply.
|
||||
//!
|
||||
//! Verified live (2026-09-17): the detail endpoint answers **anonymously**
|
||||
//! (no cookie, no WBI signature) to a browser-ish `User-Agent` + `Referer`;
|
||||
//! `build`-taking variants and bilibili's own `bilibili_pc/…` UA got `-352`.
|
||||
//! Risk control escalates with request volume from one IP — first a plain
|
||||
//! request starts answering `-352`, then adding the anonymous device cookies
|
||||
//! (`buvid3` + `buvid4`, fetched from [`SPI_URL`] and sent by [`cookie`])
|
||||
//! restores `code: 0`, and a heavily flagged IP stays blocked until
|
||||
//! `BILIBILI_COOKIE` supplies a logged-in session.
|
||||
|
||||
use super::model;
|
||||
use crate::media::Media;
|
||||
use crate::site::{FetchError, Fetched, RenderData, Site, SiteFuture};
|
||||
use html_escape::{encode_double_quoted_attribute, encode_text};
|
||||
use regex::Regex;
|
||||
use std::sync::LazyLock;
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
|
||||
const API_URL: &str = "https://api.bilibili.com/x/polymer/web-dynamic/v1/detail";
|
||||
|
||||
/// Anonymous device-fingerprint endpoint handing out `buvid3`/`buvid4`.
|
||||
const SPI_URL: &str = "https://api.bilibili.com/x/frontend/finger/spi";
|
||||
|
||||
/// Sent with every API request: requests without it are the ones bilibili
|
||||
/// risk-controls (`code -352`, HTTP 412).
|
||||
const REFERER: &str = "https://www.bilibili.com/";
|
||||
|
||||
/// hdslb image variant used as thumbnail (and as the oversized fallback): a
|
||||
/// downscaled still of the same image, ~30 KB instead of ~570 KB. Verified
|
||||
/// live for `.jpg` and `.webp` sources. Not applied to `.gif` (unverified for
|
||||
/// animated sources, and Telegram generates its own frame preview).
|
||||
const THUMB_SUFFIX: &str = "@518w.jpg";
|
||||
|
||||
/// Optional `Cookie` header value (`SESSDATA=…; bili_jct=…`) for deployments
|
||||
/// that need a logged-in session. Unset by default: the adapter fetches
|
||||
/// bilibili's anonymous device cookies itself (see [`cookie`]) and works
|
||||
/// without any operator setup, so unlike pixiv the site stays enabled.
|
||||
static COOKIE: LazyLock<Option<String>> = LazyLock::new(|| {
|
||||
std::env::var("BILIBILI_COOKIE")
|
||||
.ok()
|
||||
.map(|s| s.trim().to_string())
|
||||
.filter(|s| !s.is_empty())
|
||||
});
|
||||
|
||||
/// Cached `buvid3`/`buvid4` header value from [`SPI_URL`], or `None` when the
|
||||
/// fingerprint endpoint was unavailable (requests then go out without a
|
||||
/// cookie, as before).
|
||||
static BUVID: LazyLock<tokio::sync::Mutex<Option<String>>> = LazyLock::new(Default::default);
|
||||
|
||||
/// Registry entry for the bilibili adapter (see [`crate::site::Site`]).
|
||||
pub struct BilibiliSite;
|
||||
|
||||
impl Site for BilibiliSite {
|
||||
fn id(&self) -> &'static str {
|
||||
"bilibili"
|
||||
}
|
||||
|
||||
fn pattern(&self) -> &'static Regex {
|
||||
&PATTERN
|
||||
}
|
||||
|
||||
fn cache_key(&self, url: &str) -> Option<String> {
|
||||
cache_key(url)
|
||||
}
|
||||
|
||||
fn fetch_from_url<'a>(&'a self, url: &'a str) -> SiteFuture<'a, Fetched> {
|
||||
Box::pin(async move { fetch_from_url(url).await })
|
||||
}
|
||||
}
|
||||
|
||||
/// Every supported link form, with the dynamic id in group 1: the direct
|
||||
/// dynamic (web / mobile / h5 share) and opus URLs.
|
||||
pub static PATTERN: LazyLock<Regex> = LazyLock::new(|| {
|
||||
Regex::new(
|
||||
r"^(?:https?://)?(?:www|t|m)\.bilibili\.com/(?:opus/|dynamic/|h5/dynamic/detail/)?(\d+)",
|
||||
)
|
||||
.unwrap()
|
||||
});
|
||||
|
||||
pub fn enabled() -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
pub async fn fetch_from_url(url: &str) -> Result<Fetched, FetchError> {
|
||||
let dynamic_id = PATTERN
|
||||
.captures(url)
|
||||
.and_then(|caps| caps.get(1))
|
||||
.ok_or(FetchError::NotFound)?
|
||||
.as_str();
|
||||
let item = fetch(dynamic_id).await?;
|
||||
Ok(item.into())
|
||||
}
|
||||
|
||||
/// Cache key for a bilibili URL: `"bilibili:<dynamic id>"`. The prefix is the
|
||||
/// site id used for caption-format lookup and link-cache keys.
|
||||
pub fn cache_key(url: &str) -> Option<String> {
|
||||
PATTERN
|
||||
.captures(url)
|
||||
.map(|caps| format!("bilibili:{}", &caps[1]))
|
||||
}
|
||||
|
||||
/// Bilibili's fetch-retry policy: transient classes only. Not-found, blocked
|
||||
/// and parse failures are permanent.
|
||||
pub fn is_retryable(err: &FetchError) -> bool {
|
||||
matches!(err, FetchError::Http(_) | FetchError::Transient(_))
|
||||
}
|
||||
|
||||
/// hdslb media serves without a `Referer` (verified live 2026-09-17 on
|
||||
/// `i0.hdslb.com` image URLs, requested both with and without one), so no
|
||||
/// extra headers.
|
||||
pub fn media_headers(_url: &str) -> Option<Vec<(&'static str, String)>> {
|
||||
None
|
||||
}
|
||||
|
||||
/// `Cookie` header for bilibili requests: the operator's `BILIBILI_COOKIE`
|
||||
/// when set, otherwise the anonymous device cookies.
|
||||
///
|
||||
/// Device cookies are the adapter's own fix for risk control, not merely
|
||||
/// insurance — verified 2026-09-17 on an IP bilibili had flagged: every
|
||||
/// request answered `-352` without them and `code: 0` with `buvid3` +
|
||||
/// `buvid4`. A logged-in `BILIBILI_COOKIE` carries its own device cookies,
|
||||
/// hence precedence rather than concatenation.
|
||||
async fn cookie() -> Option<String> {
|
||||
if let Some(cookie) = COOKIE.as_deref() {
|
||||
return Some(cookie.to_string());
|
||||
}
|
||||
// ponytail: cached for the process lifetime. Refetching after a `-352`
|
||||
// would mint a new device id for the same flagged IP — the escalation
|
||||
// path is BILIBILI_COOKIE.
|
||||
let mut cached = BUVID.lock().await;
|
||||
if cached.is_none() {
|
||||
*cached = match fetch_buvid().await {
|
||||
Ok(cookie) => cookie,
|
||||
Err(e) => {
|
||||
log::debug!("bilibili fingerprint unavailable: {e}");
|
||||
None
|
||||
}
|
||||
};
|
||||
}
|
||||
cached.clone()
|
||||
}
|
||||
|
||||
/// Fetches the device cookies bilibili hands to any visitor. The result is
|
||||
/// deliberately not an error: an unavailable fingerprint endpoint just means
|
||||
/// requests go out without a cookie.
|
||||
async fn fetch_buvid() -> Result<Option<String>, FetchError> {
|
||||
let response = crate::site::CLIENT.get(SPI_URL).send().await?;
|
||||
let fingerprint: model::Fingerprint = response.json().await.map_err(|e| FetchError::Site {
|
||||
site: "bilibili",
|
||||
error: Box::new(e),
|
||||
})?;
|
||||
Ok(buvid_cookie(&fingerprint))
|
||||
}
|
||||
|
||||
/// `buvid3=B; buvid4=B` from a fingerprint response; `None` when it carried
|
||||
/// no device ids.
|
||||
fn buvid_cookie(fingerprint: &model::Fingerprint) -> Option<String> {
|
||||
let data = fingerprint.data.as_ref()?;
|
||||
if data.buvid3.is_empty() && data.buvid4.is_empty() {
|
||||
return None;
|
||||
}
|
||||
Some(format!("buvid3={}; buvid4={}", data.buvid3, data.buvid4))
|
||||
}
|
||||
|
||||
async fn request(url: &str) -> reqwest::RequestBuilder {
|
||||
let request = crate::site::CLIENT.get(url).header("Referer", REFERER);
|
||||
match cookie().await {
|
||||
Some(cookie) => request.header("Cookie", cookie),
|
||||
None => request,
|
||||
}
|
||||
}
|
||||
|
||||
/// Fetches one dynamic by id and returns its item.
|
||||
///
|
||||
/// The API answers client-level failures with HTTP 200 + a business `code`
|
||||
/// (see [`code_error`]); HTTP 412 is bilibili's risk-control page.
|
||||
pub async fn fetch(dynamic_id: &str) -> Result<model::Item, FetchError> {
|
||||
let response = request(API_URL)
|
||||
.await
|
||||
.query(&[("id", dynamic_id)])
|
||||
.send()
|
||||
.await?;
|
||||
let status = response.status();
|
||||
if !status.is_success() {
|
||||
return Err(match status.as_u16() {
|
||||
412 => risk_control("412"),
|
||||
_ => FetchError::Transient(format!("bilibili status {status}")),
|
||||
});
|
||||
}
|
||||
let detail: model::Detail = response.json().await.map_err(|e| FetchError::Site {
|
||||
site: "bilibili",
|
||||
error: Box::new(e),
|
||||
})?;
|
||||
if let Some(err) = code_error(detail.code, detail.message.as_deref().unwrap_or_default()) {
|
||||
return Err(err);
|
||||
}
|
||||
detail
|
||||
.data
|
||||
.and_then(|data| data.item)
|
||||
.map(|item| *item)
|
||||
.ok_or(FetchError::NotFound)
|
||||
}
|
||||
|
||||
/// Maps the API's business code to an error; `None` means success.
|
||||
///
|
||||
/// `-352`/`-412` are bilibili's risk control and are mapped to a *transient*
|
||||
/// error on purpose: the queue retries them with backoff instead of dropping
|
||||
/// the post. A removed or nonexistent dynamic answers `500` (verified
|
||||
/// 2026-09-17; the older `4101147` is kept because nazurin still documents
|
||||
/// it) — permanent, so a dead link is not retried.
|
||||
fn code_error(code: i64, message: &str) -> Option<FetchError> {
|
||||
match code {
|
||||
0 => None,
|
||||
-352 | -412 => Some(risk_control(&code.to_string())),
|
||||
500 | 4101147 => Some(FetchError::NotFound),
|
||||
_ => Some(FetchError::Site {
|
||||
site: "bilibili",
|
||||
error: format!("code {code}: {message}").into(),
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
/// Risk control: retryable, but retrying rarely helps on its own — the
|
||||
/// operator's lever is `BILIBILI_COOKIE`, hence the hint. Logged once so a
|
||||
/// blocked deployment does not flood the log with one line per link.
|
||||
fn risk_control(code: &str) -> FetchError {
|
||||
if !RISK_CONTROL_LOGGED.swap(true, Ordering::Relaxed) {
|
||||
log::warn!("bilibili risk control ({code}); set BILIBILI_COOKIE if this persists");
|
||||
}
|
||||
FetchError::Transient(format!("bilibili risk control ({code})"))
|
||||
}
|
||||
|
||||
static RISK_CONTROL_LOGGED: AtomicBool = AtomicBool::new(false);
|
||||
|
||||
impl From<model::Item> for Fetched {
|
||||
fn from(item: model::Item) -> Self {
|
||||
let url = format!("https://www.bilibili.com/opus/{}", item.id_str);
|
||||
let author = author_name(&item).to_string();
|
||||
let author_url = author_url(&item).unwrap_or_else(|| url.clone());
|
||||
let text = text_of(&item);
|
||||
let tags = topic_name(&item).to_string();
|
||||
|
||||
let caption = caption(&url, &author_url, &author, &text);
|
||||
let media = media_of(&item);
|
||||
|
||||
Fetched {
|
||||
source_url: url.clone(),
|
||||
caption,
|
||||
title: text.clone(),
|
||||
media,
|
||||
sensitive: false,
|
||||
site_id: "bilibili",
|
||||
render_data: Some(RenderData {
|
||||
url,
|
||||
author: encode_text(&author).into_owned(),
|
||||
author_url,
|
||||
title: encode_text(&text).into_owned(),
|
||||
tags: encode_text(&tags).into_owned(),
|
||||
}),
|
||||
_keep_alive: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn author_name(item: &model::Item) -> &str {
|
||||
item.modules
|
||||
.as_ref()
|
||||
.and_then(|modules| modules.module_author.as_ref())
|
||||
.map(|author| author.name.as_str())
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
fn author_url(item: &model::Item) -> Option<String> {
|
||||
item.modules
|
||||
.as_ref()
|
||||
.and_then(|modules| modules.module_author.as_ref())
|
||||
.and_then(|author| author.mid)
|
||||
.map(|mid| format!("https://space.bilibili.com/{mid}"))
|
||||
}
|
||||
|
||||
fn desc_text(item: &model::Item) -> &str {
|
||||
item.modules
|
||||
.as_ref()
|
||||
.and_then(|modules| modules.module_dynamic.as_ref())
|
||||
.and_then(|dynamic| dynamic.desc.as_ref())
|
||||
.map(|desc| desc.text.as_str())
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
fn topic_name(item: &model::Item) -> &str {
|
||||
item.modules
|
||||
.as_ref()
|
||||
.and_then(|modules| modules.module_dynamic.as_ref())
|
||||
.and_then(|dynamic| dynamic.topic.as_ref())
|
||||
.map(|topic| topic.name.as_str())
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
/// The post's text: its own plus the quoted original's when it is a forward,
|
||||
/// marked the way bilibili's web UI does (`//@author:`).
|
||||
fn text_of(item: &model::Item) -> String {
|
||||
let own = desc_text(item);
|
||||
let Some(orig) = item.orig.as_deref() else {
|
||||
return own.to_string();
|
||||
};
|
||||
let orig_text = desc_text(orig);
|
||||
if orig_text.is_empty() {
|
||||
return own.to_string();
|
||||
}
|
||||
let name = author_name(orig);
|
||||
let mut text = own.to_string();
|
||||
if !text.is_empty() {
|
||||
text.push('\n');
|
||||
}
|
||||
if name.is_empty() {
|
||||
text.push_str(orig_text);
|
||||
} else {
|
||||
text.push_str(&format!("//@{name}:\n{orig_text}"));
|
||||
}
|
||||
text
|
||||
}
|
||||
|
||||
/// The dynamic's media: its own grid (or video cover), falling back to the
|
||||
/// quoted original's when a forward shell has none (mirrors misskey's
|
||||
/// `effective()` handling of renotes).
|
||||
fn media_of(item: &model::Item) -> Vec<Media> {
|
||||
let own = own_media(item);
|
||||
if !own.is_empty() {
|
||||
return own;
|
||||
}
|
||||
item.orig.as_deref().map(own_media).unwrap_or_default()
|
||||
}
|
||||
|
||||
fn own_media(item: &model::Item) -> Vec<Media> {
|
||||
let Some(major) = item
|
||||
.modules
|
||||
.as_ref()
|
||||
.and_then(|modules| modules.module_dynamic.as_ref())
|
||||
.and_then(|dynamic| dynamic.major.as_ref())
|
||||
else {
|
||||
return Vec::new();
|
||||
};
|
||||
if let Some(draw) = major.draw.as_ref() {
|
||||
return draw
|
||||
.items
|
||||
.iter()
|
||||
.filter_map(|pic| image(&pic.src))
|
||||
.collect();
|
||||
}
|
||||
major
|
||||
.archive
|
||||
.as_ref()
|
||||
.and_then(|archive| archive.cover.as_deref())
|
||||
.and_then(image)
|
||||
.into_iter()
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Maps one image URL: bilibili serves it as `http://`, and a `.gif` source
|
||||
/// is an animation rather than a photo.
|
||||
fn image(url: &str) -> Option<Media> {
|
||||
let url = to_https(url);
|
||||
if !url.starts_with("https://") {
|
||||
return None;
|
||||
}
|
||||
Some(if url.ends_with(".gif") {
|
||||
Media::Animated {
|
||||
title: None,
|
||||
url,
|
||||
// Left empty on purpose: the `@518w.jpg` variant is unverified for
|
||||
// animated sources, and Telegram generates a frame preview itself.
|
||||
thumbnail_url: String::new(),
|
||||
}
|
||||
} else {
|
||||
Media::Illustration {
|
||||
title: None,
|
||||
// Written before `url` moves so the formatting borrows it.
|
||||
thumbnail_url: Some(format!("{url}{THUMB_SUFFIX}")),
|
||||
url,
|
||||
fallback_url: None,
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
/// Bilibili serves media as `http://` (and sometimes protocol-relative
|
||||
/// `//host/…`); Telegram only accepts `https://`.
|
||||
fn to_https(url: &str) -> String {
|
||||
if let Some(rest) = url.strip_prefix("http://") {
|
||||
format!("https://{rest}")
|
||||
} else if let Some(rest) = url.strip_prefix("//") {
|
||||
format!("https://{rest}")
|
||||
} else {
|
||||
url.to_string()
|
||||
}
|
||||
}
|
||||
|
||||
fn caption(url: &str, author_url: &str, author: &str, text: &str) -> String {
|
||||
let url = encode_double_quoted_attribute(url);
|
||||
let author_url = encode_double_quoted_attribute(author_url);
|
||||
let author = encode_text(author);
|
||||
if text.is_empty() {
|
||||
return format!("{url}\n<a href=\"{author_url}\">{author}</a>");
|
||||
}
|
||||
format!(
|
||||
"{url}\n<a href=\"{author_url}\">{author}</a>: {}",
|
||||
encode_text(text)
|
||||
)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn item_json(major: serde_json::Value, text: &str) -> serde_json::Value {
|
||||
serde_json::json!({
|
||||
"id_str": "1245284537985925159",
|
||||
"modules": {
|
||||
"module_author": { "name": "索尼音乐中国", "mid": 486906719 },
|
||||
"module_dynamic": {
|
||||
"desc": { "text": text },
|
||||
"major": major,
|
||||
"topic": { "id": 1347638, "name": "音乐" },
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
fn parse(json: serde_json::Value) -> Fetched {
|
||||
serde_json::from_value::<model::Item>(json).unwrap().into()
|
||||
}
|
||||
|
||||
fn draw_item(src: &str) -> serde_json::Value {
|
||||
serde_json::json!({
|
||||
"type": "MAJOR_TYPE_DRAW",
|
||||
"draw": { "items": [{ "src": src, "width": 2304, "height": 2880, "size": 566.7 }] },
|
||||
})
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pattern_matches_supported_forms() {
|
||||
for url in [
|
||||
"https://t.bilibili.com/1245284537985925159",
|
||||
"t.bilibili.com/1245284537985925159",
|
||||
"https://t.bilibili.com/h5/dynamic/detail/1245284537985925159",
|
||||
"https://www.bilibili.com/opus/1245284537985925159",
|
||||
"https://m.bilibili.com/dynamic/1245284537985925159",
|
||||
"https://www.bilibili.com/opus/1245284537985925159?share_source=copy_web",
|
||||
] {
|
||||
assert!(PATTERN.is_match(url), "{url}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pattern_rejects_other_bilibili_pages() {
|
||||
for url in [
|
||||
"https://www.bilibili.com/video/BV1JTtt6JEZu",
|
||||
"https://www.bilibili.com/",
|
||||
"https://space.bilibili.com/486906719/dynamic",
|
||||
"https://live.bilibili.com/22632424",
|
||||
"https://www.bilibili.com/read/cv123456",
|
||||
"https://t.bilibili.com/",
|
||||
"https://x.com/user/status/1234567890",
|
||||
] {
|
||||
assert!(!PATTERN.is_match(url), "{url}");
|
||||
}
|
||||
}
|
||||
|
||||
/// Short links stay unmatched on purpose (most point at videos) so the
|
||||
/// bot keeps ignoring them instead of answering with a failure.
|
||||
#[test]
|
||||
fn pattern_ignores_short_links() {
|
||||
assert!(!PATTERN.is_match("https://b23.tv/abc123"));
|
||||
assert_eq!(cache_key("https://b23.tv/abc123"), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cache_key_normalizes_direct_forms() {
|
||||
for url in [
|
||||
"https://t.bilibili.com/1245284537985925159",
|
||||
"https://t.bilibili.com/h5/dynamic/detail/1245284537985925159?utm_source=share",
|
||||
"https://www.bilibili.com/opus/1245284537985925159",
|
||||
"https://m.bilibili.com/dynamic/1245284537985925159",
|
||||
] {
|
||||
assert_eq!(
|
||||
cache_key(url),
|
||||
Some("bilibili:1245284537985925159".to_string()),
|
||||
"{url}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn from_item_maps_draw_images_and_topic() {
|
||||
let fetched = parse(item_json(
|
||||
draw_item("http://i0.hdslb.com/bfs/new_dyn/a.jpg"),
|
||||
"新歌上线",
|
||||
));
|
||||
|
||||
assert_eq!(
|
||||
fetched.source_url,
|
||||
"https://www.bilibili.com/opus/1245284537985925159"
|
||||
);
|
||||
assert_eq!(fetched.site_id, "bilibili");
|
||||
assert_eq!(fetched.title, "新歌上线");
|
||||
assert!(!fetched.sensitive);
|
||||
assert_eq!(fetched.media.len(), 1);
|
||||
match &fetched.media[0] {
|
||||
Media::Illustration {
|
||||
url,
|
||||
thumbnail_url,
|
||||
fallback_url,
|
||||
..
|
||||
} => {
|
||||
assert_eq!(url, "https://i0.hdslb.com/bfs/new_dyn/a.jpg");
|
||||
assert_eq!(
|
||||
thumbnail_url.as_deref(),
|
||||
Some("https://i0.hdslb.com/bfs/new_dyn/a.jpg@518w.jpg")
|
||||
);
|
||||
assert_eq!(*fallback_url, None);
|
||||
}
|
||||
other => panic!("{other:?}"),
|
||||
}
|
||||
let caption = &fetched.caption;
|
||||
assert!(
|
||||
caption.starts_with("https://www.bilibili.com/opus/1245284537985925159\n"),
|
||||
"{caption}"
|
||||
);
|
||||
assert!(
|
||||
caption.contains("https://space.bilibili.com/486906719"),
|
||||
"{caption}"
|
||||
);
|
||||
assert!(caption.contains("索尼音乐中国"), "{caption}");
|
||||
assert!(caption.ends_with(": 新歌上线"), "{caption}");
|
||||
assert_eq!(
|
||||
fetched.render_fields(),
|
||||
Some((
|
||||
"索尼音乐中国",
|
||||
"https://space.bilibili.com/486906719",
|
||||
"新歌上线",
|
||||
"音乐"
|
||||
))
|
||||
);
|
||||
}
|
||||
|
||||
/// `.gif` sources are animations; they must not be sent as photos, and
|
||||
/// their thumbnail is left for Telegram to generate.
|
||||
#[test]
|
||||
fn from_item_maps_gif_as_animation() {
|
||||
let fetched = parse(item_json(
|
||||
draw_item("http://i0.hdslb.com/bfs/new_dyn/a.gif"),
|
||||
"",
|
||||
));
|
||||
match &fetched.media[0] {
|
||||
Media::Animated {
|
||||
url, thumbnail_url, ..
|
||||
} => {
|
||||
assert_eq!(url, "https://i0.hdslb.com/bfs/new_dyn/a.gif");
|
||||
assert_eq!(thumbnail_url, "");
|
||||
}
|
||||
other => panic!("{other:?}"),
|
||||
}
|
||||
// No text: the caption is the link plus the author line only.
|
||||
assert_eq!(
|
||||
fetched.caption,
|
||||
"https://www.bilibili.com/opus/1245284537985925159\n<a href=\"https://space.bilibili.com/486906719\">索尼音乐中国</a>"
|
||||
);
|
||||
}
|
||||
|
||||
/// Bilibili's media URLs arrive as `http://` or protocol-relative; both
|
||||
/// must become `https://` before they reach Telegram.
|
||||
#[test]
|
||||
fn media_urls_are_normalized_to_https() {
|
||||
let fetched = parse(item_json(draw_item("//i0.hdslb.com/bfs/new_dyn/p.jpg"), ""));
|
||||
assert_eq!(
|
||||
fetched.media[0].url(),
|
||||
"https://i0.hdslb.com/bfs/new_dyn/p.jpg"
|
||||
);
|
||||
|
||||
let fetched = parse(item_json(draw_item("not-a-url"), ""));
|
||||
assert!(fetched.media.is_empty());
|
||||
}
|
||||
|
||||
/// The video stream is out of scope; an AV dynamic still yields its cover.
|
||||
#[test]
|
||||
fn from_item_maps_archive_cover() {
|
||||
let major = serde_json::json!({
|
||||
"type": "MAJOR_TYPE_ARCHIVE",
|
||||
"archive": {
|
||||
"aid": 117189055087158_i64,
|
||||
"bvid": "BV1JTtt6JEZu",
|
||||
"cover": "http://i0.hdslb.com/bfs/archive/c.jpg",
|
||||
"title": "Supersubmarina",
|
||||
"duration_text": "03:45",
|
||||
},
|
||||
});
|
||||
let fetched = parse(item_json(major, "投稿了视频"));
|
||||
assert_eq!(fetched.media.len(), 1);
|
||||
assert_eq!(
|
||||
fetched.media[0].url(),
|
||||
"https://i0.hdslb.com/bfs/archive/c.jpg"
|
||||
);
|
||||
assert!(matches!(fetched.media[0], Media::Illustration { .. }));
|
||||
}
|
||||
|
||||
/// A forward shell carries the quote's text and, when it has no media of
|
||||
/// its own, the quote's images.
|
||||
#[test]
|
||||
fn from_item_forward_uses_orig_media_and_text() {
|
||||
let mut json = item_json(serde_json::Value::Null, "转发理由");
|
||||
json["orig"] = serde_json::json!({
|
||||
"id_str": "1246767523595026450",
|
||||
"modules": {
|
||||
"module_author": { "name": "A-SOUL_Official", "mid": 703007996 },
|
||||
"module_dynamic": {
|
||||
"desc": { "text": "原动态正文" },
|
||||
"major": draw_item("http://i0.hdslb.com/bfs/new_dyn/o.jpg"),
|
||||
},
|
||||
},
|
||||
});
|
||||
let fetched = parse(json);
|
||||
|
||||
assert_eq!(fetched.media.len(), 1);
|
||||
assert_eq!(
|
||||
fetched.media[0].url(),
|
||||
"https://i0.hdslb.com/bfs/new_dyn/o.jpg"
|
||||
);
|
||||
assert_eq!(fetched.title, "转发理由\n//@A-SOUL_Official:\n原动态正文");
|
||||
// The forwarder stays the author; the quote appears in the text.
|
||||
assert!(
|
||||
fetched.caption.contains("索尼音乐中国"),
|
||||
"{}",
|
||||
fetched.caption
|
||||
);
|
||||
assert!(
|
||||
fetched.caption.contains("原动态正文"),
|
||||
"{}",
|
||||
fetched.caption
|
||||
);
|
||||
}
|
||||
|
||||
/// A text-only dynamic has no media — the bot replies "No media found".
|
||||
#[test]
|
||||
fn from_item_without_major_has_no_media() {
|
||||
let fetched = parse(item_json(serde_json::Value::Null, "只有文字"));
|
||||
assert!(fetched.media.is_empty());
|
||||
assert_eq!(fetched.title, "只有文字");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn caption_escapes_site_text() {
|
||||
let fetched = parse(item_json(
|
||||
draw_item("http://i0.hdslb.com/bfs/new_dyn/a.jpg"),
|
||||
"<b>\"x\" & y</b>",
|
||||
));
|
||||
assert_eq!(fetched.title, "<b>\"x\" & y</b>");
|
||||
// `encode_text` escapes markup only; a bare quote is text, not an
|
||||
// attribute delimiter, and stays as-is.
|
||||
assert!(
|
||||
fetched.caption.contains("<b>\"x\" & y</b>"),
|
||||
"{}",
|
||||
fetched.caption
|
||||
);
|
||||
let (_, _, title, _) = fetched.render_fields().unwrap();
|
||||
assert_eq!(title, "<b>\"x\" & y</b>");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn code_error_classifies_api_codes() {
|
||||
assert!(code_error(0, "0").is_none());
|
||||
// Risk control must be retryable so the queue backs off instead of
|
||||
// dropping the post.
|
||||
for code in [-352, -412] {
|
||||
let err = code_error(code, "-352").unwrap();
|
||||
assert!(is_retryable(&err), "{err}");
|
||||
}
|
||||
// A removed dynamic is permanent.
|
||||
assert!(matches!(code_error(500, ""), Some(FetchError::NotFound)));
|
||||
assert!(matches!(
|
||||
code_error(4101147, ""),
|
||||
Some(FetchError::NotFound)
|
||||
));
|
||||
let err = code_error(-400, "param parsing failed").unwrap();
|
||||
assert!(!is_retryable(&err), "{err}");
|
||||
assert!(err.to_string().contains("-400"), "{err}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn buvid_cookie_needs_device_ids() {
|
||||
let fingerprint: model::Fingerprint = serde_json::from_value(serde_json::json!({
|
||||
"code": 0,
|
||||
"data": { "b_3": "ABCinfoc", "b_4": "DEF-Au1eCYnrGyhSrD" },
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
buvid_cookie(&fingerprint).as_deref(),
|
||||
Some("buvid3=ABCinfoc; buvid4=DEF-Au1eCYnrGyhSrD")
|
||||
);
|
||||
|
||||
// A response without device ids must not produce a `Cookie` header
|
||||
// with empty values.
|
||||
for json in [
|
||||
serde_json::json!({ "code": 0, "data": { "b_3": "", "b_4": "" } }),
|
||||
serde_json::json!({ "code": -352 }),
|
||||
] {
|
||||
let fingerprint: model::Fingerprint = serde_json::from_value(json).unwrap();
|
||||
assert_eq!(buvid_cookie(&fingerprint), None);
|
||||
}
|
||||
}
|
||||
|
||||
/// The device-cookie half of the adapter: the fingerprint endpoint keeps
|
||||
/// answering even when the dynamic endpoint risk-controls this IP, so it
|
||||
/// stays a meaningful live check on its own.
|
||||
#[tokio::test]
|
||||
#[ignore = "live network: requires outbound HTTPS to api.bilibili.com"]
|
||||
async fn live_fingerprint_yields_device_cookies() {
|
||||
let cookie = fetch_buvid().await.unwrap();
|
||||
let cookie = cookie.expect("fingerprint endpoint returned no device ids");
|
||||
assert!(cookie.contains("buvid3="), "{cookie}");
|
||||
assert!(cookie.contains("buvid4="), "{cookie}");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[ignore = "live network: requires outbound HTTPS to api.bilibili.com"]
|
||||
async fn live_fetch_draw_dynamic() {
|
||||
// 索尼音乐中国, a two-picture dynamic.
|
||||
let Some(fetched) = live_fetch("https://www.bilibili.com/opus/1245284537985925159").await
|
||||
else {
|
||||
return;
|
||||
};
|
||||
assert_eq!(fetched.site_id, "bilibili");
|
||||
assert_eq!(
|
||||
fetched.source_url,
|
||||
"https://www.bilibili.com/opus/1245284537985925159"
|
||||
);
|
||||
let urls: Vec<&str> = fetched.media.iter().map(|m| m.url()).collect();
|
||||
assert_eq!(urls.len(), 2, "{urls:?}");
|
||||
assert!(
|
||||
urls.iter().all(|u| u.starts_with("https://i0.hdslb.com/")),
|
||||
"{urls:?}"
|
||||
);
|
||||
assert!(
|
||||
fetched.caption.contains("space.bilibili.com"),
|
||||
"{}",
|
||||
fetched.caption
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[ignore = "live network: requires outbound HTTPS to api.bilibili.com"]
|
||||
async fn live_fetch_text_only_dynamic() {
|
||||
// A text-only dynamic: no media, so the bot answers "No media found".
|
||||
let Some(fetched) = live_fetch("https://t.bilibili.com/1246767523595026450").await else {
|
||||
return;
|
||||
};
|
||||
assert!(fetched.media.is_empty());
|
||||
assert!(!fetched.title.trim().is_empty());
|
||||
}
|
||||
|
||||
/// Fetches a live dynamic, skipping the assertion when bilibili
|
||||
/// risk-controls this IP (the site blocks datacenter/over-used addresses
|
||||
/// with `-352` regardless of cookies — a real failure would surface as a
|
||||
/// parse error or a not-found instead). Mirrors the token-gated pixiv
|
||||
/// tests' "skipping: …" convention.
|
||||
async fn live_fetch(url: &str) -> Option<Fetched> {
|
||||
match fetch_from_url(url).await {
|
||||
Ok(fetched) => Some(fetched),
|
||||
Err(e) if e.to_string().contains("risk control") => {
|
||||
eprintln!("skipping: {e}");
|
||||
None
|
||||
}
|
||||
Err(e) => panic!("{e}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
mod interface;
|
||||
mod model;
|
||||
|
||||
pub use interface::{
|
||||
BilibiliSite, PATTERN, cache_key, enabled, fetch_from_url, is_retryable, media_headers,
|
||||
};
|
||||
@@ -0,0 +1,124 @@
|
||||
//! Serde DTOs for the Bilibili dynamic detail endpoint
|
||||
//! (`/x/polymer/web-dynamic/v1/detail`), mirroring live responses
|
||||
//! (field paths verified 2026-09-17). Every field is optional so an API
|
||||
//! shape change degrades to "no media" instead of a parse failure.
|
||||
|
||||
use serde::Deserialize;
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Detail {
|
||||
/// Business code: `0` = OK, `-352`/`-412` = risk control, `500`/`4101147`
|
||||
/// = gone.
|
||||
pub(crate) code: i64,
|
||||
#[serde(default)]
|
||||
pub(crate) message: Option<String>,
|
||||
#[serde(default)]
|
||||
pub(crate) data: Option<Data>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Data {
|
||||
#[serde(default)]
|
||||
pub(crate) item: Option<Box<Item>>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Item {
|
||||
/// The dynamic id, same numeric id as in the URL.
|
||||
#[serde(default)]
|
||||
pub(crate) id_str: String,
|
||||
#[serde(default)]
|
||||
pub(crate) modules: Option<Modules>,
|
||||
/// The quoted dynamic when this item is a forward. A forward shell often
|
||||
/// carries no media of its own — the original holds it.
|
||||
#[serde(default)]
|
||||
pub(crate) orig: Option<Box<Item>>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Modules {
|
||||
#[serde(default)]
|
||||
pub(crate) module_author: Option<Author>,
|
||||
#[serde(default)]
|
||||
pub(crate) module_dynamic: Option<Dynamic>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Author {
|
||||
#[serde(default)]
|
||||
pub(crate) name: String,
|
||||
#[serde(default)]
|
||||
pub(crate) mid: Option<i64>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Dynamic {
|
||||
#[serde(default)]
|
||||
pub(crate) desc: Option<Desc>,
|
||||
#[serde(default)]
|
||||
pub(crate) major: Option<Major>,
|
||||
/// A single topic (`{"id":…,"name":…}`), the dynamic's only tag source.
|
||||
#[serde(default)]
|
||||
pub(crate) topic: Option<Topic>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Desc {
|
||||
#[serde(default)]
|
||||
pub(crate) text: String,
|
||||
}
|
||||
|
||||
/// `major` is a tagged union: `type` (`MAJOR_TYPE_DRAW` / `_ARCHIVE` / …)
|
||||
/// plus one payload object per type. Only the two payloads this adapter reads
|
||||
/// are modeled; an unknown major simply yields no media.
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Major {
|
||||
#[serde(default)]
|
||||
pub(crate) draw: Option<Draw>,
|
||||
#[serde(default)]
|
||||
pub(crate) archive: Option<Archive>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Draw {
|
||||
#[serde(default)]
|
||||
pub(crate) items: Vec<Pic>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Pic {
|
||||
/// Image URL, served as `http://` — normalized to https by the adapter.
|
||||
pub(crate) src: String,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Archive {
|
||||
/// The attached video's cover — the only image an AV dynamic has (the
|
||||
/// video itself is deliberately not resolved, see the module docs).
|
||||
#[serde(default)]
|
||||
pub(crate) cover: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Topic {
|
||||
#[serde(default)]
|
||||
pub(crate) name: String,
|
||||
}
|
||||
|
||||
/// Response of the anonymous fingerprint endpoint (`/x/frontend/finger/spi`),
|
||||
/// the source of the adapter's device cookies.
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct Fingerprint {
|
||||
#[serde(default)]
|
||||
pub(crate) data: Option<FingerprintData>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize, Debug)]
|
||||
pub(crate) struct FingerprintData {
|
||||
/// Sent as the `buvid3` cookie.
|
||||
#[serde(default, rename = "b_3")]
|
||||
pub(crate) buvid3: String,
|
||||
/// Sent as the `buvid4` cookie.
|
||||
#[serde(default, rename = "b_4")]
|
||||
pub(crate) buvid4: String,
|
||||
}
|
||||
@@ -1,8 +1,8 @@
|
||||
//! Site fetching dispatcher and unified result types.
|
||||
//!
|
||||
//! Dispatch order: twitter → bsky → misskey → pixiv. Each site module
|
||||
//! exports a `PATTERN`, `enabled()` and `fetch_from_url()`; a future site
|
||||
//! plugs in by adding one guarded entry in `SITES`.
|
||||
//! Dispatch order: twitter → bsky → misskey → pixiv → bilibili. Each site
|
||||
//! module exports a `PATTERN`, `enabled()` and `fetch_from_url()`; a future
|
||||
//! site plugs in by adding one guarded entry in `SITES`.
|
||||
|
||||
use std::future::Future;
|
||||
use std::pin::Pin;
|
||||
@@ -13,6 +13,7 @@ use std::time::Duration;
|
||||
use regex::Regex;
|
||||
use thiserror::Error;
|
||||
|
||||
pub mod bilibili;
|
||||
pub mod bsky;
|
||||
pub mod misskey;
|
||||
pub mod pixiv;
|
||||
@@ -26,7 +27,8 @@ pub use pixiv::PixivError;
|
||||
pub struct Fetched {
|
||||
/// Canonical URL: `x.com/{author}/status/{id}` |
|
||||
/// `https://www.pixiv.net/artworks/{id}` |
|
||||
/// `https://bsky.app/profile/{handle}/post/{rkey}`
|
||||
/// `https://bsky.app/profile/{handle}/post/{rkey}` |
|
||||
/// `https://www.bilibili.com/opus/{id}`
|
||||
pub source_url: String,
|
||||
/// The exact HTML produced by the site's caption().
|
||||
pub caption: String,
|
||||
@@ -35,7 +37,7 @@ pub struct Fetched {
|
||||
pub media: Vec<crate::media::Media>,
|
||||
/// Spoiler flag for all media of this post.
|
||||
pub sensitive: bool,
|
||||
/// Site id (`"twitter"` / `"bsky"` / `"pixiv"`): the single source of
|
||||
/// Site id (`"twitter"` / `"bsky"` / `"pixiv"` / `"bilibili"`): the single source of
|
||||
/// truth for site identity — caption-format lookup, cache-key prefix and
|
||||
/// the SetFormat whitelist all derive from it. Set by the producing site.
|
||||
pub site_id: &'static str,
|
||||
@@ -283,7 +285,7 @@ pub(crate) fn log_once_ffmpeg_missing() {
|
||||
}
|
||||
|
||||
/// Site adapter: one impl per supported site (twitter / bsky / misskey /
|
||||
/// pixiv), registered in `SITES`. All site-specific knowledge — URL pattern,
|
||||
/// pixiv / bilibili), registered in `SITES`. All site-specific knowledge — URL pattern,
|
||||
/// cache-key format, fetch, retry policy, media-host headers, startup
|
||||
/// validation — lives in the site module; the central dispatcher only
|
||||
/// iterates the registry.
|
||||
@@ -294,9 +296,9 @@ pub(crate) fn log_once_ffmpeg_missing() {
|
||||
/// site structs are stateless unit structs, so the boxed futures never
|
||||
/// borrow from `self` beyond the call's scope.
|
||||
pub trait Site: Send + Sync {
|
||||
/// Stable site id (`"twitter"` / `"bsky"` / `"misskey"` / `"pixiv"`):
|
||||
/// caption-format lookup, cache-key prefixes and the SetFormat whitelist
|
||||
/// derive from it.
|
||||
/// Stable site id (`"twitter"` / `"bsky"` / `"misskey"` / `"pixiv"` /
|
||||
/// `"bilibili"`): caption-format lookup, cache-key prefixes and the
|
||||
/// SetFormat whitelist derive from it.
|
||||
fn id(&self) -> &'static str;
|
||||
/// URL pattern; the dispatcher's first match wins (dispatch order).
|
||||
fn pattern(&self) -> &'static Regex;
|
||||
@@ -332,14 +334,15 @@ pub trait Site: Send + Sync {
|
||||
type SiteFuture<'a, T, E = FetchError> = Pin<Box<dyn Future<Output = Result<T, E>> + Send + 'a>>;
|
||||
|
||||
/// The one registry of supported sites, in dispatch order (twitter → bsky →
|
||||
/// misskey → pixiv). Adding a site = new module + one `Box::new(...)` entry
|
||||
/// here; the bot crate never lists sites itself.
|
||||
/// misskey → pixiv → bilibili). Adding a site = new module + one
|
||||
/// `Box::new(...)` entry here; the bot crate never lists sites itself.
|
||||
static SITES: LazyLock<Vec<Box<dyn Site>>> = LazyLock::new(|| {
|
||||
vec![
|
||||
Box::new(twitter::TwitterSite),
|
||||
Box::new(bsky::BskySite),
|
||||
Box::new(misskey::MisskeySite),
|
||||
Box::new(pixiv::PixivSite),
|
||||
Box::new(bilibili::BilibiliSite),
|
||||
]
|
||||
});
|
||||
|
||||
@@ -541,6 +544,10 @@ mod tests {
|
||||
cache_key("https://bsky.app/profile/handle.example/post/3lorem"),
|
||||
Some("bsky:handle.example/3lorem".into())
|
||||
);
|
||||
assert_eq!(
|
||||
cache_key("https://t.bilibili.com/1245284537985925159"),
|
||||
Some("bilibili:1245284537985925159".into())
|
||||
);
|
||||
assert_eq!(cache_key("https://example.com/not-a-post"), None);
|
||||
}
|
||||
|
||||
@@ -549,16 +556,21 @@ mod tests {
|
||||
assert_eq!(site_id_from_key("twitter:123"), "twitter");
|
||||
assert_eq!(site_id_from_key("pixiv:123"), "pixiv");
|
||||
assert_eq!(site_id_from_key("bsky:handle.example/3lorem"), "bsky");
|
||||
assert_eq!(site_id_from_key("bilibili:123"), "bilibili");
|
||||
assert_eq!(site_id_from_key("unknown:1"), "unknown");
|
||||
assert_eq!(site_id_from_key("no-colon"), "unknown");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn registry_lists_all_sites_in_dispatch_order() {
|
||||
assert_eq!(site_ids(), vec!["twitter", "bsky", "misskey", "pixiv"]);
|
||||
assert_eq!(
|
||||
site_ids(),
|
||||
vec!["twitter", "bsky", "misskey", "pixiv", "bilibili"]
|
||||
);
|
||||
// Enabled sites dispatch; unsupported URLs never match.
|
||||
assert!(find_site("https://x.com/u/status/1").is_some());
|
||||
assert!(find_site("https://misskey.io/notes/abc").is_some());
|
||||
assert!(find_site("https://t.bilibili.com/1245284537985925159").is_some());
|
||||
assert!(find_site("https://example.com/x").is_none());
|
||||
// Cache keys are pattern-driven, independent of the enabled() gate
|
||||
// (pixiv is disabled in tests without PIXIV_REFRESH_TOKEN).
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
//! Central env handling. The only other places that read env are
|
||||
//! `Bot::from_env` (TELOXIDE_TOKEN) and x-media (PIXIV_REFRESH_TOKEN).
|
||||
//! `Bot::from_env` (TELOXIDE_TOKEN) and x-media (PIXIV_REFRESH_TOKEN,
|
||||
//! TWITTER_AUTH_TOKEN, BILIBILI_COOKIE).
|
||||
|
||||
use std::env;
|
||||
use std::net::IpAddr;
|
||||
|
||||
@@ -279,7 +279,7 @@ pub(crate) async fn execute_command(
|
||||
bot,
|
||||
message.chat.id.0,
|
||||
message.id,
|
||||
"Unknown site. Use twitter, bsky, pixiv or misskey.",
|
||||
"Unknown site. Use twitter, bsky, pixiv, misskey or bilibili.",
|
||||
)
|
||||
.await?;
|
||||
return Ok(());
|
||||
@@ -320,7 +320,7 @@ pub(crate) async fn execute_command(
|
||||
bot,
|
||||
message.chat.id.0,
|
||||
message.id,
|
||||
"Unrecognized link. Use a twitter/x, pixiv, bsky or misskey post URL.",
|
||||
"Unrecognized link. Use a twitter/x, pixiv, bsky, misskey or bilibili post URL.",
|
||||
)
|
||||
.await?;
|
||||
return Ok(());
|
||||
@@ -358,7 +358,7 @@ pub(crate) async fn execute_command(
|
||||
bot,
|
||||
message.chat.id.0,
|
||||
message.id,
|
||||
"No enabled site matches this link (twitter/x, pixiv, bsky or misskey).",
|
||||
"No enabled site matches this link (twitter/x, pixiv, bsky, misskey or bilibili).",
|
||||
)
|
||||
.await?;
|
||||
return Ok(());
|
||||
@@ -400,7 +400,7 @@ pub(crate) async fn execute_command(
|
||||
bot,
|
||||
message.chat.id.0,
|
||||
message.id,
|
||||
"No enabled site matches this link (twitter/x, pixiv, bsky or misskey).",
|
||||
"No enabled site matches this link (twitter/x, pixiv, bsky, misskey or bilibili).",
|
||||
)
|
||||
.await?;
|
||||
}
|
||||
|
||||
@@ -60,8 +60,8 @@ pub(crate) async fn reply_html(
|
||||
|
||||
/// Log prefix tying the whole lifecycle of one link (fetch → send → cache →
|
||||
/// forward) together: the normalized cache key (`twitter:123…`, `pixiv:123`,
|
||||
/// `bsky:handle/rkey`) instead of the raw URL, so logs stay short and do not
|
||||
/// echo full user-submitted URLs at info level.
|
||||
/// `bsky:handle/rkey`, `bilibili:123…`) instead of the raw URL, so logs stay
|
||||
/// short and do not echo full user-submitted URLs at info level.
|
||||
pub fn log_key(url: &str) -> String {
|
||||
x_media::site::cache_key(url).unwrap_or_else(|| "<unsupported>".to_string())
|
||||
}
|
||||
|
||||
@@ -17,7 +17,7 @@ pub struct ChatData {
|
||||
pub edit_message: HashMap<i64, EditMessage>,
|
||||
/// name -> HTML template containing "[]"
|
||||
pub template: HashMap<String, String>,
|
||||
/// site name (twitter/bsky/misskey/pixiv) -> user-supplied caption format
|
||||
/// site name (twitter/bsky/misskey/pixiv/bilibili) -> user-supplied caption format
|
||||
/// with {url} {author} {author_url} {title} {tags} placeholders.
|
||||
pub message_format: HashMap<String, String>,
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user