Commit Graph
12 Commits
Author SHA1 Message Date
YoursFunny c1f5d3ca54 feat(sites): add bilibili dynamic support (images and animated images)
Fetch `t.bilibili.com/<id>`, `www.bilibili.com/opus/<id>`,
`t.bilibili.com/h5/dynamic/detail/<id>` and `m.bilibili.com/dynamic/<id>`
through the anonymous `/x/polymer/web-dynamic/v1/detail` endpoint (no
cookie, no WBI signature; the site adds the device cookies
`/x/frontend/finger/spi` hands out, which is what lifts bilibili's
`-352` risk control).

Media: the `major.draw` grid (`.gif` sources become animations, the rest
photos with a downscaled `@518w.jpg` thumbnail used both as preview and
as the oversized fallback), an attached video's cover, and the quoted
dynamic's media for forwards. The video stream itself is not resolved;
`b23.tv` short links stay unmatched (they mostly point at videos, so
matching them would turn a silently ignored link into a failure reply).

`-352`/`-412` map to a retryable error so the queue backs off instead of
dropping the post; a removed dynamic (`500`) is permanent.

Registry-driven, so no bot-side code changes beyond the site lists in the
command replies; found while researching nazurin and
telegram-bili-feed-helper (see BILIBILI_PLAN.md).
2026-09-17 20:48:15 +08:00
YoursFunny abdc27ed5e docs: resync AGENTS.md and the doc comments with the code
AGENTS.md:
- db.rs row claimed a per-store connection pool; there is one shared pool for
  all three tables (statics.rs builds it once).
- retry enqueue moved to send.rs, noted in both handlers rows.
- queue row now names both notifies (workers' + the sweep's).
- Retries bullet documents fetch vs fetch_once.
- test count ~125 -> ~135, the untested-files list no longer claims state.rs
  and handlers.rs are untested, and the live-test inventory mentions the
  token-gated, not-#[ignore]d pixiv download test that makes a local
  `cargo test --workspace` hit the network.
- /bot_dict is admin-only now.

Code docs:
- site/mod.rs: the module doc pointed new sites at `fetch_once` (a name that
  did not exist then and now means a single-attempt fetch) -> `SITES`; the
  cache_key/SITES/Site docs still said "twitter -> bsky -> pixiv" (misskey
  is registered third); RenderData now documents which fields are escaped
  and why url/author_url are not.
- state.rs, callback.rs: drop the pre-misskey site list and the `<name>`
  that rustdoc read as an HTML tag.
- Fixed the remaining rustdoc links/warnings: `cargo doc --workspace
  --no-deps` is now warning-free (was 8).
- docs/site-registry-refactor.md: §1 describes the pre-refactor state; said so.

No behavior change. fmt/clippy clean, 55 + 68 tests pass (the live pixiv
download test flaked on a CDN body timeout, as before).
2026-09-16 23:28:46 +08:00
YoursFunny 0a577600fd fix: repair four correctness defects in the send/state/handler paths
- send: the PHOTO_INVALID_DIMENSIONS marker never matched (the description is
  lower-cased, the marker was not), so oversized photos sent by URL were
  classified Permanent instead of taking the download-and-downscale fallback.
- inline: the debounce state was one global slot, so a second user's query
  cancelled the first user's pending answer entirely; it is now per user.
- state: prune_expired wrote back a stale snapshot without the per-chat lock,
  clobbering a concurrent update() (lost edit-message record -> "Expired");
  it now re-reads and prunes under the same lock update() uses.
- send/queue: a task dead-lettered on retry exhaustion kept its keep-alive
  temp media (ugoira MP4) alive until process exit; dead_letter_notify now
  releases it, and enqueue_retry releases when the enqueue itself fails.

Also folds the duplicated retry enqueue in post_send_actions into
send::enqueue_retry (single clock source, single place that releases).

Tests: +6 (marker, per-user debounce x3, keep-alive release, prune contract
x2); the marker and keep-alive cases were verified to fail before the fix.
cargo fmt/clippy clean, 52 + 69 tests pass.
2026-09-16 21:11:39 +08:00
YoursFunny 2f741e5f4b refactor(send): apply ponytail audit cuts 2, 4, 6
- updated_sequence_task: clone the Task and mutate the two fields
  instead of rebuilding all 12 by hand (-22 lines; new fields no
  longer need a sync here)
- unify unix_now with db::now_f64 (unix_now() = now_f64() as i64),
  moved to db.rs next to its clock source
- classify_to_send_error takes the MediaFetchFailure label, folding
  the duplicated inline match in send_batch_via_upload (-8 lines)
2026-09-07 19:19:56 +08:00
YoursFunny 1e77bb0478 refactor(db): share one DbPool across stores; merge schema init 2026-08-14 19:37:18 +08:00
YoursFunny 2e2d1b3506 style: rustfmt the DbPool call sites from the connection-pool change
Formatting-only; the pool commit landed before cargo fmt was run.
2026-08-13 22:25:13 +08:00
YoursFunny edb32c23b4 perf(db): reuse SQLite connections via a small per-store pool
Every DB operation (queue lease/enqueue, chat_state get/set, link_cache
read/write) used to open a fresh connection — including the busy timeout
and WAL pragma — then close it, on every message, URL job and callback.

Replace with DbPool: a tiny pool (4 connections max, semaphore-bounded
concurrency for backpressure) whose with_conn() method runs the closure on
a pooled connection inside spawn_blocking. Steady-state cost of an
operation is a list pop + semaphore acquire instead of a connection open.
2026-08-13 22:12:34 +08:00
YoursFunny ee6f3e4a27 state: evict idle chats from the ChatStore cache
prune_expired only shrank edit_message maps, so the cache kept one
ChatData per chat forever (a leak proportional to chat count). Chats
without live edit records are now dropped from the cache and their
per-chat lock (DB row persists; get() reloads). Lock order kept safe:
prune never holds the cache lock while taking the per-chat locks.
2026-08-08 20:16:26 +08:00
YoursFunny 1d9e3629c9 state: serialize per-chat get→mutate→set with ChatStore::update
Concurrent handler tasks (the batch-forward design spawns several per
chat) each snapshotted the same ChatData and last-writer-wins silently
dropped mutations — e.g. a second edit_message record, leaving one
prompt's Forward button dead. All write cycles now run under a per-chat
async lock; read-only callers keep get().
2026-08-08 20:12:51 +08:00
YoursFunny b0ced34b4c refactor: share sqlite open/with_conn helpers in db.rs
Converge the duplicated open_db (open + busy_timeout) and the
spawn_blocking + expect ceremony that every table access repeated
into one db.rs module. ChatStore no longer creates the tasks table
(schema ownership: queue.rs owns tasks, state.rs chat_state,
link_cache.rs link_cache). No schema or behavior change - all
CREATE TABLE statements are byte-identical, IF NOT EXISTS stays
idempotent, so existing data/task_queue.db files need no migration.
2026-08-07 16:00:12 +08:00
YoursFunny ab2306002a perf: handle batch-forwarded URLs concurrently with queue workers 2026-08-05 01:26:24 +08:00
YoursFunny 84ab146069 finish rust rewrite, add docker, drop python 2026-08-03 23:30:42 +08:00