Commit Graph
100 Commits
Author SHA1 Message Date
YoursFunny 09656810fe fix: prune persisted expired prompts 2026-09-25 01:49:30 +08:00
YoursFunny 786bafca42 fix: account for photo processing peak memory 2026-09-25 01:49:30 +08:00
YoursFunny 18da6e97c2 fix: reject oversized local media before upload 2026-09-25 01:49:29 +08:00
YoursFunny 7b816d4153 fix: split long message forwards 2026-09-25 01:49:29 +08:00
YoursFunny d8b9a06453 fix: bound direct command fetches 2026-09-25 01:40:09 +08:00
YoursFunny 4e0f1b834e fix: restrict media download hosts 2026-09-25 01:40:09 +08:00
YoursFunny 21437d1843 fix: bound queue panic retries 2026-09-25 01:11:28 +08:00
YoursFunny a54029c19b fix: propagate startup repair database errors 2026-09-25 01:06:09 +08:00
YoursFunny c6120b7cc0 fix: make sqlite migrations atomic 2026-09-24 21:47:48 +08:00
YoursFunny 7b4e273533 fix: bound site JSON response bodies 2026-09-24 21:25:03 +08:00
YoursFunny 6a87583cc0 fix: honor bsky HLS retry-after 2026-09-24 21:01:36 +08:00
YoursFunny 1ba64e858d fix: enforce inline result limit centrally 2026-09-24 21:00:00 +08:00
YoursFunny 008d31b9bd test: cover inline answer edge cases 2026-09-24 20:57:08 +08:00
YoursFunny 66ec14662c style: simplify inline answer returns 2026-09-24 20:57:08 +08:00
YoursFunny 8ae7d9947d fix: honor media retry-after delay 2026-09-24 20:57:08 +08:00
YoursFunny 28e8fec68d fix: rate limit plain chat operations 2026-09-24 20:57:08 +08:00
YoursFunny b2b01721b2 fix: debounce inline generations and cap results 2026-09-24 20:40:27 +08:00
YoursFunny 85c3be5ef5 fix: always answer inline query 2026-09-24 20:33:23 +08:00
YoursFunny 5e7a87c0dd fix: send text posts as HTML 2026-09-24 20:29:04 +08:00
YoursFunny 6a2ba6a50a fix: handle empty media and prompt persistence 2026-09-24 19:51:50 +08:00
YoursFunny fb354c9434 test: handle chat state Result in callback 2026-09-24 19:46:23 +08:00
YoursFunny bc7f249c20 fix: filter local animation cache sources 2026-09-24 19:44:48 +08:00
YoursFunny 403b84996f test: cover ugoira expansion budgets 2026-09-24 19:42:50 +08:00
YoursFunny f08b187964 fix: drop unsafe upstream media entries 2026-09-24 19:39:46 +08:00
YoursFunny 8428468712 fix: surface chat state read failures 2026-09-24 19:37:29 +08:00
YoursFunny 412359e84c fix: do not cache local media paths 2026-09-24 18:52:18 +08:00
YoursFunny 438ced278f fix: align CI tags and webhook healthcheck 2026-09-24 18:49:20 +08:00
YoursFunny 704b14f05a fix: bound ffmpeg encoding processes 2026-09-24 18:46:09 +08:00
YoursFunny 1f9da14772 fix: bound ugoira archive expansion 2026-09-24 18:44:21 +08:00
YoursFunny 4694e60ee3 fix: reject upstream local media paths 2026-09-24 18:41:23 +08:00
YoursFunny 88dc0ccb9e fix: stop persistent queue before URL workers 2026-09-24 18:37:17 +08:00
YoursFunny b563b949bb fix: avoid overwriting chat state after read failure 2026-09-24 18:35:52 +08:00
YoursFunny e7855b02fe fix: fence queue dead-letter side effects 2026-09-24 18:33:21 +08:00
YoursFunny 5e265f2413 chore(log): align levels with the documented convention
Three places disagreed with AGENTS' own level rules: get_chat/get_chat_administrators failures during /set_channel printed at error even though the usual cause is a mistyped channel name the user fixes on the spot (error is reserved for dead letters and the like — now warn, same as other user-served degradations); the edit-before-forward callback arrival line logged at info while the sibling arrival lines for commands and inline queries log at debug (now debug, still carries the escaped payload); and the two upload-fallback lines logged at info while photo.rs's parallel degradations log at warn (now warn both — Telegram failing to fetch our media is a degradation that leaves the user served). AGENTS' convention sentence is now explicit about all three instead of the ambiguous '(fallback, retry enqueue, dead-letter is error)'.
2026-09-24 15:38:25 +08:00
YoursFunny 5060b760ba fix(fetch): cap one fetch at a 900-second total budget
The per-request CLIENT timeout and the per-chunk DOWNLOAD_IDLE_TIMEOUT both reset on progress, so neither bounded the fetch as a whole: a URL that keeps dripping (a trickle the idle timeout reads as life) could hold one of the eight FETCH_SLOTS for effectively ever, and the queue behind the slots is what then stalls. The attempts loop now runs inside tokio::time::timeout(900 s) — generous for a genuinely large ugoira zip on a honest slow link (minutes), fatal for a drip — and answers Transient with the cache key, so the retry and its jittered backoff take over. The permit is taken outside the timeout: the slot is released on either path. Audit low finding 'url_workers long tail' (process-wide total-deadline option).
2026-09-24 15:38:25 +08:00
YoursFunny 7019f34801 fix(send): refuse a file id at the upload fallback's door
download_to_temp guarded against a file id, but prepare_upload_item answered a non-http media_url first — and item_url maps FileId to the id itself, so the id was presented as a local *path* and died later with a confusing file-open error instead of a classification. The guard now sits at the entry, download_to_temp takes the already-narrowed URL (the inner guard deleted as dead), and the test pins the refusal as Permanent before any I/O. The file also carries item10's download-classification arm (RateLimited is retryable there too) — same file, landed here to keep each tree compiling.
2026-09-24 15:35:55 +08:00
YoursFunny d05450d88a fix(fetch): jitter the retry backoff and honor a429's Retry-After
Two gaps in fetch_with_attempts: the sleep was the bare 1 << attempt, so every worker that failed together (a source coming back, a shared proxy blip) also recovered on the same tick and re-stamped the source; and a429 arrived as an anonymous Transient, its Retry-After header — the one place a source tells you exactly how long it wants silence — dropped on the floor.

retry_wait(attempt, roll, err) is the pure decision: doubling base plus a random slice of itself ([base, 2x base)), floored at a named Retry-After. status_error now takes the response, reads the seconds-form header and returns the new FetchError::RateLimited { site, retry_after_secs } (HTTP-date parses to None and stays transient); the pure table moved to classify_status so tests still build it from bare status codes. The delay is capped at MAX_RETRY_AFTER_SECS = 60 — the header is server-supplied and must not park one of the eight fetch slots.

Everywhere a Transient meant 'retryable' the new variant joins: the Site trait default, pixiv's override (plus its ugoira-zip mapping), bsky's HLS second attempt, the download classifier, and the user-facing message arm. Tests pin the429 rows (with and without the header, cap included) and retry_wait's math; AGENTS' retries bullet and variant list follow.
2026-09-24 15:35:55 +08:00
YoursFunny 053ae0ec25 fix(send): charge the photo download window to the memory budget
A non-photo body charged the process-wide budget for as long as download_to_temp held it; a photo charged nothing until its prepare step, so the download itself — up to MAX_PHOTO_DOWNLOAD_BYTES per item, six items per batch, eight prep slots process-wide — was memory the MEMORY_BUDGET comment promised was bounded but was not (~6x32 MiB on top of the accounted512 MiB). Photos now reserve their own download cap for that window; the prepare step still charges header probe + decode buffer, the only overlap that is actually held in memory together. Net accounting stays within the declared budget instead of exceeding it whenever a batch of large photos downloads at once (audit: upload.rs photo window uncharged).
2026-09-24 15:29:17 +08:00
YoursFunny d1f2ae09b2 fix(rate-limit): charge plain messages and chat actions to the global bucket
send_media_group, send_animation and copy_messages paced themselves against both buckets; send_message and send_chat_action paced against neither. A dead-letter storm — or one error reply per failed item, plus the action refresh loop's repeated typing requests — could therefore burst past Telegram's per-bot30/s ceiling with only429s left to absorb it, exactly what the bot-wide bucket exists to prevent. Both now acquire one global token like their siblings (per-chat pacing for messages already happens at their call sites; chat actions are cheap and frequent, so only the global bucket applies to them).
2026-09-24 15:28:59 +08:00
YoursFunny 715d6a59b0 fix(photo): sum dimensions in u64 so a huge header cannot wrap
plan_photo, target_dims and both pipeline branches added the two u32 dimensions before comparing against PHOTO_MAX_DIMENSION_SUM: u32::MAX + 2 wrapped to 1 and read 'within limits' — the plan said AsIs, and an absurd-sized image would go to Telegram untouched (in debug builds the addition panics instead). The four production sums now widen to u64 first; the test pins the wrap case as TooLarge. PNG caps each dimension at 2^31-1, so a spec-valid file cannot reach this today — the raw header is parsed before any crate validation, which is exactly where a hostile file lands (audit: photo.rs u32 wrap).
2026-09-24 15:25:19 +08:00
YoursFunny ef8a1dae15 fix(handlers): honour the TTL when a caption-edit reply arrives
A button press on an edit-before-forward prompt already dropped records past EDIT_MESSAGE_TTL lazily (before the 300 s sweep clears them); a text reply to the same prompt did not — it rewrote the caption from a record the sweep was about to treat as dead, so the two paths disagreed about what 'expired' means. The reply path now applies the same rule: drop the stale record, return false, and let the message flow on as if no prompt existed. The test seeds a prompt90000 s old (past the TTL under any config a parallel test can hold) and pins all three effects: no edit consumed, no API call, record gone.
2026-09-24 15:23:02 +08:00
YoursFunny ac34759b02 fix(state): report a failed chat-state write to /set_format
set()/update() logged a failed DB write and returned nothing: the cache already held the new value, so /set_format answered 'Format set' for a change that vanishes on the next restart — a promise the retry queue's enqueue path is already forbidden from making. Both now return whether the write landed, and /set_format's two save paths (set and reset) answer 'in memory only … lost on restart' when it did not. The flag travels as the second tuple element; callers that only relay the value destructure it, callers that ignore it are untouched. A false cannot be forced in a test without a pool-failure seam, so the honest path is pinned by the code, not a fixture.
2026-09-24 15:21:05 +08:00
YoursFunny c420c95165 fix(docker): refuse a root or non-numeric LOCAL_USER_ID
LOCAL_USER_ID=0 passed every check and reached `setpriv --reuid=0`: the bot would run root while looking properly configured, and useradd -o accepted the duplicate uid without complaint. A non-numeric value failed later inside useradd behind `|| true`, which hid the real cause. The entrypoint now rejects both up front with a message naming the variable, and the id/uid expansions are quoted so a value with spaces cannot word-split into extra useradd arguments (audit SEC-007).

Verified by sourcing the entrypoint under a faked `id`: uid 0, 'abc' and '12 3' each exit 1 with the refusal line, while a non-root caller passes straight through to exec. AGENTS and .env.example state the constraint.
2026-09-24 15:12:22 +08:00
YoursFunny 4d6ec7924b fix(handlers): escape user-supplied text in log lines
A Telegram display name, callback payload or channel handle is attacker-controlled text, and it went straight into info/error lines: one embedded newline forged a second log entry, and control characters could hide inside a line (log injection, audit SEC-006). handlers::log_escape replaces newlines, carriage returns and other control characters with visible escapes — clean input borrows, so logging allocates nothing extra — and every site that prints such a value uses it: the callback arrival line, the set-forward-channel info line (name + handle) and both get_chat error lines. The test pins that no raw newline can survive; AGENTS' logging convention names the helper.
2026-09-24 15:02:31 +08:00
YoursFunny 9bae46eb9a chore(compose): pin acme-companion to its paired release
The companion was the file's only unpinned image: it runs with a rw docker.sock, so 'latest' let tomorrow's upstream change deploy itself on the next pull, with full visibility into this stack's containers and environment. 2.8.2 is the release paired with the already-pinned nginx-proxy 1.11.6 — the two projects cut releases a day apart (1.11.6 on 2026-07-16, 2.8.2 on 2026-07-17).
2026-09-24 15:00:45 +08:00
YoursFunny cb91913088 docs(agents): correct the CI, coverage and cert-mount claims
Five statements had drifted from the repo:

- the test-suite comment claimed no CI test step exists; ci.yml has run
  cargo test --workspace --locked all along (the audit's H5).
- the untested list still named config.rs as fully untested although three
  parsing tests pin its webhook truth table, TTL fallback and blank secrets.
- the commands.rs executor was said to need a real Bot; its three tests run
  through the scripted MockSender, and it joins the driven-through-mocks list.
- model.rs was called untested although every adapter fixture deserializes
  into those DTOs.
- the volume line claimed ./cert is mounted; the shipped compose mounts only
  ./data, and WEBHOOK_CERT (a Telegram-facing self-signed upload) needs the
  operator to add mount and env line themselves, which .env.example already
  says. The live-test census also gained site/download.rs (2) and the bot
  crate's repair.rs/urls.rs pair.
2026-09-24 14:54:07 +08:00
YoursFunny 2b6cb506c7 refactor(download): fold apply_media_headers into media_request
apply_media_headers had exactly one caller — media_request, the documented 'one choke point' every download already goes through — so the site-header walk sits inside that choke point now instead of one indirection past it, and the two doc comments merge into the one that describes both rules (guard + headers).
2026-09-24 04:37:54 +08:00
YoursFunny c49e2175a1 refactor(site): drop RenderData's url field, source_url already carries it
Every adapter built RenderData { url } from the exact value it then moved into Fetched.source_url — a copy of the same canonical URL held in two places, feeding one placeholder. caption_with reads {url} from self.source_url now and the field is gone from the struct (bsky's and twitter's halves of this change landed with their caption commits; their render builders no longer set it either).
2026-09-24 04:37:54 +08:00
YoursFunny 2ec13a0624 refactor(send): inline hidden_template_count into its two call sites
A one-line saturating_sub wrapper exported for one production line (the prompt text) and two assertions — the expression is shorter than its name at every use.
2026-09-24 04:37:54 +08:00
YoursFunny 0765b7deec refactor(send): media_from reads the thumbnail off the item itself
All three callers passed item.thumbnail_url() as the fourth argument — a parameter that could never vary without defeating its own purpose, and only the video arm ever reads it. The function takes the item already; it asks the item. Three call sites lose an argument.
2026-09-24 04:37:53 +08:00
YoursFunny e72cb1b98f refactor(send): inline the per-kind media builders into media_from
photo_media, video_media and animation_media each wrapped the same five lines (build the kind, attach the caption, attach the spoiler) for exactly one caller — media_from's own match arms. The arms carry those lines now; three pub(super) functions and their call indirection are gone, and the dispatch the doc already described as 'the one place' actually is the one place.
2026-09-24 04:37:53 +08:00
YoursFunny 9fc5533676 refactor(twitter): delegate the tweet caption to the shared builder
Tweet::caption duplicated site::caption byte for byte for non-empty text (and dropped the shared empty-text rule, leaving a dangling ': ' behind the author link when a tweet's text comes out empty after link expansion). It delegates now, like bsky's Post::caption a commit earlier and bilibili/misskey before them; encode_double_quoted_attribute loses its last twitter user.
2026-09-24 04:37:53 +08:00
YoursFunny 46bbe8639d refactor(bsky): delegate the post caption to the shared builder
Post::caption hand-formatted exactly the string site::caption produces — same template, same three escapes (the shared fn even handles the empty-text case these two lines did not: no dangling ': ' after the author link). One line now, matching what bilibili and misskey already do; the encode_double_quoted_attribute import loses its last bsky user.
2026-09-24 04:37:53 +08:00
YoursFunny 2ff213e22f refactor(inline): delete InlineKind, convert straight to CachedMediaKind
InlineKind mirrored CachedMediaKind variant for variant (Photo/Video/Gif) and existed only to feed url_result, with two conversions kept in step: Media → InlineKind at the fetch call and CachedMediaKind → InlineKind at the cache call (plus the enum and the mapper). url_result takes CachedMediaKind now — the type the cache path already carries and the fetch path maps to with the same three-arm match — and the degraded-video skip collapses to a guard before the call. One enum fewer between a media kind and its Telegram result.
2026-09-24 04:37:53 +08:00
YoursFunny 2c45b3491d refactor(main): the sweep takes one AppContext instead of five collaborators
periodic_sweep listed sender, chat_store, link_cache, task_queue and config as separate parameters — exactly the five fields AppContext already carries, and exactly what handle_message and the workers pass around as one value. It takes &AppContext now (production via from_statics(&bot), the test via stores.ctx(&sender), which that test already had); CHAT_STORE and LINK_CACHE lose their last direct use in main and leave the import. The collaborator bundle is unchanged, so the paused-clock test drives the same loop.
2026-09-24 04:37:52 +08:00
YoursFunny a3d03cf30f refactor(site): one ffmpeg gate instead of a probe plus a log call
ffmpeg_available() and log_once_ffmpeg_missing() only ever appeared together (both encoders, three lines each: check, log, return), and calling one without the other was a bug in waiting — a probe that never logged, or a log that never gated. ffmpeg_missing() is the single gate: false when the binary is there, otherwise log once per process and say yes. Both call sites shrink to one condition.
2026-09-24 04:37:52 +08:00
YoursFunny e64b48a453 refactor(site): match the site pattern once, then check enabled
find_site walked the registry with "enabled && matches" and disabled_site walked it again with "!enabled && matches" — two passes re-running the same regexes to answer one question. matching_site finds the first match and the fetch decides what a disabled site means; the patterns are disjoint (one domain each), so "first enabled match" and "first match, then check" never disagree.
2026-09-24 04:37:52 +08:00
YoursFunny 022f63916c refactor(site): drop site_id_from_key, split the key at its caller
site_id_from_key split the prefix out and then walked the registry to confirm the prefix was a registered site id — a round trip over a value cache_key itself produced from that registry: the unknown and no-colon branches were unreachable for any key the bot makes. The single caller (the link-cache hit path) splits the prefix directly; the registry-echo test and the misskey assertion of it go with it.
2026-09-24 04:06:13 +08:00
YoursFunny f984428169 refactor(photo): fold photo_plan into its only caller
photo_plan wrapped the header dispatch for exactly one consumer (decode_budget_bytes); plan_photo itself has three callers and stays. The dispatch is inlined into the budget function with its doc merged, saving the wrapper's signature, doc duplication and call indirection.
2026-09-24 04:03:59 +08:00
YoursFunny 85fb12edc7 refactor(photo): read the PNG header with the png crate
parse_png_header hand-decoded the IHDR — byte offsets, the depth byte's five legal values, the color byte's five — roughly thirty lines the png crate already implements (and validates properly: CRC included). It is now Decoder::new + read_info, which is all the header a plan needs; no pixels are decoded. The synthetic test fixture gained the IHDR CRC and an IDAT header (read_info stops at the first IDAT; the hand parser stopped four bytes earlier and checked no CRC), with a nine-line reflected CRC-32 alongside it; the assertions on width/height/depth/color are unchanged, as are the real-file cases.
2026-09-24 04:03:08 +08:00
YoursFunny 7bdf760c7b ops(compose): align the deploy template defaults and gate the healthcheck
Two template landmines for anyone running docker compose from the tracked files. (1) LOCAL_USER_ID defaulted to 1000 in compose and .env.example while the entrypoint and both READMEs say 9001 — the ./data owner on the host silently depended on which doc you read; all four now say 9001. (2) The healthcheck probed WEBHOOK_PORT unconditionally, so the template's own WEBHOOK=false (polling, no listener) shipped a permanently unhealthy container — the probe is now conditional on the interpolated WEBHOOK value (test '<v>' != true || exec 3<>/dev/tcp/…): polling deployments answer healthy without a port, webhook deployments still surface a dead listener to the orchestrator. Both expressions verified locally; compose YAML parses.
2026-09-24 03:47:34 +08:00
YoursFunny 3dbcb45899 test(config): pin the webhook truth table, TTL fallback and blank secrets
Config::load had exactly one test (BOT_ADMIN parsing) while carrying the branches an operator is most likely to mistype: the WEBHOOK truth table (case-insensitive true|yes|1 — 'on' must not enable it), an unparseable EDIT_MESSAGE_TTL_SECONDS (warn + default, not a silent 0 that expires prompts instantly), and empty WEBHOOK_CERT/WEBHOOK_SECRET_TOKEN (compose injects ${VAR:-} as an empty string, which must read as unset, not as a one-character secret). Three table-style tests reuse the existing env save/restore pattern; none of them asserts a field another test reads, so they are safe under parallel execution.
2026-09-24 03:46:59 +08:00
YoursFunny aee1cf35ae ci(docker): run the image build check on PRs that touch crate sources
The pull_request paths filter listed only Dockerfile/entrypoint/.dockerignore/Cargo.{toml,lock}/workflow/crate manifests, so a PR changing only Rust sources never triggered the build-only check — and the Dockerfile's stub-plus-touch layering is exactly the thing that breaks against source structure it has never seen (a new crate directory, a build script). Such PRs now get the check before master/tag reveals the problem; crates/**/Cargo.toml is subsumed by crates/**.
2026-09-24 03:45:35 +08:00
YoursFunny 7b50ad82b0 ci: make the live job honest about what it ran and skipped
Three holes in the live job. (1) It ran -p x-media only, on the comment that everything network-gated lives there — false: xmedia-bot carries two #[ignore]d live tests (repair refetch, text-only link) that CI never executed; both crates run now, still behind the 'live' name filter, and the bot crate's tests need no token (their sends go through MockSender). (2) BILIBILI_COOKIE was never passed, so the bilibili live tests always hit the risk-control early return. (3) That early return — like the pixiv download test's token gate — printed an indistinguishable 'skipping:' line that --show-output never showed, so with continue-on-error a fully-skipped weekly run read exactly like full coverage; skips now print a 'SKIP ' prefix, the steps tee with --show-output, and a summary step lists every skip in the run summary plus a workflow warning. The redundant 'Run token-gated tests' step (the whole non-ignored suite, a rerun of the test job) goes away with the comment that justified it.
2026-09-24 03:45:17 +08:00
YoursFunny a49bb1d1af test(x-media): gate the pixiv download test as live-network too
The pixiv download test was token-gated but not #[ignore]d, so a shell with PIXIV_REFRESH_TOKEN exported made every local cargo test --workspace hit i.pximg.net — where it demonstrably flaked (twice in one session: TLS EOF mid-body) — and CI's non-ignored step ran it as a side effect of having a secret. It carries the convention's #[ignore = "live network: …"] now and is renamed with the live_ prefix so the live job's --ignored live name filter actually picks it up (the old name did not contain 'live' and would have been filtered out). Its skip prints the SKIP prefix the next commit's live job greps, and AGENTS.md stops claiming a local workspace run is not fully offline.
2026-09-24 03:44:06 +08:00
YoursFunny e99f8b0bc0 build: pin ffmpeg to a versioned URL and verify its sha256 unconditionally
The default build fetched /redirect/latest/ — a floating URL that changes under every build — and skipped the sha256 check whenever FFMPEG_SHA256 was empty, which it always was: neither the Dockerfile nor docker.yml carried a hash, so any binary the CDN served reached the image unverified and then processed untrusted media bytes. Both now pin the 9.0.2 release build with the hash the mirror publishes beside it (verified here by direct digest of the downloaded zip: fa8ecf4a…909d7f matches the sidecar), docker.yml's build-args fall back to the same pair, and the check is unconditional — an FFMPEG_URL override without its matching hash fails the build at the download step. The ffmpeg layer also moves above the dependency layer so a manifest/lock edit no longer re-downloads it (the old comment claimed caching 'unless FFMPEG_URL changes', which was never true), and the stale step reference in the sources comment goes with it. Local note: Git-Bash's sha256sum -c reads files in text mode here and cannot roundtrip anything binary — the -c line itself is standard and is exercised by the PR build in CI.
2026-09-24 03:43:04 +08:00
YoursFunny 3fd04e4420 feat(x-media): gate every fetch entry with one process-wide semaphore
The URL workers bound only their own path (8 workers, bounded channel); inline queries, /debug and /test spawned fetches straight from handler tasks with no limit at all — and the expensive part runs inside the fetch (ugoira: up to 512 MiB plus ffmpeg; bsky: an HLS remux), so N users could mean N concurrent encodes. fetch_with_attempts now takes one permit from an 8-slot gate (the count matching URL_WORKERS, so the bot's own pipeline keeps its full width), which covers every entry at once: message links, queue-side repair, inline, /debug, /test, and retries. Unsupported and disabled links answer before the gate, and the shared-fetch dedup already waits outside x-media, so nothing double-counts.
2026-09-24 03:38:53 +08:00
YoursFunny 0e91753402 perf(x-media): give the slot-holding fallback download its own budget
download_media_limited had one hard-coded total (600s) for every caller, and its heaviest caller — the bot's upload fallback — holds a PREP slot (and its memory reservation) for the whole transfer: six slow-but-alive downloads (a byte every 29s satisfies the idle window) could stall the fallback chain for ten minutes, queue retries included. The budget is a parameter now: the fallback passes 300s of its own (50 MiB in 300s ≈ 1.4 Mbit/s; a slower link is better served by retrying toward the item's smaller URL than by pinning a slot), while bsky's in-fetch HLS segments keep the generous 600s DOWNLOAD_TOTAL_TIMEOUT, now pub(crate) and re-exported for them. download_too_slow reports whichever budget it got.
2026-09-24 03:37:38 +08:00
YoursFunny 8f254c3779 perf(x-media): stream the frame zip through a tokio file handle
download_media_to_file took &mut std::fs::File and wrote every network chunk with a sync write_all on the executor thread — for a ugoira frame zip (up to 512 MiB) that is the whole download stalling a runtime worker, while bsky's remux had already been moved to tokio::fs for exactly this reason (its comment: a multi-megabyte std::fs::write blocks the executor thread). The signature takes &mut tokio::fs::File now; the single caller (pixiv's ugoira path) clones the NamedTempFile's handle — the clone shares the file offset, so the ZipArchive extraction in spawn_blocking reads what the download wrote — and drops it after the download hands the bytes to the OS.
2026-09-24 03:33:23 +08:00
YoursFunny e9dd3aece2 perf(send): write the fallback temp file on a blocking thread
download_to_temp wrote the whole buffered body — up to MAX_MEDIA_UPLOAD_BYTES, 50 MiB — with a sync write_all on whatever executor thread ran the prep task; six PREP slots could stall six runtime threads at once on a slow volume (Docker volume, SD card), pausing unrelated handlers and chat-action refreshes with them. The write now runs in spawn_blocking, the pattern the photo decode in the same file already uses; the failure classes are unchanged (write error = retryable resource exhaustion, panic = permanent, worker bugs must be loud).
2026-09-24 03:30:30 +08:00
YoursFunny 67b6bd3038 fix(commands): stop /debug from nesting blockquotes on long posts
debug_report wrapped every caption in <blockquote>, but preview_caption for a post over CAPTION_QUOTE_TEXT_CHARS already carries quote_long_caption's expandable blockquote — and the API rejects nested entities, so /debug on any long post answered 400. The wrapper now skips a caption that already carries a blockquote, exactly the rule quote_long_caption itself applies; the new test pins that the quoted caption passes through with a single blockquote while the existing test keeps pinning the wrap for ordinary captions.
2026-09-24 02:53:49 +08:00
YoursFunny 3fb6b3b4da fix(fetch): one status table — a persistent 4xx is permanent everywhere
status_error's catch-all called every unlisted status Transient, so a 400/405/418/451 got three retries per link before the same answer (twitter syndication's broken-token 400 being the live example), and download_status_error plus misskey's and bilibili's local fallbacks each carried their own copy of the table — bilibili and misskey classifying a 404 as Transient while the center classified it NotFound. The center now makes any client error except 408/429 a refusal (permanent), the media path delegates to it as status_error("media", ...) and its duplicated fn is deleted, and misskey/bilibili fall through to the center after their own special statuses (misskey's 400 body, bilibili's 412). A table test pins every class.
2026-09-24 02:52:10 +08:00
YoursFunny e6800fd27b fix(pixiv): keep a frame-zip download hiccup retryable
download_media_to_file's non-transport errors all folded into PixivError::Api, and Api is permanent in pixiv_error_is_retryable — so one 429/5xx, stalled transfer or temp-file write failure while fetching the ugoira frame zip permanently failed the whole post, while the bot's own upload downloads classify the very same classes as retryable (classify_download_error). A new PixivError::Transient carries those classes into the existing retry policy (and into startup validation's 'stays enabled' branch); Http keeps its arm and everything else stays permanent. The retryable table and the startup-validation loop pin both halves.
2026-09-24 02:50:22 +08:00
YoursFunny 38e65a3791 fix(queue): close the lost-wakeup window on shutdown
stop() flags the shutdown and fires notify_waiters, but a worker parked between its loop-top stop check and its notified() registration — i.e. inside earliest_run_after's DB await — was not registered when the notification fired, so it slept until the next enqueue that never comes; stop() then blocked until main's 30s shutdown timeout force-killed the drain. The sweep had the same window before recover_expired's await and would sit out a full 30s tick. Both loops now enable() the waiter first and re-check the stop flag: either the stop already happened (recheck returns) or the waiter is registered (notify_waiters reaches it) — no gap. The window itself is a scheduling race with no test seam, so this is pinned by reasoning rather than a regression test; the existing stop tests cover the ordinary path.
2026-09-24 02:49:10 +08:00
YoursFunny eecd4320f8 fix(state): keep failed and stale reads from poisoning the chat cache
Three related races in ChatStore. A read that errored (busy/IO) was indistinguishable from an absent row, so the default got cached — and the next update would write that default back over the chat's real settings (forward channel, templates, formats). A cache-miss backfill inserted unconditionally, letting a stale DB snapshot overwrite the value a concurrent set had just written. And sweep eviction removed per-chat locks unconditionally, so a lock pulled out from under an in-flight update let a second writer create a fresh one and enter the critical section concurrently; eviction now keeps any lock with a holder (the same rule rate_limit's prune applies), which still bounds the map because an uncontended leftover is caught by a later sweep. Two regression tests pin the failed-read and contended-lock cases.
2026-09-24 02:47:49 +08:00
YoursFunny d774f0b37c fix(pixiv): treat an empty refresh token as unset
docker-compose injects PIXIV_REFRESH_TOKEN='' for a blank .env value and enabled() only checked that the variable exists, so a default deployment read pixiv as configured, sent an empty refresh token to OAuth and failed login validation on every boot (error line + admin DM) — and it disagreed with the empty-as-unset gating the tests already use. Filter empty strings in PIXIV_CLIENT and enabled(), like twitter and bilibili; a unit test pins empty == unset.
2026-09-24 00:48:29 +08:00
YoursFunny 5d44946690 perf(db): size the pool to cover every DB consumer
POOL_SIZE = 4 was the process-wide cap on concurrent DB operations while 4 queue workers, 8 URL workers, dispatcher handlers and the sweep all share the pool — WAL readers queued behind writers and every hot-path round trip (3-5 per message) contended for four permits. 16 covers every consumer at once; SQLite's single writer serializes writes regardless.
2026-09-24 00:48:29 +08:00
YoursFunny 33b1f04f0e fix(send): release one keep-alive reference per settled task
release_keep_alive retained every KEEP_ALIVE entry whose path matched the settling task, deleting the shared TempDir out from under a concurrent duplicate of the same post (a shared fetch pushes one Arc per pipeline): the duplicate's queued retry then dead-lettered on local media that no longer existed. Release now removes exactly one matching entry, which requires settle to run once per task — handle_task settled on Permanent right before the queue invoked dead_letter_notify, which settles the same payload again, so the redundant settle is dropped. A regression test pins one settle to one entry removed.
2026-09-24 00:48:28 +08:00
YoursFunny 4c1fa857c4 fix(webhook): refuse to start without a secret token
Webhook mode passed the secret token to the axum listener only when WEBHOOK_SECRET_TOKEN was set, and docker-compose defaults it to empty (an empty string counts as unset) — the default deployment therefore ran its listener on a public port with no check on X-Telegram-Bot-Api-Secret-Token, so anyone could POST forged updates and impersonate admins (/bot_dict, /clear_cache, /test). Startup now fails when webhook mode has no secret; .env.example and AGENTS.md spell out the requirement.
2026-09-24 00:48:21 +08:00
YoursFunny 3ebc1e4a8f feat(inline): answer from the link cache, and put the answer on the trait
Inline results were URL-only: Telegram fetches an inline result's URL itself
and cannot send site headers, so every pixiv item (and every locally encoded
ugoira/bsky MP4) was skipped and such a query answered empty. A post that is
already in the link cache now answers with InlineQueryResultCached* built
from its Telegram file ids — no fetch, no upload, and the hotlink-protected
case simply works. A degraded entry (file ids gone) falls back to URLs, and
there a video with no poster is skipped (Telegram would try to render the
mp4 as its own thumbnail).

The answer call moves onto MediaSender (answer_inline_query, mirroring the
other user-flow methods), which is what makes the path testable at all: the
two new tests drive the cache answer and the degraded/empty answer through
TestStores + MockSender, which recorded nothing about inline before.

The result builders are shared by both paths now (url_result/cached_result +
inline_kind), so the fetch path's behaviour is unchanged.
2026-09-21 20:35:18 +08:00
YoursFunny 20a0cf3ea3 feat(pixiv): fall back to the large variant when a page has no original
A page whose `original` the API left out (the restricted ones) was dropped
whole by the multi-page branch's filter_map — a missing picture in the album
while `large` sat right there — and the single-page branch produced no media
at all when both places it looks for an original were empty. Both now share
one builder that falls back to `large` (the same picture, lower resolution).
The two tests that pinned the old behaviour now pin the fallback.
2026-09-21 20:29:48 +08:00
YoursFunny f6523e021d feat(urls): deliver a media-less post as its text
A post with no media was answered with "No media found or media type is not
supported.", throwing away text the fetch had already parsed, escaped and
built a caption for (the per-site format and the long-post quoting
included). It now goes out as a message through the same caption the media
path would attach — the senders' own quoting is applied here, since there is
no sender to do it. No queue entry: there is no Task shape for text and a
post with nothing to download is cheap to paste again, so a failure is
reported (send::send_text_post) rather than retried.

Proven live: live_a_text_only_link_is_sent_as_text fetches a real text-only
tweet through url_media and asserts one send_message carrying the post link
and no media send.
2026-09-21 20:24:40 +08:00
YoursFunny c549a6d35e feat(upload): give videos and animations Telegram's real 50 MB cap
One MAX_UPLOAD_BYTES (10 MiB) bounded every upload, but that is the *photo*
limit: Telegram's own docs say sendVideo/sendAnimation/sendDocument take up
to 50 MB, and RequestEntityTooLarge is "larger than 50 MB". So a 10-50 MB
video that Telegram refused to fetch by URL was refused a download too, and
a video has no smaller variant — the post was lost. The non-photo cap is now
MAX_MEDIA_UPLOAD_BYTES, and such a body charges the process-wide budget for
the length of the preparation (one 64 MiB unit covers the cap), since
PREP_SLOTS alone no longer bounds their added RAM. A const test pins both
caps against Telegram's numbers.
2026-09-21 20:21:01 +08:00
YoursFunny 9e5b6df21c docs: note the payload's MediaRef in AGENTS.md 2026-09-21 18:44:42 +08:00
YoursFunny aa70b45ae6 refactor(send): model a payload's media as MediaRef, not media + file_id
MediaItemPayload carried one String field with two meanings and a file_id
bool beside it to say which, in all three variants; every reader re-checked
the flag (input_file had a three-arm pattern just to find the file-id case).
MediaRef::Source/FileId says it once, and the readers now match on it —
local_media_paths, item_url, input_file and the download path each shrank to
the one branch they care about.

Fixes a real failure that the flag was hiding: send_animation built its
InputFile with input_file_for(media_url), which read a cached file id as a
local path and answered "local media file missing" — permanent. So the
second request for a single-gif post always failed (the third worked, from
the degraded entry). It now uses the payload's own input_file, and
a_cached_animation_sends_by_file_id fails without that line.

Wire shape: a queued row from before this change no longer parses, and the
queue already handles that shape (handle_task dead-letters it as an invalid
payload, and dead_letter_notify still names the post and drops the stale
cache entry).
2026-09-21 18:44:26 +08:00
YoursFunny 70d4f2ec88 refactor(x-media): move the download stack into site/download.rs
site/mod.rs held the registry, the Site trait, Fetched, the caption helpers,
FetchError and, at the end, the whole media-download stack: two HTTP clients
with different timeouts, the guard that refuses a URL inside the host's own
network (start URL and every redirect hop), and the two streaming entry
points. That stack is its own reason to change and moves whole, with its
tests; CLIENT and the download fns stay re-exported at site::… so no adapter
or bot call site moved. SITES becomes pub(crate) for the header rule.
2026-09-21 18:35:58 +08:00
YoursFunny b7763a6572 refactor(send): move the Bot API error policy into send/error.rs
classify_request_error, the marker tables it matches on, Classification and
SendError (with its fallback conversion) are one policy — which failures are
retried, which are permanent, which the reupload fallback owns — and were
interleaved with the payload types and the senders. They move whole into
send/error.rs and are re-exported, so every existing send::… path is
unchanged.
2026-09-21 18:32:07 +08:00
YoursFunny 459bfe5803 refactor(media_sender): move the test support into its own file
Two thirds of media_sender.rs was cfg(test)-only scaffolding (MockSender and
the fake_api), which made the trait's own surface hard to find. The module
becomes media_sender/mod.rs (trait + the Bot impl, 260 lines) plus
media_sender/test_support.rs (445); the path
crate::media_sender::test_support is unchanged, so no caller moved.
2026-09-21 18:30:01 +08:00
YoursFunny 01e097a8b6 refactor(handlers): split urls.rs into workers, pipeline and startup repair
urls.rs carried five reasons to change: the job channel and its worker pool,
the single-flight fetch, URL parsing, the per-URL pipeline, and the startup
repair of queued retries. The two with their own lifecycle move out:

- url_workers.rs: the bounded channel, its supervised pool and
  start/stop_url_workers (the pipeline stays in urls.rs, which the workers
  call).
- repair.rs: needs_refetch/apply_refresh/refetch/repair_lost_local_media with
  their tests, moved whole (the live one keeps its #[ignore]).

Also folds the two byte-identical render_fields -> CachedPost mappings
(urls.rs and repair.rs) into urls::cached_snapshot, and moves the shared
permanent_error test fixture into ctx::test_support.
2026-09-21 18:28:41 +08:00
YoursFunny 435c8c4cc4 refactor(commands): the executor takes its context instead of the statics
execute_command reached for CHAT_STORE/LINK_CACHE/CONFIG in 16 places, which
is why the one handler body with no test could not have one: those statics
point at the real $DATA_DIR/task_queue.db, so any test would have written to
the developer's state. It now takes &AppContext (the shape handle_message
uses) and answers through the given sender; bot stays for what the
MediaSender surface does not carry (channel admin lookups, the HTML report).

Three tests cover what that unlocks: the settings/set_format round-trip
against the chat's own store (including the refused placeholder), the admin
gate on both admin-only commands, and /debug answering without sending.
2026-09-21 18:25:08 +08:00
YoursFunny df8fd77d29 docs: sync AGENTS.md with the refactors
download_media is gone (the bot uses download_media_limited), the per-site
modules no longer export enabled/is_retryable/media_headers for sites that
inherit the trait defaults, and the shared fetch is shared_fetch.
2026-09-21 17:43:43 +08:00
YoursFunny f7179d65ee refactor(db): open_store no longer creates the directory
handlers::db_path already creates the DATA_DIR before it opens
$DATA_DIR/task_queue.db, and every other caller (tests) passes a path whose
parent exists, so the guard plus its one-caller rusqlite_error mapper were
duplicated work. The doc now says which caller owns the directory.
2026-09-21 17:42:56 +08:00
YoursFunny 65c9aa6c5c refactor(send): inline parse_media_url, drop photos_first's rebind
parse_media_url had a single caller (input_file_for) and read better as the
one expression it wrapped; photos_first rebound its argument only to gain
mut.
2026-09-21 17:41:57 +08:00
YoursFunny c58202e683 refactor(send): call send_animation directly
send_animation_inner only reshuffled its arguments into MediaSender's
send_animation; both call sites (URL send and the reupload fallback) now call
the sender themselves, which also drops the InputFile and MediaSender imports.
2026-09-21 17:40:43 +08:00
YoursFunny ea63dbb9f9 refactor(send): inline Task::media_items into its only caller
local_media_paths was the sole caller; the match that flattens the batches
(or the lone animation) moves into it.
2026-09-21 17:39:24 +08:00
YoursFunny 9db1a6093b refactor(main): drop the no-op dispatcher dependency
dptree::deps![""] inserted a &'static str no handler ever asked for;
Dispatcher::builder already starts from an empty DependencyMap, so the call
was the default written out.
2026-09-21 17:38:11 +08:00
YoursFunny bd98001d42 refactor(urls): inline fetch_shared into its one caller
It bound the process-wide map and the site fetch for shared_fetch and had a
single call site; the caller now passes both directly.
2026-09-21 17:37:31 +08:00
YoursFunny 90663a51cc refactor(handlers): inline reply_html into its only caller
The /debug arm was the sole caller; the HTML parse mode and reply decoration
move there with the Requester-disambiguation comment.
2026-09-21 17:36:44 +08:00
YoursFunny 7ecb61399d refactor(handlers): reply returns unit
Every one of its call sites discarded the message id (a bare ?; or let _ =),
so the Result carried a value nothing could use.
2026-09-21 17:35:50 +08:00
YoursFunny 40ab07882d fix(handlers): restore the group hint's link filter
Replacing is_group(kind) with teloxide's predicates dropped the parentheses:
&& binds tighter than ||, so the branch read as is_group() OR (is_supergroup()
AND has-link) -- a plain group got the hint for any link, supported or not.
The e2e test only covered a group with a supported link (which is true either
way); it now also covers an unsupported link in a group.
2026-09-21 17:35:07 +08:00