The docker workflow only builds/pushes; tests were a local responsibility.
Add .github/workflows/ci.yml with two layers:
- test: cargo fmt --check + cargo clippy --workspace --all-targets -D
warnings + cargo test --workspace (fully offline, no secrets) on every
push/PR, including forks.
- live: the #[ignore]d live-network tests plus the pixiv token-gated
tests, run on schedule / manual dispatch / tag pushes only (fork PRs
cannot read repository secrets), with PIXIV_REFRESH_TOKEN /
TWITTER_AUTH_TOKEN injected and continue-on-error for flaky sites.
Test gating (documented in AGENTS.md):
- live-network tests now carry #[ignore = "live network: ..."] (twitter 3,
bsky 2, pixiv bogus-token 1) and run via -- --ignored live.
- pixiv token tests early-return when PIXIV_REFRESH_TOKEN is absent or
empty (an unset GitHub secret arrives as ""); test_fetch previously
failed without a token.
Also fixes the three clippy assertions_on_constants warnings in send.rs
(required for -D warnings).
A task whose media is a locally produced file (pixiv ugoira MP4, bsky HLS
remux MP4) references a path inside a tempfile TempDir owned by Fetched.
The retry ran after that TempDir was dropped, so the file was already gone
and the retry always failed permanently ("local media file missing") — and
the upload fallback even tried to GET the local path as a URL.
Keep the temp dirs in a process-wide registry (Fetched::take_keep_alive ->
send::KEEP_ALIVE) that is only released when the task settles (sent or
permanently failed); the upload fallback now uploads local files directly
instead of attempting to download them.
Restart-mid-queue still loses the files (documented behavior in
input_file_for) — only the in-process retry path is fixed here.
OpenSSL came from two removable defaults: teloxide's `default` feature
(native-tls) and x-media's reqwest default features (default-tls). Switch
both to rustls (webpki-roots) so the binary links no system TLS libs:
- teloxide: default-features=false + rustls + ctrlc_handler (was part of
the removed default)
- reqwest (x-media): default-features=false + rustls-tls
Verified: openssl-sys/native-tls gone from the tree, cargo check clean,
all 5 live twitter/bsky fetches pass over rustls, full container startup
works. The runtime image now needs no libssl.so.3/libcrypto.so.3 or CA
bundle (ffmpeg only processes local files; all downloads go through
reqwest), saving ~8MB.
x-media's reqwest 0.13 dragged in quinn/rustls/aws-lc-rs (cmake C
build) alongside teloxide's 0.12; unifying on 0.12 removes the whole
second TLS/QUIC stack from the build and image. The unused regex dep in
xmedia-bot is gone; x-media keeps url (the bsky HLS remux uses it).
Post URLs and author URLs were interpolated raw into <a href> attributes
(and the raw user URL from empty_fetched into caption text), so crafted
links could break the HTML parse and fail the send with a 400. All
attribute interpolations now use encode_double_quoted_attribute; text
stays encode_text.
The shared HTTP client ignored the proxy the Bot API uses, so on
proxy-required networks (e.g. behind the GFW) every site fetch failed
while the bot itself worked. Explicit proxy overrides reqwest's system
detection; unset keeps direct connections.
Both matched URL substrings anywhere in text, so a link like
https://evil.com/?u=pixiv.net/artworks/1 triggered a real fetch and
cache-key pollution. Prefix ^(?:https?://)? like the twitter pattern.
Twitter (syndication + auth GraphQL) mapped every non-2xx to NotFound,
killing retries on 429/5xx; bsky never checked status; pixiv network
errors arrived wrapped in PixivError and were excluded from the retry
loop. New FetchError::Transient covers 429/5xx from all sites, the
retry loop now also retries Pixiv errors, and 404/410 stay permanent.
download_to_temp now uses download_media_limited: non-photos abort the
moment the 10 MiB upload cap is crossed mid-stream (no more full-body
buffering before the size check), photos cap at the 512 MiB decode
budget, and the ugoira frame zip gets a 512 MiB cap. MediaTooLarge
routes to the existing smaller-URL fallback.
Sniff the frame extension from magic bytes instead of the entry filename,
cap each frame at 64 MiB (declared size + streamed read), and map a
panicked encode worker to the existing degrade path instead of
expect()-panicking the whole fetch handler.
bsky video embeds expose only an m3u8 playlist, which Telegram cannot
fetch. Download the master/variant playlists and TS segments through the
shared client (proxy-aware, size-capped), then concat-remux locally;
keep the temp dir alive via Fetched._keep_alive like the ugoira path.
Also adds site::download_media_limited (streaming size cap), status
checks on media_size, and moves the ffmpeg probe to site/mod.rs for
pixiv/bsky to share.
Apply rustfmt to the 11 files that had drifted (86 hunks): x-media
site modules (bsky/pixiv/twitter) and xmedia-bot (config/main/
photo/send). Formatting only - no semantic changes; full test suite
still green.
Replace display_text_range slicing with FxEmbed-style content matching:
expand mapped t.co links to their real URLs (dropping internal
x.com/i/web/status pages), then strip every leftover t.co short link
(appended media link, unmapped links).
The old code cut by display_text_range, whose index unit differs per
endpoint (UTF-16 on the syndication endpoint, code points in the
GraphQL fallback), so slicing by either unit left a partial
"https://t." caption tail on the other path. Content matching is
unit-agnostic and also keeps user-posted/quote links at the end of
the text that the trailing cut previously dropped.
YoursFunny