Exact-string dedup let https://x.com/u/status/1 and
https://x.com/u/status/1/photo/1 (or the same link in text and caption)
through twice, causing duplicate fetches and sends. Dedup now uses
cache_key, falling back to the raw URL for unsupported links.
prune_expired only shrank edit_message maps, so the cache kept one
ChatData per chat forever (a leak proportional to chat count). Chats
without live edit records are now dropped from the cache and their
per-chat lock (DB row persists; get() reloads). Lock order kept safe:
prune never holds the cache lock while taking the per-chat locks.
The 8-permit semaphore was acquired inside the spawned task, so a burst
queued unlimited tasks (each cloning Bot+Message) and nothing tracked
them at shutdown — in-flight sends fired after the stop notice. URL work
now flows through a 256-slot mpsc drained by 8 workers started from
main; a full channel backpressures the per-chat handler, and shutdown
sets URL_STOP so workers stop pulling.
Concurrent handler tasks (the batch-forward design spawns several per
chat) each snapshotted the same ChatData and last-writer-wins silently
dropped mutations — e.g. a second edit_message record, leaving one
prompt's Forward button dead. All write cycles now run under a per-chat
async lock; read-only callers keep get().
A retried ugoira/bsky temp MP4 was already deleted with its TempDir, so
the retry failed at multipart-build time with a confusing Io error and
wasted all three attempts. input_file_for now rejects a missing local
path up front as a clean permanent error.
A resumed SendMediaSequence (batch_index>0 or already-sent ids) ran
post_send_actions again, opening a second edit prompt and inserting a
second edit_message record for the same messages — both Forward buttons
worked, enabling double forwards. Resumes now skip it.
A lease error (e.g. persistent SQLITE_BUSY) while rows are due made the
worker spin with sleep(0), hammering SQLite and flooding the log.
lease_next now returns the error and the loop sleeps 1s before retrying.
notify_waiters drops the notification when every worker is between its
DB reads and registering notified(); a task enqueued in that window sat
until a stale timer fired. notify_one stores a permit, so the next
worker to wait wakes immediately and re-leases. stop() still wakes all
workers with notify_waiters.
Every network retry hard-coded retry_delay_seconds(0), so backoff was
flat at 1.2-1.8s regardless of attempt; a multi-minute outage dead-
lettered after three rapid tries. The queue now scales the handler's
delay by 2^attempts (cap 300s) before rescheduling.
The shared HTTP client ignored the proxy the Bot API uses, so on
proxy-required networks (e.g. behind the GFW) every site fetch failed
while the bot itself worked. Explicit proxy overrides reqwest's system
detection; unset keeps direct connections.
Both matched URL substrings anywhere in text, so a link like
https://evil.com/?u=pixiv.net/artworks/1 triggered a real fetch and
cache-key pollution. Prefix ^(?:https?://)? like the twitter pattern.
Twitter (syndication + auth GraphQL) mapped every non-2xx to NotFound,
killing retries on 429/5xx; bsky never checked status; pixiv network
errors arrived wrapped in PixivError and were excluded from the retry
loop. New FetchError::Transient covers 429/5xx from all sites, the
retry loop now also retries Pixiv errors, and 404/410 stay permanent.
download_to_temp now uses download_media_limited: non-photos abort the
moment the 10 MiB upload cap is crossed mid-stream (no more full-body
buffering before the size check), photos cap at the 512 MiB decode
budget, and the ugoira frame zip gets a 512 MiB cap. MediaTooLarge
routes to the existing smaller-URL fallback.
Sniff the frame extension from magic bytes instead of the entry filename,
cap each frame at 64 MiB (declared size + streamed read), and map a
panicked encode worker to the existing degrade path instead of
expect()-panicking the whole fetch handler.
Compare the sender's user id (not the chat id, which only matches in
private chats) and require the bot to actually be an admin with post
rights instead of silently passing when it is missing from the list.
Also stops panicking on get_me network failures.
Rows left in_progress by a panicked/crashed worker were only recovered at
start(); a runtime sweep (30s interval, woken by the same notify) now
re-queues them once the 120s lock TTL expires. Workers run under a
supervisor that respawns a panicked loop instead of silently shrinking
the pool of 4.
bsky video embeds expose only an m3u8 playlist, which Telegram cannot
fetch. Download the master/variant playlists and TS segments through the
shared client (proxy-aware, size-capped), then concat-remux locally;
keep the temp dir alive via Fetched._keep_alive like the ugoira path.
Also adds site::download_media_limited (streaming size cap), status
checks on media_size, and moves the ffmpeg probe to site/mod.rs for
pixiv/bsky to share.
Call setMyCommands at startup so clients show the command list in the
/ menu. handlers::register_commands wraps Command::bot_commands()
(teloxide derives it from the #[command(description)] attributes);
a registration failure only warns and does not stop the bot.
Apply rustfmt to the 11 files that had drifted (86 hunks): x-media
site modules (bsky/pixiv/twitter) and xmedia-bot (config/main/
photo/send). Formatting only - no semantic changes; full test suite
still green.
/clear_cache with no argument wipes the whole link_cache table;
with a post URL it removes that single entry (normalized via
site::cache_key so fxtwitter/mobile/photo variants collide with
the write-side key). Non-admins get 'Admin only.'. LinkCache gains
clear(Option<&str>) -> usize reporting removed rows.
Converge the duplicated open_db (open + busy_timeout) and the
spawn_blocking + expect ceremony that every table access repeated
into one db.rs module. ChatStore no longer creates the tasks table
(schema ownership: queue.rs owns tasks, state.rs chat_state,
link_cache.rs link_cache). No schema or behavior change - all
CREATE TABLE statements are byte-identical, IF NOT EXISTS stays
idempotent, so existing data/task_queue.db files need no migration.
Replace display_text_range slicing with FxEmbed-style content matching:
expand mapped t.co links to their real URLs (dropping internal
x.com/i/web/status pages), then strip every leftover t.co short link
(appended media link, unmapped links).
The old code cut by display_text_range, whose index unit differs per
endpoint (UTF-16 on the syndication endpoint, code points in the
GraphQL fallback), so slicing by either unit left a partial
"https://t." caption tail on the other path. Content matching is
unit-agnostic and also keeps user-posted/quote links at the end of
the text that the trailing cut previously dropped.
All bot commands now carry English descriptions, shown in the Telegram
command menu and by /help (which prints Command::descriptions()). The
README command table explains each command's arguments and behavior:
forward channel (@channel or ID), edit-before-forward flow, template
[] placeholder semantics and per-site caption format placeholders.
certs, html and acme are shared across nginx-proxy, acme-companion and
the acme-ip service, so they become named volumes (dropping the nginx-
prefix). The vhost.d mount is removed: acme-companion only needs it with
ACME_HTTP_CHALLENGE_LOCATION=true or letsencrypt_user_data, and users
who customize nginx config can mount their own. ./data stays a bind
mount so the SQLite state stays directly backup-able. README updated.
compose files now carry a single comment: the commented ACME_HOST line
(uncomment to enable domain cert issuance). Everything else moved to the
README env table and the deployment sections, which explain ACME_HOST,
DEFAULT_HOST and every other variable.
Domain deployment now uses ACME_HOST (acme-companion auto issue/renew)
instead of the legacy LETSENCRYPT_HOST name. IP-only deployments can use
acme.sh to obtain Let's Encrypt IP certificates (shortlived profile,
~7-day validity, http-01 only) instead of self-signed manual certs:
documented the acme-ip service, first-issue command with install-cert
into nginx-certs and nginx reload via docker socket.
Removed the manual WEBHOOK_CERT flow and the ./cert volume from the
compose example.
acme-companion prefers com.github.nginx-proxy.nginx over the legacy
jrcs.letsencrypt_nginx_proxy_companion namespace (functions.sh checks
the new label first, falls back to the old one).
docker compose restart reuses the container, so the overlay fs keeps the
user created on first boot; a second useradd fails with exit code 9 and
set -e kills the container on every restart (crash loop, bot never comes
back). Create the user only if missing, align UID otherwise, and never
let chown failure on bind-mounted volumes kill the container.
Also comment out nginx-proxy's empty environment block in the compose
example: an empty mapping fails validation on newer compose versions
(must be a mapping).
YoursFunny