TelegramTwitterMediaBot/README.en.md

TelegramXMediaBot

A Telegram bot that turns post links from X / Twitter, Pixiv, Bluesky, Misskey (misskey.io), and Bilibili dynamics into media messages (images, video, GIF) with the post's title, author, and tags.

Features

• Sending a link in a private chat fetches and sends the images, videos and GIFs automatically; oversized media is split into batches (10 items per group)
• Text-only posts report "no media"; unsupported links are silently ignored. Fetch failures name the reason (post gone / content withheld / source risk control / site not enabled)
• Long posts (text ≥ CAPTION_QUOTE_TEXT_CHARS, default 200) show the text part of their caption inside a collapsible blockquote, with the link and author line left outside it
• Inline queries (@bot <link>) — except Pixiv images and locally transcoded animations, which Telegram cannot fetch (no Referer) and would show broken, so they are skipped; a supported link posted in a group gets a one-line hint to use the private chat or inline mode (channels stay silent)
• /start explains the supported sites and how to use it; /help lists the commands plus argument syntax, the caption placeholders and the private-chat rule; the bot's profile description texts are set at startup
• /settings shows this chat's configuration (forward channel, edit-before-forward, per-site caption formats, saved templates); templates are added with /set_template and removed with /remove_template
• Bind a forward channel for automatic forwarding; edit the caption before forwarding and apply custom templates (the prompt carries Confirm / Skip buttons, states its expiry, and is marked expired in place once it lapses)
• Failed sends are retried automatically with persistence; the notice names which link failed, how long the retry waits, or the final cause
• The chat action stays on screen for the whole fetch, so long jobs (ugoira transcode, large uploads) do not look stalled
• Pixiv ugoira animations are transcoded to MP4; Bluesky videos are remuxed (HLS stream → MP4)
• Photos exceeding Telegram's size/dimension limits are compressed automatically (original format kept, JPEG fallback only when needed)
• Link-result cache: after a successful send the Telegram file ids and caption fields are cached locally, so a repeated link is re-sent from local state — no source-site request, no media file stored (expiry controlled by LINK_CACHE_TTL_SECONDS, default 7 days)

Quick start

# Required: BotFather token; optional: PIXIV_REFRESH_TOKEN (Pixiv is disabled without it)
export TELOXIDE_TOKEN=<token>
export PIXIV_REFRESH_TOKEN=<token>

cargo run -p xmedia-bot

Docker deployment (docker-compose.yml in this repo is the orchestration; instance values live in the .env next to it, and compose substitutes every ${VAR} from there):

docker build -t tgxmb .
docker run --rm -d --name tgxmb --env-file .env -v ./data:/app/data tgxmb

Environment variables: TELOXIDE_TOKEN (required), PIXIV_REFRESH_TOKEN, BOT_ADMIN, EDIT_MESSAGE_TTL_SECONDS, LINK_CACHE_TTL_SECONDS, RUST_LOG, TELOXIDE_PROXY, WEBHOOK*, TWITTER_AUTH_TOKEN (optional), BILIBILI_COOKIE (optional).

NSFW tweets: the public syndication endpoint does not return sensitive content. Setting TWITTER_AUTH_TOKEN (the auth_token cookie value of a logged-in x.com session) lets the bot fetch NSFW media in the logged-in state only when it hits a withheld tweet; without it the bot answers that the post's media is withheld and needs TWITTER_AUTH_TOKEN.

Bilibili dynamics are fetched anonymously by default (no login; the bot fetches bilibili's anonymous buvid3/buvid4 device cookies itself to raise the success rate). If the server's egress IP gets hard-flagged by bilibili (persistent risk control (-352) log lines or HTTP 412), set BILIBILI_COOKIE (the whole cookie string from a logged-in browser, e.g. SESSDATA=…; bili_jct=…) to restore access. Only a dynamic's images and animations are sent; an attached video degrades to its cover image.

Webhook deployment (needs a reverse proxy)

docker-compose.yml ships an nginx-proxy + acme-companion reverse-proxy orchestration. The committed file needs no editing: domain, tokens and admins are instance values and live in the .env beside it (compose reads and substitutes ${VAR} at startup). Pick one deployment shape:

With a domain

1. Point a DNS A record at the server
2. In .env set VIRTUAL_HOST and WEBHOOK_URL to the domain; to have acme-companion issue the certificate, also uncomment the ACME_HOST line in docker-compose.yml and set ACME_HOST in .env
3. acme-companion issues and renews certificates automatically — nothing manual

IP only Let's Encrypt can issue certificates for public IPs (available since 2026, validity ~7 days, requires the shortlived profile). Use acme.sh to issue and renew automatically, no manual certificates:

1. Add an acme-ip service to compose (issue + daily auto-renewal check):

   acme-ip:
     image: neilpang/acme.sh
     container_name: acme-ip
     command: daemon
     restart: always
     volumes:
       - certs:/acme.sh
       - html:/usr/share/nginx/html
       - /var/run/docker.sock:/var/run/docker.sock:ro
     networks: [proxy]
   

2. First issuance (replace <SERVER_IP> with the server's public IP; IPv6 works too, repeat -d for more):

   docker compose exec acme-ip acme.sh --issue --server letsencrypt \
     -d <SERVER_IP> --cert-profile shortlived --days 3 \
     --webroot /usr/share/nginx/html \
     --install-cert --cert-file /acme.sh/<SERVER_IP>.crt \
     --key-file /acme.sh/<SERVER_IP>.key \
     --reloadcmd "curl --unix-socket /var/run/docker.sock -X POST http://localhost/containers/nginx-proxy/kill?signal=HUP"
   

3. In .env set VIRTUAL_HOST=<SERVER_IP> and WEBHOOK_URL=https://<SERVER_IP>/; no WEBHOOK_CERT needed. Renewal is handled by the acme.sh daemon (--days 3 = renew every 3 days, buffer against the 7-day validity), and a successful renewal HUP-notifies nginx-proxy to load the new certificate.

   Limitations: certificate validity ~7 days; only http-01/tls-alpn-01 validation (port 80 must be publicly reachable); no DNS-01, private IPs or IP ranges; at most 5 certificates per 168 hours for the same IP set. It is recommended to trial-issue with --server letsencrypt_test first, then switch to the production server.

Telegram only accepts ports 443/80/88/8443.

Environment variables

Variable	Description
TELOXIDE_TOKEN	Bot token (required)
PIXIV_REFRESH_TOKEN	Pixiv refresh token; Pixiv is disabled without it (a pixiv link then gets an explicit "site not enabled" reply instead of silence)
BILIBILI_COOKIE	Optional bilibili cookie string (SESSDATA=…; bili_jct=…); only needed when the egress IP stays risk-controlled (device cookies are fetched automatically)
BOT_ADMIN	Admin chat IDs, comma-separated; receives start/stop notifications
EDIT_MESSAGE_TTL_SECONDS	Edit-before-forward record expiry in seconds, default 86400; once lapsed the prompt is rewritten in place to "expired — nothing was forwarded" (no extra message)
LINK_CACHE_TTL_SECONDS	Link-result cache expiry in seconds, default 604800 (7 days)
CAPTION_QUOTE_TEXT_CHARS	The text part of the caption (the joined {title} + {content}) is wrapped in a collapsible blockquote once it reaches this many characters, default 200; 0 disables
DATA_DIR	Data directory (where the SQLite task_queue.db lives), default data (relative to the working directory, created automatically)
RUST_LOG	Log level, default info,hyper_util=warn,reqwest=warn (an unset variable no longer silences the log). Recipes: info,xmedia_bot=debug,x_media=debug (app detail, no dependency noise) / debug,hyper_util=off (everything) / trace (also prints full links and message text — user data)
TELOXIDE_PROXY	HTTP proxy (e.g. http://127.0.0.1:10808); applies to both the Telegram Bot API and site fetches — required on restricted networks (e.g. behind the GFW). Never leave it blank (TELOXIDE_PROXY=) — teloxide panics on a value it cannot parse; omit the line when unused. docker-compose.yml deliberately does not pass it to the container (a 127.0.0.1 proxy there is the container itself): add the line and use host.docker.internal:<port> when a deployment needs one
LOCAL_USER_ID	UID the container runs as, default 9001
VIRTUAL_HOST	Public domain or IP; nginx-proxy routes by this (set it in .env, which compose reads)
VIRTUAL_PORT	Port the bot listens on inside the container; nginx-proxy's forwarding target
ACME_HOST	Domain deployment: when set to the domain, acme-companion issues/renews certificates automatically
DEFAULT_HOST	nginx-proxy routes requests with unknown Host headers to this vhost (needed for IP access)
DEFAULT_EMAIL	acme-companion certificate notification email
WEBHOOK	true enables webhook mode (polling by default)
WEBHOOK_LISTEN / WEBHOOK_PORT	Listen address/port inside the bot container
WEBHOOK_URL	Public HTTPS URL (https://domain/ or https://IP/)
WEBHOOK_CERT	Optional; self-signed certificate path, only used for Telegram-side validation (TLS is terminated by the reverse proxy)
WEBHOOK_SECRET_TOKEN	Update validation token (X-Telegram-Bot-Api-Secret-Token)

Commands

Command	Description
/start	Welcome message
/help	List all commands and usage (this command table)
/set_forward_channel <channel>	Set the forward channel: @channel or channel ID; media messages are forwarded to it automatically afterwards
/remove_forward_channel	Remove the forward channel
/edit_before_forward	Toggle "edit before forward": when enabled, the bot posts a prompt after forwarding; replying to it edits the first forwarded message's caption (or tapping a template button applies one), then ↩️ Confirm forwards and 🛑 Skip drops this forward; the prompt states its expiry and is marked expired in place when it lapses (nothing is forwarded)
/set_template <name>	Reply to a message containing [] to save it as a named template; [] is replaced by the original post link when forwarding (used with "edit before forward")
/remove_template <name>	Remove a template (names are listed by /settings; the prompt's keyboard shows at most 60)
/settings	Show this chat's configuration: forward channel, edit-before-forward, per-site caption formats, saved templates
/set_format <site> <format>	Customize the caption format for one site. Sites: twitter / bsky / pixiv / misskey / bilibili. Placeholders: {url} {author} {author_url} {title} {content} {tags}; unknown placeholders are rejected with the list of valid ones, and - restores the site's built-in format (preview with /debug <link>)
/clear_cache [link]	Clear the link cache (admin only); with a link only that entry, otherwise everything
/bot_dict	Show the current chat state (debugging; admin only)
/test <link>	Parse a link and send its media; no channel forward, no edit-before-forward prompt (send only)
/debug <link>	Debug: parse a link and report the parse result only (site, title, author, tags, media list) — no media is sent

Link processing works only in private chats; commands work in any chat. A supported link posted in a group gets a one-line hint to use the private chat or inline mode; channels stay silent.

Notes

• State is persisted in data/task_queue.db; compose deployments use the bind mount ./data (keep it a directory for easy backups)
• The runtime needs ffmpeg (built into the Docker image)
• Tests: cargo test --workspace