mirror of
https://github.com/TheFunny/TelegramTwitterMediaBot.git
synced 2026-10-06 01:22:16 +00:00
fix(retry): let slow downloads finish, and never re-run a finished task
P1 of the retry audit, from the report's "reliability and diagnosis" batch. - Media downloads no longer share the 30s *total* timeout of metadata fetches. The size caps allowed 10 MiB (reupload fallback) and 512 MiB (ugoira frame zip) while the clock allowed 30s, so a slow link made those posts impossible: `MEDIA_CLIENT` has no total timeout and instead bounds the response head and every chunk with a 30s *idle* window, which keeps the stalled-connection protection. Verified against a local probe: the old policy aborts a 40s download at 30.0s, the new one completes it (2 MiB, 40.1s), and a body that stops delivering still fails after exactly 30s. - A finished row's write-back is no longer best-effort. `delete_row` failing left the row `in_progress` with a live lease, so the next sweep flipped it back to `pending` and re-ran a completed task — a second album, a second prompt, a second channel copy. Both terminal writes are now retried, and a delete that still fails falls back to a `done` tombstone that neither the lease query nor the sweep looks at; reschedule (no safe tombstone: marking it done would drop the retry silently) logs what the sweep will do. - bsky and pixiv no longer present a *failed* video conversion as a post with no media: the remux/ugoira error propagates (pixiv keeps its retry class, bsky reports Transient), so the user sees the real cause and `fetch` gets its retries. bsky's "no ffmpeg" case stays a degradation — retrying a deployment gap cannot help. - pixiv's token exchange checks the HTTP status before parsing the body, so a 429/5xx from the OAuth endpoint stays retryable instead of becoming a permanent Api/Json error (via the shared `pixiv_error_is_retryable`), and startup validation only disables pixiv for a rejected credential — one 503 while the container came up used to turn every later pixiv link into "pixiv support is disabled".
This commit is contained in:
@@ -55,6 +55,10 @@ pub async fn fetch_from_url(url: &str) -> Result<Fetched, FetchError> {
|
||||
// working on: the media URL is derived from what the user pasted, and
|
||||
// `warn` is a level operators share.
|
||||
let key = cache_key(url).unwrap_or_else(|| "?".into());
|
||||
// A failed remux is remembered: if it leaves the post with no media at
|
||||
// all, returning `Ok` would read as "this post has no media" and skip the
|
||||
// retry that a transient segment-download failure deserves.
|
||||
let mut remux_failure: Option<String> = None;
|
||||
for item in fetched.media {
|
||||
let is_hls = matches!(&item, Media::Video { url, .. }
|
||||
if url.contains("playlist") || url.ends_with(".m3u8"));
|
||||
@@ -76,10 +80,23 @@ pub async fn fetch_from_url(url: &str) -> Result<Fetched, FetchError> {
|
||||
});
|
||||
fetched._keep_alive = Some(keep_alive);
|
||||
}
|
||||
// No ffmpeg: a deployment gap, not a bad moment — retrying it
|
||||
// would only waste the fetch budget, so the post degrades (and an
|
||||
// all-video post reports the media type as unsupported).
|
||||
Ok(None) => log::warn!("bsky video remux unavailable for [key={key}]"),
|
||||
Err(e) => log::warn!("bsky video remux failed for [key={key}]: {e}"),
|
||||
Err(e) => {
|
||||
log::warn!("bsky video remux failed for [key={key}]: {e}");
|
||||
remux_failure = Some(e);
|
||||
}
|
||||
}
|
||||
}
|
||||
if media.is_empty()
|
||||
&& let Some(reason) = remux_failure
|
||||
{
|
||||
return Err(FetchError::Transient(format!(
|
||||
"bsky video remux failed: {reason}"
|
||||
)));
|
||||
}
|
||||
fetched.media = media;
|
||||
Ok(fetched)
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user