mirror of
https://github.com/TheFunny/TelegramTwitterMediaBot.git
synced 2026-10-02 00:42:10 +00:00
fix(retry): let slow downloads finish, and never re-run a finished task
P1 of the retry audit, from the report's "reliability and diagnosis" batch. - Media downloads no longer share the 30s *total* timeout of metadata fetches. The size caps allowed 10 MiB (reupload fallback) and 512 MiB (ugoira frame zip) while the clock allowed 30s, so a slow link made those posts impossible: `MEDIA_CLIENT` has no total timeout and instead bounds the response head and every chunk with a 30s *idle* window, which keeps the stalled-connection protection. Verified against a local probe: the old policy aborts a 40s download at 30.0s, the new one completes it (2 MiB, 40.1s), and a body that stops delivering still fails after exactly 30s. - A finished row's write-back is no longer best-effort. `delete_row` failing left the row `in_progress` with a live lease, so the next sweep flipped it back to `pending` and re-ran a completed task — a second album, a second prompt, a second channel copy. Both terminal writes are now retried, and a delete that still fails falls back to a `done` tombstone that neither the lease query nor the sweep looks at; reschedule (no safe tombstone: marking it done would drop the retry silently) logs what the sweep will do. - bsky and pixiv no longer present a *failed* video conversion as a post with no media: the remux/ugoira error propagates (pixiv keeps its retry class, bsky reports Transient), so the user sees the real cause and `fetch` gets its retries. bsky's "no ffmpeg" case stays a degradation — retrying a deployment gap cannot help. - pixiv's token exchange checks the HTTP status before parsing the body, so a 429/5xx from the OAuth endpoint stays retryable instead of becoming a permanent Api/Json error (via the shared `pixiv_error_is_retryable`), and startup validation only disables pixiv for a rejected credential — one 503 while the container came up used to turn every later pixiv link into "pixiv support is disabled".
This commit is contained in:
@@ -76,6 +76,13 @@ impl PixivAPI {
|
||||
.header("User-Agent", AUTH_USER_AGENT)
|
||||
.send()
|
||||
.await?;
|
||||
// Check the status *before* reading the body: a 429/5xx from the
|
||||
// token endpoint is worth retrying (the class comes from
|
||||
// `is_retryable`), while parsing a maintenance page as JSON turned it
|
||||
// into a permanent `Api`/`Json` error with no retry at all.
|
||||
if !response.status().is_success() {
|
||||
return Err(PixivError::Status(response.status().as_u16()));
|
||||
}
|
||||
let json: serde_json::Value = serde_json::from_str(&response.text().await?)?;
|
||||
let access_token = json
|
||||
.get("access_token")
|
||||
@@ -134,8 +141,11 @@ impl PixivAPI {
|
||||
let mut illustration = Illustration::from_model(&model);
|
||||
if matches!(&model.r#type, TypeModel::Ugoira) {
|
||||
// Real ugoira support: download the frame zip and encode an MP4.
|
||||
// Without ffmpeg (or on encode failure) the post stays
|
||||
// unsupported (empty media, like Python).
|
||||
// Without ffmpeg the post stays unsupported (empty media, like
|
||||
// Python) — but a *failed* download/encode is reported instead:
|
||||
// a ugoira post has no static image to fall back to, so
|
||||
// swallowing it would present a transient zip-download error as
|
||||
// "this post has no media", with the retries skipped.
|
||||
match self.ugoira_video(illust_id).await {
|
||||
Ok(Some((mp4_path, _keep_alive))) => {
|
||||
illustration.media.push(Media::Video {
|
||||
@@ -146,7 +156,10 @@ impl PixivAPI {
|
||||
illustration._keep_alive = Some(_keep_alive);
|
||||
}
|
||||
Ok(None) => {}
|
||||
Err(e) => log::error!("ugoira encode failed for {illust_id}: {e}"),
|
||||
Err(e) => {
|
||||
log::error!("ugoira encode failed for {illust_id}: {e}");
|
||||
return Err(FetchError::Pixiv(e));
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(illustration)
|
||||
|
||||
Reference in New Issue
Block a user