fix(retry): let slow downloads finish, and never re-run a finished task

P1 of the retry audit, from the report's "reliability and diagnosis" batch.

- Media downloads no longer share the 30s *total* timeout of metadata
  fetches. The size caps allowed 10 MiB (reupload fallback) and 512 MiB
  (ugoira frame zip) while the clock allowed 30s, so a slow link made those
  posts impossible: `MEDIA_CLIENT` has no total timeout and instead bounds
  the response head and every chunk with a 30s *idle* window, which keeps the
  stalled-connection protection. Verified against a local probe: the old
  policy aborts a 40s download at 30.0s, the new one completes it (2 MiB,
  40.1s), and a body that stops delivering still fails after exactly 30s.
- A finished row's write-back is no longer best-effort. `delete_row` failing
  left the row `in_progress` with a live lease, so the next sweep flipped it
  back to `pending` and re-ran a completed task — a second album, a second
  prompt, a second channel copy. Both terminal writes are now retried, and a
  delete that still fails falls back to a `done` tombstone that neither the
  lease query nor the sweep looks at; reschedule (no safe tombstone: marking
  it done would drop the retry silently) logs what the sweep will do.
- bsky and pixiv no longer present a *failed* video conversion as a post with
  no media: the remux/ugoira error propagates (pixiv keeps its retry class,
  bsky reports Transient), so the user sees the real cause and `fetch` gets
  its retries. bsky's "no ffmpeg" case stays a degradation — retrying a
  deployment gap cannot help.
- pixiv's token exchange checks the HTTP status before parsing the body, so a
  429/5xx from the OAuth endpoint stays retryable instead of becoming a
  permanent Api/Json error (via the shared `pixiv_error_is_retryable`), and
  startup validation only disables pixiv for a rejected credential — one 503
  while the container came up used to turn every later pixiv link into
  "pixiv support is disabled".
This commit is contained in:
2026-09-20 21:02:46 +08:00
parent 4cf793cd7e
commit 0a82ca5a42
6 changed files with 269 additions and 53 deletions
+32 -10
View File
@@ -50,8 +50,14 @@ impl Site for PixivSite {
match super::api::validate().await {
Ok(()) => Ok(()),
Err(e) => {
// Keep the old behavior: a failed login disables pixiv
// for the rest of this process.
// A rejected credential disables pixiv for the rest of
// this process (it will not fix itself). A bad *moment* —
// a 5xx or a network error while the container comes up —
// must not: disabling on any error turned every later
// pixiv link into "pixiv support is disabled".
if pixiv_error_is_retryable(&e) {
return Err(format!("{e} (transient — pixiv stays enabled)"));
}
super::api::disable();
Err(format!("{e}"))
}
@@ -84,18 +90,24 @@ pub fn cache_key(url: &str) -> Option<String> {
pub fn is_retryable(err: &FetchError) -> bool {
match err {
FetchError::Http(_) | FetchError::Transient(_) => true,
FetchError::Pixiv(e) => match e {
PixivError::Http(_) => true,
PixivError::Status(code) if *code == 429 || *code >= 500 => true,
PixivError::Status(_)
| PixivError::Api(_)
| PixivError::Json(_)
| PixivError::NoAuth => false,
},
FetchError::Pixiv(e) => pixiv_error_is_retryable(e),
_ => false,
}
}
/// The pixiv-specific half of the retry policy, shared with startup
/// validation: a bad moment (429/5xx, a network error) is retryable, a
/// rejected credential is not.
fn pixiv_error_is_retryable(err: &PixivError) -> bool {
match err {
PixivError::Http(_) => true,
PixivError::Status(code) if *code == 429 || *code >= 500 => true,
PixivError::Status(_) | PixivError::Api(_) | PixivError::Json(_) | PixivError::NoAuth => {
false
}
}
}
/// pximg.net is hotlink-protected: downloads must carry the pixiv Referer.
/// The match is on the media host, not the site PATTERN — pixiv's PATTERN
/// only matches `pixiv.net/artworks/...`, never `i.pximg.net`.
@@ -410,6 +422,16 @@ mod tests {
}
}
#[test]
fn startup_validation_only_disables_on_a_definitive_failure() {
// A bad moment: the site must stay enabled for later links.
assert!(pixiv_error_is_retryable(&PixivError::Status(503)));
assert!(pixiv_error_is_retryable(&PixivError::Status(429)));
// A rejected credential is what `disable()` is for.
assert!(!pixiv_error_is_retryable(&PixivError::Status(403)));
assert!(!pixiv_error_is_retryable(&PixivError::NoAuth));
}
#[test]
fn is_retryable_classifies_transient_and_permanent() {
// Transient: network errors, explicit transient, pixiv 429/5xx.