mirror of
https://github.com/TheFunny/TelegramTwitterMediaBot.git
synced 2026-10-06 01:22:16 +00:00
fix(retry): stop losing posts to transient failures and broken promises
P0 of the retry audit. The main finding: a Telegram 5xx was classified
Permanent, so one Telegram-side blip dead-lettered the post.
- `classify_request_error`: a server error is retryable again. teloxide sleeps
10s on a 5xx and then parses the body, so the HTTP status is gone by the
time the error arrives; it is recognised by shape instead — a JSON
server-error description, or an `InvalidJson` whose raw body is not JSON
(a proxy/error page). A JSON body of the wrong shape stays permanent, since
retrying a type mismatch cannot help. Reproduced end to end: with the old
classification a fake 502 (HTML body) logged "failed permanently" and
dead-lettered; now it logs "queued for retry" and the retry delivers.
- The same class of mistake elsewhere: `is_media_fetch_failure` was missing
`failed to get HTTP url content`, the description single-media URL sends
answer with, so hotlink-rejected media failed permanently instead of going
through the reupload fallback.
- `enqueue_retry` now reports whether the row was written, and the callers
only promise a retry when it was — a failed enqueue (DB write) used to tell
the user "retrying in Ns" and then deliver nothing, ever.
- A forward that fails retryably now settles the prompt instead of leaving it
live: the queued row carries the message ids itself, and a live prompt let
a second Confirm copy the same messages to the channel twice and let Skip
answer "nothing was forwarded" while the row still delivered.
- A prompt that could not be sent no longer swallows the gated forward
silently: the chat is told, since nothing would ever forward.
- `scaled_retry_delay` only scales up, so a server-asked `retry_after` above
the 300s cap is honoured instead of retried early (which earned another 429
and then dead-lettered the post).
- Download classification: a 4xx media download is permanent (the media is
gone or refused) while transport errors and 429/5xx retry — previously every
download error counted as retryable and burned the whole budget. A temp-file
*write* failure retries too (resource exhaustion clears; a temp dir that
cannot be created stays permanent).
- Site status mapping: 401/403 are `Blocked` (permanent) rather than
`Transient`, so a refusal is reported at once instead of after three
wasted attempts; and a twitter 200 that is not a tweet is no longer
reported as withheld content (the empty `{}` withheld shape keeps
`Sensitive`, which is what triggers the auth fallback).
This commit is contained in:
@@ -71,19 +71,7 @@ async fn download_to_temp(
|
||||
};
|
||||
let bytes = match x_media::site::download_media_limited(media_url, limit).await {
|
||||
Ok(bytes) => bytes,
|
||||
Err(FetchError::Http(_)) => {
|
||||
return Err(FallbackError::Retryable {
|
||||
delay_seconds: retry_delay_seconds(0),
|
||||
});
|
||||
}
|
||||
Err(FetchError::TooLarge) => {
|
||||
return Err(FallbackError::MediaTooLarge);
|
||||
}
|
||||
Err(e) => {
|
||||
return Err(FallbackError::Permanent {
|
||||
message: format!("download failed: {e}"),
|
||||
});
|
||||
}
|
||||
Err(e) => return Err(classify_download_error(e)),
|
||||
};
|
||||
let ext = sniff_ext(&bytes);
|
||||
let mut file = tempfile::Builder::new()
|
||||
@@ -93,14 +81,37 @@ async fn download_to_temp(
|
||||
message: format!("temp file failed: {e}"),
|
||||
})?;
|
||||
use std::io::Write;
|
||||
file.as_file_mut()
|
||||
.write_all(&bytes)
|
||||
.map_err(|e| FallbackError::Permanent {
|
||||
message: format!("temp file write failed: {e}"),
|
||||
})?;
|
||||
// A write failure is resource exhaustion far more often than a broken temp
|
||||
// dir (ENOSPC / EDQUOT), and that clears on its own — worth an attempt
|
||||
// instead of dropping the post on the first try. Creating the file (above)
|
||||
// stays permanent: a temp dir that cannot be created at all is a
|
||||
// deployment fault that should fail loudly and immediately. `Retryable`
|
||||
// carries no message, so the cause is logged here.
|
||||
file.as_file_mut().write_all(&bytes).map_err(|e| {
|
||||
log::error!("temp file write failed: {e}");
|
||||
FallbackError::Retryable {
|
||||
delay_seconds: retry_delay_seconds(0),
|
||||
}
|
||||
})?;
|
||||
Ok((file, bytes))
|
||||
}
|
||||
|
||||
/// Which failure class a media download belongs to. Transport errors and
|
||||
/// server-side hiccups (429/5xx, see `download_media_limited`) are worth
|
||||
/// another attempt; a 4xx means the media itself is gone or refused, and a
|
||||
/// retry could only ask the same URL again.
|
||||
fn classify_download_error(err: FetchError) -> FallbackError {
|
||||
match err {
|
||||
FetchError::Http(_) | FetchError::Transient(_) => FallbackError::Retryable {
|
||||
delay_seconds: retry_delay_seconds(0),
|
||||
},
|
||||
FetchError::TooLarge => FallbackError::MediaTooLarge,
|
||||
e => FallbackError::Permanent {
|
||||
message: format!("download failed: {e}"),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/// Builds the media group item from an uploaded file.
|
||||
fn media_from_file(
|
||||
item: &MediaItemPayload,
|
||||
@@ -343,3 +354,31 @@ pub(super) async fn send_batch_via_upload(
|
||||
Err(e) => Err(classify_to_send_error(&e, task, "upload failed")),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod download_class_tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn download_errors_split_by_whether_a_retry_can_help() {
|
||||
// Transport failure and a server-side hiccup: try again.
|
||||
assert!(matches!(
|
||||
classify_download_error(FetchError::Transient("media status 503".into())),
|
||||
FallbackError::Retryable { .. }
|
||||
));
|
||||
// The media is gone / the host refuses us: a retry repeats the 4xx.
|
||||
assert!(matches!(
|
||||
classify_download_error(FetchError::NotFound),
|
||||
FallbackError::Permanent { .. }
|
||||
));
|
||||
assert!(matches!(
|
||||
classify_download_error(FetchError::Blocked),
|
||||
FallbackError::Permanent { .. }
|
||||
));
|
||||
// Over the cap: degrade to the smaller URL, never retry.
|
||||
assert!(matches!(
|
||||
classify_download_error(FetchError::TooLarge),
|
||||
FallbackError::MediaTooLarge
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user