mirror of
https://github.com/TheFunny/TelegramTwitterMediaBot.git
synced 2026-09-23 23:32:05 +00:00
fix(retry): fence the queue lease, clean up after a kill, name dead-lettered posts
P2 (hardening) of the retry audit, closing the report's remaining findings. - Lease fencing. `lease_next` now stamps a random `lease_token`, and every write-back a worker makes (the 30s heartbeat, `delete_row`, `reschedule`, `mark_done`) is guarded by it. A lease that expired while its holder was stalled and was then re-leased used to let *both* holders write the same row: one duplicated the send, the other silently discarded the new holder's retry (a 0-row update was not even logged). Now a worker that no longer holds the lease drops its attempt at the next heartbeat and writes nothing. Reaching existing databases needed a migration chain, which `db.rs` had been pre-committed to: `MIGRATIONS` + `migrate` track `PRAGMA user_version`, with `schema_init` as the version-0 baseline. Verified on a database created before this change: user_version 0 -> 1, column added, rows intact. - Dead-letter notifications no longer mislabel an unparsable payload. A row whose payload no longer deserializes as a `Task` (an older version's shape, corruption) used to skip the cache invalidation *and* report "Forward failed permanently" for a send task, because both were derived from the parsed value. The identity now comes off the raw JSON, so the stale link-cache entry is dropped and the message names the post. - Temp files are marked and swept. Every temp file/dir the project creates now carries `x_media::TEMP_FILE_PREFIX`, and startup removes entries with that prefix older than an hour — a killed process leaves its downloads (up to hundreds of MB) behind because no destructor runs, and the age gate keeps the sweep away from a second instance's in-flight files. Verified live: the log reports the sweep, an aged leftover goes, a fresh prefixed file and an unrelated file stay.
This commit is contained in:
@@ -190,8 +190,14 @@ async fn resolve_bsky_video(
|
||||
return Err("bsky video has too many segments".to_string());
|
||||
}
|
||||
|
||||
let frames_dir = tempfile::tempdir().map_err(|e| e.to_string())?;
|
||||
let out_dir = tempfile::tempdir().map_err(|e| e.to_string())?;
|
||||
let frames_dir = tempfile::Builder::new()
|
||||
.prefix(crate::TEMP_FILE_PREFIX)
|
||||
.tempdir()
|
||||
.map_err(|e| e.to_string())?;
|
||||
let out_dir = tempfile::Builder::new()
|
||||
.prefix(crate::TEMP_FILE_PREFIX)
|
||||
.tempdir()
|
||||
.map_err(|e| e.to_string())?;
|
||||
let mut total: u64 = 0;
|
||||
let mut list = String::new();
|
||||
for (i, seg) in segments.iter().enumerate() {
|
||||
|
||||
@@ -221,6 +221,7 @@ impl PixivAPI {
|
||||
// memory: ugoira zips can be hundreds of MB, and the old
|
||||
// download_media_limited path spiked RAM up to the size cap.
|
||||
let mut zip_file = tempfile::Builder::new()
|
||||
.prefix(crate::TEMP_FILE_PREFIX)
|
||||
.suffix(".zip")
|
||||
.tempfile()
|
||||
.map_err(|e| PixivError::Api(format!("temp zip failed: {e}")))?;
|
||||
@@ -233,8 +234,14 @@ impl PixivAPI {
|
||||
let frame_delays = metadata.frames.iter().map(|f| f.delay).collect::<Vec<_>>();
|
||||
let result =
|
||||
tokio::task::spawn_blocking(move || -> Result<(String, tempfile::TempDir), String> {
|
||||
let frames_dir = tempfile::tempdir().map_err(|e| e.to_string())?;
|
||||
let out_dir = tempfile::tempdir().map_err(|e| e.to_string())?;
|
||||
let frames_dir = tempfile::Builder::new()
|
||||
.prefix(crate::TEMP_FILE_PREFIX)
|
||||
.tempdir()
|
||||
.map_err(|e| e.to_string())?;
|
||||
let out_dir = tempfile::Builder::new()
|
||||
.prefix(crate::TEMP_FILE_PREFIX)
|
||||
.tempdir()
|
||||
.map_err(|e| e.to_string())?;
|
||||
|
||||
// Extract frames to canonical zero-padded names; pixiv ugoira
|
||||
// frames are uniformly jpg or png per artwork. The zip is read
|
||||
|
||||
Reference in New Issue
Block a user