mirror of
https://github.com/TheFunny/TelegramTwitterMediaBot.git
synced 2026-10-04 01:02:12 +00:00
fix(retry): fence the queue lease, clean up after a kill, name dead-lettered posts
P2 (hardening) of the retry audit, closing the report's remaining findings. - Lease fencing. `lease_next` now stamps a random `lease_token`, and every write-back a worker makes (the 30s heartbeat, `delete_row`, `reschedule`, `mark_done`) is guarded by it. A lease that expired while its holder was stalled and was then re-leased used to let *both* holders write the same row: one duplicated the send, the other silently discarded the new holder's retry (a 0-row update was not even logged). Now a worker that no longer holds the lease drops its attempt at the next heartbeat and writes nothing. Reaching existing databases needed a migration chain, which `db.rs` had been pre-committed to: `MIGRATIONS` + `migrate` track `PRAGMA user_version`, with `schema_init` as the version-0 baseline. Verified on a database created before this change: user_version 0 -> 1, column added, rows intact. - Dead-letter notifications no longer mislabel an unparsable payload. A row whose payload no longer deserializes as a `Task` (an older version's shape, corruption) used to skip the cache invalidation *and* report "Forward failed permanently" for a send task, because both were derived from the parsed value. The identity now comes off the raw JSON, so the stale link-cache entry is dropped and the message names the post. - Temp files are marked and swept. Every temp file/dir the project creates now carries `x_media::TEMP_FILE_PREFIX`, and startup removes entries with that prefix older than an hour — a killed process leaves its downloads (up to hundreds of MB) behind because no destructor runs, and the age gate keeps the sweep away from a second instance's in-flight files. Verified live: the log reports the sweep, an aged leftover goes, a fresh prefixed file and an unrelated file stay.
This commit is contained in:
@@ -124,9 +124,39 @@ pub fn open_store(path: &str) -> rusqlite::Result<Arc<DbPool>> {
|
||||
}
|
||||
let conn = open_db(path)?;
|
||||
schema_init(&conn)?;
|
||||
migrate(&conn)?;
|
||||
Ok(Arc::new(DbPool::new(path)))
|
||||
}
|
||||
|
||||
/// Schema migrations, applied in order and tracked by `PRAGMA user_version`
|
||||
/// (the index in this array + 1 is the version a statement brings the
|
||||
/// database to). Append only — never edit or reorder an entry, or databases
|
||||
/// already past it would skip or repeat work.
|
||||
const MIGRATIONS: &[&str] = &[
|
||||
// 1: lease fencing. A worker's write-backs (`delete`/`reschedule`/the
|
||||
// lease heartbeat) are guarded by the token it was leased with, so a
|
||||
// lease that expired and was re-leased by another worker can no longer be
|
||||
// written by its former holder — which used to duplicate a send or drop
|
||||
// the new holder's retry state, silently.
|
||||
"ALTER TABLE tasks ADD COLUMN lease_token TEXT",
|
||||
];
|
||||
|
||||
/// Brings an existing database up to [`MIGRATIONS`]. Idempotent: a database
|
||||
/// already at the latest version does no work.
|
||||
fn migrate(conn: &Connection) -> rusqlite::Result<()> {
|
||||
let version: i64 = conn.query_row("PRAGMA user_version", [], |row| row.get(0))?;
|
||||
for (index, statement) in MIGRATIONS.iter().enumerate() {
|
||||
let target = index as i64 + 1;
|
||||
if version >= target {
|
||||
continue;
|
||||
}
|
||||
conn.execute_batch(statement)?;
|
||||
// `PRAGMA` does not take bind parameters; the value is our own index.
|
||||
conn.execute_batch(&format!("PRAGMA user_version = {target}"))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn rusqlite_error(e: std::io::Error) -> rusqlite::Error {
|
||||
rusqlite::Error::ToSqlConversionFailure(Box::new(e))
|
||||
}
|
||||
@@ -135,11 +165,11 @@ fn rusqlite_error(e: std::io::Error) -> rusqlite::Error {
|
||||
/// The three stores used to own their own schema; keeping it in one place
|
||||
/// means one initialization for the whole database file.
|
||||
///
|
||||
/// ⚠️ Schema-change reminder (deferred, see `docs/architecture-refactor.md`
|
||||
/// §5): this is a plain `CREATE TABLE IF NOT EXISTS` with no versioning.
|
||||
/// Before any column/table change that must migrate existing databases, land
|
||||
/// the `PRAGMA user_version` migration chain first (`MIGRATIONS: &[&str]` +
|
||||
/// `migrate(conn)`), then restructure this function.
|
||||
/// This is the **baseline** schema (version 0): a fresh database is created
|
||||
/// exactly like this, and anything that must *change* an existing one is
|
||||
/// appended to [`MIGRATIONS`] instead of being edited in here — otherwise a
|
||||
/// database created before the change would never gain the new column and a
|
||||
/// freshly created one would try to apply the migration a second time.
|
||||
pub fn schema_init(conn: &Connection) -> rusqlite::Result<()> {
|
||||
conn.execute_batch(
|
||||
"CREATE TABLE IF NOT EXISTS tasks (id TEXT PRIMARY KEY, payload TEXT NOT NULL, \
|
||||
|
||||
Reference in New Issue
Block a user