Poll for run completion every 5 minutes instead of checking once
All checks were successful
Deploy / deploy (push) Successful in 28s

The one-shot VERIFY_DELAY_MINUTES check was too rigid: MassPrice's
processing time varies widely in practice (observed ~1 to over 20
minutes, and its queue can back up further under load — confirmed
live, a run still sitting in "Queued" 15+ minutes in), so a single
check window either misses slow-but-fine completions or has to be set
needlessly long.

Replaced with a poll loop: scheduler.py reschedules another one-off
check every VERIFY_POLL_MINUTES (default 5) until automation's
check_completion() reports "complete", up to VERIFY_MAX_ATTEMPTS
(default 12, ~1 hour) before giving up and marking "not_confirmed" for
good. verify_run() is split into a pure check_completion() (no storage
side effects) so scheduler.py's loop owns the retry/give-up decision;
automation.py just answers "is it done yet".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H6TgffxTvYhqxykAbVQg2v
This commit is contained in:
Barely Removable 2026-08-31 23:58:23 -07:00
parent 8b58dadc5a
commit 213837ff97
6 changed files with 151 additions and 90 deletions

View file

@ -39,12 +39,22 @@ tool is confirmed working, so there's exactly one thing triggering runs.
`workers = 1` for this reason — a second worker would instantiate a
second scheduler and duplicate every run. Fine for a low-traffic
single-user tool.
- Also schedules **one-off** `verify-<entry_id>` jobs (APScheduler
`date` trigger, not recurring) `VERIFY_DELAY_MINUTES` after a run is
confirmed sent — see `automation.verify_run()` below.
`rebuild_jobs()` only ever removes `rule-*` jobs, deliberately, so
editing a schedule mid-flight can't cancel a verification that's
already in progress for an earlier run.
- Also drives the **completion poll loop**: after a run is confirmed
sent, a one-off `verify-<entry_id>-<attempt>` job (APScheduler
`date` trigger, not recurring) fires `VERIFY_POLL_MINUTES` later and
calls `automation.check_completion()`. Not complete yet? Another
one-off job reschedules the next check the same interval later —
`_run_verification_check()` in `scheduler.py` owns this loop — up to
`VERIFY_MAX_ATTEMPTS` times, after which it's marked
`"not_confirmed"` for good. `rebuild_jobs()` only ever removes
`rule-*` jobs, deliberately, so editing a schedule mid-flight can't
cancel a poll loop that's already in progress for an earlier run.
These one-off jobs live only in the scheduler's in-memory jobstore
(no persistent jobstore is configured) — a container restart mid-poll
silently drops that one entry's remaining checks, leaving it stuck
on whatever state it was last in (usually "pending"). Low-impact
(an hour-long window, restarts are infrequent) but worth knowing if
an entry is ever seen stuck on "Pending" long after the fact.
- **`automation.py`** — all Playwright logic. `list_rules()` scrapes
`https://massprice.tcgplayer.com/Rule/List` for the live rule
id/name pairs (cached to `rules_cache.json`, refreshed via the
@ -71,23 +81,34 @@ tool is confirmed working, so there's exactly one thing triggering runs.
drives the real page rather than replaying a raw HTTP POST.
- **Run verification** — confirming the modal only means MassPrice
*accepted* the run; it still processes asynchronously on their side
(queued → running → Complete), taking anywhere from ~1 to ~22 minutes
in observed practice. `run_rule()` captures a "watermark" — the
`data-activitylogid` of that rule's current top entry on
`/Activity/List` (a stable, monotonically increasing ID; deliberately
*not* parsing the displayed timestamp string, which would need
fragile date/timezone handling) — **before** submitting, logs the run
as `status: "sent"` with `verification: "pending"`, and the caller in
`scheduler.py` schedules `verify_run()` to run `VERIFY_DELAY_MINUTES`
later. `verify_run()` re-checks `/Activity/List`: a new entry (higher
`data-activitylogid` than the watermark) whose status text contains
"Complete" → `verification: "complete"`; otherwise
`verification: "not_confirmed"` (which can mean "still processing",
not necessarily "failed" — the delay is a reasonable middle ground,
not a guarantee it's had enough time). `_find_latest_activity()` is
shared by the watermark capture and the follow-up check. The Activity
page's "Complete" column surfaces this; "Status" shows whether the
click itself was sent, independent of whether MassPrice finished it.
(queued → running → Complete), taking anywhere from ~1 minute to over
20 in observed practice, and its queue can back up further under
load (observed directly: a run still sitting in "Queued" 15+ minutes
in). `run_rule()` captures a "watermark" — the `data-activitylogid` of
that rule's current top entry on `/Activity/List` (a stable,
monotonically increasing ID; deliberately *not* parsing the displayed
timestamp string, which would need fragile date/timezone handling) —
**before** submitting, logs the run as `status: "sent"` with
`verification: "pending"`, and the caller in `scheduler.py` starts a
poll loop (`automation.check_completion()`, every
`VERIFY_POLL_MINUTES`, up to `VERIFY_MAX_ATTEMPTS` times — see
`scheduler.py`'s own section above for the mechanics). Each check
re-scans `/Activity/List`: a new entry (higher `data-activitylogid`
than the watermark) whose status text contains "Complete" →
`verification: "complete"`, loop stops; otherwise the loop keeps
`verification: "pending"` and just updates `verify_message` with the
latest note, until attempts run out and it's marked
`verification: "not_confirmed"` for good — which can mean "still
processing somewhere past our patience", not necessarily "failed".
Also worth knowing: a still-"Queued" MassPrice entry has no
`data-activitylogid` yet (no "View" link exists until it finishes),
so `_find_latest_activity()` — shared by the watermark capture and
every poll — correctly falls through to the *previous* completed
entry rather than matching the in-flight one; that's why "no ID
higher than the watermark" is the right "not done yet" signal, not a
bug. The Activity page's "Complete" column surfaces all of this;
"Status" shows whether the click itself was sent, independent of
whether MassPrice finished it.
- **Session persistence** — a Playwright `storage_state` (cookies +
localStorage) is saved to `/data/auth_state.json` and reused across
runs: scheduled runs never touch the login form at all, they just load