Poll for run completion every 5 minutes instead of checking once
All checks were successful
Deploy / deploy (push) Successful in 28s
All checks were successful
Deploy / deploy (push) Successful in 28s
The one-shot VERIFY_DELAY_MINUTES check was too rigid: MassPrice's processing time varies widely in practice (observed ~1 to over 20 minutes, and its queue can back up further under load — confirmed live, a run still sitting in "Queued" 15+ minutes in), so a single check window either misses slow-but-fine completions or has to be set needlessly long. Replaced with a poll loop: scheduler.py reschedules another one-off check every VERIFY_POLL_MINUTES (default 5) until automation's check_completion() reports "complete", up to VERIFY_MAX_ATTEMPTS (default 12, ~1 hour) before giving up and marking "not_confirmed" for good. verify_run() is split into a pure check_completion() (no storage side effects) so scheduler.py's loop owns the retry/give-up decision; automation.py just answers "is it done yet". Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H6TgffxTvYhqxykAbVQg2v
This commit is contained in:
parent
8b58dadc5a
commit
213837ff97
6 changed files with 151 additions and 90 deletions
67
CLAUDE.md
67
CLAUDE.md
|
|
@ -39,12 +39,22 @@ tool is confirmed working, so there's exactly one thing triggering runs.
|
|||
`workers = 1` for this reason — a second worker would instantiate a
|
||||
second scheduler and duplicate every run. Fine for a low-traffic
|
||||
single-user tool.
|
||||
- Also schedules **one-off** `verify-<entry_id>` jobs (APScheduler
|
||||
`date` trigger, not recurring) `VERIFY_DELAY_MINUTES` after a run is
|
||||
confirmed sent — see `automation.verify_run()` below.
|
||||
`rebuild_jobs()` only ever removes `rule-*` jobs, deliberately, so
|
||||
editing a schedule mid-flight can't cancel a verification that's
|
||||
already in progress for an earlier run.
|
||||
- Also drives the **completion poll loop**: after a run is confirmed
|
||||
sent, a one-off `verify-<entry_id>-<attempt>` job (APScheduler
|
||||
`date` trigger, not recurring) fires `VERIFY_POLL_MINUTES` later and
|
||||
calls `automation.check_completion()`. Not complete yet? Another
|
||||
one-off job reschedules the next check the same interval later —
|
||||
`_run_verification_check()` in `scheduler.py` owns this loop — up to
|
||||
`VERIFY_MAX_ATTEMPTS` times, after which it's marked
|
||||
`"not_confirmed"` for good. `rebuild_jobs()` only ever removes
|
||||
`rule-*` jobs, deliberately, so editing a schedule mid-flight can't
|
||||
cancel a poll loop that's already in progress for an earlier run.
|
||||
These one-off jobs live only in the scheduler's in-memory jobstore
|
||||
(no persistent jobstore is configured) — a container restart mid-poll
|
||||
silently drops that one entry's remaining checks, leaving it stuck
|
||||
on whatever state it was last in (usually "pending"). Low-impact
|
||||
(an hour-long window, restarts are infrequent) but worth knowing if
|
||||
an entry is ever seen stuck on "Pending" long after the fact.
|
||||
- **`automation.py`** — all Playwright logic. `list_rules()` scrapes
|
||||
`https://massprice.tcgplayer.com/Rule/List` for the live rule
|
||||
id/name pairs (cached to `rules_cache.json`, refreshed via the
|
||||
|
|
@ -71,23 +81,34 @@ tool is confirmed working, so there's exactly one thing triggering runs.
|
|||
drives the real page rather than replaying a raw HTTP POST.
|
||||
- **Run verification** — confirming the modal only means MassPrice
|
||||
*accepted* the run; it still processes asynchronously on their side
|
||||
(queued → running → Complete), taking anywhere from ~1 to ~22 minutes
|
||||
in observed practice. `run_rule()` captures a "watermark" — the
|
||||
`data-activitylogid` of that rule's current top entry on
|
||||
`/Activity/List` (a stable, monotonically increasing ID; deliberately
|
||||
*not* parsing the displayed timestamp string, which would need
|
||||
fragile date/timezone handling) — **before** submitting, logs the run
|
||||
as `status: "sent"` with `verification: "pending"`, and the caller in
|
||||
`scheduler.py` schedules `verify_run()` to run `VERIFY_DELAY_MINUTES`
|
||||
later. `verify_run()` re-checks `/Activity/List`: a new entry (higher
|
||||
`data-activitylogid` than the watermark) whose status text contains
|
||||
"Complete" → `verification: "complete"`; otherwise
|
||||
`verification: "not_confirmed"` (which can mean "still processing",
|
||||
not necessarily "failed" — the delay is a reasonable middle ground,
|
||||
not a guarantee it's had enough time). `_find_latest_activity()` is
|
||||
shared by the watermark capture and the follow-up check. The Activity
|
||||
page's "Complete" column surfaces this; "Status" shows whether the
|
||||
click itself was sent, independent of whether MassPrice finished it.
|
||||
(queued → running → Complete), taking anywhere from ~1 minute to over
|
||||
20 in observed practice, and its queue can back up further under
|
||||
load (observed directly: a run still sitting in "Queued" 15+ minutes
|
||||
in). `run_rule()` captures a "watermark" — the `data-activitylogid` of
|
||||
that rule's current top entry on `/Activity/List` (a stable,
|
||||
monotonically increasing ID; deliberately *not* parsing the displayed
|
||||
timestamp string, which would need fragile date/timezone handling) —
|
||||
**before** submitting, logs the run as `status: "sent"` with
|
||||
`verification: "pending"`, and the caller in `scheduler.py` starts a
|
||||
poll loop (`automation.check_completion()`, every
|
||||
`VERIFY_POLL_MINUTES`, up to `VERIFY_MAX_ATTEMPTS` times — see
|
||||
`scheduler.py`'s own section above for the mechanics). Each check
|
||||
re-scans `/Activity/List`: a new entry (higher `data-activitylogid`
|
||||
than the watermark) whose status text contains "Complete" →
|
||||
`verification: "complete"`, loop stops; otherwise the loop keeps
|
||||
`verification: "pending"` and just updates `verify_message` with the
|
||||
latest note, until attempts run out and it's marked
|
||||
`verification: "not_confirmed"` for good — which can mean "still
|
||||
processing somewhere past our patience", not necessarily "failed".
|
||||
Also worth knowing: a still-"Queued" MassPrice entry has no
|
||||
`data-activitylogid` yet (no "View" link exists until it finishes),
|
||||
so `_find_latest_activity()` — shared by the watermark capture and
|
||||
every poll — correctly falls through to the *previous* completed
|
||||
entry rather than matching the in-flight one; that's why "no ID
|
||||
higher than the watermark" is the right "not done yet" signal, not a
|
||||
bug. The Activity page's "Complete" column surfaces all of this;
|
||||
"Status" shows whether the click itself was sent, independent of
|
||||
whether MassPrice finished it.
|
||||
- **Session persistence** — a Playwright `storage_state` (cookies +
|
||||
localStorage) is saved to `/data/auth_state.json` and reused across
|
||||
runs: scheduled runs never touch the login form at all, they just load
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue