2fdb9be9b9
Some checks failed
Deploy / deploy (push) Failing after 51s
Drives massprice.tcgplayer.com's Run Rule action per-rule on a user-configured schedule (multiple times/day, day-of-week toggles), replacing MassPrice's native scheduler (capped at 15 schedules account-wide, one run/day, with observed clock drift). Deployed via Forgejo push-to-deploy on Unraid, matching arr-summary/card-grader. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H6TgffxTvYhqxykAbVQg2v
117 lines
6.1 KiB
Markdown
117 lines
6.1 KiB
Markdown
# MassPrice Scheduler
|
||
|
||
A Flask app that drives a real headless-Chromium session against
|
||
`massprice.tcgplayer.com` (TCGplayer's bulk pricing-rule tool) to click
|
||
"Run Rule" for each of the user's pricing rules on a schedule set through
|
||
this app's own web UI. Runs as one Docker container on the user's Unraid
|
||
home server, deployed the same way as the other projects on that box
|
||
(`arr-summary`, `card-grader`, `lunch-notifier`, `n64-tracker`).
|
||
|
||
## Why this exists
|
||
|
||
MassPrice has its own native per-rule scheduler (day-of-week + one
|
||
time-of-day), but it's capped at 15 active schedules account-wide and
|
||
only allows **one run per day per rule** — no multiple runs/day. Native
|
||
runs also showed real clock drift during discovery (6–22 minutes late
|
||
against their scheduled time). This tool **replaces** native scheduling
|
||
entirely: each rule's native schedule should be set to Inactive once this
|
||
tool is confirmed working, so there's exactly one thing triggering runs.
|
||
|
||
## Architecture
|
||
|
||
- **`app.py`** — Flask app, single shared-password session auth
|
||
(`LOGIN_PASSWORD` env var, `hmac.compare_digest`) — this protects the
|
||
*admin UI itself*, unrelated to the MassPrice/TCGplayer login below.
|
||
Routes: `/` (Dashboard — rule cards with schedule editor), `/activity`
|
||
(run history), and a small JSON API
|
||
(`/api/rules/refresh`, `/api/rules/<id>/schedule`,
|
||
`/api/rules/<id>/run`) that the Dashboard's JS calls without full page
|
||
reloads.
|
||
- **`scheduler.py`** — an in-process APScheduler `BackgroundScheduler`
|
||
holding one `CronTrigger` job per `(rule, schedule entry)` pair,
|
||
rebuilt from `schedules.json` on startup and whenever the webui saves
|
||
a change. Triggered jobs (and manual "Run Now" clicks) are pushed onto
|
||
one `queue.Queue`, drained by a single dedicated worker thread — this
|
||
guarantees only one Playwright browser session is ever active at a
|
||
time, matching how a human would actually operate the site and
|
||
avoiding concurrent-session weirdness.
|
||
- **Must run in exactly one process.** `gunicorn.conf.py` pins
|
||
`workers = 1` for this reason — a second worker would instantiate a
|
||
second scheduler and duplicate every run. Fine for a low-traffic
|
||
single-user tool.
|
||
- **`automation.py`** — all Playwright logic. `list_rules()` scrapes
|
||
`https://massprice.tcgplayer.com/Rule/List` for the live rule
|
||
id/name pairs (cached to `rules_cache.json`, refreshed via the
|
||
Dashboard's "Refresh Rules" button or on first boot). `run_rule(id)`
|
||
finds that rule's `li.rule.widget-rule` row, clicks its
|
||
`.dropdown-ellipsis` three-dot toggle, then clicks the row's
|
||
`a.run-rule[data-rule-id=...]` item — mirroring exactly what a human
|
||
does in the UI. **No public API was found or used** — the "Run Rule"
|
||
click handler is JS/AJAX-bound (no plain href) and its request is
|
||
almost certainly anti-forgery-token protected, so automation drives
|
||
the real page rather than replaying a raw HTTP POST.
|
||
- **Session persistence** — a Playwright `storage_state` (cookies +
|
||
localStorage) is saved to `/data/auth_state.json` and reused across
|
||
runs: the tool logs in **once**, then slides the session forward after
|
||
every successful run, rather than hitting the login form on every
|
||
scheduled click. A fresh login (via `MASSPRICE_USERNAME` /
|
||
`MASSPRICE_PASSWORD`) only happens when the saved state is missing or
|
||
has expired (detected by landing back on a login page).
|
||
- **`storage.py`** — tiny JSON read/write helpers (temp-file +
|
||
`os.replace`, so a crash mid-write can't corrupt state) for
|
||
everything under `/data`: `schedules.json`, `auth_state.json`,
|
||
`run_history.json` (bounded to the last 200 entries), `rules_cache.json`.
|
||
- **`templates/` / `static/`** — a small, plain dark dashboard. This is
|
||
a deliberately different, simpler visual identity than Arr Summary's
|
||
teletext system — that design language is specific to that app, not a
|
||
house style to reuse here.
|
||
|
||
## Known limitation — automated login
|
||
|
||
`MASSPRICE_USERNAME`/`MASSPRICE_PASSWORD` drive a fully automated login
|
||
(the user's explicit choice over a manual hand-off flow). **If
|
||
TCGplayer's login ever presents a CAPTCHA or a 2FA challenge, automated
|
||
login will fail outright** — there is deliberately no fallback UI for
|
||
this yet; a failed login just shows up as a `failure` entry in Activity
|
||
with the error message. If this starts happening routinely, it needs a
|
||
design change (e.g. a Chrome-remote-debugging-based manual re-auth flow),
|
||
not a bigger retry loop.
|
||
|
||
**The real logged-out MassPrice login form was never inspected** during
|
||
development — the account was already signed in when the site was
|
||
explored, and logging it out to look at the form would have disrupted
|
||
the user's real, live MassPrice session. `automation._login()` uses
|
||
resilient role/label/placeholder-based Playwright locators rather than
|
||
brittle CSS, but **expect to need a one-time adjustment against the real
|
||
form on first deploy** if the generic locators don't match.
|
||
|
||
## Local development
|
||
|
||
```bash
|
||
pip install -r requirements.txt
|
||
playwright install chromium # only needed if testing automation.py directly
|
||
python3 app.py
|
||
```
|
||
|
||
Runs on port 5000 (`PORT` env to change). Without `MASSPRICE_USERNAME`/
|
||
`MASSPRICE_PASSWORD` set and without Chromium installed, the Dashboard
|
||
still boots — `_bootstrap()` in `app.py` best-effort-catches the initial
|
||
rule sync, so local UI/schedule-editing work doesn't require Docker or
|
||
real MassPrice credentials at all. `DATA_DIR` env var overrides where
|
||
JSON state is read/written (defaults to `/data`, which won't exist
|
||
locally — point it at a scratch directory).
|
||
|
||
**Every commit should be matched with a version bump** —
|
||
`APP_VERSION`/`APP_VERSION_DATE` near the top of `app.py` — the same
|
||
"does the footer prove this deploy landed" signal used on Arr Summary.
|
||
|
||
## Deployment
|
||
|
||
See the `deploy-massprice` skill for the Forgejo push-to-deploy
|
||
pipeline and manual container restart steps.
|
||
|
||
**Never put real credentials in the repo.** `.env` and `data/` are
|
||
gitignored and excluded from the deploy rsync precisely so
|
||
`MASSPRICE_USERNAME`/`MASSPRICE_PASSWORD`/`LOGIN_PASSWORD` and the saved
|
||
MassPrice session (`data/auth_state.json`) have somewhere to live
|
||
outside git.
|