Initial scaffold: Flask webui + APScheduler + Playwright automation for MassPrice rule scheduling
Some checks failed
Deploy / deploy (push) Failing after 51s
Some checks failed
Deploy / deploy (push) Failing after 51s
Drives massprice.tcgplayer.com's Run Rule action per-rule on a user-configured schedule (multiple times/day, day-of-week toggles), replacing MassPrice's native scheduler (capped at 15 schedules account-wide, one run/day, with observed clock drift). Deployed via Forgejo push-to-deploy on Unraid, matching arr-summary/card-grader. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H6TgffxTvYhqxykAbVQg2v
This commit is contained in:
commit
2fdb9be9b9
19 changed files with 1197 additions and 0 deletions
117
CLAUDE.md
Normal file
117
CLAUDE.md
Normal file
|
|
@ -0,0 +1,117 @@
|
|||
# MassPrice Scheduler
|
||||
|
||||
A Flask app that drives a real headless-Chromium session against
|
||||
`massprice.tcgplayer.com` (TCGplayer's bulk pricing-rule tool) to click
|
||||
"Run Rule" for each of the user's pricing rules on a schedule set through
|
||||
this app's own web UI. Runs as one Docker container on the user's Unraid
|
||||
home server, deployed the same way as the other projects on that box
|
||||
(`arr-summary`, `card-grader`, `lunch-notifier`, `n64-tracker`).
|
||||
|
||||
## Why this exists
|
||||
|
||||
MassPrice has its own native per-rule scheduler (day-of-week + one
|
||||
time-of-day), but it's capped at 15 active schedules account-wide and
|
||||
only allows **one run per day per rule** — no multiple runs/day. Native
|
||||
runs also showed real clock drift during discovery (6–22 minutes late
|
||||
against their scheduled time). This tool **replaces** native scheduling
|
||||
entirely: each rule's native schedule should be set to Inactive once this
|
||||
tool is confirmed working, so there's exactly one thing triggering runs.
|
||||
|
||||
## Architecture
|
||||
|
||||
- **`app.py`** — Flask app, single shared-password session auth
|
||||
(`LOGIN_PASSWORD` env var, `hmac.compare_digest`) — this protects the
|
||||
*admin UI itself*, unrelated to the MassPrice/TCGplayer login below.
|
||||
Routes: `/` (Dashboard — rule cards with schedule editor), `/activity`
|
||||
(run history), and a small JSON API
|
||||
(`/api/rules/refresh`, `/api/rules/<id>/schedule`,
|
||||
`/api/rules/<id>/run`) that the Dashboard's JS calls without full page
|
||||
reloads.
|
||||
- **`scheduler.py`** — an in-process APScheduler `BackgroundScheduler`
|
||||
holding one `CronTrigger` job per `(rule, schedule entry)` pair,
|
||||
rebuilt from `schedules.json` on startup and whenever the webui saves
|
||||
a change. Triggered jobs (and manual "Run Now" clicks) are pushed onto
|
||||
one `queue.Queue`, drained by a single dedicated worker thread — this
|
||||
guarantees only one Playwright browser session is ever active at a
|
||||
time, matching how a human would actually operate the site and
|
||||
avoiding concurrent-session weirdness.
|
||||
- **Must run in exactly one process.** `gunicorn.conf.py` pins
|
||||
`workers = 1` for this reason — a second worker would instantiate a
|
||||
second scheduler and duplicate every run. Fine for a low-traffic
|
||||
single-user tool.
|
||||
- **`automation.py`** — all Playwright logic. `list_rules()` scrapes
|
||||
`https://massprice.tcgplayer.com/Rule/List` for the live rule
|
||||
id/name pairs (cached to `rules_cache.json`, refreshed via the
|
||||
Dashboard's "Refresh Rules" button or on first boot). `run_rule(id)`
|
||||
finds that rule's `li.rule.widget-rule` row, clicks its
|
||||
`.dropdown-ellipsis` three-dot toggle, then clicks the row's
|
||||
`a.run-rule[data-rule-id=...]` item — mirroring exactly what a human
|
||||
does in the UI. **No public API was found or used** — the "Run Rule"
|
||||
click handler is JS/AJAX-bound (no plain href) and its request is
|
||||
almost certainly anti-forgery-token protected, so automation drives
|
||||
the real page rather than replaying a raw HTTP POST.
|
||||
- **Session persistence** — a Playwright `storage_state` (cookies +
|
||||
localStorage) is saved to `/data/auth_state.json` and reused across
|
||||
runs: the tool logs in **once**, then slides the session forward after
|
||||
every successful run, rather than hitting the login form on every
|
||||
scheduled click. A fresh login (via `MASSPRICE_USERNAME` /
|
||||
`MASSPRICE_PASSWORD`) only happens when the saved state is missing or
|
||||
has expired (detected by landing back on a login page).
|
||||
- **`storage.py`** — tiny JSON read/write helpers (temp-file +
|
||||
`os.replace`, so a crash mid-write can't corrupt state) for
|
||||
everything under `/data`: `schedules.json`, `auth_state.json`,
|
||||
`run_history.json` (bounded to the last 200 entries), `rules_cache.json`.
|
||||
- **`templates/` / `static/`** — a small, plain dark dashboard. This is
|
||||
a deliberately different, simpler visual identity than Arr Summary's
|
||||
teletext system — that design language is specific to that app, not a
|
||||
house style to reuse here.
|
||||
|
||||
## Known limitation — automated login
|
||||
|
||||
`MASSPRICE_USERNAME`/`MASSPRICE_PASSWORD` drive a fully automated login
|
||||
(the user's explicit choice over a manual hand-off flow). **If
|
||||
TCGplayer's login ever presents a CAPTCHA or a 2FA challenge, automated
|
||||
login will fail outright** — there is deliberately no fallback UI for
|
||||
this yet; a failed login just shows up as a `failure` entry in Activity
|
||||
with the error message. If this starts happening routinely, it needs a
|
||||
design change (e.g. a Chrome-remote-debugging-based manual re-auth flow),
|
||||
not a bigger retry loop.
|
||||
|
||||
**The real logged-out MassPrice login form was never inspected** during
|
||||
development — the account was already signed in when the site was
|
||||
explored, and logging it out to look at the form would have disrupted
|
||||
the user's real, live MassPrice session. `automation._login()` uses
|
||||
resilient role/label/placeholder-based Playwright locators rather than
|
||||
brittle CSS, but **expect to need a one-time adjustment against the real
|
||||
form on first deploy** if the generic locators don't match.
|
||||
|
||||
## Local development
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
playwright install chromium # only needed if testing automation.py directly
|
||||
python3 app.py
|
||||
```
|
||||
|
||||
Runs on port 5000 (`PORT` env to change). Without `MASSPRICE_USERNAME`/
|
||||
`MASSPRICE_PASSWORD` set and without Chromium installed, the Dashboard
|
||||
still boots — `_bootstrap()` in `app.py` best-effort-catches the initial
|
||||
rule sync, so local UI/schedule-editing work doesn't require Docker or
|
||||
real MassPrice credentials at all. `DATA_DIR` env var overrides where
|
||||
JSON state is read/written (defaults to `/data`, which won't exist
|
||||
locally — point it at a scratch directory).
|
||||
|
||||
**Every commit should be matched with a version bump** —
|
||||
`APP_VERSION`/`APP_VERSION_DATE` near the top of `app.py` — the same
|
||||
"does the footer prove this deploy landed" signal used on Arr Summary.
|
||||
|
||||
## Deployment
|
||||
|
||||
See the `deploy-massprice` skill for the Forgejo push-to-deploy
|
||||
pipeline and manual container restart steps.
|
||||
|
||||
**Never put real credentials in the repo.** `.env` and `data/` are
|
||||
gitignored and excluded from the deploy rsync precisely so
|
||||
`MASSPRICE_USERNAME`/`MASSPRICE_PASSWORD`/`LOGIN_PASSWORD` and the saved
|
||||
MassPrice session (`data/auth_state.json`) have somewhere to live
|
||||
outside git.
|
||||
Loading…
Add table
Add a link
Reference in a new issue