# MassPrice Scheduler A Flask app that drives a real headless-Chromium session against `massprice.tcgplayer.com` (TCGplayer's bulk pricing-rule tool) to click "Run Rule" for each of the user's pricing rules on a schedule set through this app's own web UI. Runs as one Docker container on the user's Unraid home server, deployed the same way as the other projects on that box (`arr-summary`, `card-grader`, `lunch-notifier`, `n64-tracker`). ## Why this exists MassPrice has its own native per-rule scheduler (day-of-week + one time-of-day), but it's capped at 15 active schedules account-wide and only allows **one run per day per rule** — no multiple runs/day. Native runs also showed real clock drift during discovery (6–22 minutes late against their scheduled time). This tool **replaces** native scheduling entirely: each rule's native schedule should be set to Inactive once this tool is confirmed working, so there's exactly one thing triggering runs. ## Architecture - **`app.py`** — Flask app, single shared-password session auth (`LOGIN_PASSWORD` env var, `hmac.compare_digest`) — this protects the *admin UI itself*, unrelated to the MassPrice/TCGplayer login below. Routes: `/` (Dashboard — rule cards with schedule editor), `/activity` (run history), and a small JSON API (`/api/rules/refresh`, `/api/rules//schedule`, `/api/rules//run`) that the Dashboard's JS calls without full page reloads. - **`scheduler.py`** — an in-process APScheduler `BackgroundScheduler` holding one `CronTrigger` job per `(rule, schedule entry)` pair, rebuilt from `schedules.json` on startup and whenever the webui saves a change. Triggered jobs (and manual "Run Now" clicks) are pushed onto one `queue.Queue`, drained by a single dedicated worker thread — this guarantees only one Playwright browser session is ever active at a time, matching how a human would actually operate the site and avoiding concurrent-session weirdness. - **Must run in exactly one process.** `gunicorn.conf.py` pins `workers = 1` for this reason — a second worker would instantiate a second scheduler and duplicate every run. Fine for a low-traffic single-user tool. - **`automation.py`** — all Playwright logic. `list_rules()` scrapes `https://massprice.tcgplayer.com/Rule/List` for the live rule id/name pairs (cached to `rules_cache.json`, refreshed via the Dashboard's "Refresh Rules" button or on first boot). `run_rule(id)` finds that rule's `li.rule.widget-rule` row, clicks its `.dropdown-ellipsis` three-dot toggle, then clicks the row's `a.run-rule[data-rule-id=...]` item — mirroring exactly what a human does in the UI. **No public API was found or used** — the "Run Rule" click handler is JS/AJAX-bound (no plain href) and its request is almost certainly anti-forgery-token protected, so automation drives the real page rather than replaying a raw HTTP POST. - **Session persistence** — a Playwright `storage_state` (cookies + localStorage) is saved to `/data/auth_state.json` and reused across runs: scheduled runs never touch the login form at all, they just load this saved session. When it's missing or has expired (detected by an unauthenticated visit to `/Rule/List` silently redirecting to the public marketing homepage `/` — there's no `/login` URL to check against), the run fails with a clear "session expired or missing" error rather than attempting to log in — see below for why. - **`storage.py`** — tiny JSON read/write helpers (temp-file + `os.replace`, so a crash mid-write can't corrupt state) for everything under `/data`: `schedules.json`, `auth_state.json`, `run_history.json` (bounded to the last 200 entries), `rules_cache.json`. - **`templates/` / `static/`** — a small, plain dark dashboard. This is a deliberately different, simpler visual identity than Arr Summary's teletext system — that design language is specific to that app, not a house style to reuse here. ## Login — why it's manual, and how it works MassPrice's login (`store.tcgplayer.com/oauth/login`, TCGplayer's shared OAuth identity provider) is protected by **hCaptcha**. This was confirmed by live testing during development: filling in the real username/password and clicking Submit never navigates anywhere — the invisible captcha check silently never resolves for a headless/automated browser, and the form just refuses to submit. There is no selector fix for this; **fully automated login is not achievable here**, full stop. An earlier version of this app tried `MASSPRICE_USERNAME`/ `MASSPRICE_PASSWORD` credential-based auto-login — removed once this was confirmed, since keeping dead code that can never succeed is worse than having none. What actually works: a human only needs to pass the hCaptcha, not automate around it, and that's a one-time (well, periodic — however long the session cookie lasts) cost, not a per-run one. The **Session** page in the webui drives this: 1. "Start Login Session" launches headless Chromium with its CDP debug port exposed (`DEBUG_PORT`, default `9222`). 2. The user connects their own Chrome to it via `chrome://inspect`, which renders and accepts real input against that same running browser instance — no VNC/Xvfb needed, headless Chromium's debug protocol already supports this. 3. The user logs in normally in that inspected tab, solving the hCaptcha themselves since it's a real click from a real browser. 4. "Capture Session" saves that browser's cookies to `/data/auth_state.json` — the same file scheduled runs read. **All of this lives on one dedicated thread** (`automation._debug_worker`), not arbitrary Flask request threads — Playwright's sync API isn't safe to call across threads, and gunicorn's `gthread` workers hand each request to a different one. Flask routes only ever put a command on `_debug_cmd_queue` and block for the result; they never touch the Playwright browser/context/page objects directly. An un-captured debug session auto-closes after 10 minutes so an open, unauthenticated CDP port doesn't sit exposed on the LAN indefinitely. The Dashboard shows a red banner (`_session_warning()` in `app.py`) when the most recent run/refresh failed specifically because of a missing/ expired session, so this doesn't just look like "no rules found." ## Local development ```bash pip install -r requirements.txt playwright install chromium # only needed if testing automation.py directly python3 app.py ``` Runs on port 5000 (`PORT` env to change). Without Chromium installed and without a saved `auth_state.json`, the Dashboard still boots — `_bootstrap()` in `app.py` best-effort-catches the initial rule sync, so local UI/schedule-editing work doesn't require Docker or a real MassPrice session at all. `DATA_DIR` env var overrides where JSON state is read/written (defaults to `/data`, which won't exist locally — point it at a scratch directory). **Every commit should be matched with a version bump** — `APP_VERSION`/`APP_VERSION_DATE` near the top of `app.py` — the same "does the footer prove this deploy landed" signal used on Arr Summary. ## Deployment See the `deploy-massprice` skill for the Forgejo push-to-deploy pipeline and manual container restart steps. **Never put real credentials in the repo.** `.env` and `data/` are gitignored and excluded from the deploy rsync precisely so `LOGIN_PASSWORD`/`SECRET_KEY` and the saved MassPrice session (`data/auth_state.json`) have somewhere to live outside git.