# MassPrice Scheduler A Flask app that drives a real headless-Chromium session against `massprice.tcgplayer.com` (TCGplayer's bulk pricing-rule tool) to click "Run Rule" for each of the user's pricing rules on a schedule set through this app's own web UI. Runs as one Docker container on the user's Unraid home server, deployed the same way as the other projects on that box (`arr-summary`, `card-grader`, `lunch-notifier`, `n64-tracker`). ## Why this exists MassPrice has its own native per-rule scheduler (day-of-week + one time-of-day), but it's capped at 15 active schedules account-wide and only allows **one run per day per rule** — no multiple runs/day. Native runs also showed real clock drift during discovery (6–22 minutes late against their scheduled time). This tool **replaces** native scheduling entirely: each rule's native schedule should be set to Inactive once this tool is confirmed working, so there's exactly one thing triggering runs. ## Architecture - **`app.py`** — Flask app, single shared-password session auth (`LOGIN_PASSWORD` env var, `hmac.compare_digest`) — this protects the *admin UI itself*, unrelated to the MassPrice/TCGplayer login below. Routes: `/` (Dashboard — rule cards with schedule editor), `/activity` (run history), and a small JSON API (`/api/rules/refresh`, `/api/rules//schedule`, `/api/rules//run`) that the Dashboard's JS calls without full page reloads. - **`scheduler.py`** — an in-process APScheduler `BackgroundScheduler` holding one `CronTrigger` job per `(rule, schedule entry)` pair, rebuilt from `schedules.json` on startup and whenever the webui saves a change. Triggered jobs (and manual "Run Now" clicks) are pushed onto one `queue.Queue`, drained by a single dedicated worker thread — this guarantees only one Playwright browser session is ever active at a time, matching how a human would actually operate the site and avoiding concurrent-session weirdness. - **Must run in exactly one process.** `gunicorn.conf.py` pins `workers = 1` for this reason — a second worker would instantiate a second scheduler and duplicate every run. Fine for a low-traffic single-user tool. - **`automation.py`** — all Playwright logic. `list_rules()` scrapes `https://massprice.tcgplayer.com/Rule/List` for the live rule id/name pairs (cached to `rules_cache.json`, refreshed via the Dashboard's "Refresh Rules" button or on first boot). `run_rule(id)` finds that rule's `li.rule.widget-rule` row, clicks its `.dropdown-ellipsis` three-dot toggle, then clicks the row's `a.run-rule[data-rule-id=...]` item — mirroring exactly what a human does in the UI. **No public API was found or used** — the "Run Rule" click handler is JS/AJAX-bound (no plain href) and its request is almost certainly anti-forgery-token protected, so automation drives the real page rather than replaying a raw HTTP POST. - **Session persistence** — a Playwright `storage_state` (cookies + localStorage) is saved to `/data/auth_state.json` and reused across runs: scheduled runs never touch the login form at all, they just load this saved session. When it's missing or has expired (detected by an unauthenticated visit to `/Rule/List` silently redirecting to the public marketing homepage `/` — there's no `/login` URL to check against), the run fails with a clear "session expired or missing" error rather than attempting to log in — see below for why. - **`storage.py`** — tiny JSON read/write helpers (temp-file + `os.replace`, so a crash mid-write can't corrupt state) for everything under `/data`: `schedules.json`, `auth_state.json`, `run_history.json` (bounded to the last 200 entries), `rules_cache.json`. - **`templates/` / `static/`** — a small, plain dark dashboard. This is a deliberately different, simpler visual identity than Arr Summary's teletext system — that design language is specific to that app, not a house style to reuse here. ## Login — why it's manual, and how it works MassPrice's login (`store.tcgplayer.com/oauth/login`, TCGplayer's shared OAuth identity provider) is protected by **hCaptcha**. Two automated approaches were tried and confirmed broken by live testing during development — there is no selector fix for either, **fully automated login is not achievable here**, full stop: 1. Filling in the real username/password and clicking Submit — the invisible captcha check silently never resolves for a headless/ automated browser, and the form just refuses to submit. An earlier version of this app tried `MASSPRICE_USERNAME`/`MASSPRICE_PASSWORD` this way; removed once confirmed broken. 2. Driving a real human's clicks through a headless Chromium exposed via Chrome remote debugging (`chrome://inspect`) — also removed after testing. Still fails hCaptcha, because the browser itself (a `HeadlessChrome` user-agent, `navigator.webdriver = true`) is what gets flagged, independent of who's actually clicking. Deliberately not pursued further than that one test: disguising that browser to get past hCaptcha's bot detection would be captcha evasion, not a session-handling technique, regardless of how legitimate running automation on your own account is. What actually works: log in with an ordinary, non-automated browser — hCaptcha passes the way it always does for a real person, because nothing about that browser looks automated — then copy that session's cookies over. The **Session** page in the webui drives this: 1. The user logs into massprice.tcgplayer.com in their own regular Chrome and exports its cookies with a browser extension (Cookie-Editor, "Export as JSON"). 2. They paste that JSON into the Session page and click "Import Session". 3. `automation.import_cookies()` parses it, converts each cookie from Cookie-Editor's shape into Playwright's `storage_state` cookie shape (`_convert_cookie()` — mainly the `sameSite` enum values and the `expirationDate` → `expires` field rename), and — critically — **verifies it actually authenticates** (a real headless request to `/Rule/List`, checked with the same `_is_logged_out()` used everywhere else) before saving it to `/data/auth_state.json`. A bad paste never overwrites a working session. This session is a real, live bearer credential for the user's MassPrice account for as long as it's valid — same trust model as `LOGIN_PASSWORD`/ `SECRET_KEY` in `.env` (server-only, gitignored, never in git), not something to treat as inert config. It isn't permanent: every successful automated run re-saves it (`_save_state()`), which tends to slide a "remember me"-style cookie's expiration forward, but it will eventually expire (inactivity, a password change, TCGplayer forcing re-auth) and need re-importing. The Dashboard shows a red banner (`_session_warning()` in `app.py`) when the most recent run/refresh failed specifically because of a missing/ expired session, so this doesn't just look like "no rules found." ## Local development ```bash pip install -r requirements.txt playwright install chromium # only needed if testing automation.py directly python3 app.py ``` Runs on port 5000 (`PORT` env to change). Without Chromium installed and without a saved `auth_state.json`, the Dashboard still boots — `_bootstrap()` in `app.py` best-effort-catches the initial rule sync, so local UI/schedule-editing work doesn't require Docker or a real MassPrice session at all. `DATA_DIR` env var overrides where JSON state is read/written (defaults to `/data`, which won't exist locally — point it at a scratch directory). **Every commit should be matched with a version bump** — `APP_VERSION`/`APP_VERSION_DATE` near the top of `app.py` — the same "does the footer prove this deploy landed" signal used on Arr Summary. ## Deployment See the `deploy-massprice` skill for the Forgejo push-to-deploy pipeline and manual container restart steps. **Never put real credentials in the repo.** `.env` and `data/` are gitignored and excluded from the deploy rsync precisely so `LOGIN_PASSWORD`/`SECRET_KEY` and the saved MassPrice session (`data/auth_state.json`) have somewhere to live outside git.