ninja_hippo/Massprice
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
Massprice/.claude/skills/deploy-massprice/SKILL.md
Barely Removable e2eaabca06
All checks were successful
Deploy / deploy (push) Successful in 30s
Details
Fix 15 audit findings: browser concurrency, silent verification failures, auth gaps 
Full read-through of the project. Grouped by what actually breaks:

Browser concurrency (the invariant the design rests on)
- Every Playwright session now goes through scheduler.submit() onto the
  single worker queue. "Refresh Rules", cookie import and the boot sync
  were calling automation.py directly from Flask request threads, so
  with gunicorn's threads=4 a refresh during a scheduled run opened a
  second Chromium and both wrote /data/auth_state.json — which
  Playwright writes itself, non-atomically. A torn write there reads
  back as "session expired" and costs a pointless cookie re-import.
- Request handlers that need a result use run_browser_task(), which
  waits REQUEST_WAIT_SECONDS (90) then raises BrowserBusy -> 503
  without cancelling the work. Previously a slow in-request scrape could
  exceed gunicorn's 120s timeout; killing that worker takes the
  APScheduler jobstore and run queue with it, silently dropping every
  pending verification poll.

Verification correctness (the feature that exists to catch silent failures)
- run_rule() no longer starts a poll loop when the watermark capture
  itself failed. watermark_id was None both for "capture failed" and
  "rule has no prior activity", and check_completion() skips its
  comparison on None — so a failed capture matched the *previous* run's
  Complete entry and reported a false "verified", the exact class of bug
  the verification step was added to catch.
- A refresh now reconciles schedules.json against the live rule list.
  Rule ids are stable, names are not, and the cached name is what the
  Activity-page match keys on: a renamed rule kept running while
  reporting "Not Confirmed" forever, training you to ignore the signal.
  A deleted rule was worse — its card disappears while its cron jobs
  keep firing, with no UI left to disable them. Skipped on an empty
  scrape so a MassPrice redesign can't delete every saved schedule.
- The dashboard's session banner reads a session_state.json flag set by
  _ensure_authenticated() instead of guessing from run_history.json.
  list_rules() writes no history entry, so a failed refresh — often the
  first symptom — never raised the banner, despite the docs saying it
  covered "run/refresh".

Scheduler robustness
- rebuild_jobs() parses and adds each entry under its own guard. It runs
  *after* clearing existing jobs, so one malformed entry escaping as an
  exception left the scheduler holding no jobs at all — every schedule
  silently dead until the next restart. _parse_entry() also accepts
  HH:MM:SS (some browsers emit seconds, which the old two-value unpack
  crashed on) and range-checks hour/minute.
- The API validates entries at save time rather than storing something
  every rebuild skips, and the frontend surfaces save failures instead
  of discarding them.
- An unknown TZ logs and falls back to UTC instead of crash-looping the
  container with a traceback that doesn't obviously point at .env.

Auth and web hygiene
- Empty-but-set LOGIN_PASSWORD/SECRET_KEY now fall back to their
  defaults. docker-compose always sets these, so a blank .env value
  arrived as "" — and compare_digest("", "") is True, i.e. no login at
  all. An empty submitted password is rejected outright too.
- Password compared as bytes: compare_digest raises TypeError on
  non-ASCII str, turning any accented character into a 500.
- CSRF token per session, in a meta tag, echoed as X-CSRF-Token on all
  four POST endpoints; explicit SameSite=Lax/HttpOnly.
- /api/* answers 401 instead of redirecting. fetch() follows a 302 to
  /login, gets 200 HTML, and reads it as success — "Run Now" reported
  "Queued" for runs that were never queued.
- ?next= is restricted to local paths (open redirect).

Smaller correctness fixes
- Timestamps render in TZ via a localtime filter; they were shown as raw
  UTC while schedules run in America/Phoenix.
- Activity's 24h filter excludes unparseable timestamps rather than
  including them, and treats naive values as UTC instead of raising.
- SameSite=None cookies are forced Secure — Chromium drops them
  otherwise, silently, possibly losing the one cookie carrying the login.
- The Activity selector escapes the rule name, so a name containing a
  quote no longer produces an unparseable selector.
- pytz pinned explicitly; it only ever resolved as an apscheduler 3.x
  transitive dep, which 4.x drops.

Docs corrected where they disagreed with the code: auth_state.json is
written by Playwright, not through storage.py's atomic helper; the
banner's real trigger; and the deploy skill's claim that a rename causes
"rule id not found" (it can't — schedules are keyed by the stable id).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H6TgffxTvYhqxykAbVQg2v
2026-09-01 00:28:17 -07:00

6 KiB
Raw Permalink Blame History

name description
deploy-massprice Use when deploying MassPrice Scheduler, restarting the massprice container on Unraid, confirming a deploy landed, logging into MassPrice via the Session page, or troubleshooting its scheduled-rule automation.

Deployment

Target: Unraid server, container path /mnt/user/appdata/massprice, built from Dockerfile/docker-compose.yml in this repo. Container name massprice, port 5057 (overridable via HOST_PORT — 5056 was already taken by arr-summary, despite its own docs saying 5055) → container port 5000.

Deploy mechanism: Forgejo push-to-deploy, the same pattern as arr-summary, card-grader, lunch-notifier, n64-tracker — see arr-summary's deploy-arr-summary skill or card-grader's CLAUDE.md for the full runner mechanics. Push to main on the forgejo remote (https://git.hippofam.com/ninja_hippo/massprice.git) triggers .forgejo/workflows/deploy.yml, which syncs into /mnt/user/appdata/massprice (excluding .git, .env, data/) and runs docker compose up -d --build.

Restarting the container manually: use Compose, not the Unraid Docker UI — same reasoning as the other projects (every deploy recreates the container with a new Docker ID; Unraid's UI caches the old one). From the Unraid terminal:

cd /mnt/user/appdata/massprice && docker compose restart

docker compose up -d --force-recreate for a guaranteed-clean recreate; docker ps --filter name=massprice to check real state.

Confirming a deploy landed: the footer renders v<APP_VERSION> · <APP_VERSION_DATE> from constants near the top of app.py. Bump them every commit.

First deploy checklist

  1. Create /mnt/user/appdata/massprice on Unraid and a real .env there (copy .env.example, fill in LOGIN_PASSWORD, SECRET_KEY, HOST_PORT, TZ — and optionally GMAIL_USER/GMAIL_APP_PASSWORD/ ALERT_EMAIL for the session-expiry email alert, same Gmail-app- password pattern as lunch-notifier).
  2. First docker compose up -d --build — confirm the container starts and /login responds on HOST_PORT.
  3. Log into the webui, go to the Session page, and follow its instructions: log into massprice.tcgplayer.com in your own regular Chrome, export its cookies with the Cookie-Editor extension, paste them into the box, and click "Import Session". MassPrice's login is behind hCaptcha, which blocks any form of automated login (confirmed by testing — see CLAUDE.md), so this manual export is the only path in, and needs repeating whenever the session eventually expires.
  4. Click "Refresh Rules" on the Dashboard — confirms the saved session actually works and pulls the real rule list.
  5. Set up at least one schedule and use "Run Now" once to confirm a click actually lands — cross-check by looking at MassPrice's own Activity Log (https://massprice.tcgplayer.com/Activity/List) for a fresh timestamp on that rule.
  6. Only after step 5 succeeds, go to each existing rule on massprice.tcgplayer.com (three-dot menu → Edit Schedule) and set Status to Inactive, so native scheduling stops double-triggering runs and this tool becomes the sole source of scheduled runs.

Persistent state, never touched by deploys

.env holds LOGIN_PASSWORD, SECRET_KEY, and (optionally) GMAIL_USER/GMAIL_APP_PASSWORD/ALERT_EMAIL. data/ (bind-mounted to /data) holds schedules.json (the schedules set from the webui), auth_state.json (the saved MassPrice session imported via the Session page — a live bearer credential for the user's MassPrice account while valid, see CLAUDE.md; losing this just costs one re-import, not data), run_history.json, rules_cache.json, session_state.json (the "session is known-bad" flag behind the dashboard banner), and notify_state.json (the session-expiry alert's send-once dedupe flag). Both are excluded from the deploy rsync (which runs --delete) and from git.

If scheduled runs stop working

Check, in order: 0. Local Activity says success but nothing shows up in MassPrice's own Activity Log — this exact symptom happened in production once already: run_rule() was only opening MassPrice's "Are you sure?" confirmation modal, never clicking its #Confirmation_Run button, so zero requests ever fired despite no error being thrown. Fixed, but if it recurs (e.g. a MassPrice redesign changes the modal/selectors), this is the first thing to suspect — see CLAUDE.md.

  1. Dashboard banner / Activity page — a "session expired or missing" message means go log in again via the Session page. "Rule id … not found on /Rule/List" means that rule was deleted on MassPrice (a rename can't cause it — schedules are keyed by the stable rule id): click "Refresh Rules", which re-syncs schedules.json against the live list, drops schedules for deleted rules, and updates cached names for renamed ones. Renames matter because run verification matches the Activity page by name — a stale one shows every run as "Not Confirmed" while it's actually succeeding, so refresh after renaming anything.
  2. Is the container still running — docker ps --filter name=massprice — a Chromium crash inside the container (rare, but possible if shm_size ever gets dropped from docker-compose.yml) would show as the container restarting.
  3. Timezone — TZ in .env controls what "6:00 AM" means for every schedule; confirm it matches what the user actually expects before assuming a schedule itself is wrong.
  4. Session import fails with "didn't authenticate" — the export was probably taken from the wrong tab or before login finished. Re-export only after you're actually looking at massprice.tcgplayer.com's own Dashboard/Rules pages, not the login page or store.tcgplayer.com.
  5. Session import fails with "none of the pasted cookies are for tcgplayer.com" — the extension exported the wrong site's cookies; make sure the active tab was massprice.tcgplayer.com when you clicked Export.