Replace credential auto-login with a Chrome-remote-debugging session capture flow
All checks were successful
Deploy / deploy (push) Successful in 38s

Live testing against the real site found MassPrice's login is behind
hCaptcha, which silently blocks fully automated username/password
login outright (the form never submits) — no selector fix could work
around this. Also fixed the broken _is_logged_out() heuristic: an
unauthenticated visit to /Rule/List redirects to the public marketing
homepage, not a /login URL, so run_rule/list_rules were silently
scraping the wrong page instead of erroring.

New flow: a Session page lets the user start a headless-but-CDP-exposed
Chromium instance, log in themselves via chrome://inspect (solving the
captcha as a real click), then capture that browser's cookies into the
same auth_state.json scheduled runs already use. Required a socat proxy
in the container image, since Chromium's debug port binds loopback-only
regardless of --remote-debugging-address (confirmed by live testing),
unreachable through Docker's port mapping without it. All Playwright
objects for this now live on one dedicated thread (automation.py's
_debug_worker), since the sync API isn't safe to call across the
arbitrary threads gunicorn's gthread workers hand out per request.

Dashboard now shows a banner when the latest run failed specifically
due to a missing/expired session, rather than just looking like an
empty rule list.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H6TgffxTvYhqxykAbVQg2v
This commit is contained in:
Barely Removable 2026-08-31 19:55:08 -07:00
parent cb027167a7
commit bec399dd11
12 changed files with 473 additions and 121 deletions

View file

@ -1,6 +1,6 @@
--- ---
name: deploy-massprice name: deploy-massprice
description: Use when deploying MassPrice Scheduler, restarting the massprice container on Unraid, confirming a deploy landed, or troubleshooting its scheduled-rule automation. description: Use when deploying MassPrice Scheduler, restarting the massprice container on Unraid, confirming a deploy landed, logging into MassPrice via the Session page, or troubleshooting its scheduled-rule automation.
--- ---
## Deployment ## Deployment
@ -9,15 +9,14 @@ description: Use when deploying MassPrice Scheduler, restarting the massprice co
built from `Dockerfile`/`docker-compose.yml` in this repo. Container name built from `Dockerfile`/`docker-compose.yml` in this repo. Container name
`massprice`, port `5057` (overridable via `HOST_PORT` — `5056` was `massprice`, port `5057` (overridable via `HOST_PORT` — `5056` was
already taken by `arr-summary`, despite its own docs saying `5055`) → already taken by `arr-summary`, despite its own docs saying `5055`) →
container port container port `5000`. Also exposes `DEBUG_PORT` (default `9222`) for the
`5000`. Chrome-remote-debugging login flow — see below.
**Deploy mechanism: Forgejo push-to-deploy**, the same pattern as **Deploy mechanism: Forgejo push-to-deploy**, the same pattern as
`arr-summary`, `card-grader`, `lunch-notifier`, `n64-tracker` — see `arr-summary`, `card-grader`, `lunch-notifier`, `n64-tracker` — see
`arr-summary`'s `deploy-arr-summary` skill or `card-grader`'s `CLAUDE.md` `arr-summary`'s `deploy-arr-summary` skill or `card-grader`'s `CLAUDE.md`
for the full runner mechanics. Push to `main` on the `forgejo` remote for the full runner mechanics. Push to `main` on the `forgejo` remote
(`https://git.hippofam.com/ninja_hippo/massprice.git` — create this repo (`https://git.hippofam.com/ninja_hippo/massprice.git`) triggers
on the Forgejo instance first if it doesn't exist yet) triggers
`.forgejo/workflows/deploy.yml`, which syncs into `.forgejo/workflows/deploy.yml`, which syncs into
`/mnt/user/appdata/massprice` (excluding `.git`, `.env`, `data/`) and `/mnt/user/appdata/massprice` (excluding `.git`, `.env`, `data/`) and
runs `docker compose up -d --build`. runs `docker compose up -d --build`.
@ -40,41 +39,41 @@ cd /mnt/user/appdata/massprice && docker compose restart
1. Create `/mnt/user/appdata/massprice` on Unraid and a real `.env` 1. Create `/mnt/user/appdata/massprice` on Unraid and a real `.env`
there (copy `.env.example`, fill in `LOGIN_PASSWORD`, `SECRET_KEY`, there (copy `.env.example`, fill in `LOGIN_PASSWORD`, `SECRET_KEY`,
`MASSPRICE_USERNAME`, `MASSPRICE_PASSWORD`, `TZ`). `HOST_PORT`, `DEBUG_PORT`, `TZ`).
2. First `docker compose up -d --build` — confirm the container starts 2. First `docker compose up -d --build` — confirm the container starts
and `/login` responds on `HOST_PORT`. and `/login` responds on `HOST_PORT`.
3. Log into the webui, click "Refresh Rules" — confirms Playwright can 3. Log into the webui, go to the **Session** page, and follow its
reach massprice.tcgplayer.com and log in automatically. **If this instructions: click "Start Login Session", connect your own Chrome to
fails**, check `data/run_history.json`/container logs for the login `<unraid-ip>:<DEBUG_PORT>` via `chrome://inspect`, log in (solving the
error — the login form locators in `automation._login()` were written hCaptcha yourself — this cannot be automated, see `CLAUDE.md`), then
generically (never verified against the real logged-out form, see click "Capture Session" back in the webui.
`CLAUDE.md`) and may need adjusting to match TCGplayer's actual field 4. Click "Refresh Rules" on the Dashboard — confirms the saved session
names/labels. actually works and pulls the real rule list.
4. Set up at least one schedule and use "Run Now" once to confirm a 5. Set up at least one schedule and use "Run Now" once to confirm a
click actually lands — cross-check by looking at MassPrice's own click actually lands — cross-check by looking at MassPrice's own
Activity Log (`https://massprice.tcgplayer.com/Activity/List`) for a Activity Log (`https://massprice.tcgplayer.com/Activity/List`) for a
fresh timestamp on that rule. fresh timestamp on that rule.
5. **Only after step 4 succeeds**, go to each of the 5 existing rules on 6. **Only after step 5 succeeds**, go to each existing rule on
massprice.tcgplayer.com (three-dot menu → Edit Schedule) and set massprice.tcgplayer.com (three-dot menu → Edit Schedule) and set
Status to Inactive, so native scheduling stops double-triggering runs Status to Inactive, so native scheduling stops double-triggering runs
and this tool becomes the sole source of scheduled runs. and this tool becomes the sole source of scheduled runs.
## Persistent state, never touched by deploys ## Persistent state, never touched by deploys
`.env` holds `LOGIN_PASSWORD`, `SECRET_KEY`, and the MassPrice `.env` holds `LOGIN_PASSWORD` and `SECRET_KEY`. `data/` (bind-mounted to
credentials. `data/` (bind-mounted to `/data`) holds `schedules.json` `/data`) holds `schedules.json` (the schedules set from the webui),
(the schedules set from the webui), `auth_state.json` (the saved `auth_state.json` (the saved MassPrice browser session captured via the
MassPrice browser session — losing this just costs one re-login, not Session page — losing this just costs one re-login, not data),
data), `run_history.json`, and `rules_cache.json`. Both are excluded `run_history.json`, and `rules_cache.json`. Both are excluded from the
from the deploy rsync (which runs `--delete`) and from git. deploy rsync (which runs `--delete`) and from git.
## If scheduled runs stop working ## If scheduled runs stop working
Check, in order: Check, in order:
1. **Activity page in the webui** — does the failure message say 1. **Dashboard banner / Activity page** — a "session expired or missing"
"still on a login page" (credentials/CAPTCHA problem) or "rule id not message means go log in again via the Session page. "rule id not
found" (a rule was renamed/deleted on MassPrice — click "Refresh found" means a rule was renamed/deleted on MassPrice — click "Refresh
Rules")? Rules".
2. **Is the container still running** — `docker ps --filter 2. **Is the container still running** — `docker ps --filter
name=massprice` — a Chromium crash inside the container (rare, but name=massprice` — a Chromium crash inside the container (rare, but
possible if `shm_size` ever gets dropped from `docker-compose.yml`) possible if `shm_size` ever gets dropped from `docker-compose.yml`)
@ -82,3 +81,8 @@ Check, in order:
3. **Timezone** — `TZ` in `.env` controls what "6:00 AM" means for every 3. **Timezone** — `TZ` in `.env` controls what "6:00 AM" means for every
schedule; confirm it matches what the user actually expects before schedule; confirm it matches what the user actually expects before
assuming a schedule itself is wrong. assuming a schedule itself is wrong.
4. **Session capture stuck at "Still not logged in"** — the debug
session's page needs to actually be on `/Rule/List` or somewhere under
`/Rule` after login for the capture check to pass; if the inspected
tab is still sitting on the marketing homepage or the OAuth login
page, finish clicking through to the dashboard/rules first.

View file

@ -10,17 +10,16 @@ LOGIN_PASSWORD=changeme
# python3 -c "import secrets; print(secrets.token_hex(32))" # python3 -c "import secrets; print(secrets.token_hex(32))"
SECRET_KEY=change_this_to_a_random_secret_key SECRET_KEY=change_this_to_a_random_secret_key
# ── MassPrice (TCGplayer) login ─────────────────────────────────────
# Used by Playwright to log in automatically the first time, and again
# any time the saved session expires. Stored here only — gitignored,
# never committed, never logged.
MASSPRICE_USERNAME=your_tcgplayer_email@example.com
MASSPRICE_PASSWORD=your_tcgplayer_password
# ── Host port (what port Unraid exposes) ──────────────────────────── # ── Host port (what port Unraid exposes) ────────────────────────────
# Default is 5057 (arr-summary already uses 5056) — change if in use # Default is 5057 (arr-summary already uses 5056) — change if in use
HOST_PORT=5057 HOST_PORT=5057
# ── Debug port (Chrome remote debugging, for logging in) ───────────
# MassPrice's login is behind hCaptcha, so there's no way to fully
# automate signing in. Log in via the webui's Session page instead,
# which connects your own Chrome (chrome://inspect) to this port.
DEBUG_PORT=9222
# ── Timezone ───────────────────────────────────────────────────────── # ── Timezone ─────────────────────────────────────────────────────────
# Controls what "6:00 AM" means for your schedules # Controls what "6:00 AM" means for your schedules
TZ=America/New_York TZ=America/New_York

View file

@ -52,11 +52,12 @@ tool is confirmed working, so there's exactly one thing triggering runs.
the real page rather than replaying a raw HTTP POST. the real page rather than replaying a raw HTTP POST.
- **Session persistence** — a Playwright `storage_state` (cookies + - **Session persistence** — a Playwright `storage_state` (cookies +
localStorage) is saved to `/data/auth_state.json` and reused across localStorage) is saved to `/data/auth_state.json` and reused across
runs: the tool logs in **once**, then slides the session forward after runs: scheduled runs never touch the login form at all, they just load
every successful run, rather than hitting the login form on every this saved session. When it's missing or has expired (detected by an
scheduled click. A fresh login (via `MASSPRICE_USERNAME` / unauthenticated visit to `/Rule/List` silently redirecting to the
`MASSPRICE_PASSWORD`) only happens when the saved state is missing or public marketing homepage `/` — there's no `/login` URL to check
has expired (detected by landing back on a login page). against), the run fails with a clear "session expired or missing"
error rather than attempting to log in — see below for why.
- **`storage.py`** — tiny JSON read/write helpers (temp-file + - **`storage.py`** — tiny JSON read/write helpers (temp-file +
`os.replace`, so a crash mid-write can't corrupt state) for `os.replace`, so a crash mid-write can't corrupt state) for
everything under `/data`: `schedules.json`, `auth_state.json`, everything under `/data`: `schedules.json`, `auth_state.json`,
@ -66,24 +67,47 @@ tool is confirmed working, so there's exactly one thing triggering runs.
teletext system — that design language is specific to that app, not a teletext system — that design language is specific to that app, not a
house style to reuse here. house style to reuse here.
## Known limitation — automated login ## Login — why it's manual, and how it works
`MASSPRICE_USERNAME`/`MASSPRICE_PASSWORD` drive a fully automated login MassPrice's login (`store.tcgplayer.com/oauth/login`, TCGplayer's shared
(the user's explicit choice over a manual hand-off flow). **If OAuth identity provider) is protected by **hCaptcha**. This was
TCGplayer's login ever presents a CAPTCHA or a 2FA challenge, automated confirmed by live testing during development: filling in the real
login will fail outright** — there is deliberately no fallback UI for username/password and clicking Submit never navigates anywhere — the
this yet; a failed login just shows up as a `failure` entry in Activity invisible captcha check silently never resolves for a headless/automated
with the error message. If this starts happening routinely, it needs a browser, and the form just refuses to submit. There is no selector fix
design change (e.g. a Chrome-remote-debugging-based manual re-auth flow), for this; **fully automated login is not achievable here**, full stop.
not a bigger retry loop. An earlier version of this app tried `MASSPRICE_USERNAME`/
`MASSPRICE_PASSWORD` credential-based auto-login — removed once this was
confirmed, since keeping dead code that can never succeed is worse than
having none.
**The real logged-out MassPrice login form was never inspected** during What actually works: a human only needs to pass the hCaptcha, not
development — the account was already signed in when the site was automate around it, and that's a one-time (well, periodic — however long
explored, and logging it out to look at the form would have disrupted the session cookie lasts) cost, not a per-run one. The **Session** page
the user's real, live MassPrice session. `automation._login()` uses in the webui drives this:
resilient role/label/placeholder-based Playwright locators rather than 1. "Start Login Session" launches headless Chromium with its CDP debug
brittle CSS, but **expect to need a one-time adjustment against the real port exposed (`DEBUG_PORT`, default `9222`).
form on first deploy** if the generic locators don't match. 2. The user connects their own Chrome to it via `chrome://inspect`,
which renders and accepts real input against that same running
browser instance — no VNC/Xvfb needed, headless Chromium's debug
protocol already supports this.
3. The user logs in normally in that inspected tab, solving the hCaptcha
themselves since it's a real click from a real browser.
4. "Capture Session" saves that browser's cookies to
`/data/auth_state.json` — the same file scheduled runs read.
**All of this lives on one dedicated thread** (`automation._debug_worker`),
not arbitrary Flask request threads — Playwright's sync API isn't safe
to call across threads, and gunicorn's `gthread` workers hand each
request to a different one. Flask routes only ever put a command on
`_debug_cmd_queue` and block for the result; they never touch the
Playwright browser/context/page objects directly. An un-captured debug
session auto-closes after 10 minutes so an open, unauthenticated CDP
port doesn't sit exposed on the LAN indefinitely.
The Dashboard shows a red banner (`_session_warning()` in `app.py`) when
the most recent run/refresh failed specifically because of a missing/
expired session, so this doesn't just look like "no rules found."
## Local development ## Local development
@ -93,13 +117,13 @@ playwright install chromium # only needed if testing automation.py directly
python3 app.py python3 app.py
``` ```
Runs on port 5000 (`PORT` env to change). Without `MASSPRICE_USERNAME`/ Runs on port 5000 (`PORT` env to change). Without Chromium installed and
`MASSPRICE_PASSWORD` set and without Chromium installed, the Dashboard without a saved `auth_state.json`, the Dashboard still boots —
still boots — `_bootstrap()` in `app.py` best-effort-catches the initial `_bootstrap()` in `app.py` best-effort-catches the initial rule sync, so
rule sync, so local UI/schedule-editing work doesn't require Docker or local UI/schedule-editing work doesn't require Docker or a real MassPrice
real MassPrice credentials at all. `DATA_DIR` env var overrides where session at all. `DATA_DIR` env var overrides where JSON state is
JSON state is read/written (defaults to `/data`, which won't exist read/written (defaults to `/data`, which won't exist locally — point it
locally — point it at a scratch directory). at a scratch directory).
**Every commit should be matched with a version bump** — **Every commit should be matched with a version bump** —
`APP_VERSION`/`APP_VERSION_DATE` near the top of `app.py` — the same `APP_VERSION`/`APP_VERSION_DATE` near the top of `app.py` — the same
@ -112,6 +136,6 @@ pipeline and manual container restart steps.
**Never put real credentials in the repo.** `.env` and `data/` are **Never put real credentials in the repo.** `.env` and `data/` are
gitignored and excluded from the deploy rsync precisely so gitignored and excluded from the deploy rsync precisely so
`MASSPRICE_USERNAME`/`MASSPRICE_PASSWORD`/`LOGIN_PASSWORD` and the saved `LOGIN_PASSWORD`/`SECRET_KEY` and the saved
MassPrice session (`data/auth_state.json`) have somewhere to live MassPrice session (`data/auth_state.json`) have somewhere to live
outside git. outside git.

View file

@ -5,6 +5,13 @@ LABEL maintainer="MassPrice Scheduler"
WORKDIR /app WORKDIR /app
# socat proxies the login-session debug port: Chromium's remote-debugging
# port binds to 127.0.0.1 only, regardless of --remote-debugging-address
# (deliberate Chromium security hardening, confirmed by testing) — a
# loopback-only listener is unreachable through Docker's port mapping,
# so socat re-exposes it on all interfaces. See automation.py.
RUN apt-get update -qq && apt-get install -y -qq socat && rm -rf /var/lib/apt/lists/*
COPY requirements.txt . COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt RUN pip install --no-cache-dir -r requirements.txt

59
app.py
View file

@ -8,8 +8,8 @@ import automation
import scheduler import scheduler
import storage import storage
APP_VERSION = "0.1.0" APP_VERSION = "0.2.0"
APP_VERSION_DATE = "2026-08-31" APP_VERSION_DATE = "2026-09-01"
app = Flask(__name__) app = Flask(__name__)
app.secret_key = os.environ.get("SECRET_KEY", "dev-secret-change-me") app.secret_key = os.environ.get("SECRET_KEY", "dev-secret-change-me")
@ -81,6 +81,19 @@ def _merged_rules():
return rules, cache.get("updated_at") return rules, cache.get("updated_at")
def _session_warning():
"""True if the most recent run/refresh failed specifically because
the saved MassPrice session is missing/expired — surfaced as a
dashboard banner so a dead session doesn't just look like "no rules"."""
history = storage.load("run_history.json", [])
if not history:
return False
latest = history[0]
return latest.get("status") == "failure" and "session expired or missing" in (
latest.get("message") or ""
)
@app.route("/") @app.route("/")
@login_required @login_required
def dashboard(): def dashboard():
@ -91,9 +104,51 @@ def dashboard():
rules_updated_at=rules_updated_at, rules_updated_at=rules_updated_at,
days=DAYS, days=DAYS,
day_labels=DAY_LABELS, day_labels=DAY_LABELS,
session_warning=_session_warning(),
) )
@app.route("/session")
@login_required
def session_page():
return render_template(
"session.html",
status=automation.session_status(),
debug_active=automation.debug_session_active(),
debug_port=os.environ.get("DEBUG_PORT", "9222"),
host=request.host.split(":")[0],
)
@app.route("/api/session/start", methods=["POST"])
@login_required
def api_session_start():
try:
automation.start_debug_session()
except Exception as exc:
return jsonify({"error": str(exc)}), 502
return jsonify({"ok": True})
@app.route("/api/session/capture", methods=["POST"])
@login_required
def api_session_capture():
try:
automation.capture_debug_session()
except automation.AutomationError as exc:
return jsonify({"error": str(exc)}), 409
except Exception as exc:
return jsonify({"error": str(exc)}), 502
return jsonify({"ok": True})
@app.route("/api/session/stop", methods=["POST"])
@login_required
def api_session_stop():
automation.stop_debug_session()
return jsonify({"ok": True})
@app.route("/activity") @app.route("/activity")
@login_required @login_required
def activity(): def activity():

View file

@ -5,21 +5,32 @@ is bound to a JS/AJAX click handler with no plain href, and the request
it fires is almost certainly anti-forgery-token protected. it fires is almost certainly anti-forgery-token protected.
Session handling: a Playwright `storage_state` (cookies + localStorage) Session handling: a Playwright `storage_state` (cookies + localStorage)
is persisted to AUTH_STATE_PATH and reused across runs, so the tool logs is persisted to AUTH_STATE_PATH and reused across runs, so scheduled runs
in once and slides the session forward on every successful run rather never touch the login form at all. That's deliberate, not just an
than hitting the login form every time. A fresh login (via optimization — MassPrice's real login (on store.tcgplayer.com, TCGplayer's
MASSPRICE_USERNAME/MASSPRICE_PASSWORD) only happens when the saved state shared OAuth identity provider) is behind hCaptcha, confirmed by live
is missing or has expired. testing: filling the real username/password and submitting silently
never navigates away, because the invisible captcha check never resolves
for a headless/automated browser. **There is no way to fully automate
logging in.** A human has to solve that captcha at least once per
session lifetime, via the "Chrome remote debugging" capture flow below.
NOTE — the real logged-out login form was not inspected during Capture flow: `start_debug_session()` launches a headless-but-CDP-exposed
development (the account was already signed in during discovery, and Chromium instance (`--remote-debugging-port`), which the user connects to
forcing a logout to inspect it would have disrupted the user's real from their own Chrome via chrome://inspect to log in interactively —
MassPrice session). The locators in _login() are written to be resilient solving the hCaptcha themselves, since it's a real click. Once logged in,
(role/label/placeholder based, not brittle CSS) but may need a one-time `capture_debug_session()` saves that browser's cookies to
adjustment against the live form on first deploy. See CLAUDE.md. AUTH_STATE_PATH. All of this must happen on a single dedicated thread —
Playwright's sync API is not safe to call from arbitrary threads, and
Flask/gunicorn's gthread workers hand each request to a different one —
so `_debug_worker()` owns every Playwright object here and talks to
Flask request threads only through `_debug_cmd_queue`.
""" """
import logging import logging
import os import os
import queue
import subprocess
import threading
from datetime import datetime, timezone from datetime import datetime, timezone
from playwright.sync_api import sync_playwright from playwright.sync_api import sync_playwright
@ -31,8 +42,19 @@ logger = logging.getLogger("massprice.automation")
BASE_URL = "https://massprice.tcgplayer.com" BASE_URL = "https://massprice.tcgplayer.com"
AUTH_STATE_PATH = os.path.join(storage.DATA_DIR, "auth_state.json") AUTH_STATE_PATH = os.path.join(storage.DATA_DIR, "auth_state.json")
MASSPRICE_USERNAME = os.environ.get("MASSPRICE_USERNAME") # Chromium's remote-debugging port binds to 127.0.0.1 only inside the
MASSPRICE_PASSWORD = os.environ.get("MASSPRICE_PASSWORD") # container, no matter what --remote-debugging-address is set to —
# deliberate Chromium security hardening (confirmed by live testing, not
# a flag mistake). A loopback-only listener is unreachable through
# Docker's port mapping, so `socat` re-exposes it on all interfaces at a
# second, different internal port (they can't share one — Chromium's
# 127.0.0.1 bind and a 0.0.0.0 bind on the same port conflict). Neither
# of these is the DEBUG_PORT env var app.py reads for display — that one
# is the user-facing HOST port for chrome://inspect instructions, mapped
# in docker-compose.yml to _PROXY_PORT, not to Chromium's own port.
_INTERNAL_DEBUG_PORT = 9222
_PROXY_PORT = 9333
DEBUG_SESSION_TIMEOUT = 600 # auto-close an un-captured debug session after 10 minutes
# Standard, necessary mitigations for headless Chromium inside Docker. # Standard, necessary mitigations for headless Chromium inside Docker.
LAUNCH_ARGS = ["--no-sandbox", "--disable-dev-shm-usage"] LAUNCH_ARGS = ["--no-sandbox", "--disable-dev-shm-usage"]
@ -54,54 +76,28 @@ def _save_state(context):
def _is_logged_out(page): def _is_logged_out(page):
if "login" in page.url.lower() or "signin" in page.url.lower(): """An unauthenticated visit to any /Rule/* page silently redirects to
return True the public marketing homepage (`/`) — there's no `/login` URL and no
return page.locator('input[type="password"]').count() > 0 password field to detect, so "did we get bounced off of /Rule" is the
only reliable signal (verified by live inspection)."""
return "/Rule" not in page.url
def _login(page):
if not MASSPRICE_USERNAME or not MASSPRICE_PASSWORD:
raise AutomationError(
"MASSPRICE_USERNAME/MASSPRICE_PASSWORD not set — cannot log in"
)
page.goto(f"{BASE_URL}/Dashboard")
if not _is_logged_out(page):
return
email_field = (
page.get_by_label("Email", exact=False)
.or_(page.get_by_placeholder("Email", exact=False))
.or_(page.locator('input[type="email"]'))
.or_(page.locator('input[name="username" i]'))
)
password_field = page.locator('input[type="password"]')
email_field.first.fill(MASSPRICE_USERNAME)
password_field.first.fill(MASSPRICE_PASSWORD)
submit = (
page.get_by_role("button", name="Log In", exact=False)
.or_(page.get_by_role("button", name="Sign In", exact=False))
.or_(page.locator('button[type="submit"]'))
)
submit.first.click()
page.wait_for_load_state("networkidle")
if _is_logged_out(page):
raise AutomationError(
"Login failed — still on a login page after submit "
"(bad credentials, or a CAPTCHA/2FA challenge that automated "
"login cannot get past — see CLAUDE.md)"
)
def _ensure_authenticated(page): def _ensure_authenticated(page):
page.goto(f"{BASE_URL}/Rule/List") page.goto(f"{BASE_URL}/Rule/List")
if _is_logged_out(page): if _is_logged_out(page):
_login(page) raise AutomationError(
page.goto(f"{BASE_URL}/Rule/List") "MassPrice session expired or missing — log in via the "
if _is_logged_out(page): "Session page (chrome://inspect) and click Capture Session."
raise AutomationError("Still logged out after a login attempt") )
def session_status():
if not os.path.exists(AUTH_STATE_PATH):
return {"state": "missing", "saved_at": None}
mtime = os.path.getmtime(AUTH_STATE_PATH)
saved_at = datetime.fromtimestamp(mtime, tz=timezone.utc).isoformat()
return {"state": "saved", "saved_at": saved_at}
def list_rules(): def list_rules():
@ -200,3 +196,146 @@ def _log_history(rule_id, rule_name, status, message):
}, },
) )
storage.save("run_history.json", history[:200]) storage.save("run_history.json", history[:200])
# ── Interactive login-session capture (Chrome remote debugging) ─────────
#
# Everything below runs exclusively on `_debug_worker`'s thread. Flask
# routes call start_debug_session()/capture_debug_session()/
# stop_debug_session(), which just enqueue a command and block for the
# result — never touch the Playwright objects directly.
_debug_cmd_queue: "queue.Queue" = queue.Queue()
_debug_thread = None
_debug_active = False # read from other threads; plain bool assignment is atomic in CPython
def _debug_worker():
global _debug_active
playwright = browser = context = page = timer = socat_proc = None
def do_stop():
nonlocal playwright, browser, context, page, timer, socat_proc
global _debug_active
if timer:
timer.cancel()
timer = None
if socat_proc:
try:
socat_proc.terminate()
except Exception:
pass
socat_proc = None
if browser:
try:
browser.close()
except Exception:
pass
if playwright:
try:
playwright.stop()
except Exception:
pass
playwright = browser = context = page = None
_debug_active = False
while True:
cmd, result_q = _debug_cmd_queue.get()
try:
if cmd == "start":
if browser is None:
playwright = sync_playwright().start()
browser = playwright.chromium.launch(
headless=True,
args=LAUNCH_ARGS
+ [
f"--remote-debugging-port={_INTERNAL_DEBUG_PORT}",
"--remote-allow-origins=*",
],
)
# Re-expose Chromium's loopback-only debug port on
# all interfaces so Docker's port mapping (which
# forwards to _PROXY_PORT, not Chromium's own port —
# see the comment above) can actually reach it.
socat_proc = subprocess.Popen(
[
"socat",
f"TCP-LISTEN:{_PROXY_PORT},fork,reuseaddr",
f"TCP:127.0.0.1:{_INTERNAL_DEBUG_PORT}",
]
)
context = _new_context(browser)
page = context.new_page()
page.goto(f"{BASE_URL}/")
_debug_active = True
timer = threading.Timer(
DEBUG_SESSION_TIMEOUT, lambda: _debug_cmd_queue.put(("stop", None))
)
timer.daemon = True
timer.start()
if result_q:
result_q.put(("ok", None))
elif cmd == "capture":
if context is None:
if result_q:
result_q.put(
("error", "No active login session — click Start Login Session first")
)
else:
page.goto(f"{BASE_URL}/Rule/List")
page.wait_for_timeout(1000)
if _is_logged_out(page):
if result_q:
result_q.put(
(
"error",
"Still not logged in — finish logging in via "
"chrome://inspect, then try Capture Session again",
)
)
else:
_save_state(context)
do_stop()
if result_q:
result_q.put(("ok", None))
elif cmd == "stop":
do_stop()
if result_q:
result_q.put(("ok", None))
except Exception as exc:
logger.exception("debug session command %r failed", cmd)
if result_q:
result_q.put(("error", str(exc)))
def _ensure_debug_thread():
global _debug_thread
if _debug_thread is None or not _debug_thread.is_alive():
_debug_thread = threading.Thread(target=_debug_worker, daemon=True)
_debug_thread.start()
def _debug_call(cmd, timeout=30):
result_q: "queue.Queue" = queue.Queue()
_debug_cmd_queue.put((cmd, result_q))
status, payload = result_q.get(timeout=timeout)
if status == "error":
raise AutomationError(payload)
def start_debug_session():
_ensure_debug_thread()
_debug_call("start")
def capture_debug_session():
_debug_call("capture")
def stop_debug_session():
_ensure_debug_thread()
_debug_call("stop")
def debug_session_active():
return _debug_active

View file

@ -7,6 +7,16 @@ services:
shm_size: "1gb" shm_size: "1gb"
ports: ports:
- "${HOST_PORT:-5057}:5000" - "${HOST_PORT:-5057}:5000"
# MassPrice's login is behind hCaptcha, which blocks fully
# automated sign-in — this port reaches (via an in-container socat
# proxy, see automation.py) headless Chromium's CDP debug endpoint,
# so a human can log in via chrome://inspect from their own machine
# (see the Session page / deploy-massprice skill). Maps to 9333,
# NOT Chromium's own 9222 — Chromium's debug port binds loopback-
# only regardless of config, so socat re-exposes it on a different
# internal port. Only listens while a login session is actively
# started from the webui, and only reachable on the LAN.
- "${DEBUG_PORT:-9222}:9333"
volumes: volumes:
# Schedules, run history, the rule-name cache, and the saved # Schedules, run history, the rule-name cache, and the saved
# MassPrice login session must outlive `up -d --build`, which # MassPrice login session must outlive `up -d --build`, which
@ -15,7 +25,6 @@ services:
environment: environment:
- LOGIN_PASSWORD=${LOGIN_PASSWORD} - LOGIN_PASSWORD=${LOGIN_PASSWORD}
- SECRET_KEY=${SECRET_KEY} - SECRET_KEY=${SECRET_KEY}
- MASSPRICE_USERNAME=${MASSPRICE_USERNAME} - DEBUG_PORT=${DEBUG_PORT:-9222}
- MASSPRICE_PASSWORD=${MASSPRICE_PASSWORD}
- TZ=${TZ:-America/New_York} - TZ=${TZ:-America/New_York}
restart: unless-stopped restart: unless-stopped

View file

@ -154,6 +154,33 @@ footer {
.empty-state { color: var(--text-dim); } .empty-state { color: var(--text-dim); }
.banner {
border-radius: var(--radius);
padding: 12px 16px;
margin-bottom: 20px;
font-size: 0.88rem;
}
.banner-warn {
background: rgba(248, 113, 113, 0.12);
border: 1px solid var(--failure);
color: #f2c2c2;
}
.banner a { color: var(--text); }
.panel {
background: var(--panel);
border: 1px solid var(--border);
border-radius: var(--radius);
padding: 20px;
margin-bottom: 16px;
}
.panel h2 { font-size: 1rem; margin: 0 0 12px; }
.panel-narrow { max-width: 560px; }
.session-steps { padding-left: 20px; display: flex; flex-direction: column; gap: 6px; font-size: 0.88rem; }
.session-steps code { background: var(--panel-alt); border: 1px solid var(--border); border-radius: 4px; padding: 1px 6px; }
.session-actions { display: flex; gap: 8px; margin-top: 16px; }
.activity-table { width: 100%; border-collapse: collapse; font-size: 0.85rem; } .activity-table { width: 100%; border-collapse: collapse; font-size: 0.85rem; }
.activity-table th, .activity-table td { text-align: left; padding: 8px 12px; border-bottom: 1px solid var(--border); } .activity-table th, .activity-table td { text-align: left; padding: 8px 12px; border-bottom: 1px solid var(--border); }
.activity-table th { color: var(--text-dim); font-weight: 500; font-size: 0.75rem; text-transform: uppercase; } .activity-table th { color: var(--text-dim); font-weight: 500; font-size: 0.75rem; text-transform: uppercase; }

View file

@ -21,8 +21,40 @@ document.addEventListener("DOMContentLoaded", () => {
} }
document.querySelectorAll(".rule-card").forEach(initRuleCard); document.querySelectorAll(".rule-card").forEach(initRuleCard);
initSessionControls();
}); });
function initSessionControls() {
const startBtn = document.getElementById("session-start");
const captureBtn = document.getElementById("session-capture");
const stopBtn = document.getElementById("session-stop");
if (!startBtn) return; // not on the Session page
async function call(url, btn) {
btn.disabled = true;
const original = btn.textContent;
btn.textContent = "Working…";
try {
const res = await fetch(url, { method: "POST" });
const data = await res.json().catch(() => ({}));
if (!res.ok) {
alert(data.error || "Request failed");
btn.disabled = false;
btn.textContent = original;
return;
}
window.location.reload();
} catch (err) {
btn.disabled = false;
btn.textContent = original;
}
}
startBtn.addEventListener("click", () => call("/api/session/start", startBtn));
captureBtn.addEventListener("click", () => call("/api/session/capture", captureBtn));
stopBtn.addEventListener("click", () => call("/api/session/stop", stopBtn));
}
function initRuleCard(card) { function initRuleCard(card) {
const ruleId = card.dataset.ruleId; const ruleId = card.dataset.ruleId;

View file

@ -13,6 +13,7 @@
<nav> <nav>
<a href="{{ url_for('dashboard') }}" class="{{ 'active' if request.endpoint == 'dashboard' else '' }}">Dashboard</a> <a href="{{ url_for('dashboard') }}" class="{{ 'active' if request.endpoint == 'dashboard' else '' }}">Dashboard</a>
<a href="{{ url_for('activity') }}" class="{{ 'active' if request.endpoint == 'activity' else '' }}">Activity</a> <a href="{{ url_for('activity') }}" class="{{ 'active' if request.endpoint == 'activity' else '' }}">Activity</a>
<a href="{{ url_for('session_page') }}" class="{{ 'active' if request.endpoint == 'session_page' else '' }}">Session</a>
<a href="{{ url_for('logout') }}">Log Out</a> <a href="{{ url_for('logout') }}">Log Out</a>
</nav> </nav>
{% endif %} {% endif %}

View file

@ -9,6 +9,13 @@
</div> </div>
</div> </div>
{% if session_warning %}
<div class="banner banner-warn">
Your MassPrice login session has expired or was never captured — scheduled runs and rule
refreshes will fail until you log in again. <a href="{{ url_for('session_page') }}">Go to the Session page</a>.
</div>
{% endif %}
{% if not rules %} {% if not rules %}
<p class="empty-state">No rules loaded yet. Click "Refresh Rules" to pull the current rule list from MassPrice.</p> <p class="empty-state">No rules loaded yet. Click "Refresh Rules" to pull the current rule list from MassPrice.</p>
{% endif %} {% endif %}

48
templates/session.html Normal file
View file

@ -0,0 +1,48 @@
{% extends "_shell.html" %}
{% block title %}Session &middot; MassPrice Scheduler{% endblock %}
{% block content %}
<h1>MassPrice Login Session</h1>
<div class="panel panel-narrow">
<h2>Status</h2>
{% if status.state == 'saved' %}
<p><span class="status-dot status-success"></span>Session saved {{ status.saved_at }}</p>
<p class="muted small">If scheduled runs start failing with a "session expired" error, come back here and log in again.</p>
{% else %}
<p><span class="status-dot status-failure"></span>No session saved yet &mdash; scheduled runs will fail until you log in.</p>
{% endif %}
</div>
<div class="panel panel-narrow">
<h2>Log in</h2>
<p class="muted small">
MassPrice's login is protected by hCaptcha, which blocks fully automated
sign-in. You need to log in yourself, at least once per session lifetime,
using your own Chrome browser connected to a debug browser running in
this container.
</p>
<ol class="session-steps">
<li>Click <strong>Start Login Session</strong> below.</li>
<li>On your own computer, open Chrome and go to <code>chrome://inspect/#devices</code>.</li>
<li>Under "Discover network targets", click <strong>Configure&hellip;</strong> and add
<code>{{ host }}:{{ debug_port }}</code>, then close the dialog.</li>
<li>A MassPrice tab should appear under "Remote Target" &mdash; click
<strong>inspect</strong> to open it, then log in normally (including solving the
hCaptcha challenge yourself).</li>
<li>Once you're looking at the MassPrice dashboard or rules list in that inspected
tab, come back here and click <strong>Capture Session</strong>.</li>
</ol>
<div class="session-actions">
<button id="session-start" type="button" class="btn btn-primary" {{ 'disabled' if debug_active else '' }}>Start Login Session</button>
<button id="session-capture" type="button" class="btn" {{ 'disabled' if not debug_active else '' }}>Capture Session</button>
<button id="session-stop" type="button" class="btn" {{ 'disabled' if not debug_active else '' }}>Stop</button>
</div>
{% if debug_active %}
<p class="muted small">Login session is active &mdash; it auto-closes after 10 minutes if not captured.</p>
{% endif %}
</div>
{% endblock %}
{% block scripts %}
<script src="{{ url_for('static', filename='js/app.js') }}"></script>
{% endblock %}