Replace Chrome-remote-debugging login capture with a cookie-paste import flow
All checks were successful
Deploy / deploy (push) Successful in 25s
All checks were successful
Deploy / deploy (push) Successful in 25s
Live testing showed the CDP approach still fails hCaptcha even with a real human clicking through chrome://inspect: the headless Chromium's own fingerprint (HeadlessChrome user-agent, navigator.webdriver=true) is what gets flagged, independent of who's driving it. Trying to disguise that browser to get past bot detection would be captcha evasion, not a session-handling technique, so that path is dropped entirely rather than patched further. New flow: log into MassPrice with an ordinary, non-automated browser (hCaptcha passes normally), export the session's cookies with a extension, and paste them into the Session page. import_cookies() converts the export into Playwright's storage_state shape and verifies it actually authenticates against the live site before saving it, so a bad paste never clobbers a working session. This also removes the now-unnecessary socat proxy / debug port exposure — Chromium's own CDP port was only ever needed for the dropped approach. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H6TgffxTvYhqxykAbVQg2v
This commit is contained in:
parent
bec399dd11
commit
48d8bac332
10 changed files with 214 additions and 311 deletions
|
|
@ -9,8 +9,7 @@ description: Use when deploying MassPrice Scheduler, restarting the massprice co
|
|||
built from `Dockerfile`/`docker-compose.yml` in this repo. Container name
|
||||
`massprice`, port `5057` (overridable via `HOST_PORT` — `5056` was
|
||||
already taken by `arr-summary`, despite its own docs saying `5055`) →
|
||||
container port `5000`. Also exposes `DEBUG_PORT` (default `9222`) for the
|
||||
Chrome-remote-debugging login flow — see below.
|
||||
container port `5000`.
|
||||
|
||||
**Deploy mechanism: Forgejo push-to-deploy**, the same pattern as
|
||||
`arr-summary`, `card-grader`, `lunch-notifier`, `n64-tracker` — see
|
||||
|
|
@ -39,14 +38,16 @@ cd /mnt/user/appdata/massprice && docker compose restart
|
|||
|
||||
1. Create `/mnt/user/appdata/massprice` on Unraid and a real `.env`
|
||||
there (copy `.env.example`, fill in `LOGIN_PASSWORD`, `SECRET_KEY`,
|
||||
`HOST_PORT`, `DEBUG_PORT`, `TZ`).
|
||||
`HOST_PORT`, `TZ`).
|
||||
2. First `docker compose up -d --build` — confirm the container starts
|
||||
and `/login` responds on `HOST_PORT`.
|
||||
3. Log into the webui, go to the **Session** page, and follow its
|
||||
instructions: click "Start Login Session", connect your own Chrome to
|
||||
`<unraid-ip>:<DEBUG_PORT>` via `chrome://inspect`, log in (solving the
|
||||
hCaptcha yourself — this cannot be automated, see `CLAUDE.md`), then
|
||||
click "Capture Session" back in the webui.
|
||||
instructions: log into massprice.tcgplayer.com in your own regular
|
||||
Chrome, export its cookies with the Cookie-Editor extension, paste
|
||||
them into the box, and click "Import Session". MassPrice's login is
|
||||
behind hCaptcha, which blocks any form of automated login (confirmed
|
||||
by testing — see `CLAUDE.md`), so this manual export is the only path
|
||||
in, and needs repeating whenever the session eventually expires.
|
||||
4. Click "Refresh Rules" on the Dashboard — confirms the saved session
|
||||
actually works and pulls the real rule list.
|
||||
5. Set up at least one schedule and use "Run Now" once to confirm a
|
||||
|
|
@ -62,8 +63,9 @@ cd /mnt/user/appdata/massprice && docker compose restart
|
|||
|
||||
`.env` holds `LOGIN_PASSWORD` and `SECRET_KEY`. `data/` (bind-mounted to
|
||||
`/data`) holds `schedules.json` (the schedules set from the webui),
|
||||
`auth_state.json` (the saved MassPrice browser session captured via the
|
||||
Session page — losing this just costs one re-login, not data),
|
||||
`auth_state.json` (the saved MassPrice session imported via the Session
|
||||
page — a live bearer credential for the user's MassPrice account while
|
||||
valid, see `CLAUDE.md`; losing this just costs one re-import, not data),
|
||||
`run_history.json`, and `rules_cache.json`. Both are excluded from the
|
||||
deploy rsync (which runs `--delete`) and from git.
|
||||
|
||||
|
|
@ -81,8 +83,11 @@ Check, in order:
|
|||
3. **Timezone** — `TZ` in `.env` controls what "6:00 AM" means for every
|
||||
schedule; confirm it matches what the user actually expects before
|
||||
assuming a schedule itself is wrong.
|
||||
4. **Session capture stuck at "Still not logged in"** — the debug
|
||||
session's page needs to actually be on `/Rule/List` or somewhere under
|
||||
`/Rule` after login for the capture check to pass; if the inspected
|
||||
tab is still sitting on the marketing homepage or the OAuth login
|
||||
page, finish clicking through to the dashboard/rules first.
|
||||
4. **Session import fails with "didn't authenticate"** — the export was
|
||||
probably taken from the wrong tab or before login finished. Re-export
|
||||
only after you're actually looking at massprice.tcgplayer.com's own
|
||||
Dashboard/Rules pages, not the login page or `store.tcgplayer.com`.
|
||||
5. **Session import fails with "none of the pasted cookies are for
|
||||
tcgplayer.com"** — the extension exported the wrong site's cookies;
|
||||
make sure the active tab was massprice.tcgplayer.com when you clicked
|
||||
Export.
|
||||
|
|
|
|||
|
|
@ -14,12 +14,6 @@ SECRET_KEY=change_this_to_a_random_secret_key
|
|||
# Default is 5057 (arr-summary already uses 5056) — change if in use
|
||||
HOST_PORT=5057
|
||||
|
||||
# ── Debug port (Chrome remote debugging, for logging in) ───────────
|
||||
# MassPrice's login is behind hCaptcha, so there's no way to fully
|
||||
# automate signing in. Log in via the webui's Session page instead,
|
||||
# which connects your own Chrome (chrome://inspect) to this port.
|
||||
DEBUG_PORT=9222
|
||||
|
||||
# ── Timezone ─────────────────────────────────────────────────────────
|
||||
# Controls what "6:00 AM" means for your schedules
|
||||
TZ=America/New_York
|
||||
|
|
|
|||
74
CLAUDE.md
74
CLAUDE.md
|
|
@ -70,40 +70,50 @@ tool is confirmed working, so there's exactly one thing triggering runs.
|
|||
## Login — why it's manual, and how it works
|
||||
|
||||
MassPrice's login (`store.tcgplayer.com/oauth/login`, TCGplayer's shared
|
||||
OAuth identity provider) is protected by **hCaptcha**. This was
|
||||
confirmed by live testing during development: filling in the real
|
||||
username/password and clicking Submit never navigates anywhere — the
|
||||
invisible captcha check silently never resolves for a headless/automated
|
||||
browser, and the form just refuses to submit. There is no selector fix
|
||||
for this; **fully automated login is not achievable here**, full stop.
|
||||
An earlier version of this app tried `MASSPRICE_USERNAME`/
|
||||
`MASSPRICE_PASSWORD` credential-based auto-login — removed once this was
|
||||
confirmed, since keeping dead code that can never succeed is worse than
|
||||
having none.
|
||||
OAuth identity provider) is protected by **hCaptcha**. Two automated
|
||||
approaches were tried and confirmed broken by live testing during
|
||||
development — there is no selector fix for either, **fully automated
|
||||
login is not achievable here**, full stop:
|
||||
1. Filling in the real username/password and clicking Submit — the
|
||||
invisible captcha check silently never resolves for a headless/
|
||||
automated browser, and the form just refuses to submit. An earlier
|
||||
version of this app tried `MASSPRICE_USERNAME`/`MASSPRICE_PASSWORD`
|
||||
this way; removed once confirmed broken.
|
||||
2. Driving a real human's clicks through a headless Chromium exposed via
|
||||
Chrome remote debugging (`chrome://inspect`) — also removed after
|
||||
testing. Still fails hCaptcha, because the browser itself (a
|
||||
`HeadlessChrome` user-agent, `navigator.webdriver = true`) is what
|
||||
gets flagged, independent of who's actually clicking. Deliberately
|
||||
not pursued further than that one test: disguising that browser to
|
||||
get past hCaptcha's bot detection would be captcha evasion, not a
|
||||
session-handling technique, regardless of how legitimate running
|
||||
automation on your own account is.
|
||||
|
||||
What actually works: a human only needs to pass the hCaptcha, not
|
||||
automate around it, and that's a one-time (well, periodic — however long
|
||||
the session cookie lasts) cost, not a per-run one. The **Session** page
|
||||
in the webui drives this:
|
||||
1. "Start Login Session" launches headless Chromium with its CDP debug
|
||||
port exposed (`DEBUG_PORT`, default `9222`).
|
||||
2. The user connects their own Chrome to it via `chrome://inspect`,
|
||||
which renders and accepts real input against that same running
|
||||
browser instance — no VNC/Xvfb needed, headless Chromium's debug
|
||||
protocol already supports this.
|
||||
3. The user logs in normally in that inspected tab, solving the hCaptcha
|
||||
themselves since it's a real click from a real browser.
|
||||
4. "Capture Session" saves that browser's cookies to
|
||||
`/data/auth_state.json` — the same file scheduled runs read.
|
||||
What actually works: log in with an ordinary, non-automated browser —
|
||||
hCaptcha passes the way it always does for a real person, because
|
||||
nothing about that browser looks automated — then copy that session's
|
||||
cookies over. The **Session** page in the webui drives this:
|
||||
1. The user logs into massprice.tcgplayer.com in their own regular
|
||||
Chrome and exports its cookies with a browser extension (Cookie-Editor,
|
||||
"Export as JSON").
|
||||
2. They paste that JSON into the Session page and click "Import Session".
|
||||
3. `automation.import_cookies()` parses it, converts each cookie from
|
||||
Cookie-Editor's shape into Playwright's `storage_state` cookie shape
|
||||
(`_convert_cookie()` — mainly the `sameSite` enum values and the
|
||||
`expirationDate` → `expires` field rename), and — critically —
|
||||
**verifies it actually authenticates** (a real headless request to
|
||||
`/Rule/List`, checked with the same `_is_logged_out()` used
|
||||
everywhere else) before saving it to `/data/auth_state.json`. A bad
|
||||
paste never overwrites a working session.
|
||||
|
||||
**All of this lives on one dedicated thread** (`automation._debug_worker`),
|
||||
not arbitrary Flask request threads — Playwright's sync API isn't safe
|
||||
to call across threads, and gunicorn's `gthread` workers hand each
|
||||
request to a different one. Flask routes only ever put a command on
|
||||
`_debug_cmd_queue` and block for the result; they never touch the
|
||||
Playwright browser/context/page objects directly. An un-captured debug
|
||||
session auto-closes after 10 minutes so an open, unauthenticated CDP
|
||||
port doesn't sit exposed on the LAN indefinitely.
|
||||
This session is a real, live bearer credential for the user's MassPrice
|
||||
account for as long as it's valid — same trust model as `LOGIN_PASSWORD`/
|
||||
`SECRET_KEY` in `.env` (server-only, gitignored, never in git), not
|
||||
something to treat as inert config. It isn't permanent: every successful
|
||||
automated run re-saves it (`_save_state()`), which tends to slide a
|
||||
"remember me"-style cookie's expiration forward, but it will eventually
|
||||
expire (inactivity, a password change, TCGplayer forcing re-auth) and
|
||||
need re-importing.
|
||||
|
||||
The Dashboard shows a red banner (`_session_warning()` in `app.py`) when
|
||||
the most recent run/refresh failed specifically because of a missing/
|
||||
|
|
|
|||
|
|
@ -5,13 +5,6 @@ LABEL maintainer="MassPrice Scheduler"
|
|||
|
||||
WORKDIR /app
|
||||
|
||||
# socat proxies the login-session debug port: Chromium's remote-debugging
|
||||
# port binds to 127.0.0.1 only, regardless of --remote-debugging-address
|
||||
# (deliberate Chromium security hardening, confirmed by testing) — a
|
||||
# loopback-only listener is unreachable through Docker's port mapping,
|
||||
# so socat re-exposes it on all interfaces. See automation.py.
|
||||
RUN apt-get update -qq && apt-get install -y -qq socat && rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
|
|
|
|||
34
app.py
34
app.py
|
|
@ -8,7 +8,7 @@ import automation
|
|||
import scheduler
|
||||
import storage
|
||||
|
||||
APP_VERSION = "0.2.0"
|
||||
APP_VERSION = "0.3.0"
|
||||
APP_VERSION_DATE = "2026-09-01"
|
||||
|
||||
app = Flask(__name__)
|
||||
|
|
@ -111,30 +111,15 @@ def dashboard():
|
|||
@app.route("/session")
|
||||
@login_required
|
||||
def session_page():
|
||||
return render_template(
|
||||
"session.html",
|
||||
status=automation.session_status(),
|
||||
debug_active=automation.debug_session_active(),
|
||||
debug_port=os.environ.get("DEBUG_PORT", "9222"),
|
||||
host=request.host.split(":")[0],
|
||||
)
|
||||
return render_template("session.html", status=automation.session_status())
|
||||
|
||||
|
||||
@app.route("/api/session/start", methods=["POST"])
|
||||
@app.route("/api/session/import", methods=["POST"])
|
||||
@login_required
|
||||
def api_session_start():
|
||||
def api_session_import():
|
||||
body = request.get_json(force=True, silent=True) or {}
|
||||
try:
|
||||
automation.start_debug_session()
|
||||
except Exception as exc:
|
||||
return jsonify({"error": str(exc)}), 502
|
||||
return jsonify({"ok": True})
|
||||
|
||||
|
||||
@app.route("/api/session/capture", methods=["POST"])
|
||||
@login_required
|
||||
def api_session_capture():
|
||||
try:
|
||||
automation.capture_debug_session()
|
||||
automation.import_cookies(body.get("cookies", ""))
|
||||
except automation.AutomationError as exc:
|
||||
return jsonify({"error": str(exc)}), 409
|
||||
except Exception as exc:
|
||||
|
|
@ -142,13 +127,6 @@ def api_session_capture():
|
|||
return jsonify({"ok": True})
|
||||
|
||||
|
||||
@app.route("/api/session/stop", methods=["POST"])
|
||||
@login_required
|
||||
def api_session_stop():
|
||||
automation.stop_debug_session()
|
||||
return jsonify({"ok": True})
|
||||
|
||||
|
||||
@app.route("/activity")
|
||||
@login_required
|
||||
def activity():
|
||||
|
|
|
|||
269
automation.py
269
automation.py
|
|
@ -6,31 +6,32 @@ it fires is almost certainly anti-forgery-token protected.
|
|||
|
||||
Session handling: a Playwright `storage_state` (cookies + localStorage)
|
||||
is persisted to AUTH_STATE_PATH and reused across runs, so scheduled runs
|
||||
never touch the login form at all. That's deliberate, not just an
|
||||
never touch the login form at all. That's a hard requirement, not just an
|
||||
optimization — MassPrice's real login (on store.tcgplayer.com, TCGplayer's
|
||||
shared OAuth identity provider) is behind hCaptcha, confirmed by live
|
||||
testing: filling the real username/password and submitting silently
|
||||
never navigates away, because the invisible captcha check never resolves
|
||||
for a headless/automated browser. **There is no way to fully automate
|
||||
logging in.** A human has to solve that captcha at least once per
|
||||
session lifetime, via the "Chrome remote debugging" capture flow below.
|
||||
shared OAuth identity provider) is behind hCaptcha. Two automated
|
||||
approaches were tried and confirmed broken during development:
|
||||
1. Filling the real username/password and submitting — the invisible
|
||||
captcha check never resolves for a headless/automated browser, so
|
||||
the form silently never navigates.
|
||||
2. Driving a real human's clicks through a headless Chromium exposed
|
||||
via Chrome remote debugging (chrome://inspect) — still fails, because
|
||||
that Chromium's fingerprint (a `HeadlessChrome` user-agent,
|
||||
`navigator.webdriver = true`) is itself what hCaptcha flags,
|
||||
independent of who's actually clicking.
|
||||
Deliberately not pursued further: trying to disguise that browser to get
|
||||
past hCaptcha's bot detection is captcha-evasion, not a session-handling
|
||||
technique, regardless of how legitimate the underlying goal is.
|
||||
|
||||
Capture flow: `start_debug_session()` launches a headless-but-CDP-exposed
|
||||
Chromium instance (`--remote-debugging-port`), which the user connects to
|
||||
from their own Chrome via chrome://inspect to log in interactively —
|
||||
solving the hCaptcha themselves, since it's a real click. Once logged in,
|
||||
`capture_debug_session()` saves that browser's cookies to
|
||||
AUTH_STATE_PATH. All of this must happen on a single dedicated thread —
|
||||
Playwright's sync API is not safe to call from arbitrary threads, and
|
||||
Flask/gunicorn's gthread workers hand each request to a different one —
|
||||
so `_debug_worker()` owns every Playwright object here and talks to
|
||||
Flask request threads only through `_debug_cmd_queue`.
|
||||
What actually works: the user logs into MassPrice in their own ordinary,
|
||||
non-automated Chrome (hCaptcha passes the way it always does for a real
|
||||
person), exports that session's cookies with a browser extension, and
|
||||
pastes them into the Session page. `import_cookies()` below converts that
|
||||
paste into Playwright's storage_state format and verifies it actually
|
||||
authenticates before accepting it.
|
||||
"""
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import queue
|
||||
import subprocess
|
||||
import threading
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from playwright.sync_api import sync_playwright
|
||||
|
|
@ -42,23 +43,18 @@ logger = logging.getLogger("massprice.automation")
|
|||
BASE_URL = "https://massprice.tcgplayer.com"
|
||||
AUTH_STATE_PATH = os.path.join(storage.DATA_DIR, "auth_state.json")
|
||||
|
||||
# Chromium's remote-debugging port binds to 127.0.0.1 only inside the
|
||||
# container, no matter what --remote-debugging-address is set to —
|
||||
# deliberate Chromium security hardening (confirmed by live testing, not
|
||||
# a flag mistake). A loopback-only listener is unreachable through
|
||||
# Docker's port mapping, so `socat` re-exposes it on all interfaces at a
|
||||
# second, different internal port (they can't share one — Chromium's
|
||||
# 127.0.0.1 bind and a 0.0.0.0 bind on the same port conflict). Neither
|
||||
# of these is the DEBUG_PORT env var app.py reads for display — that one
|
||||
# is the user-facing HOST port for chrome://inspect instructions, mapped
|
||||
# in docker-compose.yml to _PROXY_PORT, not to Chromium's own port.
|
||||
_INTERNAL_DEBUG_PORT = 9222
|
||||
_PROXY_PORT = 9333
|
||||
DEBUG_SESSION_TIMEOUT = 600 # auto-close an un-captured debug session after 10 minutes
|
||||
|
||||
# Standard, necessary mitigations for headless Chromium inside Docker.
|
||||
LAUNCH_ARGS = ["--no-sandbox", "--disable-dev-shm-usage"]
|
||||
|
||||
# Chrome's internal sameSite cookie values (what export extensions like
|
||||
# Cookie-Editor report) mapped to what Playwright's storage_state expects.
|
||||
_SAME_SITE_MAP = {
|
||||
"no_restriction": "None",
|
||||
"lax": "Lax",
|
||||
"strict": "Strict",
|
||||
"unspecified": "Lax",
|
||||
}
|
||||
|
||||
|
||||
class AutomationError(Exception):
|
||||
pass
|
||||
|
|
@ -88,7 +84,7 @@ def _ensure_authenticated(page):
|
|||
if _is_logged_out(page):
|
||||
raise AutomationError(
|
||||
"MassPrice session expired or missing — log in via the "
|
||||
"Session page (chrome://inspect) and click Capture Session."
|
||||
"Session page and paste in a fresh cookie export."
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -198,144 +194,75 @@ def _log_history(rule_id, rule_name, status, message):
|
|||
storage.save("run_history.json", history[:200])
|
||||
|
||||
|
||||
# ── Interactive login-session capture (Chrome remote debugging) ─────────
|
||||
#
|
||||
# Everything below runs exclusively on `_debug_worker`'s thread. Flask
|
||||
# routes call start_debug_session()/capture_debug_session()/
|
||||
# stop_debug_session(), which just enqueue a command and block for the
|
||||
# result — never touch the Playwright objects directly.
|
||||
|
||||
_debug_cmd_queue: "queue.Queue" = queue.Queue()
|
||||
_debug_thread = None
|
||||
_debug_active = False # read from other threads; plain bool assignment is atomic in CPython
|
||||
# ── Login-session import (paste cookies exported from a real browser) ───
|
||||
|
||||
|
||||
def _debug_worker():
|
||||
global _debug_active
|
||||
playwright = browser = context = page = timer = socat_proc = None
|
||||
def _convert_cookie(raw):
|
||||
"""Convert one cookie from a Cookie-Editor-style export into
|
||||
Playwright's storage_state cookie shape."""
|
||||
expires = raw.get("expirationDate")
|
||||
same_site_raw = (raw.get("sameSite") or "unspecified").lower()
|
||||
return {
|
||||
"name": raw["name"],
|
||||
"value": raw["value"],
|
||||
"domain": raw["domain"],
|
||||
"path": raw.get("path", "/"),
|
||||
"expires": expires if expires is not None else -1,
|
||||
"httpOnly": bool(raw.get("httpOnly", False)),
|
||||
"secure": bool(raw.get("secure", False)),
|
||||
"sameSite": _SAME_SITE_MAP.get(same_site_raw, "Lax"),
|
||||
}
|
||||
|
||||
def do_stop():
|
||||
nonlocal playwright, browser, context, page, timer, socat_proc
|
||||
global _debug_active
|
||||
if timer:
|
||||
timer.cancel()
|
||||
timer = None
|
||||
if socat_proc:
|
||||
try:
|
||||
socat_proc.terminate()
|
||||
except Exception:
|
||||
pass
|
||||
socat_proc = None
|
||||
if browser:
|
||||
try:
|
||||
browser.close()
|
||||
except Exception:
|
||||
pass
|
||||
if playwright:
|
||||
try:
|
||||
playwright.stop()
|
||||
except Exception:
|
||||
pass
|
||||
playwright = browser = context = page = None
|
||||
_debug_active = False
|
||||
|
||||
while True:
|
||||
cmd, result_q = _debug_cmd_queue.get()
|
||||
def import_cookies(raw_text):
|
||||
"""Parse a pasted cookie export, convert it, verify it actually
|
||||
authenticates against MassPrice, and only then save it as the new
|
||||
session. Raises AutomationError with a user-facing message on any
|
||||
failure; never overwrites a working session with a bad paste."""
|
||||
raw_text = (raw_text or "").strip()
|
||||
if not raw_text:
|
||||
raise AutomationError("Paste your exported cookies first — the box was empty")
|
||||
|
||||
try:
|
||||
parsed = json.loads(raw_text)
|
||||
except json.JSONDecodeError:
|
||||
raise AutomationError(
|
||||
"Couldn't parse that as JSON — make sure you copied the full "
|
||||
"export from the cookie extension"
|
||||
)
|
||||
|
||||
if not isinstance(parsed, list) or not parsed:
|
||||
raise AutomationError(
|
||||
"Expected a JSON array of cookies (Cookie-Editor's export format) — "
|
||||
"got something else"
|
||||
)
|
||||
|
||||
relevant = [c for c in parsed if "tcgplayer.com" in (c.get("domain") or "")]
|
||||
if not relevant:
|
||||
raise AutomationError(
|
||||
"None of the pasted cookies are for tcgplayer.com — make sure you "
|
||||
"exported while on the massprice.tcgplayer.com tab, after logging in"
|
||||
)
|
||||
|
||||
try:
|
||||
cookies = [_convert_cookie(c) for c in relevant]
|
||||
except KeyError as exc:
|
||||
raise AutomationError(f"A pasted cookie was missing its {exc} field")
|
||||
|
||||
candidate_state = {"cookies": cookies, "origins": []}
|
||||
|
||||
with sync_playwright() as p:
|
||||
browser = p.chromium.launch(headless=True, args=LAUNCH_ARGS)
|
||||
try:
|
||||
if cmd == "start":
|
||||
if browser is None:
|
||||
playwright = sync_playwright().start()
|
||||
browser = playwright.chromium.launch(
|
||||
headless=True,
|
||||
args=LAUNCH_ARGS
|
||||
+ [
|
||||
f"--remote-debugging-port={_INTERNAL_DEBUG_PORT}",
|
||||
"--remote-allow-origins=*",
|
||||
],
|
||||
)
|
||||
# Re-expose Chromium's loopback-only debug port on
|
||||
# all interfaces so Docker's port mapping (which
|
||||
# forwards to _PROXY_PORT, not Chromium's own port —
|
||||
# see the comment above) can actually reach it.
|
||||
socat_proc = subprocess.Popen(
|
||||
[
|
||||
"socat",
|
||||
f"TCP-LISTEN:{_PROXY_PORT},fork,reuseaddr",
|
||||
f"TCP:127.0.0.1:{_INTERNAL_DEBUG_PORT}",
|
||||
]
|
||||
)
|
||||
context = _new_context(browser)
|
||||
page = context.new_page()
|
||||
page.goto(f"{BASE_URL}/")
|
||||
_debug_active = True
|
||||
timer = threading.Timer(
|
||||
DEBUG_SESSION_TIMEOUT, lambda: _debug_cmd_queue.put(("stop", None))
|
||||
)
|
||||
timer.daemon = True
|
||||
timer.start()
|
||||
if result_q:
|
||||
result_q.put(("ok", None))
|
||||
elif cmd == "capture":
|
||||
if context is None:
|
||||
if result_q:
|
||||
result_q.put(
|
||||
("error", "No active login session — click Start Login Session first")
|
||||
)
|
||||
else:
|
||||
page.goto(f"{BASE_URL}/Rule/List")
|
||||
page.wait_for_timeout(1000)
|
||||
if _is_logged_out(page):
|
||||
if result_q:
|
||||
result_q.put(
|
||||
(
|
||||
"error",
|
||||
"Still not logged in — finish logging in via "
|
||||
"chrome://inspect, then try Capture Session again",
|
||||
)
|
||||
)
|
||||
else:
|
||||
_save_state(context)
|
||||
do_stop()
|
||||
if result_q:
|
||||
result_q.put(("ok", None))
|
||||
elif cmd == "stop":
|
||||
do_stop()
|
||||
if result_q:
|
||||
result_q.put(("ok", None))
|
||||
except Exception as exc:
|
||||
logger.exception("debug session command %r failed", cmd)
|
||||
if result_q:
|
||||
result_q.put(("error", str(exc)))
|
||||
|
||||
|
||||
def _ensure_debug_thread():
|
||||
global _debug_thread
|
||||
if _debug_thread is None or not _debug_thread.is_alive():
|
||||
_debug_thread = threading.Thread(target=_debug_worker, daemon=True)
|
||||
_debug_thread.start()
|
||||
|
||||
|
||||
def _debug_call(cmd, timeout=30):
|
||||
result_q: "queue.Queue" = queue.Queue()
|
||||
_debug_cmd_queue.put((cmd, result_q))
|
||||
status, payload = result_q.get(timeout=timeout)
|
||||
if status == "error":
|
||||
raise AutomationError(payload)
|
||||
|
||||
|
||||
def start_debug_session():
|
||||
_ensure_debug_thread()
|
||||
_debug_call("start")
|
||||
|
||||
|
||||
def capture_debug_session():
|
||||
_debug_call("capture")
|
||||
|
||||
|
||||
def stop_debug_session():
|
||||
_ensure_debug_thread()
|
||||
_debug_call("stop")
|
||||
|
||||
|
||||
def debug_session_active():
|
||||
return _debug_active
|
||||
context = browser.new_context(storage_state=candidate_state)
|
||||
page = context.new_page()
|
||||
page.goto(f"{BASE_URL}/Rule/List")
|
||||
if _is_logged_out(page):
|
||||
raise AutomationError(
|
||||
"Imported cookies didn't authenticate — still logged out. "
|
||||
"Make sure you exported them from massprice.tcgplayer.com "
|
||||
"itself, after finishing login, not from the login page."
|
||||
)
|
||||
_save_state(context)
|
||||
finally:
|
||||
browser.close()
|
||||
|
|
|
|||
|
|
@ -7,16 +7,6 @@ services:
|
|||
shm_size: "1gb"
|
||||
ports:
|
||||
- "${HOST_PORT:-5057}:5000"
|
||||
# MassPrice's login is behind hCaptcha, which blocks fully
|
||||
# automated sign-in — this port reaches (via an in-container socat
|
||||
# proxy, see automation.py) headless Chromium's CDP debug endpoint,
|
||||
# so a human can log in via chrome://inspect from their own machine
|
||||
# (see the Session page / deploy-massprice skill). Maps to 9333,
|
||||
# NOT Chromium's own 9222 — Chromium's debug port binds loopback-
|
||||
# only regardless of config, so socat re-exposes it on a different
|
||||
# internal port. Only listens while a login session is actively
|
||||
# started from the webui, and only reachable on the LAN.
|
||||
- "${DEBUG_PORT:-9222}:9333"
|
||||
volumes:
|
||||
# Schedules, run history, the rule-name cache, and the saved
|
||||
# MassPrice login session must outlive `up -d --build`, which
|
||||
|
|
@ -25,6 +15,5 @@ services:
|
|||
environment:
|
||||
- LOGIN_PASSWORD=${LOGIN_PASSWORD}
|
||||
- SECRET_KEY=${SECRET_KEY}
|
||||
- DEBUG_PORT=${DEBUG_PORT:-9222}
|
||||
- TZ=${TZ:-America/New_York}
|
||||
restart: unless-stopped
|
||||
|
|
|
|||
|
|
@ -178,9 +178,22 @@ footer {
|
|||
.panel-narrow { max-width: 560px; }
|
||||
|
||||
.session-steps { padding-left: 20px; display: flex; flex-direction: column; gap: 6px; font-size: 0.88rem; }
|
||||
.session-steps code { background: var(--panel-alt); border: 1px solid var(--border); border-radius: 4px; padding: 1px 6px; }
|
||||
.session-steps a { color: var(--accent); }
|
||||
.session-actions { display: flex; gap: 8px; margin-top: 16px; }
|
||||
|
||||
.cookie-textarea {
|
||||
width: 100%;
|
||||
margin-top: 12px;
|
||||
background: var(--panel-alt);
|
||||
border: 1px solid var(--border);
|
||||
color: var(--text);
|
||||
border-radius: 6px;
|
||||
padding: 10px;
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
|
||||
font-size: 0.8rem;
|
||||
resize: vertical;
|
||||
}
|
||||
|
||||
.activity-table { width: 100%; border-collapse: collapse; font-size: 0.85rem; }
|
||||
.activity-table th, .activity-table td { text-align: left; padding: 8px 12px; border-bottom: 1px solid var(--border); }
|
||||
.activity-table th { color: var(--text-dim); font-weight: 500; font-size: 0.75rem; text-transform: uppercase; }
|
||||
|
|
|
|||
|
|
@ -25,34 +25,31 @@ document.addEventListener("DOMContentLoaded", () => {
|
|||
});
|
||||
|
||||
function initSessionControls() {
|
||||
const startBtn = document.getElementById("session-start");
|
||||
const captureBtn = document.getElementById("session-capture");
|
||||
const stopBtn = document.getElementById("session-stop");
|
||||
if (!startBtn) return; // not on the Session page
|
||||
const importBtn = document.getElementById("session-import");
|
||||
const textarea = document.getElementById("cookie-paste");
|
||||
if (!importBtn) return; // not on the Session page
|
||||
|
||||
async function call(url, btn) {
|
||||
btn.disabled = true;
|
||||
const original = btn.textContent;
|
||||
btn.textContent = "Working…";
|
||||
importBtn.addEventListener("click", async () => {
|
||||
const original = importBtn.textContent;
|
||||
importBtn.disabled = true;
|
||||
importBtn.textContent = "Verifying…";
|
||||
try {
|
||||
const res = await fetch(url, { method: "POST" });
|
||||
const res = await fetch("/api/session/import", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ cookies: textarea.value }),
|
||||
});
|
||||
const data = await res.json().catch(() => ({}));
|
||||
if (!res.ok) {
|
||||
alert(data.error || "Request failed");
|
||||
btn.disabled = false;
|
||||
btn.textContent = original;
|
||||
alert(data.error || "Import failed");
|
||||
return;
|
||||
}
|
||||
window.location.reload();
|
||||
} catch (err) {
|
||||
btn.disabled = false;
|
||||
btn.textContent = original;
|
||||
} finally {
|
||||
importBtn.disabled = false;
|
||||
importBtn.textContent = original;
|
||||
}
|
||||
}
|
||||
|
||||
startBtn.addEventListener("click", () => call("/api/session/start", startBtn));
|
||||
captureBtn.addEventListener("click", () => call("/api/session/capture", captureBtn));
|
||||
stopBtn.addEventListener("click", () => call("/api/session/stop", stopBtn));
|
||||
});
|
||||
}
|
||||
|
||||
function initRuleCard(card) {
|
||||
|
|
|
|||
|
|
@ -7,40 +7,37 @@
|
|||
<h2>Status</h2>
|
||||
{% if status.state == 'saved' %}
|
||||
<p><span class="status-dot status-success"></span>Session saved {{ status.saved_at }}</p>
|
||||
<p class="muted small">If scheduled runs start failing with a "session expired" error, come back here and log in again.</p>
|
||||
<p class="muted small">If scheduled runs start failing with a "session expired" error, come back here and import a fresh export.</p>
|
||||
{% else %}
|
||||
<p><span class="status-dot status-failure"></span>No session saved yet — scheduled runs will fail until you log in.</p>
|
||||
<p><span class="status-dot status-failure"></span>No session saved yet — scheduled runs will fail until you import one.</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
|
||||
<div class="panel panel-narrow">
|
||||
<h2>Log in</h2>
|
||||
<p class="muted small">
|
||||
MassPrice's login is protected by hCaptcha, which blocks fully automated
|
||||
sign-in. You need to log in yourself, at least once per session lifetime,
|
||||
using your own Chrome browser connected to a debug browser running in
|
||||
this container.
|
||||
MassPrice's login is protected by hCaptcha, which blocks any automated
|
||||
sign-in attempt — including a real click relayed through a headless
|
||||
browser, since the browser itself gets flagged. There's no way around
|
||||
this: log in with your own ordinary Chrome instead, then copy that
|
||||
session over.
|
||||
</p>
|
||||
<ol class="session-steps">
|
||||
<li>Click <strong>Start Login Session</strong> below.</li>
|
||||
<li>On your own computer, open Chrome and go to <code>chrome://inspect/#devices</code>.</li>
|
||||
<li>Under "Discover network targets", click <strong>Configure…</strong> and add
|
||||
<code>{{ host }}:{{ debug_port }}</code>, then close the dialog.</li>
|
||||
<li>A MassPrice tab should appear under "Remote Target" — click
|
||||
<strong>inspect</strong> to open it, then log in normally (including solving the
|
||||
hCaptcha challenge yourself).</li>
|
||||
<li>Once you're looking at the MassPrice dashboard or rules list in that inspected
|
||||
tab, come back here and click <strong>Capture Session</strong>.</li>
|
||||
<li>Log into <a href="https://massprice.tcgplayer.com/" target="_blank" rel="noopener">massprice.tcgplayer.com</a>
|
||||
normally, in your own Chrome.</li>
|
||||
<li>Install the <strong>Cookie-Editor</strong> extension (free, from the
|
||||
Chrome Web Store) if you don't already have it.</li>
|
||||
<li>While still on the massprice.tcgplayer.com tab (not the login page),
|
||||
click the Cookie-Editor icon, then <strong>Export</strong> →
|
||||
<strong>Export as JSON</strong> — this copies the cookies to your
|
||||
clipboard.</li>
|
||||
<li>Paste them into the box below and click <strong>Import Session</strong>.</li>
|
||||
</ol>
|
||||
|
||||
<textarea id="cookie-paste" class="cookie-textarea" placeholder='[{"domain": ".tcgplayer.com", "name": "...", ...}]' rows="8"></textarea>
|
||||
<div class="session-actions">
|
||||
<button id="session-start" type="button" class="btn btn-primary" {{ 'disabled' if debug_active else '' }}>Start Login Session</button>
|
||||
<button id="session-capture" type="button" class="btn" {{ 'disabled' if not debug_active else '' }}>Capture Session</button>
|
||||
<button id="session-stop" type="button" class="btn" {{ 'disabled' if not debug_active else '' }}>Stop</button>
|
||||
<button id="session-import" type="button" class="btn btn-primary">Import Session</button>
|
||||
</div>
|
||||
{% if debug_active %}
|
||||
<p class="muted small">Login session is active — it auto-closes after 10 minutes if not captured.</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endblock %}
|
||||
{% block scripts %}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue