Replace Chrome-remote-debugging login capture with a cookie-paste import flow
All checks were successful
Deploy / deploy (push) Successful in 25s

Live testing showed the CDP approach still fails hCaptcha even with a
real human clicking through chrome://inspect: the headless Chromium's
own fingerprint (HeadlessChrome user-agent, navigator.webdriver=true)
is what gets flagged, independent of who's driving it. Trying to
disguise that browser to get past bot detection would be captcha
evasion, not a session-handling technique, so that path is dropped
entirely rather than patched further.

New flow: log into MassPrice with an ordinary, non-automated browser
(hCaptcha passes normally), export the session's cookies with a
extension, and paste them into the Session page. import_cookies()
converts the export into Playwright's storage_state shape and verifies
it actually authenticates against the live site before saving it,
so a bad paste never clobbers a working session. This also removes the
now-unnecessary socat proxy / debug port exposure — Chromium's own
CDP port was only ever needed for the dropped approach.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H6TgffxTvYhqxykAbVQg2v
This commit is contained in:
Barely Removable 2026-08-31 20:10:51 -07:00
parent bec399dd11
commit 48d8bac332
10 changed files with 214 additions and 311 deletions

View file

@ -70,40 +70,50 @@ tool is confirmed working, so there's exactly one thing triggering runs.
## Login — why it's manual, and how it works
MassPrice's login (`store.tcgplayer.com/oauth/login`, TCGplayer's shared
OAuth identity provider) is protected by **hCaptcha**. This was
confirmed by live testing during development: filling in the real
username/password and clicking Submit never navigates anywhere — the
invisible captcha check silently never resolves for a headless/automated
browser, and the form just refuses to submit. There is no selector fix
for this; **fully automated login is not achievable here**, full stop.
An earlier version of this app tried `MASSPRICE_USERNAME`/
`MASSPRICE_PASSWORD` credential-based auto-login — removed once this was
confirmed, since keeping dead code that can never succeed is worse than
having none.
OAuth identity provider) is protected by **hCaptcha**. Two automated
approaches were tried and confirmed broken by live testing during
development — there is no selector fix for either, **fully automated
login is not achievable here**, full stop:
1. Filling in the real username/password and clicking Submit — the
invisible captcha check silently never resolves for a headless/
automated browser, and the form just refuses to submit. An earlier
version of this app tried `MASSPRICE_USERNAME`/`MASSPRICE_PASSWORD`
this way; removed once confirmed broken.
2. Driving a real human's clicks through a headless Chromium exposed via
Chrome remote debugging (`chrome://inspect`) — also removed after
testing. Still fails hCaptcha, because the browser itself (a
`HeadlessChrome` user-agent, `navigator.webdriver = true`) is what
gets flagged, independent of who's actually clicking. Deliberately
not pursued further than that one test: disguising that browser to
get past hCaptcha's bot detection would be captcha evasion, not a
session-handling technique, regardless of how legitimate running
automation on your own account is.
What actually works: a human only needs to pass the hCaptcha, not
automate around it, and that's a one-time (well, periodic — however long
the session cookie lasts) cost, not a per-run one. The **Session** page
in the webui drives this:
1. "Start Login Session" launches headless Chromium with its CDP debug
port exposed (`DEBUG_PORT`, default `9222`).
2. The user connects their own Chrome to it via `chrome://inspect`,
which renders and accepts real input against that same running
browser instance — no VNC/Xvfb needed, headless Chromium's debug
protocol already supports this.
3. The user logs in normally in that inspected tab, solving the hCaptcha
themselves since it's a real click from a real browser.
4. "Capture Session" saves that browser's cookies to
`/data/auth_state.json` — the same file scheduled runs read.
What actually works: log in with an ordinary, non-automated browser —
hCaptcha passes the way it always does for a real person, because
nothing about that browser looks automated — then copy that session's
cookies over. The **Session** page in the webui drives this:
1. The user logs into massprice.tcgplayer.com in their own regular
Chrome and exports its cookies with a browser extension (Cookie-Editor,
"Export as JSON").
2. They paste that JSON into the Session page and click "Import Session".
3. `automation.import_cookies()` parses it, converts each cookie from
Cookie-Editor's shape into Playwright's `storage_state` cookie shape
(`_convert_cookie()` — mainly the `sameSite` enum values and the
`expirationDate` → `expires` field rename), and — critically —
**verifies it actually authenticates** (a real headless request to
`/Rule/List`, checked with the same `_is_logged_out()` used
everywhere else) before saving it to `/data/auth_state.json`. A bad
paste never overwrites a working session.
**All of this lives on one dedicated thread** (`automation._debug_worker`),
not arbitrary Flask request threads — Playwright's sync API isn't safe
to call across threads, and gunicorn's `gthread` workers hand each
request to a different one. Flask routes only ever put a command on
`_debug_cmd_queue` and block for the result; they never touch the
Playwright browser/context/page objects directly. An un-captured debug
session auto-closes after 10 minutes so an open, unauthenticated CDP
port doesn't sit exposed on the LAN indefinitely.
This session is a real, live bearer credential for the user's MassPrice
account for as long as it's valid — same trust model as `LOGIN_PASSWORD`/
`SECRET_KEY` in `.env` (server-only, gitignored, never in git), not
something to treat as inert config. It isn't permanent: every successful
automated run re-saves it (`_save_state()`), which tends to slide a
"remember me"-style cookie's expiration forward, but it will eventually
expire (inactivity, a password change, TCGplayer forcing re-auth) and
need re-importing.
The Dashboard shows a red banner (`_session_warning()` in `app.py`) when
the most recent run/refresh failed specifically because of a missing/