card-grader/CLAUDE.md
mattie726 0757e95ad3
Some checks failed
Deploy / deploy (push) Failing after 13s
Add Forgejo deploy workflow and update deploy docs
2026-08-25 23:19:09 -07:00

162 lines
9.1 KiB
Markdown

# Card Grader
Self-hosted web app that estimates PSA trading card grades from photos, using
vision-capable LLMs (Claude and/or GPT) as the grading engine. Runs as a
single Docker container on the user's Unraid home server, exposed at
`hippofam.com/cards` through Nginx Proxy Manager.
No framework: the whole backend is Python stdlib
(`http.server.ThreadingHTTPServer`) plus `sqlite3`. Optional third-party
packages are `anthropic`, `openai`, and `Pillow` (image processing only —
never required for the server to boot).
## Architecture
- **`app.py`** — HTTP routing/handlers. Reads Basic Auth username forwarded
by the reverse proxy for *attribution only* (who graded what), never for
access control — auth itself is enforced by Nginx Proxy Manager in front
of the container via htpasswd. Admin-only settings are gated by
`ADMIN_USERNAME` (`CARD_GRADER_ADMIN_USER` env var). Request bodies are
capped (`MAX_BODY_BYTES`); oversized/malformed bodies get a 413 and the
connection is closed rather than trusting a client-supplied length.
- **`vision.py`** — Provider abstraction over Anthropic and OpenAI vision
APIs (`_call_anthropic` / `_call_openai`, dispatched from a common
entrypoint). `MODELS` holds per-model pricing/capabilities.
`GRADING_SYSTEM` is the large system prompt encoding PSA's actual grading
rubric (centering tolerances, corner/edge/surface criteria, qualifiers —
MC/OC/PD/ST/MK, half-grades, Altered Authentic) — verified against
psacard.com, beckett.com, and cgccards.com, not guessed. Self-reported
"high" confidence is capped to "medium" when neither centering nor edge
whitening was independently measurable from the image geometry.
- **`cardimage.py`** — Pillow-based image pipeline: card boundary detection,
corner/edge/surface crop generation, centering/edge-whitening/aspect-ratio
measurement from pixels (not just left to the model's eye). Includes
holder/toploader-aware detection (`detect_card_box`): a combinatorial
search over candidate edge positions constrained to standard card aspect
ratios, so the app can locate the actual card inside a slab/toploader
rather than just detecting the holder's outline. When a holder is
detected, edge-whitening measurement is refused (the holder's own inner
edge corrupts that specific measurement) but centering still runs
(border-width geometry, unaffected by clear plastic), and aspect-ratio
measurement deliberately uses the *unrefined* box to avoid circularity
(refinement selects for standard-ratio rectangles, so measuring the
refined box's ratio would trivially always look "normal").
- **`store.py`** — SQLite (WAL mode) persistence. `grades` and
`grade_events` tables, JSON columns for structured data, schema
migrations via idempotent `ALTER TABLE` in `store.init()`. List/detail
API responses explicitly exclude `source_images_json` to keep payloads
small. DB path overridable via `CARD_GRADER_DB_PATH`.
- **`static/`** — Vanilla JS/CSS/HTML frontend, PWA-enabled (manifest,
service worker). "PSA slab" visual theme. `__BASE__` is templated at
request time so the app works correctly when served from a sub-path
(`/cards`) behind the proxy.
## Local development
```bash
python3 app.py
```
Needs `ANTHROPIC_API_KEY` and/or `OPENAI_API_KEY` in the environment to
actually grade; the server itself has no required third-party deps. A local
`grades.db` is gitignored and is scratch/test data only — it is **not** the
production database (that lives on the server, see below).
## Deployment
**Target:** Unraid server, container path `/mnt/user/appdata/card-grader`,
built from `Dockerfile` / `docker-compose.yml` in this repo. Non-root
container user (`99:100` / `nobody:users`), healthcheck, log rotation,
`init: true`. Reverse-proxied by Nginx Proxy Manager at `hippofam.com/cards`
with per-user HTTP Basic Auth.
**Current deploy mechanism (as of 2026-08-25): manual.** Changes are copied
to the server by hand (`scp`) and the container is rebuilt over SSH; the two
copies of the repo (local Mac, server) are kept in sync by committing on
both sides. There is currently no automated push-to-deploy path connected —
see below.
**Superseded approach, fully torn down: git-shell restricted deploy.** A
`carddeploy` service account (login shell `git-shell`, one bare repo, a
`post-receive` hook triggering a single root-owned deploy script via one
narrowly-scoped `sudo` rule) was built, tested, and verified working
end-to-end — then explicitly torn down at the user's request in favor of
Forgejo (below). Nothing from it remains: the account, sudoers rule, bare
repo, `/boot/config/go` persistence block, and the `sshd_config`
`AllowUsers` addition were all removed and verified reverted. The local git
remote and SSH keypair for it were deleted too. If this pattern (restricted
git-shell account + sudo-scoped deploy script) is useful as a reference for
scoping *other* access requests later, it's a solid model — it was
abandoned for tooling reasons (moving to a real git host), not because it
didn't work.
**In progress: Forgejo.** Decision made 2026-08-25 to self-host Forgejo
(soft-fork of Gitea) as the git remote and CI/CD trigger, replacing the
git-shell approach entirely. Plan, run by the user themselves (not by
Claude — user explicitly wants to execute server-side setup steps
personally):
1. Forgejo itself runs as its own Docker container on the same Unraid box
(`/mnt/user/appdata/forgejo`), proxied at `git.hippofam.com` via Nginx
Proxy Manager.
2. This repo gets a `forgejo` remote and is pushed there (full history,
not a fresh copy).
3. Forgejo Actions (instance + repo level) is enabled, and a self-hosted
runner is registered **at the repo level** (not instance-wide) as its
own container (`/mnt/user/appdata/forgejo-runner`), with the host
Docker socket mounted in so it can rebuild the `card-grader` container.
4. A `.forgejo/workflows/deploy.yml` workflow (checkout + `docker compose
up -d --build` on push to `main`) does the actual deploy.
**Known tradeoff, not yet resolved:** unlike the git-shell mailbox (which
could only ever trigger one hardcoded script), a CI runner executes
whatever a workflow YAML says — mounting the Docker socket gives it control
over *any* container on the box, not just this one. The access boundary
becomes "who can push a workflow file to this repo" rather than a sudoers
rule. If tighter isolation is wanted later, a `docker-socket-proxy` in
front of the runner (scoped to just the `card-grader` compose project) was
proposed but not yet built — revisit if this matters more once the basic
pipeline is working.
**Status:** walkthrough delivered to the user 2026-08-25; not yet executed.
**Do not assume any of this is live** — confirm current state before
relying on it or making claims about deploy mechanism to the user.
A prior root SSH key/access to the whole box was explicitly revoked by the
user before any of the above; every access grant since then (including a
one-time temporary root key used only to run the git-shell setup script,
itself removed immediately after) has been narrowly scoped and time-boxed
on request. Treat that as the standing expectation for any future
server-access ask on this project — default to the smallest scope that
does the job, time-box it, and confirm teardown/persistence explicitly
rather than leaving it open-ended.
## Conventions and constraints established for this project
- PSA grading rules must be verified against a real source (PSA's own site,
Beckett, CGC) before being encoded into `GRADING_SYSTEM` — this rubric has
had real accuracy bugs (e.g. a self-contradictory centering tolerance, a
missing 5% front-centering leeway rule for grades 7+) found and fixed
through actual verification, not assumption.
- The vision prompt's `cannot_assess` escape hatch is deliberately narrow:
surface condition genuinely can't be separated from dust/scratches on a
toploader/slab's plastic, so it keeps the escape hatch. Corner and edge
geometry (sharpness, whitening, chipping) **does** read through clear
plastic and must be judged normally — an earlier prompt version gave the
model an escape hatch for corners/edges too, which caused false
"cannot tell from photo" results on cards that were plainly visible
through the holder. Don't reintroduce that.
- No pre-flight/"precheck" step before a paid grading call. This was tried
and explicitly rejected by the user — their workflow is screenshots that
can't be pulled out of the toploader, so a pre-check step doesn't fit and
was reverted.
- No browser automation (Playwright or otherwise) to scrape eBay listings.
Tried a plain `curl`-based fetch once (not Playwright) and got an
immediate 403; iterating on headers/fingerprinting to get past that would
be bot-detection evasion regardless of which tool performs it, and is
out of scope. This feature is paused; if revisited, the legitimate path
is a client-side bookmarklet/extension that uses the *user's own*
authenticated browser session rather than a server-side fetch.
- Never trust the reverse proxy's forwarded auth header for anything beyond
attribution (whose name to log against a grade) — access control is the
proxy's job, not the app's.