From 2fdb9be9b9803934129d9bd1dd2148080a37b374 Mon Sep 17 00:00:00 2001 From: Barely Removable Date: Mon, 31 Aug 2026 17:31:53 -0700 Subject: [PATCH] Initial scaffold: Flask webui + APScheduler + Playwright automation for MassPrice rule scheduling Drives massprice.tcgplayer.com's Run Rule action per-rule on a user-configured schedule (multiple times/day, day-of-week toggles), replacing MassPrice's native scheduler (capped at 15 schedules account-wide, one run/day, with observed clock drift). Deployed via Forgejo push-to-deploy on Unraid, matching arr-summary/card-grader. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01H6TgffxTvYhqxykAbVQg2v --- .claude/skills/deploy-massprice/SKILL.md | 82 +++++++++ .env.example | 26 +++ .forgejo/workflows/deploy.yml | 29 ++++ .gitignore | 6 + CLAUDE.md | 117 +++++++++++++ Dockerfile | 15 ++ app.py | 157 ++++++++++++++++++ automation.py | 202 +++++++++++++++++++++++ docker-compose.yml | 21 +++ gunicorn.conf.py | 13 ++ requirements.txt | 4 + scheduler.py | 93 +++++++++++ static/css/style.css | 159 ++++++++++++++++++ static/js/app.js | 107 ++++++++++++ storage.py | 35 ++++ templates/_shell.html | 26 +++ templates/activity.html | 24 +++ templates/dashboard.html | 68 ++++++++ templates/login.html | 13 ++ 19 files changed, 1197 insertions(+) create mode 100644 .claude/skills/deploy-massprice/SKILL.md create mode 100644 .env.example create mode 100644 .forgejo/workflows/deploy.yml create mode 100644 .gitignore create mode 100644 CLAUDE.md create mode 100644 Dockerfile create mode 100644 app.py create mode 100644 automation.py create mode 100644 docker-compose.yml create mode 100644 gunicorn.conf.py create mode 100644 requirements.txt create mode 100644 scheduler.py create mode 100644 static/css/style.css create mode 100644 static/js/app.js create mode 100644 storage.py create mode 100644 templates/_shell.html create mode 100644 templates/activity.html create mode 100644 templates/dashboard.html create mode 100644 templates/login.html diff --git a/.claude/skills/deploy-massprice/SKILL.md b/.claude/skills/deploy-massprice/SKILL.md new file mode 100644 index 0000000..e5902c3 --- /dev/null +++ b/.claude/skills/deploy-massprice/SKILL.md @@ -0,0 +1,82 @@ +--- +name: deploy-massprice +description: Use when deploying MassPrice Scheduler, restarting the massprice container on Unraid, confirming a deploy landed, or troubleshooting its scheduled-rule automation. +--- + +## Deployment + +**Target:** Unraid server, container path `/mnt/user/appdata/massprice`, +built from `Dockerfile`/`docker-compose.yml` in this repo. Container name +`massprice`, port `5056` (overridable via `HOST_PORT`) → container port +`5000`. + +**Deploy mechanism: Forgejo push-to-deploy**, the same pattern as +`arr-summary`, `card-grader`, `lunch-notifier`, `n64-tracker` — see +`arr-summary`'s `deploy-arr-summary` skill or `card-grader`'s `CLAUDE.md` +for the full runner mechanics. Push to `main` on the `forgejo` remote +(`https://git.hippofam.com/ninja_hippo/massprice.git` — create this repo +on the Forgejo instance first if it doesn't exist yet) triggers +`.forgejo/workflows/deploy.yml`, which syncs into +`/mnt/user/appdata/massprice` (excluding `.git`, `.env`, `data/`) and +runs `docker compose up -d --build`. + +**Restarting the container manually: use Compose, not the Unraid Docker +UI** — same reasoning as the other projects (every deploy recreates the +container with a new Docker ID; Unraid's UI caches the old one). From the +Unraid terminal: +```bash +cd /mnt/user/appdata/massprice && docker compose restart +``` +`docker compose up -d --force-recreate` for a guaranteed-clean recreate; +`docker ps --filter name=massprice` to check real state. + +**Confirming a deploy landed:** the footer renders +`v · ` from constants near the top of +`app.py`. Bump them every commit. + +## First deploy checklist + +1. Create `/mnt/user/appdata/massprice` on Unraid and a real `.env` + there (copy `.env.example`, fill in `LOGIN_PASSWORD`, `SECRET_KEY`, + `MASSPRICE_USERNAME`, `MASSPRICE_PASSWORD`, `TZ`). +2. First `docker compose up -d --build` — confirm the container starts + and `/login` responds on `HOST_PORT`. +3. Log into the webui, click "Refresh Rules" — confirms Playwright can + reach massprice.tcgplayer.com and log in automatically. **If this + fails**, check `data/run_history.json`/container logs for the login + error — the login form locators in `automation._login()` were written + generically (never verified against the real logged-out form, see + `CLAUDE.md`) and may need adjusting to match TCGplayer's actual field + names/labels. +4. Set up at least one schedule and use "Run Now" once to confirm a + click actually lands — cross-check by looking at MassPrice's own + Activity Log (`https://massprice.tcgplayer.com/Activity/List`) for a + fresh timestamp on that rule. +5. **Only after step 4 succeeds**, go to each of the 5 existing rules on + massprice.tcgplayer.com (three-dot menu → Edit Schedule) and set + Status to Inactive, so native scheduling stops double-triggering runs + and this tool becomes the sole source of scheduled runs. + +## Persistent state, never touched by deploys + +`.env` holds `LOGIN_PASSWORD`, `SECRET_KEY`, and the MassPrice +credentials. `data/` (bind-mounted to `/data`) holds `schedules.json` +(the schedules set from the webui), `auth_state.json` (the saved +MassPrice browser session — losing this just costs one re-login, not +data), `run_history.json`, and `rules_cache.json`. Both are excluded +from the deploy rsync (which runs `--delete`) and from git. + +## If scheduled runs stop working + +Check, in order: +1. **Activity page in the webui** — does the failure message say + "still on a login page" (credentials/CAPTCHA problem) or "rule id not + found" (a rule was renamed/deleted on MassPrice — click "Refresh + Rules")? +2. **Is the container still running** — `docker ps --filter + name=massprice` — a Chromium crash inside the container (rare, but + possible if `shm_size` ever gets dropped from `docker-compose.yml`) + would show as the container restarting. +3. **Timezone** — `TZ` in `.env` controls what "6:00 AM" means for every + schedule; confirm it matches what the user actually expects before + assuming a schedule itself is wrong. diff --git a/.env.example b/.env.example new file mode 100644 index 0000000..fe75326 --- /dev/null +++ b/.env.example @@ -0,0 +1,26 @@ +# ══════════════════════════════════════════════════════════════════ +# MassPrice Scheduler — Environment Variables +# Copy this file to .env and fill in your values. +# ══════════════════════════════════════════════════════════════════ + +# ── Login (this tool's own login page, not your MassPrice account) ─ +LOGIN_PASSWORD=changeme + +# Secret key for Flask session signing — generate a random string: +# python3 -c "import secrets; print(secrets.token_hex(32))" +SECRET_KEY=change_this_to_a_random_secret_key + +# ── MassPrice (TCGplayer) login ───────────────────────────────────── +# Used by Playwright to log in automatically the first time, and again +# any time the saved session expires. Stored here only — gitignored, +# never committed, never logged. +MASSPRICE_USERNAME=your_tcgplayer_email@example.com +MASSPRICE_PASSWORD=your_tcgplayer_password + +# ── Host port (what port Unraid exposes) ──────────────────────────── +# Default is 5056 (arr-summary already uses 5055) — change if in use +HOST_PORT=5056 + +# ── Timezone ───────────────────────────────────────────────────────── +# Controls what "6:00 AM" means for your schedules +TZ=America/New_York diff --git a/.forgejo/workflows/deploy.yml b/.forgejo/workflows/deploy.yml new file mode 100644 index 0000000..66d9814 --- /dev/null +++ b/.forgejo/workflows/deploy.yml @@ -0,0 +1,29 @@ +name: Deploy + +on: + push: + branches: [main] + +jobs: + deploy: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Install rsync + run: apt-get update -qq && apt-get install -y -qq rsync + + # .env and data/ are server-only state and must survive every + # deploy — without the excludes, --delete removes them because + # they aren't in the repo. + - name: Sync source to deploy path + run: | + rsync -a --delete \ + --exclude='.git' \ + --exclude='.env' \ + --exclude='data' \ + ./ /mnt/user/appdata/massprice/ + + - name: Rebuild and restart + working-directory: /mnt/user/appdata/massprice + run: docker compose up -d --build diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..23a75d5 --- /dev/null +++ b/.gitignore @@ -0,0 +1,6 @@ +.env +data/ +__pycache__/ +*.pyc +.DS_Store +.venv/ diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..c490a8c --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,117 @@ +# MassPrice Scheduler + +A Flask app that drives a real headless-Chromium session against +`massprice.tcgplayer.com` (TCGplayer's bulk pricing-rule tool) to click +"Run Rule" for each of the user's pricing rules on a schedule set through +this app's own web UI. Runs as one Docker container on the user's Unraid +home server, deployed the same way as the other projects on that box +(`arr-summary`, `card-grader`, `lunch-notifier`, `n64-tracker`). + +## Why this exists + +MassPrice has its own native per-rule scheduler (day-of-week + one +time-of-day), but it's capped at 15 active schedules account-wide and +only allows **one run per day per rule** — no multiple runs/day. Native +runs also showed real clock drift during discovery (6–22 minutes late +against their scheduled time). This tool **replaces** native scheduling +entirely: each rule's native schedule should be set to Inactive once this +tool is confirmed working, so there's exactly one thing triggering runs. + +## Architecture + +- **`app.py`** — Flask app, single shared-password session auth + (`LOGIN_PASSWORD` env var, `hmac.compare_digest`) — this protects the + *admin UI itself*, unrelated to the MassPrice/TCGplayer login below. + Routes: `/` (Dashboard — rule cards with schedule editor), `/activity` + (run history), and a small JSON API + (`/api/rules/refresh`, `/api/rules//schedule`, + `/api/rules//run`) that the Dashboard's JS calls without full page + reloads. +- **`scheduler.py`** — an in-process APScheduler `BackgroundScheduler` + holding one `CronTrigger` job per `(rule, schedule entry)` pair, + rebuilt from `schedules.json` on startup and whenever the webui saves + a change. Triggered jobs (and manual "Run Now" clicks) are pushed onto + one `queue.Queue`, drained by a single dedicated worker thread — this + guarantees only one Playwright browser session is ever active at a + time, matching how a human would actually operate the site and + avoiding concurrent-session weirdness. + - **Must run in exactly one process.** `gunicorn.conf.py` pins + `workers = 1` for this reason — a second worker would instantiate a + second scheduler and duplicate every run. Fine for a low-traffic + single-user tool. +- **`automation.py`** — all Playwright logic. `list_rules()` scrapes + `https://massprice.tcgplayer.com/Rule/List` for the live rule + id/name pairs (cached to `rules_cache.json`, refreshed via the + Dashboard's "Refresh Rules" button or on first boot). `run_rule(id)` + finds that rule's `li.rule.widget-rule` row, clicks its + `.dropdown-ellipsis` three-dot toggle, then clicks the row's + `a.run-rule[data-rule-id=...]` item — mirroring exactly what a human + does in the UI. **No public API was found or used** — the "Run Rule" + click handler is JS/AJAX-bound (no plain href) and its request is + almost certainly anti-forgery-token protected, so automation drives + the real page rather than replaying a raw HTTP POST. +- **Session persistence** — a Playwright `storage_state` (cookies + + localStorage) is saved to `/data/auth_state.json` and reused across + runs: the tool logs in **once**, then slides the session forward after + every successful run, rather than hitting the login form on every + scheduled click. A fresh login (via `MASSPRICE_USERNAME` / + `MASSPRICE_PASSWORD`) only happens when the saved state is missing or + has expired (detected by landing back on a login page). +- **`storage.py`** — tiny JSON read/write helpers (temp-file + + `os.replace`, so a crash mid-write can't corrupt state) for + everything under `/data`: `schedules.json`, `auth_state.json`, + `run_history.json` (bounded to the last 200 entries), `rules_cache.json`. +- **`templates/` / `static/`** — a small, plain dark dashboard. This is + a deliberately different, simpler visual identity than Arr Summary's + teletext system — that design language is specific to that app, not a + house style to reuse here. + +## Known limitation — automated login + +`MASSPRICE_USERNAME`/`MASSPRICE_PASSWORD` drive a fully automated login +(the user's explicit choice over a manual hand-off flow). **If +TCGplayer's login ever presents a CAPTCHA or a 2FA challenge, automated +login will fail outright** — there is deliberately no fallback UI for +this yet; a failed login just shows up as a `failure` entry in Activity +with the error message. If this starts happening routinely, it needs a +design change (e.g. a Chrome-remote-debugging-based manual re-auth flow), +not a bigger retry loop. + +**The real logged-out MassPrice login form was never inspected** during +development — the account was already signed in when the site was +explored, and logging it out to look at the form would have disrupted +the user's real, live MassPrice session. `automation._login()` uses +resilient role/label/placeholder-based Playwright locators rather than +brittle CSS, but **expect to need a one-time adjustment against the real +form on first deploy** if the generic locators don't match. + +## Local development + +```bash +pip install -r requirements.txt +playwright install chromium # only needed if testing automation.py directly +python3 app.py +``` + +Runs on port 5000 (`PORT` env to change). Without `MASSPRICE_USERNAME`/ +`MASSPRICE_PASSWORD` set and without Chromium installed, the Dashboard +still boots — `_bootstrap()` in `app.py` best-effort-catches the initial +rule sync, so local UI/schedule-editing work doesn't require Docker or +real MassPrice credentials at all. `DATA_DIR` env var overrides where +JSON state is read/written (defaults to `/data`, which won't exist +locally — point it at a scratch directory). + +**Every commit should be matched with a version bump** — +`APP_VERSION`/`APP_VERSION_DATE` near the top of `app.py` — the same +"does the footer prove this deploy landed" signal used on Arr Summary. + +## Deployment + +See the `deploy-massprice` skill for the Forgejo push-to-deploy +pipeline and manual container restart steps. + +**Never put real credentials in the repo.** `.env` and `data/` are +gitignored and excluded from the deploy rsync precisely so +`MASSPRICE_USERNAME`/`MASSPRICE_PASSWORD`/`LOGIN_PASSWORD` and the saved +MassPrice session (`data/auth_state.json`) have somewhere to live +outside git. diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..f35789a --- /dev/null +++ b/Dockerfile @@ -0,0 +1,15 @@ +FROM mcr.microsoft.com/playwright/python:v1.47.0-jammy + +LABEL net.unraid.docker.webui="http://[IP]:[PORT:5000]/" +LABEL maintainer="MassPrice Scheduler" + +WORKDIR /app + +COPY requirements.txt . +RUN pip install --no-cache-dir -r requirements.txt + +COPY . . + +EXPOSE 5000 + +CMD ["gunicorn", "--config=gunicorn.conf.py", "app:app"] diff --git a/app.py b/app.py new file mode 100644 index 0000000..d622d6b --- /dev/null +++ b/app.py @@ -0,0 +1,157 @@ +import hmac +import os +from functools import wraps + +from flask import Flask, jsonify, redirect, render_template, request, session, url_for + +import automation +import scheduler +import storage + +APP_VERSION = "0.1.0" +APP_VERSION_DATE = "2026-08-31" + +app = Flask(__name__) +app.secret_key = os.environ.get("SECRET_KEY", "dev-secret-change-me") + +LOGIN_PASSWORD = os.environ.get("LOGIN_PASSWORD", "changeme") + +DAYS = ["mon", "tue", "wed", "thu", "fri", "sat", "sun"] +DAY_LABELS = {"mon": "Mon", "tue": "Tue", "wed": "Wed", "thu": "Thu", "fri": "Fri", "sat": "Sat", "sun": "Sun"} + + +def login_required(view): + @wraps(view) + def wrapped(*args, **kwargs): + if not session.get("authenticated"): + return redirect(url_for("login", next=request.path)) + return view(*args, **kwargs) + + return wrapped + + +@app.context_processor +def inject_version(): + return {"app_version": APP_VERSION, "app_version_date": APP_VERSION_DATE} + + +@app.route("/login", methods=["GET", "POST"]) +def login(): + error = None + if request.method == "POST": + password = request.form.get("password", "") + if hmac.compare_digest(password, LOGIN_PASSWORD): + session["authenticated"] = True + return redirect(request.args.get("next") or url_for("dashboard")) + error = "Incorrect password" + return render_template("login.html", error=error) + + +@app.route("/logout") +def logout(): + session.clear() + return redirect(url_for("login")) + + +def _merged_rules(): + """Combine the cached rule list with saved schedules and last-run status.""" + cache = storage.load("rules_cache.json", {"updated_at": None, "rules": []}) + schedules = storage.load("schedules.json", {}) + history = storage.load("run_history.json", []) + + last_run_by_rule = {} + for entry in history: + rid = entry.get("rule_id") + if rid and rid not in last_run_by_rule: + last_run_by_rule[rid] = entry + + rules = [] + for r in cache.get("rules", []): + rid = r["id"] + sched = schedules.get(rid, {}) + rules.append( + { + "id": rid, + "name": r["name"], + "enabled": sched.get("enabled", True), + "schedules": sched.get("schedules", []), + "last_run": last_run_by_rule.get(rid), + } + ) + return rules, cache.get("updated_at") + + +@app.route("/") +@login_required +def dashboard(): + rules, rules_updated_at = _merged_rules() + return render_template( + "dashboard.html", + rules=rules, + rules_updated_at=rules_updated_at, + days=DAYS, + day_labels=DAY_LABELS, + ) + + +@app.route("/activity") +@login_required +def activity(): + history = storage.load("run_history.json", []) + return render_template("activity.html", history=history) + + +@app.route("/api/rules/refresh", methods=["POST"]) +@login_required +def api_refresh_rules(): + try: + rules = automation.list_rules() + except Exception as exc: + return jsonify({"error": str(exc)}), 502 + return jsonify({"rules": rules}) + + +@app.route("/api/rules//schedule", methods=["POST"]) +@login_required +def api_save_schedule(rule_id): + body = request.get_json(force=True, silent=True) or {} + schedules = storage.load("schedules.json", {}) + cache = storage.load("rules_cache.json", {"rules": []}) + name = next((r["name"] for r in cache.get("rules", []) if r["id"] == rule_id), None) + + schedules[rule_id] = { + "name": name, + "enabled": bool(body.get("enabled", True)), + "schedules": body.get("schedules", []), + } + storage.save("schedules.json", schedules) + scheduler.rebuild_jobs() + return jsonify({"ok": True}) + + +@app.route("/api/rules//run", methods=["POST"]) +@login_required +def api_run_rule(rule_id): + cache = storage.load("rules_cache.json", {"rules": []}) + name = next((r["name"] for r in cache.get("rules", []) if r["id"] == rule_id), None) + scheduler.enqueue_run(rule_id, name) + return jsonify({"ok": True}) + + +def _bootstrap(): + scheduler.start(tz=os.environ.get("TZ")) + # Best-effort initial rule sync so the dashboard isn't blank before + # the user ever clicks "Refresh Rules". Silently skipped if + # Playwright/credentials aren't available yet (e.g. local dev). + if storage.load("rules_cache.json", None) is None: + try: + automation.list_rules() + except Exception: + pass + + +_bootstrap() + + +if __name__ == "__main__": + app.run(host="0.0.0.0", port=int(os.environ.get("PORT", 5000)), debug=False) diff --git a/automation.py b/automation.py new file mode 100644 index 0000000..2954f37 --- /dev/null +++ b/automation.py @@ -0,0 +1,202 @@ +"""Playwright automation against massprice.tcgplayer.com. + +Drives the real page rather than replaying a raw HTTP request: "Run Rule" +is bound to a JS/AJAX click handler with no plain href, and the request +it fires is almost certainly anti-forgery-token protected. + +Session handling: a Playwright `storage_state` (cookies + localStorage) +is persisted to AUTH_STATE_PATH and reused across runs, so the tool logs +in once and slides the session forward on every successful run rather +than hitting the login form every time. A fresh login (via +MASSPRICE_USERNAME/MASSPRICE_PASSWORD) only happens when the saved state +is missing or has expired. + +NOTE — the real logged-out login form was not inspected during +development (the account was already signed in during discovery, and +forcing a logout to inspect it would have disrupted the user's real +MassPrice session). The locators in _login() are written to be resilient +(role/label/placeholder based, not brittle CSS) but may need a one-time +adjustment against the live form on first deploy. See CLAUDE.md. +""" +import logging +import os +from datetime import datetime, timezone + +from playwright.sync_api import sync_playwright + +import storage + +logger = logging.getLogger("massprice.automation") + +BASE_URL = "https://massprice.tcgplayer.com" +AUTH_STATE_PATH = os.path.join(storage.DATA_DIR, "auth_state.json") + +MASSPRICE_USERNAME = os.environ.get("MASSPRICE_USERNAME") +MASSPRICE_PASSWORD = os.environ.get("MASSPRICE_PASSWORD") + +# Standard, necessary mitigations for headless Chromium inside Docker. +LAUNCH_ARGS = ["--no-sandbox", "--disable-dev-shm-usage"] + + +class AutomationError(Exception): + pass + + +def _new_context(browser): + if os.path.exists(AUTH_STATE_PATH): + return browser.new_context(storage_state=AUTH_STATE_PATH) + return browser.new_context() + + +def _save_state(context): + os.makedirs(storage.DATA_DIR, exist_ok=True) + context.storage_state(path=AUTH_STATE_PATH) + + +def _is_logged_out(page): + if "login" in page.url.lower() or "signin" in page.url.lower(): + return True + return page.locator('input[type="password"]').count() > 0 + + +def _login(page): + if not MASSPRICE_USERNAME or not MASSPRICE_PASSWORD: + raise AutomationError( + "MASSPRICE_USERNAME/MASSPRICE_PASSWORD not set — cannot log in" + ) + page.goto(f"{BASE_URL}/Dashboard") + if not _is_logged_out(page): + return + + email_field = ( + page.get_by_label("Email", exact=False) + .or_(page.get_by_placeholder("Email", exact=False)) + .or_(page.locator('input[type="email"]')) + .or_(page.locator('input[name="username" i]')) + ) + password_field = page.locator('input[type="password"]') + + email_field.first.fill(MASSPRICE_USERNAME) + password_field.first.fill(MASSPRICE_PASSWORD) + + submit = ( + page.get_by_role("button", name="Log In", exact=False) + .or_(page.get_by_role("button", name="Sign In", exact=False)) + .or_(page.locator('button[type="submit"]')) + ) + submit.first.click() + page.wait_for_load_state("networkidle") + + if _is_logged_out(page): + raise AutomationError( + "Login failed — still on a login page after submit " + "(bad credentials, or a CAPTCHA/2FA challenge that automated " + "login cannot get past — see CLAUDE.md)" + ) + + +def _ensure_authenticated(page): + page.goto(f"{BASE_URL}/Rule/List") + if _is_logged_out(page): + _login(page) + page.goto(f"{BASE_URL}/Rule/List") + if _is_logged_out(page): + raise AutomationError("Still logged out after a login attempt") + + +def list_rules(): + """Scrape /Rule/List for current rule id/name pairs. + + Returns a list of {"id": ..., "name": ...} dicts and caches it to + rules_cache.json. + """ + with sync_playwright() as p: + browser = p.chromium.launch(headless=True, args=LAUNCH_ARGS) + try: + context = _new_context(browser) + page = context.new_page() + _ensure_authenticated(page) + _save_state(context) + + rows = page.locator("li.rule.widget-rule") + rules = [] + for i in range(rows.count()): + row = rows.nth(i) + run_link = row.locator("a.run-rule").first + rule_id = run_link.get_attribute("data-rule-id") + if not rule_id: + continue + name_text = None + try: + # The rule name renders as the first line of text in + # the row, ahead of the "My Selling Prices" / + # "Schedule Active…" detail lines. + full_text = row.inner_text().strip() + name_text = full_text.splitlines()[0].strip() + except Exception: + pass + rules.append({"id": rule_id, "name": name_text or f"Rule {rule_id}"}) + + storage.save( + "rules_cache.json", + {"updated_at": datetime.now(timezone.utc).isoformat(), "rules": rules}, + ) + return rules + finally: + browser.close() + + +def run_rule(rule_id, rule_name=None): + """Click "Run Rule" for the given rule id. Logs the outcome to + run_history.json and returns (status, message).""" + status = "failure" + message = "" + with sync_playwright() as p: + browser = p.chromium.launch(headless=True, args=LAUNCH_ARGS) + try: + context = _new_context(browser) + page = context.new_page() + _ensure_authenticated(page) + _save_state(context) + + run_selector = f'a.run-rule[data-rule-id="{rule_id}"]' + row = page.locator("li.rule.widget-rule").filter( + has=page.locator(run_selector) + ) + if row.count() == 0: + raise AutomationError( + f"Rule id {rule_id} not found on /Rule/List " + "(renamed or deleted on the MassPrice side?)" + ) + + row.first.locator(".dropdown-ellipsis").click() + run_link = row.first.locator(run_selector) + run_link.wait_for(state="visible", timeout=5000) + run_link.click() + page.wait_for_timeout(1500) # let the click's AJAX call fire + + status = "success" + message = "Run Rule clicked" + except Exception as exc: + message = str(exc) + logger.exception("run_rule failed for rule %s", rule_id) + finally: + browser.close() + + _log_history(rule_id, rule_name, status, message) + return status, message + + +def _log_history(rule_id, rule_name, status, message): + history = storage.load("run_history.json", []) + history.insert( + 0, + { + "rule_id": rule_id, + "rule_name": rule_name, + "timestamp": datetime.now(timezone.utc).isoformat(), + "status": status, + "message": message, + }, + ) + storage.save("run_history.json", history[:200]) diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..ae95679 --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,21 @@ +services: + massprice: + build: . + container_name: massprice + # Headless Chromium's default /dev/shm (64MB) is too small in Docker + # and causes renderer crashes — this is the standard mitigation. + shm_size: "1gb" + ports: + - "${HOST_PORT:-5056}:5000" + volumes: + # Schedules, run history, the rule-name cache, and the saved + # MassPrice login session must outlive `up -d --build`, which + # recreates the container and discards its filesystem. + - ./data:/data + environment: + - LOGIN_PASSWORD=${LOGIN_PASSWORD} + - SECRET_KEY=${SECRET_KEY} + - MASSPRICE_USERNAME=${MASSPRICE_USERNAME} + - MASSPRICE_PASSWORD=${MASSPRICE_PASSWORD} + - TZ=${TZ:-America/New_York} + restart: unless-stopped diff --git a/gunicorn.conf.py b/gunicorn.conf.py new file mode 100644 index 0000000..26f9812 --- /dev/null +++ b/gunicorn.conf.py @@ -0,0 +1,13 @@ +# Gunicorn configuration +import os + +# Exactly one worker: APScheduler and the Playwright run-queue live in +# process memory, not a shared store — a second worker would duplicate +# every scheduled job and run each rule twice. Fine for a low-traffic +# single-user admin tool. +workers = 1 +threads = 4 +worker_class = "gthread" +bind = "0.0.0.0:" + os.environ.get("PORT", "5000") +timeout = 120 # a rule run launches Chromium, logs in, and clicks through +keepalive = 5 diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..b9546a5 --- /dev/null +++ b/requirements.txt @@ -0,0 +1,4 @@ +flask==3.1.0 +gunicorn==23.0.0 +playwright==1.47.0 +apscheduler==3.10.4 diff --git a/scheduler.py b/scheduler.py new file mode 100644 index 0000000..f1778eb --- /dev/null +++ b/scheduler.py @@ -0,0 +1,93 @@ +"""In-process cron scheduling for rule runs. + +A single background worker thread owns every Playwright run — both +APScheduler-triggered runs and manual "Run Now" clicks are pushed onto +one queue.Queue so exactly one browser session is ever active at a time. +This must run in exactly one process; see gunicorn.conf.py's workers=1. +""" +import logging +import os +import queue +import threading + +import pytz +from apscheduler.schedulers.background import BackgroundScheduler +from apscheduler.triggers.cron import CronTrigger + +import automation +import storage + +logger = logging.getLogger("massprice.scheduler") + +_work_queue: "queue.Queue" = queue.Queue() +# Timezone must be set at construction time, not via a later .configure() +# call — APScheduler resolves a timezone immediately in __init__ via +# tzlocal's system lookup, which fails inside this container image for +# an arbitrary $TZ (no zoneinfo/tzdata wiring), so the explicit TZ env +# var has to win before that lookup ever runs. +_scheduler = BackgroundScheduler(timezone=pytz.timezone(os.environ.get("TZ", "UTC"))) +_started = False + +VALID_DAYS = ["mon", "tue", "wed", "thu", "fri", "sat", "sun"] + + +def _worker_loop(): + while True: + rule_id, rule_name = _work_queue.get() + try: + automation.run_rule(rule_id, rule_name) + except Exception: + logger.exception("Unhandled error running rule %s", rule_id) + finally: + _work_queue.task_done() + + +def enqueue_run(rule_id, rule_name=None): + _work_queue.put((rule_id, rule_name)) + + +def _job_id(rule_id, index): + return f"rule-{rule_id}-{index}" + + +def rebuild_jobs(): + """Clear and re-add every APScheduler job from schedules.json.""" + for job in _scheduler.get_jobs(): + job.remove() + + schedules = storage.load("schedules.json", {}) + for rule_id, rule in schedules.items(): + if not rule.get("enabled", True): + continue + for idx, entry in enumerate(rule.get("schedules", [])): + days = [d for d in entry.get("days", []) if d in VALID_DAYS] + time_str = entry.get("time") + if not days or not time_str: + continue + hour, minute = time_str.split(":") + trigger = CronTrigger( + day_of_week=",".join(days), + hour=int(hour), + minute=int(minute), + ) + _scheduler.add_job( + enqueue_run, + trigger=trigger, + args=[rule_id, rule.get("name")], + id=_job_id(rule_id, idx), + replace_existing=True, + misfire_grace_time=300, + ) + logger.info("Rebuilt %d scheduled job(s)", len(_scheduler.get_jobs())) + + +def start(tz=None): + # tz is accepted for backwards-compat call sites but is a no-op now + # that timezone is fixed at construction time above. + global _started + if _started: + return + rebuild_jobs() + _scheduler.start() + threading.Thread(target=_worker_loop, daemon=True).start() + _started = True diff --git a/static/css/style.css b/static/css/style.css new file mode 100644 index 0000000..5869b51 --- /dev/null +++ b/static/css/style.css @@ -0,0 +1,159 @@ +:root { + --bg: #12161c; + --panel: #1b212b; + --panel-alt: #232a35; + --border: #2b3340; + --text: #e6e9ef; + --text-dim: #9aa4b2; + --accent: #3b82f6; + --accent-dim: #1e3a5f; + --success: #34d399; + --failure: #f87171; + --radius: 8px; +} + +* { box-sizing: border-box; } + +body { + margin: 0; + background: var(--bg); + color: var(--text); + font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif; + font-size: 15px; +} + +h1 { font-size: 1.4rem; margin: 0; } + +.topbar { + display: flex; + align-items: center; + justify-content: space-between; + padding: 16px 24px; + border-bottom: 1px solid var(--border); +} +.brand { font-weight: 700; font-size: 1.1rem; } +.brand span { color: var(--text-dim); font-weight: 400; } + +nav a { + color: var(--text-dim); + text-decoration: none; + margin-left: 20px; + font-size: 0.9rem; +} +nav a:hover, nav a.active { color: var(--text); } + +main { max-width: 1100px; margin: 0 auto; padding: 24px; } + +footer { + text-align: center; + color: var(--text-dim); + font-size: 0.75rem; + padding: 24px; +} + +.page-header { + display: flex; + justify-content: space-between; + align-items: center; + margin-bottom: 20px; + flex-wrap: wrap; + gap: 12px; +} +.page-header-meta { display: flex; align-items: center; gap: 12px; } +.muted { color: var(--text-dim); } +.small { font-size: 0.78rem; } + +.btn { + background: var(--panel-alt); + color: var(--text); + border: 1px solid var(--border); + border-radius: var(--radius); + padding: 8px 14px; + cursor: pointer; + font-size: 0.85rem; +} +.btn:hover { border-color: var(--accent); } +.btn:disabled { opacity: 0.6; cursor: default; } +.btn-primary { background: var(--accent); border-color: var(--accent); color: white; } +.btn-small { padding: 6px 10px; font-size: 0.8rem; } + +.rule-grid { + display: grid; + grid-template-columns: repeat(auto-fill, minmax(320px, 1fr)); + gap: 16px; +} + +.rule-card { + background: var(--panel); + border: 1px solid var(--border); + border-radius: var(--radius); + padding: 18px; + display: flex; + flex-direction: column; + gap: 12px; +} + +.rule-card-header { display: flex; justify-content: space-between; align-items: center; gap: 12px; } +.rule-card-header h2 { font-size: 1.05rem; margin: 0; } + +.switch { position: relative; display: inline-block; width: 40px; height: 22px; flex-shrink: 0; } +.switch input { opacity: 0; width: 0; height: 0; position: absolute; } +.slider { + position: absolute; inset: 0; background: var(--panel-alt); + border: 1px solid var(--border); border-radius: 22px; cursor: pointer; transition: .15s; +} +.slider::before { + content: ""; position: absolute; height: 16px; width: 16px; left: 2px; top: 2px; + background: var(--text-dim); border-radius: 50%; transition: .15s; +} +.switch input:checked + .slider { background: var(--accent-dim); border-color: var(--accent); } +.switch input:checked + .slider::before { transform: translateX(18px); background: var(--accent); } + +.rule-last-run { font-size: 0.82rem; } +.status-dot { display: inline-block; width: 8px; height: 8px; border-radius: 50%; margin-right: 6px; } +.status-dot.status-success { background: var(--success); } +.status-dot.status-failure { background: var(--failure); } +.status-row-failure td { color: #f2a3a3; } + +.schedule-list { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 6px; } +.schedule-chip { + display: flex; align-items: center; gap: 10px; + background: var(--panel-alt); border: 1px solid var(--border); border-radius: 6px; + padding: 6px 10px; font-size: 0.82rem; +} +.chip-time { font-variant-numeric: tabular-nums; font-weight: 600; } +.chip-days { color: var(--text-dim); flex: 1; text-transform: uppercase; letter-spacing: 0.03em; font-size: 0.72rem; } +.chip-remove { background: none; border: none; color: var(--text-dim); cursor: pointer; font-size: 1.1rem; line-height: 1; } +.chip-remove:hover { color: var(--failure); } + +.schedule-add { display: flex; flex-direction: column; gap: 8px; border-top: 1px dashed var(--border); padding-top: 12px; } +.schedule-add-row { display: flex; gap: 8px; } +.new-time { + background: var(--panel-alt); border: 1px solid var(--border); color: var(--text); + border-radius: 6px; padding: 6px 8px; font-size: 0.85rem; flex: 1; +} +.day-toggles { display: flex; gap: 4px; flex-wrap: wrap; } +.day-toggle { + background: var(--panel-alt); border: 1px solid var(--border); color: var(--text-dim); + border-radius: 6px; padding: 4px 8px; font-size: 0.72rem; cursor: pointer; text-transform: uppercase; +} +.day-toggle.active { background: var(--accent-dim); border-color: var(--accent); color: var(--text); } + +.rule-actions { display: flex; justify-content: flex-end; } + +.login-card { + max-width: 320px; margin: 80px auto; background: var(--panel); + border: 1px solid var(--border); border-radius: var(--radius); padding: 28px; + display: flex; flex-direction: column; gap: 12px; +} +.login-card input { + background: var(--panel-alt); border: 1px solid var(--border); color: var(--text); + border-radius: 6px; padding: 10px; font-size: 1rem; +} +.error { color: var(--failure); font-size: 0.85rem; } + +.empty-state { color: var(--text-dim); } + +.activity-table { width: 100%; border-collapse: collapse; font-size: 0.85rem; } +.activity-table th, .activity-table td { text-align: left; padding: 8px 12px; border-bottom: 1px solid var(--border); } +.activity-table th { color: var(--text-dim); font-weight: 500; font-size: 0.75rem; text-transform: uppercase; } diff --git a/static/js/app.js b/static/js/app.js new file mode 100644 index 0000000..2473193 --- /dev/null +++ b/static/js/app.js @@ -0,0 +1,107 @@ +document.addEventListener("DOMContentLoaded", () => { + const refreshBtn = document.getElementById("refresh-rules"); + if (refreshBtn) { + refreshBtn.addEventListener("click", async () => { + refreshBtn.disabled = true; + const original = refreshBtn.textContent; + refreshBtn.textContent = "Refreshing…"; + try { + const res = await fetch("/api/rules/refresh", { method: "POST" }); + if (!res.ok) { + const data = await res.json().catch(() => ({})); + alert("Refresh failed: " + (data.error || res.status)); + return; + } + window.location.reload(); + } finally { + refreshBtn.disabled = false; + refreshBtn.textContent = original; + } + }); + } + + document.querySelectorAll(".rule-card").forEach(initRuleCard); +}); + +function initRuleCard(card) { + const ruleId = card.dataset.ruleId; + + card.querySelector(".rule-enabled").addEventListener("change", (e) => { + saveSchedule(card, { enabledOverride: e.target.checked }); + }); + + card.querySelectorAll(".day-toggle").forEach((btn) => { + btn.addEventListener("click", () => btn.classList.toggle("active")); + }); + + card.querySelector(".add-schedule").addEventListener("click", () => { + const time = card.querySelector(".new-time").value; + const days = [...card.querySelectorAll(".day-toggle.active")].map((b) => b.dataset.day); + if (!time || days.length === 0) { + alert("Pick a time and at least one day."); + return; + } + addChip(card, time, days); + card.querySelectorAll(".day-toggle.active").forEach((b) => b.classList.remove("active")); + saveSchedule(card); + }); + + card.querySelectorAll(".chip-remove").forEach((btn) => bindRemove(btn)); + + card.querySelector(".run-now").addEventListener("click", async (e) => { + const btn = e.currentTarget; + const original = btn.textContent; + btn.disabled = true; + btn.textContent = "Queuing…"; + try { + const res = await fetch(`/api/rules/${ruleId}/run`, { method: "POST" }); + btn.textContent = res.ok ? "Queued" : "Failed"; + } catch (err) { + btn.textContent = "Failed"; + } finally { + setTimeout(() => { + btn.textContent = original; + btn.disabled = false; + }, 2000); + } + }); +} + +function bindRemove(btn) { + btn.addEventListener("click", () => { + const card = btn.closest(".rule-card"); + btn.closest(".schedule-chip").remove(); + saveSchedule(card); + }); +} + +function addChip(card, time, days) { + const list = card.querySelector(".schedule-list"); + const li = document.createElement("li"); + li.className = "schedule-chip"; + li.innerHTML = + `${time}` + + `${days.join(" ")}` + + ``; + list.appendChild(li); + bindRemove(li.querySelector(".chip-remove")); +} + +async function saveSchedule(card, opts = {}) { + const ruleId = card.dataset.ruleId; + const enabled = + opts.enabledOverride !== undefined ? opts.enabledOverride : card.querySelector(".rule-enabled").checked; + const schedules = [...card.querySelectorAll(".schedule-chip")].map((chip) => ({ + time: chip.querySelector(".chip-time").textContent.trim(), + days: chip + .querySelector(".chip-days") + .textContent.trim() + .split(/\s+/) + .filter(Boolean), + })); + await fetch(`/api/rules/${ruleId}/schedule`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ enabled, schedules }), + }); +} diff --git a/storage.py b/storage.py new file mode 100644 index 0000000..5c5c30f --- /dev/null +++ b/storage.py @@ -0,0 +1,35 @@ +"""Small JSON read/write helpers for the state files under DATA_DIR. + +Every file is written via a temp-file + os.replace so a crash mid-write +never leaves a half-written JSON file behind for the next read. +""" +import json +import os +import threading + +DATA_DIR = os.environ.get("DATA_DIR", "/data") + +_lock = threading.Lock() + + +def _path(name): + return os.path.join(DATA_DIR, name) + + +def load(name, default): + path = _path(name) + with _lock: + if not os.path.exists(path): + return default + with open(path) as f: + return json.load(f) + + +def save(name, data): + path = _path(name) + with _lock: + os.makedirs(DATA_DIR, exist_ok=True) + tmp = path + ".tmp" + with open(tmp, "w") as f: + json.dump(data, f, indent=2) + os.replace(tmp, path) diff --git a/templates/_shell.html b/templates/_shell.html new file mode 100644 index 0000000..63548fe --- /dev/null +++ b/templates/_shell.html @@ -0,0 +1,26 @@ + + + + + + {% block title %}MassPrice Scheduler{% endblock %} + + + +
+
MassPrice Scheduler
+ {% if session.get('authenticated') %} + + {% endif %} +
+
+ {% block content %}{% endblock %} +
+
v{{ app_version }} · {{ app_version_date }}
+ {% block scripts %}{% endblock %} + + diff --git a/templates/activity.html b/templates/activity.html new file mode 100644 index 0000000..cd9c07c --- /dev/null +++ b/templates/activity.html @@ -0,0 +1,24 @@ +{% extends "_shell.html" %} +{% block title %}Activity · MassPrice Scheduler{% endblock %} +{% block content %} +

Activity

+{% if not history %} +

No runs recorded yet.

+{% else %} + + + + + + {% for entry in history %} + + + + + + + {% endfor %} + +
TimeRuleStatusMessage
{{ entry.timestamp }}{{ entry.rule_name or entry.rule_id }}{{ entry.status }}{{ entry.message }}
+{% endif %} +{% endblock %} diff --git a/templates/dashboard.html b/templates/dashboard.html new file mode 100644 index 0000000..ea365ba --- /dev/null +++ b/templates/dashboard.html @@ -0,0 +1,68 @@ +{% extends "_shell.html" %} +{% block title %}Dashboard · MassPrice Scheduler{% endblock %} +{% block content %} +
+

Rules

+
+ {% if rules_updated_at %}Rules last synced {{ rules_updated_at }}{% endif %} + +
+
+ +{% if not rules %} +

No rules loaded yet. Click "Refresh Rules" to pull the current rule list from MassPrice.

+{% endif %} + +
+ {% for rule in rules %} +
+
+

{{ rule.name }}

+ +
+ +
+ {% if rule.last_run %} + + Last run {{ rule.last_run.timestamp }} — {{ rule.last_run.status }} + {% if rule.last_run.status == 'failure' %}
{{ rule.last_run.message }}
{% endif %} + {% else %} + No runs yet + {% endif %} +
+ +
    + {% for entry in rule.schedules %} +
  • + {{ entry.time }} + {{ entry.days | join(' ') }} + +
    • + {% endfor %} +
+ +
+
+ + +
+
+ {% for d in days %} + + {% endfor %} +
+
+ +
+ +
+
+ {% endfor %} +
+{% endblock %} +{% block scripts %} + +{% endblock %} diff --git a/templates/login.html b/templates/login.html new file mode 100644 index 0000000..1e3fec8 --- /dev/null +++ b/templates/login.html @@ -0,0 +1,13 @@ +{% extends "_shell.html" %} +{% block title %}Log In · MassPrice Scheduler{% endblock %} +{% block content %} + +{% endblock %}