Pull daily instead of on day 25; skip past dates and weekend misreads

The monthly day-25 pull pushed whatever month the district's permalink
served. September's PDF was still live on 9/29, so the 9/25 pull
re-pushed September and October would not have been pulled until 10/25.
Home Assistant now dedups lunch events per date, so pull every day and
pick up a new month the evening it's posted.

- push only today onward; skip weekend dates that have items (a misread
  date: Fri 9/25's menu once landed on Sat 9/26) and log a warning
- settings schema is just calendar_run_time; load_settings keeps only
  known keys and startup rewrites the file, dropping stale SMS-era data
- UI: remove day-of-month field and the "no dedup" warning, fix buttons
  that reverted to "This Week" labels
- docs: daily scheduler, HA queued mode + dedup, calendar is iCloud

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

.env.example

@@ -12,13 +12,9 @@ HA_WEBHOOK_URL=
 ANTHROPIC_API_KEY=
 
 # --- Optional ---
-# Day of the month (1-28) to pull whichever month is currently published
+# Time of the daily pull (24-hour HH:MM, container timezone). Each day it
-# and push it to the calendar. Re-pushing the same month creates duplicate
+# pushes today onward from whichever month is published; Home Assistant
-# events, so pick one day near month-end (the district tends to post next
+# skips dates that already have a lunch event, so daily pulls are safe.
-# month's menu a few days early).
-CALENDAR_RUN_DAY_OF_MONTH=25
-
-# Time to run the pull (24-hour HH:MM, container timezone)
 CALENDAR_RUN_TIME=18:00
 
 # HTTP server port

CLAUDE.md

@@ -11,12 +11,17 @@ user's request in favor of Home Assistant calendar events exclusively. If
 you find references to texting/SMS/phone numbers anywhere (docs, old
 commits), they describe the pre-2026-09-03 app, not the current one.
 
+**2026-09-29: monthly → daily pulls, HA-side dedup.** See "Pull scheduler"
+and "Home Assistant integration" below. Anything describing a day-25
+monthly pull or "no dedup, re-pushing creates duplicates" describes the
+pre-2026-09-29 app.
+
 **2026-09-08: pivoted from Nutrislice to a Claude-read PDF, and from
 weekly to monthly pulls.** The district discontinued Nutrislice (the old
 scraper hit `kyrene.api.nutrislice.com`, a clean per-day JSON API) in
 favor of publishing one monthly menu PDF per grade band on
 `kyrene.org/family-student-resources/food-services/school-menus`. See
-"Menu source" and "Monthly pull scheduler" below for how that's handled
+"Menu source" and "Daily pull scheduler" below for how that's handled
 now. Anything describing a weekly Nutrislice pull describes the
 pre-2026-09-08 app.
 
@@ -54,16 +59,15 @@ pre-2026-09-08 app.
   fail-soft-into-`_status`-error behavior as a Home Assistant push failure.
   `GET /api/integrations` reports both keys' configured/not-configured
   state.
-- Settings (calendar pull schedule) persist to `/data/settings.json`, not
+- Settings (just `calendar_run_time`) persist to `/data/settings.json`, not
-  environment variables — env vars (`CALENDAR_RUN_TIME`,
+  environment variables — the env var `CALENDAR_RUN_TIME` is a default used
-  `CALENDAR_RUN_DAY_OF_MONTH`) are defaults used only the very first time,
+  only before a settings file exists. `_DEFAULTS` is the complete schema:
-  before a settings file exists. `load_settings()` merges the file's
+  `load_settings()` merges the file *over* `_DEFAULTS` (so an old file
-  contents *over* `_DEFAULTS` rather than returning the file verbatim —
+  missing a key still yields a valid schedule instead of zero jobs) and
-  this matters because the server's settings.json predates several schema
+  **keeps only keys in `_DEFAULTS`**; startup rewrites the file with that
-  changes (SMS era, then the weekly-schedule era) and is missing whatever
+  result. That pruning (2026-09-29) removed the SMS era's phone numbers /
-  keys the current schema expects; without the merge, the scheduler would
+  carrier gateway / vacations and the old `calendar_run_day_of_month` from
-  silently come up with zero jobs after every deploy until someone happened
+  the server's settings.json — they had lingered, unread, since the pivots.
-  to re-save the Schedule card in the UI.
 - **No vacation/no-push date ranges (removed 2026-09-03, briefly existed).**
   A day with no posted menu already produces an empty `items` list from
   `get_month_menu()`, and `_push_calendar()` already skips pushing when
@@ -77,53 +81,51 @@ pre-2026-09-08 app.
   does anything externally-visible. On the HA side
   (`automation.lunch_menu_family_calendar`, `http://192.168.86.207:8123`),
   a `local_only` webhook trigger runs `calendar.create_event` against
-  `calendar.family_2` (the "Family" Google Calendar entity — there are two
+  `calendar.family_2` (the shared iCloud "Family" calendar, via CalDAV —
-  entities both named "Family" in this HA instance, `calendar.family` and
+  there are two entities both named "Family" in this HA instance,
-  `calendar.family_2`; this integration deliberately targets `_2` per user
+  `calendar.family` and `calendar.family_2`; this integration deliberately
-  choice). `calendar.family_2` only supports `create_event`, not
+  targets `_2` per user choice). `calendar.family_2` supports
-  `delete_event` — a bad push can't be cleaned up via the HA API, only
+  `create_event` and `get_events` but not `delete_event` — a bad push can't
-  manually in Google Calendar. No duplicate-prevention: re-pushing the
+  be cleaned up via the HA API, only by hand in the Calendar app.
-  same date's menu creates a second event — this is why the default
+  **Automation mode history — `queued` since 2026-09-29.** The app fires
-  schedule is a single day/month, not daily (see below).
+  one webhook POST per school day back-to-back (HA answers each POST
-  **`mode: parallel`, not the default `single` (fixed 2026-09-03 after a
+  immediately, so the app never waits on the calendar write). `single`
-  real bug):** a push fires one webhook POST per school day back-to-back
+  (default) dropped all but the first as `failed_single` (fixed 2026-09-03
-  with no delay. Under `mode: single`, HA rejects any webhook trigger that
+  → `parallel`); `parallel` then sent ~20 simultaneous CalDAV PUTs on a
-  arrives while a previous run is still executing (`calendar.create_event`
+  month push, overflowing iCloud (connection pool full + ~10× `500
-  itself takes ~600-700ms) — most days in a batch were silently dropped as
+  Internal Server Error` on the 2026-09-25 push). Now `queued` (max 40):
-  `failed_single`, so only one day's event ever actually got created per
+  one write at a time, nothing dropped. **Dedup (2026-09-29):** before
-  push. Found via `ha_get_automation_traces` after the user reported a
+  creating, the automation reads the date with `calendar.get_events` and
-  manual test push not showing up on the calendar. Each invocation is
+  stops if a `🍱 Mariposa Lunch` event already exists — that's what makes
-  independent (one `calendar.create_event` call, no shared state), so
+  daily pulls safe. It verifies the event saved and retries once after 15s.
-  `parallel` is the correct mode per HA's own guidance ("independent
+  Pre-dedup pushes left 2-3 copies on most early-September days; those
-  per-entity actions → parallel"), not just a workaround. The webhook id is
+  have to be deleted by hand. The webhook id is
   a random token that lives only in the server's `.env` (`HA_WEBHOOK_URL`)
   and in the automation's trigger config — never committed here. If the
   calendar push stops working, check both ends: `GET /api/integrations`
   on this app reports whether `HA_WEBHOOK_URL` is set at all, and
   `ha_get_automation_traces` on `automation.lunch_menu_family_calendar`
   shows whether HA is even receiving the webhook.
-- **Monthly pull scheduler (weekly → monthly, 2026-09-08):** calendar pulls
+- **Daily pull scheduler (monthly → daily, 2026-09-29):** `push_month_to_calendar()`
-  run on their own schedule (`calendar_run_time`/`calendar_run_day_of_month`
+  runs every day at `calendar_run_time` (default 18:00; `schedule` job
-  in settings, `schedule` jobs tagged `"calendar"`), completely decoupled
+  tagged `"calendar"` — `schedule.clear("calendar")`, not a bare
-  from anything else — the tag/isolation pattern
+  `schedule.clear()`, so a future second schedule wouldn't stomp it).
-  (`schedule.clear("calendar")` instead of a bare `schedule.clear()`) is
+  **Why daily:** the old once-a-month day-25 pull pushed whatever month was
-  deliberate so a future second schedule wouldn't stomp this one. The
+  live at the permalink; when the district posted late (September 2026: the
-  `schedule` library has no native day-of-month trigger, so
+  permalink still served September on 09-29), it silently re-pushed the
-  `apply_calendar_schedule()` registers a *daily* job
+  current month and the next month's menu wouldn't be pulled until the
-  (`_maybe_push_month`) that no-ops unless `datetime.now().day` matches
+  following 25th — most of that month gone. Daily pulls pick up a new month
-  the configured day — `push_month_to_calendar()` only actually runs on
+  the evening it's posted; HA-side dedup makes the repeats harmless, and the
-  that one day per month. `next_calendar_run()` computes the real next
+  Claude read costs well under a cent a day. Each pull pushes **only today
-  firing directly from settings (handling month/year rollover and clamping
+  onward** (past dates are skipped — they're already on the calendar or no
-  the configured day to short months, e.g. day 30 in February → the 28th)
+  longer matter) and **skips weekend dates that have items**, logging a
-  for the UI's "Next Scheduled" field, since the underlying `schedule` job's
+  warning and noting it in the status result: there's never school lunch on
-  own `next_run` is meaningless here (it's always just "tomorrow"). Default
+  a weekend, so that's a misread date (seen once: Fri 9/25's menu landed on
-  is day 25 at 18:00 — picked because the district tends to post next
+  Sat 9/26). Whatever month is live at the permalink is what gets read —
-  month's PDF a few days before month-end, so pulling then usually catches
+  there's no logic forcing "next month specifically."
-  the *upcoming* month rather than the one ending. Whatever month is live
-  at the permalink when the job fires is what gets pushed — there's no
-  logic forcing "next month specifically."
 - **Manual actions:** "Push This Month to Calendar" (`POST /api/push-calendar`
-  → `push_month_to_calendar()`, same function the schedule calls) and
+  → `push_month_to_calendar()`, same function the schedule calls — so it
+  also pushes only today onward and skips weekend misreads) and
   "Preview This Month's Menu" (`GET /api/preview` → `get_month_menu()`
   directly, pure dry run, no Home Assistant push, no side effects at all —
   use this to check the scraper before trusting a scheduled push).

app.py

@@ -2,13 +2,12 @@
 Lunch Notifier — pulls the Mariposa lunch menu and pushes it to a Home
 Assistant calendar. Visit http://localhost:2323 to manage settings.
 """
-import calendar
 import json
 import logging
 import os
 import threading
 import time
-from datetime import datetime
+from datetime import date, datetime, timedelta
 from pathlib import Path
 
 import requests
@@ -25,10 +24,10 @@ app = Flask(__name__)
 SETTINGS_FILE = Path(os.environ.get("SETTINGS_FILE", "/data/settings.json"))
 PORT = int(os.environ.get("PORT", "2323"))
 
-# Env-var defaults (used only when no settings file exists yet)
+# Env-var defaults (used only when no settings file exists yet). These are
+# also the complete settings schema: load_settings() drops any other key.
 _DEFAULTS = {
     "calendar_run_time": os.environ.get("CALENDAR_RUN_TIME", "18:00"),
-    "calendar_run_day_of_month": int(os.environ.get("CALENDAR_RUN_DAY_OF_MONTH", "25")),
 }
 
 _status = {"last_calendar_run": None, "calendar_result": None}
@@ -42,11 +41,12 @@ def load_settings() -> dict:
         try:
             with open(SETTINGS_FILE) as f:
                 # Merge over _DEFAULTS so a settings.json from before a schema
-                # change (e.g. the pre-calendar SMS-era file, or the
+                # change still gets a valid schedule rather than silently
-                # weekly-schedule era before calendar_run_day_of_month
+                # scheduling nothing, and keep only current keys so stale
-                # existed) still gets a valid schedule rather than silently
+                # ones (SMS-era phone numbers, the old day-of-month pull)
-                # scheduling nothing.
+                # don't linger. Startup rewrites the file with this result.
-                return {**_DEFAULTS, **json.load(f)}
+                stored = json.load(f)
+                return {**_DEFAULTS, **{k: v for k, v in stored.items() if k in _DEFAULTS}}
         except Exception:
             pass
     return dict(_DEFAULTS)
@@ -83,11 +83,15 @@ def _push_calendar(items: list[str], date_str: str | None) -> str:
 
 
 def push_month_to_calendar() -> None:
-    """Fetch whichever month is currently published and push every day that
+    """Fetch whichever month is currently published and push every day from
-    has a posted menu — days with nothing posted (weekends, breaks,
+    today onward that has a posted menu. Days with nothing posted (breaks,
-    holidays) are skipped automatically since there's nothing to create an
+    holidays) are skipped since there's nothing to create an event from;
-    event from. Runs on its own schedule (calendar_run_time/
+    past days are skipped since they're already on the calendar or no
-    calendar_run_day_of_month), independent of anything else."""
+    longer matter; weekend days are skipped even if the PDF read put a menu
+    on one (there's never school lunch on a weekend, so that's a misread
+    date — seen once as Fri 9/25's menu landing on Sat 9/26). Runs daily
+    (calendar_run_time); Home Assistant skips dates that already have a
+    lunch event, so repeated pulls don't create duplicates."""
     global _status
 
     try:
@@ -97,8 +101,20 @@ def push_month_to_calendar() -> None:
         _status = {**_status, "last_calendar_run": datetime.now().isoformat(), "calendar_result": f"error: {e}"}
         return
 
-    pushed = errors = no_items = 0
+    today = date.today()
+    pushed = errors = no_items = past = weekend = 0
     for day in month:
+        d = date.fromisoformat(day["date"])
+        if d < today:
+            past += 1
+            continue
+        if d.weekday() >= 5:
+            if day["items"]:
+                log.warning(f"Skipping {day['date']} ({d:%a}): menu on a weekend is a misread date")
+                weekend += 1
+            else:
+                no_items += 1
+            continue
         result = _push_calendar(day["items"], day["date"])
         if result == "ok":
             pushed += 1
@@ -107,7 +123,8 @@ def push_month_to_calendar() -> None:
         else:
             errors += 1
 
-    log.info(f"Month push: {pushed} pushed, {errors} errors, {no_items} no menu")
+    log.info(f"Month push: {pushed} pushed, {errors} errors, {no_items} no menu, "
+             f"{past} past, {weekend} weekend misreads skipped")
 
     if errors:
         summary = f"{pushed} ok, {errors} error" + ("s" if errors != 1 else "")
@@ -115,6 +132,8 @@ def push_month_to_calendar() -> None:
         summary = f"ok ({pushed} day{'s' if pushed != 1 else ''})"
     else:
         summary = "no items"
+    if weekend:
+        summary += f", {weekend} weekend misread{'s' if weekend != 1 else ''} skipped"
 
     _status = {
         **_status,
@@ -125,43 +144,20 @@ def push_month_to_calendar() -> None:
 
 # ── Scheduler ─────────────────────────────────────────────────────────────────
 
-def _maybe_push_month(day_of_month: int) -> None:
-    """schedule fires this daily at run_time; only actually pushes on the
-    configured day of the month (the `schedule` library has no native
-    day-of-month trigger)."""
-    if datetime.now().day == day_of_month:
-        push_month_to_calendar()
-
-
 def apply_calendar_schedule(settings: dict) -> None:
     with _lock:
         schedule.clear("calendar")
         run_time = settings.get("calendar_run_time", "18:00")
-        day_of_month = settings.get("calendar_run_day_of_month", 25)
+        schedule.every().day.at(run_time).do(push_month_to_calendar).tag("calendar")
-        schedule.every().day.at(run_time).do(_maybe_push_month, day_of_month).tag("calendar")
+        log.info(f"Scheduled calendar pull: daily at {run_time}")
-        log.info(f"Scheduled calendar pull: day {day_of_month} of each month at {run_time}")
 
 
 def next_calendar_run(settings: dict) -> datetime:
-    """The `schedule` job above fires daily (it has no native day-of-month
-    trigger — see _maybe_push_month), so its own next_run is just
-    "tomorrow", not the actual next push. Compute the real one directly
-    from settings for display in the UI."""
     run_time = settings.get("calendar_run_time", "18:00")
     hour, minute = (int(p) for p in run_time.split(":"))
-    day_of_month = settings.get("calendar_run_day_of_month", 25)
-
     now = datetime.now()
-    candidate = now.replace(
+    candidate = now.replace(hour=hour, minute=minute, second=0, microsecond=0)
-        day=min(day_of_month, calendar.monthrange(now.year, now.month)[1]),
+    return candidate if candidate > now else candidate + timedelta(days=1)
-        hour=hour, minute=minute, second=0, microsecond=0,
-    )
-    if candidate <= now:
-        y, m = (now.year, now.month + 1) if now.month < 12 else (now.year + 1, 1)
-        candidate = candidate.replace(
-            year=y, month=m, day=min(day_of_month, calendar.monthrange(y, m)[1]),
-        )
-    return candidate
 
 
 def _schedule_loop() -> None:
@@ -195,7 +191,7 @@ def api_get_settings():
 def api_post_settings():
     data = request.get_json(force=True)
     s = load_settings()
-    for key in ("calendar_run_time", "calendar_run_day_of_month"):
+    for key in _DEFAULTS:
         if key in data:
             s[key] = data[key]
     save_settings(s)
@@ -369,11 +365,7 @@ input:focus { outline: 2px solid var(--primary); outline-offset: -1px; border-co
       <label class="field-label">Pull Time</label>
       <input type="time" id="f-time" style="max-width:160px">
     </div>
-    <div class="field">
+    <div class="item-sub" style="margin-bottom:14px;">Pulls every day and pushes today onward from whichever month the district has published, so next month's menu lands on the calendar as soon as it's posted. Home Assistant skips dates that already have a lunch event, so daily pulls don't create duplicates.</div>
-      <label class="field-label">Pull Day of Month</label>
-      <input type="number" id="f-day" min="1" max="28" style="max-width:100px">
-      <div class="item-sub" style="margin-top:8px;">Fires once a month and pushes whichever month is currently published &mdash; e.g. day 25 catches next month's menu once the district posts it near month-end. There's no dedup, so re-pushing the same month creates duplicate calendar events.</div>
-    </div>
     <hr class="divider">
     <button class="btn btn-primary" onclick="saveSchedule()">Save Schedule</button>
   </div>
@@ -413,7 +405,6 @@ function applyIntegrations(ig) {
 
 function applySettings(s) {
   document.getElementById('f-time').value = s.calendar_run_time || '18:00';
-  document.getElementById('f-day').value = s.calendar_run_day_of_month || 25;
 }
 
 function applyStatus(st) {
@@ -428,8 +419,7 @@ function applyStatus(st) {
 
 async function saveSchedule() {
   const calendar_run_time = document.getElementById('f-time').value;
-  const calendar_run_day_of_month = parseInt(document.getElementById('f-day').value, 10);
+  await post('/api/settings', {calendar_run_time});
-  await post('/api/settings', {calendar_run_time, calendar_run_day_of_month});
   toast('Schedule saved');
   load();
 }
@@ -440,7 +430,7 @@ async function pushCalendar() {
   btn.textContent = 'Pushing…';
   await post('/api/push-calendar', {});
   toast('Triggered! Check status in a moment.');
-  setTimeout(() => { btn.disabled=false; btn.textContent='Push This Week to Calendar'; load(); }, 6000);
+  setTimeout(() => { btn.disabled=false; btn.textContent='Push This Month to Calendar'; load(); }, 6000);
 }
 
 async function previewMenu() {
@@ -465,7 +455,7 @@ async function previewMenu() {
     box.style.display = '';
   } finally {
     btn.disabled = false;
-    btn.textContent = "Preview This Week's Menu";
+    btn.textContent = "Preview This Month's Menu";
   }
 }
 
@@ -492,11 +482,12 @@ setInterval(load, 30000);
 # ── Entrypoint ────────────────────────────────────────────────────────────────
 
 if __name__ == "__main__":
-    if not SETTINGS_FILE.exists():
+    # Always rewrite: creates the file on first run, and prunes keys the
-        log.info(f"Creating default settings at {SETTINGS_FILE}")
+    # current schema no longer uses (see load_settings).
-        save_settings(_DEFAULTS)
+    settings = load_settings()
+    save_settings(settings)
 
-    apply_calendar_schedule(load_settings())
+    apply_calendar_schedule(settings)
     threading.Thread(target=_schedule_loop, daemon=True).start()
     log.info(f"Web UI → http://localhost:{PORT}")
     app.run(host="0.0.0.0", port=PORT, debug=False, use_reloader=False)