Expire stored source photos after 7 days, keeping grades forever
Storing originals for Regrade grew the database ~16MB per grade at the upload cap. Photos now expire on a configurable window (default 7 days, counted from last grade so regrading resets it) and the file is VACUUMed so the space is actually returned — clearing the column alone only moves pages to the freelist, which would have made the whole feature a no-op on disk. Grades, thumbnails and measurements are never pruned; a pruned card's Regrade falls back to the existing re-pick path, now with a banner saying why.
This commit is contained in:
parent
633cb98462
commit
602bce8ce3
5 changed files with 127 additions and 0 deletions
54
store.py
54
store.py
|
|
@ -58,6 +58,20 @@ DEFAULT_SETTINGS = {
|
|||
"vision_effort": "low",
|
||||
}
|
||||
|
||||
# How long a grade keeps the original photo(s) that produced it. They exist
|
||||
# so Regrade can re-run without asking for the photo again, which is worth
|
||||
# most right after a grading-logic change — a value that decays fast. What
|
||||
# doesn't decay is the grade record itself, so only the images are dropped
|
||||
# here; the history row, its thumbnail and every measurement stay forever.
|
||||
# A pruned card's Regrade falls back to asking for the photo, exactly as a
|
||||
# card graded before images were stored does.
|
||||
# Counted from created_at, which update_grade_result bumps — so a card you
|
||||
# regraded yesterday keeps its photos for another week, rather than being
|
||||
# pruned on the age of its first grading.
|
||||
# 0 or negative disables pruning entirely.
|
||||
SOURCE_IMAGE_RETENTION_DAYS = int(
|
||||
os.environ.get("CARD_GRADER_IMAGE_RETENTION_DAYS", "7"))
|
||||
|
||||
|
||||
def connect():
|
||||
# timeout: how long to wait for a writer's lock before giving up. The
|
||||
|
|
@ -256,6 +270,46 @@ def update_grade_result(grade_id, grade, thumbnail=None, source_images=None):
|
|||
return get_grade(grade_id)
|
||||
|
||||
|
||||
def prune_source_images(days=None):
|
||||
"""Drop stored photos older than the retention window. Returns the count.
|
||||
|
||||
Only source_images_json is cleared — the grade, its thumbnail and its
|
||||
measurements are untouched, so history stays complete and only the
|
||||
expensive part expires.
|
||||
|
||||
VACUUM afterwards because clearing a column returns its pages to
|
||||
SQLite's freelist without shrinking the file: without it the database
|
||||
would keep every byte this is meant to reclaim, and the whole feature
|
||||
would silently do nothing to disk usage. It rewrites the file, so it's
|
||||
run only when something was actually pruned, and on its own connection
|
||||
since VACUUM cannot execute inside a transaction.
|
||||
"""
|
||||
days = SOURCE_IMAGE_RETENTION_DAYS if days is None else days
|
||||
if days <= 0:
|
||||
return 0
|
||||
cutoff = time.strftime("%Y-%m-%dT%H:%M:%S",
|
||||
time.localtime(time.time() - days * 86400))
|
||||
conn = connect()
|
||||
try:
|
||||
cur = conn.execute(
|
||||
"UPDATE grades SET source_images_json = NULL "
|
||||
"WHERE source_images_json IS NOT NULL AND created_at < ?",
|
||||
(cutoff,),
|
||||
)
|
||||
pruned = cur.rowcount or 0
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
if pruned:
|
||||
vac = sqlite3.connect(DB_PATH, timeout=60.0, isolation_level=None)
|
||||
try:
|
||||
vac.execute("VACUUM")
|
||||
finally:
|
||||
vac.close()
|
||||
return pruned
|
||||
|
||||
|
||||
def get_grade_images(grade_id):
|
||||
"""The original (bytes, filename) pairs for Regrade, or None if this
|
||||
grade never had them stored (a card graded before this feature existed,
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue