Cap self-reported 'high' confidence when neither centering nor edge whitening was actually measured
Confidence was entirely the model's own self-report, based only on photo sharpness and category coverage -- it had no reliable way to account for whether centering/edge whitening came from a real pixel measurement or from its own eye, even though the app treats that distinction as a big deal everywhere else (it's the whole reason cardimage.py exists). A sharp, well-lit photo of a foil or die-cut card could claim 'high' while actually running on eye-only judgment for two of PSA's four categories. Enforced in code rather than left to the prompt alone -- this is an objective, checkable fact (did centering_profile/edge_wear_profile return a reliable result or not), exactly the kind of thing that shouldn't depend on the model correctly weighing one more instruction among many. Only caps high->medium when NEITHER measurement is available; one of two still measured is left to the model's own judgment, since that's a real legitimate basis for confidence. Also updated the prompt itself so the model's own reasoning stays consistent with what gets displayed, rather than silently contradicting its own prose. Tested all five branches (neither/both/one/one-unreliable measured, and confirmed no spurious limitation text when the model already said medium).
This commit is contained in:
parent
2b3bbaf40f
commit
3cacf2f292
1 changed files with 37 additions and 2 deletions
39
vision.py
39
vision.py
|
|
@ -692,6 +692,15 @@ centering, that range should be genuinely wide (e.g. 6-9), not cosmetic.
|
||||||
where you could assess all four categories; "medium" when one or two \
|
where you could assess all four categories; "medium" when one or two \
|
||||||
categories are unassessable; "low" when the photo mainly supports identifying \
|
categories are unassessable; "low" when the photo mainly supports identifying \
|
||||||
the card rather than grading it.
|
the card rather than grading it.
|
||||||
|
Centering and edge whitening are measured directly from the pixels when a \
|
||||||
|
MEASURED CENTERING / MEASURED EDGE WHITENING block is supplied above — that's \
|
||||||
|
a materially stronger basis than reading either by eye, and the reverse is \
|
||||||
|
true too: if NEITHER was measurable on this card (both missing or refused, \
|
||||||
|
with a reason given), don't call this "high" even if the photo itself is \
|
||||||
|
sharp and well-lit. A crisp photo of a card whose finish defeats the \
|
||||||
|
measurement (foil, refractor, die-cut, or the margin problem described \
|
||||||
|
above) still leaves you assessing two of the four categories by eye alone, \
|
||||||
|
which is exactly what "medium" is for.
|
||||||
- limitations: list each specific thing the photo prevented you from checking \
|
- limitations: list each specific thing the photo prevented you from checking \
|
||||||
("back not shown, so back centering and back corners are unknown", "resolution \
|
("back not shown, so back centering and back corners are unknown", "resolution \
|
||||||
too low to see print lines or light surface scratches").
|
too low to see print lines or light surface scratches").
|
||||||
|
|
@ -993,6 +1002,32 @@ def grade_card(images, api_key=None, model=None, effort=None, zoom_details=True)
|
||||||
if card_type not in ("pokemon", "sports", "other_tcg", "other"):
|
if card_type not in ("pokemon", "sports", "other_tcg", "other"):
|
||||||
card_type = "other"
|
card_type = "other"
|
||||||
|
|
||||||
|
confidence = parsed.get("confidence") or "low"
|
||||||
|
limitations = [l for l in (parsed.get("limitations") or []) if l]
|
||||||
|
|
||||||
|
# "high" confidence is defined to the model as being able to confidently
|
||||||
|
# assess all four categories — but centering and edge whitening are the
|
||||||
|
# two this app can measure from pixels rather than ask the model to
|
||||||
|
# judge by eye, and a self-reported "high" doesn't reliably account for
|
||||||
|
# whether that measurement was actually available on THIS card. Capped
|
||||||
|
# here rather than left to the prompt alone: the model juggles a long
|
||||||
|
# instruction list already, and this is exactly the kind of objective,
|
||||||
|
# checkable fact code should enforce rather than hope gets weighed
|
||||||
|
# correctly every time. Only caps a "high" claim down to "medium" when
|
||||||
|
# NEITHER was measured — one of two still measured is a real basis the
|
||||||
|
# model may legitimately be confident from, so that's left to its own
|
||||||
|
# judgment.
|
||||||
|
def _measured_ok(m):
|
||||||
|
return bool(m) and m.get("reliable", True) is not False
|
||||||
|
|
||||||
|
if confidence == "high" and not _measured_ok(centering) and not _measured_ok(measured):
|
||||||
|
confidence = "medium"
|
||||||
|
limitations.append(
|
||||||
|
"Confidence capped at medium: neither centering nor edge "
|
||||||
|
"whitening could be measured from the pixels on this photo, so "
|
||||||
|
"the estimate leans on the photo alone for two of PSA's four "
|
||||||
|
"categories rather than a direct pixel measurement for either.")
|
||||||
|
|
||||||
return {
|
return {
|
||||||
"closeups": len(crops),
|
"closeups": len(crops),
|
||||||
"card_type": card_type,
|
"card_type": card_type,
|
||||||
|
|
@ -1004,14 +1039,14 @@ def grade_card(images, api_key=None, model=None, effort=None, zoom_details=True)
|
||||||
"estimated_grade": _clean_grade(parsed.get("estimated_grade")),
|
"estimated_grade": _clean_grade(parsed.get("estimated_grade")),
|
||||||
"grade_low": low,
|
"grade_low": low,
|
||||||
"grade_high": high,
|
"grade_high": high,
|
||||||
"confidence": parsed.get("confidence") or "low",
|
"confidence": confidence,
|
||||||
"categories": {
|
"categories": {
|
||||||
"centering": _clean_category(parsed.get("centering")),
|
"centering": _clean_category(parsed.get("centering")),
|
||||||
"corners": _clean_category(parsed.get("corners")),
|
"corners": _clean_category(parsed.get("corners")),
|
||||||
"edges": _clean_category(parsed.get("edges")),
|
"edges": _clean_category(parsed.get("edges")),
|
||||||
"surface": _clean_category(parsed.get("surface")),
|
"surface": _clean_category(parsed.get("surface")),
|
||||||
},
|
},
|
||||||
"limitations": [l for l in (parsed.get("limitations") or []) if l],
|
"limitations": limitations,
|
||||||
"note": parsed.get("note") or "",
|
"note": parsed.get("note") or "",
|
||||||
"usage": usage,
|
"usage": usage,
|
||||||
}
|
}
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue