Facere/FacereDataset
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

crawler: filter Pro 2.x deprecated boards from sch/pcb fetch

Browse Source 
Pro 2.x project metadata's boards[] can reference sch/pcb UUIDs that
the project owner has since deprecated/deleted (e.g. "主控板V1（废弃）").
Such UUIDs are gone from ticket.schematics / ticket.pcbs but still in
boards[]. Asking schematic/lists or documents/lists for them returns
401 and aborts the whole project.

Filter both lists against the authoritative ticket dict before posting.
Verified on 7f7565ef11 (Super Dial 电机旋钮屏): 4 boards but only 3
sch entries in schematics dict, isolating the deprecated 8bc59f to a
401 we now skip.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Knowit
2026-04-29 02:23:39 +08:00
parent 8220c99f7e
commit c3cac97593
1 changed files with 13 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
crawlers/oshwhub/crawler.py
View File

@@ -758,7 +758,14 @@ def _fetch_pro_legacy(
doc_metas: list[dict] = [] doc_metas: list[dict] = []
# 2. schematic containers -> sheet UUIDs via /api/schematic/lists # 2. schematic containers -> sheet UUIDs via /api/schematic/lists
sch_container_uuids = [b["sch"] for b in boards if b.get("sch")] # Filter `boards[].sch` against `ticket.schematics` keys: a board may
# reference a deprecated/deleted sch (e.g. "主控板V1（废弃）") whose
# UUID is gone from the schematics dict. Asking schematic/lists for it
# returns 401 and aborts the whole batch. Skip it.
valid_sch_uuids = set((manifest_ticket.get("schematics") or {}).keys())
sch_container_uuids = [
b["sch"] for b in boards if b.get("sch") and b["sch"] in valid_sch_uuids
]
sheet_uuids: list[str] = [] sheet_uuids: list[str] = []
if sch_container_uuids: if sch_container_uuids:
containers = _pro_post_json( containers = _pro_post_json(
@@ -798,7 +805,11 @@ def _fetch_pro_legacy(
time.sleep(sleep) time.sleep(sleep)
# 4. PCB documents via documents/lists docType=3 # 4. PCB documents via documents/lists docType=3
pcb_uuids = [b["pcb"] for b in boards if b.get("pcb")] # Same deprecated-uuid risk as #2 — filter against ticket.pcbs keys.
valid_pcb_uuids = set((manifest_ticket.get("pcbs") or {}).keys())
pcb_uuids = [
b["pcb"] for b in boards if b.get("pcb") and b["pcb"] in valid_pcb_uuids
]
if pcb_uuids: if pcb_uuids:
pcbs = _pro_post_json( pcbs = _pro_post_json(
pro_client, pro_client,