飞控-77: 77 std flight-controller projects ingested

Topic-targeted pull from local listing index (`name OR introduction`
contains 飞控). 79 std hits in oshwhub_listing_full.jsonl, 2 already
crawled, 77 newly fetched.

dev1 (Guangzhou) walltime:
  Step 1 detail scrape ~12s, Step 4 std-source backfill ~80s
  (concurrency=5)
Source completeness: 73/77 with editor source, 4 are upstream
attachments-only (no editor session ever attached, source_documents=[]
is genuine — no editor_version on the SSR page either).

Crawler hardening (crawlers/oshwhub/crawler.py):
- count.{like,star,fork,views} are now `.get(..., 0)` defensive.
  Listing API omits zero-valued fields for some low-activity entries
  (3/77 hit this on first pass, hard-failed with KeyError 'like').
  Affects rank_score, pick_top, and metadata.json metrics block.

License mix: 65% GPL 3.0, 11% Public Domain, 11% MIT, ~6% CC variants.

Transport: dev1 → SG via tar+scp (33 MB, ~3 min over lossy
cross-region link). Bypassed gitea push from dev1 because the same
6.5%-loss link tanks single-stream throughput.
This commit is contained in:
2026-04-30 19:04:58 +08:00
parent c199840ad3
commit 29530e09d2
20 changed files with 442 additions and 19 deletions

View File

@@ -0,0 +1,75 @@
{
"project_uuid": "025eab68656a4f80992a8349ffc3ecb4",
"fetched_at": "2026-04-30T10:57:09.980190+00:00",
"editor_version": "6.5.46",
"documents": [
{
"doc_uuid": "3a7ca4d0f2164b66ab1323b221cf9c95",
"docType": 1,
"master": "589ad7a7a72c4abeb4078ab4f8a069e2",
"path": "source/3a7ca4d0f2164b66ab1323b221cf9c95.json",
"size": 62029,
"sha256": "d12c4c2020cdea08949797375011e400432f1766517ef856da4a28c7f2403bf9"
},
{
"doc_uuid": "6ddcdb6b64ad4928ba34ceb8dacb163e",
"docType": 3,
"master": "2a221c3d50554af195719255801749c4",
"path": "source/6ddcdb6b64ad4928ba34ceb8dacb163e.json",
"size": 405080,
"sha256": "45d1bf630bb8d246da26f1325994a2f94c399e8d62d7b24324a58d44c893bb0b"
},
{
"doc_uuid": "ad8d6d1dd80c4512be97e6b71b8d6448",
"docType": 3,
"master": "b32a60425fac4e13ae98ad0a2874daa3",
"path": "source/ad8d6d1dd80c4512be97e6b71b8d6448.json",
"size": 374471,
"sha256": "c76c5d6f9b36b0884f3d3a816b91ae89769d8dad3f87bed3be664a2f5e00cb0a"
}
],
"upstream_version_documents": [
{
"uuid": "3a7ca4d0f2164b66ab1323b221cf9c95",
"title": "MATEK F405 WSE 2-6S PDB",
"description": "",
"docType": 1,
"master": "589ad7a7a72c4abeb4078ab4f8a069e2",
"thumb": "//image.lceda.cn/histories/589ad7a7a72c4abeb4078ab4f8a069e2.png",
"components": [],
"updateTime": 1756801015,
"createTime": 1756801015,
"histories": [
"589ad7a7a72c4abeb4078ab4f8a069e2"
]
},
{
"uuid": "6ddcdb6b64ad4928ba34ceb8dacb163e",
"title": "PCB_MATEK F405 WSE PDB_4L",
"description": "",
"docType": 3,
"master": "2a221c3d50554af195719255801749c4",
"thumb": "//image.lceda.cn/histories/2a221c3d50554af195719255801749c4.png",
"components": [],
"updateTime": 1756801040,
"createTime": 1756801040,
"histories": [
"2a221c3d50554af195719255801749c4"
]
},
{
"uuid": "ad8d6d1dd80c4512be97e6b71b8d6448",
"title": "PCB_MATEK F405 WSE PDB_2L",
"description": "",
"docType": 3,
"master": "b32a60425fac4e13ae98ad0a2874daa3",
"thumb": "//image.lceda.cn/histories/b32a60425fac4e13ae98ad0a2874daa3.png",
"components": [],
"updateTime": 1756801052,
"createTime": 1756801052,
"histories": [
"b32a60425fac4e13ae98ad0a2874daa3"
]
}
]
}