飞控-77: 77 std flight-controller projects ingested

Topic-targeted pull from local listing index (`name OR introduction`
contains 飞控). 79 std hits in oshwhub_listing_full.jsonl, 2 already
crawled, 77 newly fetched.

dev1 (Guangzhou) walltime:
  Step 1 detail scrape ~12s, Step 4 std-source backfill ~80s
  (concurrency=5)
Source completeness: 73/77 with editor source, 4 are upstream
attachments-only (no editor session ever attached, source_documents=[]
is genuine — no editor_version on the SSR page either).

Crawler hardening (crawlers/oshwhub/crawler.py):
- count.{like,star,fork,views} are now `.get(..., 0)` defensive.
  Listing API omits zero-valued fields for some low-activity entries
  (3/77 hit this on first pass, hard-failed with KeyError 'like').
  Affects rank_score, pick_top, and metadata.json metrics block.

License mix: 65% GPL 3.0, 11% Public Domain, 11% MIT, ~6% CC variants.

Transport: dev1 → SG via tar+scp (33 MB, ~3 min over lossy
cross-region link). Bypassed gitea push from dev1 because the same
6.5%-loss link tanks single-stream throughput.
This commit is contained in:
2026-04-30 19:04:58 +08:00
parent c199840ad3
commit 29530e09d2
20 changed files with 442 additions and 19 deletions

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,75 @@
{
"project_uuid": "025eab68656a4f80992a8349ffc3ecb4",
"fetched_at": "2026-04-30T10:57:09.980190+00:00",
"editor_version": "6.5.46",
"documents": [
{
"doc_uuid": "3a7ca4d0f2164b66ab1323b221cf9c95",
"docType": 1,
"master": "589ad7a7a72c4abeb4078ab4f8a069e2",
"path": "source/3a7ca4d0f2164b66ab1323b221cf9c95.json",
"size": 62029,
"sha256": "d12c4c2020cdea08949797375011e400432f1766517ef856da4a28c7f2403bf9"
},
{
"doc_uuid": "6ddcdb6b64ad4928ba34ceb8dacb163e",
"docType": 3,
"master": "2a221c3d50554af195719255801749c4",
"path": "source/6ddcdb6b64ad4928ba34ceb8dacb163e.json",
"size": 405080,
"sha256": "45d1bf630bb8d246da26f1325994a2f94c399e8d62d7b24324a58d44c893bb0b"
},
{
"doc_uuid": "ad8d6d1dd80c4512be97e6b71b8d6448",
"docType": 3,
"master": "b32a60425fac4e13ae98ad0a2874daa3",
"path": "source/ad8d6d1dd80c4512be97e6b71b8d6448.json",
"size": 374471,
"sha256": "c76c5d6f9b36b0884f3d3a816b91ae89769d8dad3f87bed3be664a2f5e00cb0a"
}
],
"upstream_version_documents": [
{
"uuid": "3a7ca4d0f2164b66ab1323b221cf9c95",
"title": "MATEK F405 WSE 2-6S PDB",
"description": "",
"docType": 1,
"master": "589ad7a7a72c4abeb4078ab4f8a069e2",
"thumb": "//image.lceda.cn/histories/589ad7a7a72c4abeb4078ab4f8a069e2.png",
"components": [],
"updateTime": 1756801015,
"createTime": 1756801015,
"histories": [
"589ad7a7a72c4abeb4078ab4f8a069e2"
]
},
{
"uuid": "6ddcdb6b64ad4928ba34ceb8dacb163e",
"title": "PCB_MATEK F405 WSE PDB_4L",
"description": "",
"docType": 3,
"master": "2a221c3d50554af195719255801749c4",
"thumb": "//image.lceda.cn/histories/2a221c3d50554af195719255801749c4.png",
"components": [],
"updateTime": 1756801040,
"createTime": 1756801040,
"histories": [
"2a221c3d50554af195719255801749c4"
]
},
{
"uuid": "ad8d6d1dd80c4512be97e6b71b8d6448",
"title": "PCB_MATEK F405 WSE PDB_2L",
"description": "",
"docType": 3,
"master": "b32a60425fac4e13ae98ad0a2874daa3",
"thumb": "//image.lceda.cn/histories/b32a60425fac4e13ae98ad0a2874daa3.png",
"components": [],
"updateTime": 1756801052,
"createTime": 1756801052,
"histories": [
"b32a60425fac4e13ae98ad0a2874daa3"
]
}
]
}