飞控-77: 77 std flight-controller projects ingested

Topic-targeted pull from local listing index (`name OR introduction`
contains 飞控). 79 std hits in oshwhub_listing_full.jsonl, 2 already
crawled, 77 newly fetched.

dev1 (Guangzhou) walltime:
  Step 1 detail scrape ~12s, Step 4 std-source backfill ~80s
  (concurrency=5)
Source completeness: 73/77 with editor source, 4 are upstream
attachments-only (no editor session ever attached, source_documents=[]
is genuine — no editor_version on the SSR page either).

Crawler hardening (crawlers/oshwhub/crawler.py):
- count.{like,star,fork,views} are now `.get(..., 0)` defensive.
  Listing API omits zero-valued fields for some low-activity entries
  (3/77 hit this on first pass, hard-failed with KeyError 'like').
  Affects rank_score, pick_top, and metadata.json metrics block.

License mix: 65% GPL 3.0, 11% Public Domain, 11% MIT, ~6% CC variants.

Transport: dev1 → SG via tar+scp (33 MB, ~3 min over lossy
cross-region link). Bypassed gitea push from dev1 because the same
6.5%-loss link tanks single-stream throughput.
This commit is contained in:
2026-04-30 19:04:58 +08:00
parent c199840ad3
commit 29530e09d2
20 changed files with 442 additions and 19 deletions

View File

@@ -0,0 +1,53 @@
{
"source": "oshwhub",
"source_url": "https://oshwhub.com/JumperShao/m1_mh743_ada_v4",
"project_id": "21817f8fcf2d4e36b96f8e00c33a5c93",
"title": "m1_mh743_ada_v4",
"description_short": "朗宇M1改ArduPilot, MatekH743mini飞控转接板",
"description_path": "description.md",
"author": {
"username": "JumperShao",
"display_name": "JumperShao",
"user_id": "8c21dccebe594ff98970fc4521981d66"
},
"license": "GPL 3.0",
"tags": [],
"created_at": "2022-02-04T01:15:16.000Z",
"updated_at": "2022-02-25T01:30:27.000Z",
"published_at": "2022-02-25T01:30:27.000Z",
"crawled_at": "2026-04-30T10:57:07.010231+00:00",
"metrics": {
"likes": 0,
"stars": 1,
"forks": 2,
"views": 1074,
"watch": 0,
"comments": 0
},
"cover": {
"url": "https://image.lceda.cn/pullimage/haHqGHCALFdesiNBAPqK4v0aqj32O5SyRmxiDvtC.jpeg",
"path": null
},
"files": [],
"raw_fields": {
"path": "JumperShao/m1_mh743_ada_v4",
"grade": 0,
"origin": "std",
"public": true,
"publish": true,
"skipped_files": []
},
"source_format": "easyeda-std",
"source_path": "source/",
"source_documents": [
{
"doc_uuid": "e7d66b9e02b9457083f9ab563784a0f2",
"docType": 3,
"master": "d158f8681c8943c9b7ae993a21d3573b",
"path": "source/e7d66b9e02b9457083f9ab563784a0f2.json",
"size": 567060,
"sha256": "d5a7011ab048898e86cff342f8ea21f640beb0878917ca5fdae64938f97662a9"
}
],
"editor_version": "6.4.31"
}