Facere/FacereDataset
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Add EasyEDA Std project source ingestion (10 boards backfilled)

Browse Source 
打通 oshwhub origin=std 项目的工程源（schematic + PCB dataStr）抓取链路。原
plan.md §1.6 假设需要登录，实测 lceda.cn/api/documents/<doc>?uuid=<doc>&path=<doc>
对公开项目匿名可访问 —— 无需 cookie，无账号封禁风险。

调研：4 轮探测留痕在 data/state/std_probe[1-5]/（gitignored）；翻 Std 编辑器
v6.5.51 的 main.min.js bundle 找到 ajaxDetail 端点；按 docType 区分两种
响应 shape（schematic 项目视图 vs PCB 文档视图）。

Crawler:
  - make_source_client() 用浏览器 UA + lceda.cn/editor Referer，因为
    oshwhub /api/project/<uuid> 端点拒绝 FacereDataset/0.1 UA（CLAUDE.md
    UA 例外条款：目标站主动封自定义 UA + 公开静态资源）
  - fetch_std_source(): 项目元 → version_documents → 逐文档 dataStr → 落
    source/<doc>.json + source/manifest.json
  - --with-source（爬新项目时一并抓源）/ --backfill-source（仅扫已有）
  - QPS ≤ 0.2 (SLEEP_SOURCE = 5s) 自律

Schema: 加 source_format / source_path / source_documents / editor_version
（前 3 进 enum 锁定，便于后续 Pro / KiCad 源对齐）。

回填结果：10/10 成功，45 个文档，33.2 MB；schema validate 全通。
docTypes 主要是 1 (schematic) 与 3 (pcb)；USB 电压电流表只有 PCB 文档（4 个：
主板+盖板+底板+面板，作者未上传原理图源）。

完整调研：docs/sources/easyeda_std_source.md。
This commit is contained in:
Knowit
2026-04-28 20:07:40 +08:00
parent b0d3afd2a9
commit d874278bc5
71 changed files with 4389 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
data/raw/oshwhub/922c1f3a9b9a43ff98998f476e7946ca/metadata.json
View File

@@ -114,5 +114,42 @@
"public": true,
"publish": true,
"skipped_files": []
}
},
"source_format": "easyeda-std",
"source_path": "source/",
"source_documents": [
{
"doc_uuid": "c56a79c0509349b4be22d6f413ab1f69",
"docType": 1,
"master": "869b6ea265f8446595c7446ee07c618d",
"path": "source/c56a79c0509349b4be22d6f413ab1f69.json",
"size": 321179,
"sha256": "ae6026a14794ba0fe108f9b0b561ba22a4af83b16f5487f2faa1f675d804931b"
},
{
"doc_uuid": "ba150d03c64b479cb42d0ab34fd12709",
"docType": 3,
"master": "4d583c086dcd4686bab3e8103d15930a",
"path": "source/ba150d03c64b479cb42d0ab34fd12709.json",
"size": 1735811,
"sha256": "cfeee04d27a9eeee7fad9e1d7a109c7dfdb32b0f30d674da38a2fbb3bc36a68b"
},
{
"doc_uuid": "5c2114235e7f47e0998b8433ca808c30",
"docType": 3,
"master": "238bfc78d2c941bb950c329d4dcd15df",
"path": "source/5c2114235e7f47e0998b8433ca808c30.json",
"size": 1698152,
"sha256": "650639d7b3cb39a306ec64f7da155dc4cf4fc99609655c28c55723185b7d2c67"
},
{
"doc_uuid": "acfe4b335952431ea37f20b7a3000ada",
"docType": 3,
"master": "595aaccb3ff8474695bc11ec6785c419",
"path": "source/acfe4b335952431ea37f20b7a3000ada.json",
"size": 1748468,
"sha256": "770e5633d531883a7e2d3f2a7581d50b8148c8f71b1f1e206cb1e1b2cb7acd8f"
}
],
"editor_version": "6.5.37"
}

1
data/raw/oshwhub/922c1f3a9b9a43ff98998f476e7946ca/source/5c2114235e7f47e0998b8433ca808c30.json Normal file
View File

File diff suppressed because one or more lines are too long

1
data/raw/oshwhub/922c1f3a9b9a43ff98998f476e7946ca/source/acfe4b335952431ea37f20b7a3000ada.json Normal file
View File

File diff suppressed because one or more lines are too long

1
data/raw/oshwhub/922c1f3a9b9a43ff98998f476e7946ca/source/ba150d03c64b479cb42d0ab34fd12709.json Normal file
View File

File diff suppressed because one or more lines are too long

1
data/raw/oshwhub/922c1f3a9b9a43ff98998f476e7946ca/source/c56a79c0509349b4be22d6f413ab1f69.json Normal file
View File

File diff suppressed because one or more lines are too long

269
data/raw/oshwhub/922c1f3a9b9a43ff98998f476e7946ca/source/manifest.json Normal file
View File

@@ -0,0 +1,269 @@
{
"project_uuid": "922c1f3a9b9a43ff98998f476e7946ca",
"fetched_at": "2026-04-28T11:49:10.257978+00:00",
"editor_version": "6.5.37",
"documents": [
{
"doc_uuid": "c56a79c0509349b4be22d6f413ab1f69",
"docType": 1,
"master": "869b6ea265f8446595c7446ee07c618d",
"path": "source/c56a79c0509349b4be22d6f413ab1f69.json",
"size": 321179,
"sha256": "ae6026a14794ba0fe108f9b0b561ba22a4af83b16f5487f2faa1f675d804931b"
},
{
"doc_uuid": "ba150d03c64b479cb42d0ab34fd12709",
"docType": 3,
"master": "4d583c086dcd4686bab3e8103d15930a",
"path": "source/ba150d03c64b479cb42d0ab34fd12709.json",
"size": 1735811,
"sha256": "cfeee04d27a9eeee7fad9e1d7a109c7dfdb32b0f30d674da38a2fbb3bc36a68b"
},
{
"doc_uuid": "5c2114235e7f47e0998b8433ca808c30",
"docType": 3,
"master": "238bfc78d2c941bb950c329d4dcd15df",
"path": "source/5c2114235e7f47e0998b8433ca808c30.json",
"size": 1698152,
"sha256": "650639d7b3cb39a306ec64f7da155dc4cf4fc99609655c28c55723185b7d2c67"
},
{
"doc_uuid": "acfe4b335952431ea37f20b7a3000ada",
"docType": 3,
"master": "595aaccb3ff8474695bc11ec6785c419",
"path": "source/acfe4b335952431ea37f20b7a3000ada.json",
"size": 1748468,
"sha256": "770e5633d531883a7e2d3f2a7581d50b8148c8f71b1f1e206cb1e1b2cb7acd8f"
}
],
"upstream_version_documents": [
{
"uuid": "c56a79c0509349b4be22d6f413ab1f69",
"title": "YuzuMaix-YuzuAI-V831-AIoT",
"description": "",
"docType": 1,
"master": "869b6ea265f8446595c7446ee07c618d",
"thumb": "//image.lceda.cn/histories/869b6ea265f8446595c7446ee07c618d.png",
"components": {
"877456cbea5a4dc8aff903f2f077fcf3": 1,
"f29f57ac58b24353a27984c5792bbc76": 31,
"31316b41d258495d89e7f3a101813188": 2,
"bb009484ef804a24ac5d18c54b249586": 21,
"01f8feef49be43dcbfbe672254cb7b57": 23,
"55440d6ca9bd432d8dfbedd3562ce87a": 3,
"24eeb2ceca08480b92b847b6a310acda": 22,
"48afe5cba8974122bc847bddebf15e20": 3,
"b958857318a34415b2b4f846c53c4cc2": 1,
"8594922bf7656e1a593fe27b198cf7ff": 2,
"303ea8960e30b8ac9e63b17d909c778d": 1,
"d81ceb6e34884f4abc0c65fc0de793b8": 1,
"4534ebdc5b764766a8eb5b559a74d6a9": 1,
"13c6891c25859fa0190ab032be958a11": 1,
"e9e3af65fc87eef7d9725f81b11737ae": 1,
"10dccece233e5b3af8283ad9f57b7a59": 1,
"c3dc0e6b99ca4638b60af164b12c1d3a": 1,
"74bd5f8fe7b549288b044ac0d37dfa0c": 1,
"daf1806141ed4406bf82ed2b235f2421": 1,
"16ebee7377f54c6cbd349001618feb27": 1,
"ee77667cc965499f8621697b7f0c4a46": 1,
"a9763b6f47714ce386e2a084a55bf7a8": 1,
"a5284b47d796416aa7d586ceebbe0c75": 3,
"42b9229ab27b43f4a612445fbda2acdd": 1,
"6dd59536b80d4b1ba602fa1ee748c47d": 1,
"7b0f56a0556f49928379078a8b366362": 1,
"c8ddb66d646b4874947eb6ce5ac51906": 1,
"91a9a25e261a472e86c981ab4434f58e": 1,
"12217f0a00244939a9e3429d606551e4": 1,
"f80e29816fd30bb295222fb4f47bcc25": 1,
"2e56d513775448ed9b24b99aa82964c5": 1,
"03cbbe75bdfb43899c47095479dd821e": 1,
"74d819a8b43c44ef975cc62613670eae": 1
},
"updateTime": 1702518629,
"createTime": 1680667752,
"histories": [
"ae849d5a55a741dc9ad68e615330e6d3",
"5cb3dfc221dd4f2bb933e3f331299d99",
"6c69b10c7fe0414392941f1c09e865a0",
"d827db14a3004d1582414ca1eb7b9bad",
"16cfe334df694109a343097781263342",
"8dde0a0cb743445ab0c8d7b30306386b",
"96bd44c5179f41568f5c931424d97744",
"9754888857e449aa87b0ed32ac9d7e18",
"c3fb0601627a4481b6b02df02d0d1045",
"2d7ab8c2e22e4f04876f198cb4ade316",
"fed3c37a68d54c93bb4e0978fd81fc28",
"8f6dcad83185409c98604c5aa16e46aa",
"f2058a8bcaaa4fb4a70dcfa63a9c6567",
"23e320dd34fb474d8a105850a6aa2271",
"39af4b01162e4f7088096374d6965fb6",
"3c32ebbbc808428eae52ec7c891064bb",
"0b058483192c4195a8d79d0babef2c7a",
"608e3646341c4f21bcddf2d1e7d48509",
"de40e7ab8f9a4831b0f8440823630be7",
"97ca03b01aac471fb56b95601cb17d26",
"3632eae237c749939a023d0199d23e89",
"a9f3cb3d4e834c38b2081a8757d652d2",
"de5ff228762f49db890208e210ac2273",
"14e014ca9d55400ab36188e382dbf6b9",
"734f076ba63f426ead805e3947ec91ae",
"1235c885d86b488585fb962942562e82",
"88d5468ea26041279e71717b080c5a4e",
"8925259b82324c3ba01d8c5df6bb4f1e",
"869b6ea265f8446595c7446ee07c618d"
]
},
{
"uuid": "ba150d03c64b479cb42d0ab34fd12709",
"title": "YuzuAI-V831-AIoT-V22",
"description": "",
"docType": 3,
"master": "4d583c086dcd4686bab3e8103d15930a",
"thumb": "//image.lceda.cn/histories/4d583c086dcd4686bab3e8103d15930a.png",
"components": {
"f35f34deb3d94a8ba7d5f87d0a82e882": 1,
"76a5ba8d270141fb994fbfcc50ef0416": 1,
"50359fcc008846ffbdf2bc467afb79fc": 37,
"3667d18b6ca74adbb60776118af4d7b7": 12,
"bbf2d74717914929b5e52e3caa0392b2": 1,
"67f8ff6af38d4925b183030c58aed18a": 35,
"b3b82869fa924bae820e3a6cfb44d689": 2,
"740f591cd72e49729b462f97208c29e2": 2,
"308001d2388148a595891bedf0d443a5": 1,
"795419fe08b4408b81518abedfb501c7": 2,
"2913bc4dd4a6497b86bc7c16e187f46a": 3,
"bade90667a6e4731855e9a223387b8a6": 3,
"7559b1e979f64c67b5b3434f1fdc0256": 3,
"a2365841f454457390f178da5e6ade00": 1,
"7ad8a976c4434403abe61b7cb116556e": 1,
"118a205096a1444195e08feff716c648": 1,
"ea8992534e8b4fcdb80c9c0b6608b603": 1,
"1a0d1eed36744005ad9f6e257055f41b": 1,
"7d7ad256f7a94906950b482bea046d1b": 1,
"01aad49044514640b618e7541f2ae097": 2
},
"updateTime": 1688291296,
"createTime": 1688000643,
"histories": [
"faae5df5aa5340e2bcfe44ca27eb806b",
"f5089b6eadf14b88b52dceac6e6c3a55",
"944bf124afc54355b1fd9f876be7b594",
"14d70c512e8f4d4e83629f63f4fade99",
"41069e5699a34c818486f6a69e15d395",
"e57334a3786141478a8e24d45194b28a",
"dde7c10405b047e7ba42b2a8098d8859",
"b1ccd67ebd8d4d8c90c61d333a39aa52",
"3750926aaab3414ca78b46d2667c4c3c",
"c4a740f067af4c22aa3d3b2401c05c36",
"3300e8018f8241fa92491b14e9eeb40a",
"bf38b2a8c0284ad8843252dd9ffd1f3c",
"b4f5b7784d2b43bc8faf29d0ba4d32f7",
"3f9f5645c8574ef1af2db03b2cacc77f",
"7efadba90ce94676835b7a376e617e28",
"281180c6d7f7470092e83c793a1d8edc",
"0197b8789fbe47dba3291c349e98a8d1",
"28cc20adfd6f4127a583a83e5b3fe929",
"56c456c0dd2c41d08262a8391df3731b",
"65a642ed562246c6adac375918a40102",
"eb8632ae472f499cbc63ff350374c2d7",
"314e0ba25cd747f8b02ec6c5c4e99f3f",
"11d0418401cb4461befe5eac80cc7e97",
"dad6b8da151d4f1ba804c176230aa9a1",
"a3a93982d7874d65b971d10775ffc165",
"a7d97b702781470ea5656bc447d2b6eb",
"2141dbe6b7c54db8bbae0c845fc8dfcd",
"ed9baa146d1040f08edb8511b27769ee",
"06d1ab0377244cc6b100032f8d9d5879",
"b2659532f4244e5aa8b767f5057ecb24",
"5dec1cce3c804c1b82ab12dd1358f20f",
"d4a01b50ebac412db7d19cc1b60a642b",
"e12911283de64b1d917ebc21e36c4d30",
"a0c2a0d7f1f74ceb9aca17f681fde5b8",
"03a47459c05742d196de294c2b6fcdbc",
"d09c2776e9eb4000a16092d3d98ce8cd",
"4d583c086dcd4686bab3e8103d15930a"
]
},
{
"uuid": "5c2114235e7f47e0998b8433ca808c30",
"title": "YuzuAI-V831-AIoT-V24",
"description": "",
"docType": 3,
"master": "238bfc78d2c941bb950c329d4dcd15df",
"thumb": "//image.lceda.cn/histories/238bfc78d2c941bb950c329d4dcd15df.png",
"components": {
"50359fcc008846ffbdf2bc467afb79fc": 35,
"a801549ddf8f431796761cd071417387": 1,
"f35f34deb3d94a8ba7d5f87d0a82e882": 1,
"2158d3d8bb5f4c058f383290b974daeb": 1,
"76a5ba8d270141fb994fbfcc50ef0416": 1,
"3667d18b6ca74adbb60776118af4d7b7": 11,
"bbf2d74717914929b5e52e3caa0392b2": 1,
"67f8ff6af38d4925b183030c58aed18a": 33,
"b3b82869fa924bae820e3a6cfb44d689": 1,
"740f591cd72e49729b462f97208c29e2": 2,
"308001d2388148a595891bedf0d443a5": 1,
"795419fe08b4408b81518abedfb501c7": 1,
"2913bc4dd4a6497b86bc7c16e187f46a": 3,
"bade90667a6e4731855e9a223387b8a6": 3,
"7559b1e979f64c67b5b3434f1fdc0256": 3,
"a2365841f454457390f178da5e6ade00": 1,
"1a0d1eed36744005ad9f6e257055f41b": 1,
"7d7ad256f7a94906950b482bea046d1b": 1,
"01aad49044514640b618e7541f2ae097": 2
},
"updateTime": 1690690514,
"createTime": 1690258321,
"histories": [
"3f873f6fb2194f6db1de16953e48edfd",
"4e115717cdfa40eab5e880d31d84e99b",
"90fd9071a1824eb0b4e4c554c24372e2",
"f48cd0e24f874e6982a4452fad454385",
"98b20db1bcd94d62965d69211a875ca2",
"e68d98b1ac7f4e4abd8f285430c2d4f1",
"d6e3c3fa8fd34050aeb9915cc0569b6d",
"df50d6ad61574f318c4c924862cf396f",
"40952e84036141dea3ed759963c19265",
"5f92ec325c2641948efda94199be9b23",
"bc86cce8426e4d00819f0531ebdbfafa",
"90cb6cd2e9cc4ff2af5422e9ad377faa",
"467f2e169e0d48e0a35d4467efd9fd61",
"d742320e8d584ff095d127f199b04a96",
"238bfc78d2c941bb950c329d4dcd15df"
]
},
{
"uuid": "acfe4b335952431ea37f20b7a3000ada",
"title": "柚子爱AI相机-YUZUMAIX-V39",
"description": "",
"docType": 3,
"master": "595aaccb3ff8474695bc11ec6785c419",
"thumb": "//image.lceda.cn/histories/595aaccb3ff8474695bc11ec6785c419.png",
"components": {
"7ad8a976c4434403abe61b7cb116556e": 2,
"67f8ff6af38d4925b183030c58aed18a": 34,
"50359fcc008846ffbdf2bc467afb79fc": 41,
"9b95f850643a426a9063bbebb4f62193": 2,
"76a5ba8d270141fb994fbfcc50ef0416": 1,
"e7f085e4f71344109c090040716070ab": 1,
"5961548da77e4fb6940ceaeff42fdab3": 1,
"3667d18b6ca74adbb60776118af4d7b7": 8,
"8630a74813a14cdca67e7b05dd5d2bc3": 1,
"bbf2d74717914929b5e52e3caa0392b2": 1,
"b3b82869fa924bae820e3a6cfb44d689": 1,
"308001d2388148a595891bedf0d443a5": 1,
"795419fe08b4408b81518abedfb501c7": 1,
"2913bc4dd4a6497b86bc7c16e187f46a": 3,
"bade90667a6e4731855e9a223387b8a6": 3,
"7559b1e979f64c67b5b3434f1fdc0256": 3,
"a2365841f454457390f178da5e6ade00": 1,
"01aad49044514640b618e7541f2ae097": 2
},
"updateTime": 1702432393,
"createTime": 1695189180,
"histories": [
"595aaccb3ff8474695bc11ec6785c419"
]
}
]
}