Facere/FacereDataset
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Add EasyEDA Std project source ingestion (10 boards backfilled)

Browse Source 
打通 oshwhub origin=std 项目的工程源（schematic + PCB dataStr）抓取链路。原
plan.md §1.6 假设需要登录，实测 lceda.cn/api/documents/<doc>?uuid=<doc>&path=<doc>
对公开项目匿名可访问 —— 无需 cookie，无账号封禁风险。

调研：4 轮探测留痕在 data/state/std_probe[1-5]/（gitignored）；翻 Std 编辑器
v6.5.51 的 main.min.js bundle 找到 ajaxDetail 端点；按 docType 区分两种
响应 shape（schematic 项目视图 vs PCB 文档视图）。

Crawler:
  - make_source_client() 用浏览器 UA + lceda.cn/editor Referer，因为
    oshwhub /api/project/<uuid> 端点拒绝 FacereDataset/0.1 UA（CLAUDE.md
    UA 例外条款：目标站主动封自定义 UA + 公开静态资源）
  - fetch_std_source(): 项目元 → version_documents → 逐文档 dataStr → 落
    source/<doc>.json + source/manifest.json
  - --with-source（爬新项目时一并抓源）/ --backfill-source（仅扫已有）
  - QPS ≤ 0.2 (SLEEP_SOURCE = 5s) 自律

Schema: 加 source_format / source_path / source_documents / editor_version
（前 3 进 enum 锁定，便于后续 Pro / KiCad 源对齐）。

回填结果：10/10 成功，45 个文档，33.2 MB；schema validate 全通。
docTypes 主要是 1 (schematic) 与 3 (pcb)；USB 电压电流表只有 PCB 文档（4 个：
主板+盖板+底板+面板，作者未上传原理图源）。

完整调研：docs/sources/easyeda_std_source.md。
This commit is contained in:
Knowit
2026-04-28 20:07:40 +08:00
parent b0d3afd2a9
commit d874278bc5
71 changed files with 4389 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
data/raw/oshwhub/892dbc4ebca74227ac6269a1693380d8/metadata.json
View File

@@ -103,5 +103,42 @@
"public": true,
"publish": true,
"skipped_files": []
}
},
"source_format": "easyeda-std",
"source_path": "source/",
"source_documents": [
{
"doc_uuid": "144f2e5178604ab8add093d7ffec0071",
"docType": 3,
"master": "784bdf243ee64af5ac6f003506591d18",
"path": "source/144f2e5178604ab8add093d7ffec0071.json",
"size": 917551,
"sha256": "6d3d48aa287efeaf18e092d80f8a965ba7a4e501170475759bdf81f82e8d774f"
},
{
"doc_uuid": "975ab458f0ab46b4a7e9152399ffbf57",
"docType": 1,
"master": "f607e8d2bc69470fb3117da9b65d45ab",
"path": "source/975ab458f0ab46b4a7e9152399ffbf57.json",
"size": 258763,
"sha256": "f1008ce2f4812dc55212cdb34c4920b5a60bbb4f9af167f5f7dd6e58fe400e03"
},
{
"doc_uuid": "b5f17a05abcc439491b9c11861fe0ee2",
"docType": 1,
"master": "72d1f0e29e384bcba077ca3e96920dd6",
"path": "source/b5f17a05abcc439491b9c11861fe0ee2.json",
"size": 258763,
"sha256": "f1008ce2f4812dc55212cdb34c4920b5a60bbb4f9af167f5f7dd6e58fe400e03"
},
{
"doc_uuid": "bbc0cb810edd408f9c9f34adf3e171d1",
"docType": 3,
"master": "0ce027b5537d44c6991d9ed7c3e4ac50",
"path": "source/bbc0cb810edd408f9c9f34adf3e171d1.json",
"size": 5256,
"sha256": "37ca6b546c4f1267e1780a2584e839c32a96622ba42992124f6b5620c181a122"
}
],
"editor_version": "6.5.41"
}

1
data/raw/oshwhub/892dbc4ebca74227ac6269a1693380d8/source/144f2e5178604ab8add093d7ffec0071.json Normal file
View File

File diff suppressed because one or more lines are too long

1
data/raw/oshwhub/892dbc4ebca74227ac6269a1693380d8/source/975ab458f0ab46b4a7e9152399ffbf57.json Normal file
View File

File diff suppressed because one or more lines are too long

1
data/raw/oshwhub/892dbc4ebca74227ac6269a1693380d8/source/b5f17a05abcc439491b9c11861fe0ee2.json Normal file
View File

File diff suppressed because one or more lines are too long

1
data/raw/oshwhub/892dbc4ebca74227ac6269a1693380d8/source/bbc0cb810edd408f9c9f34adf3e171d1.json Normal file
View File

File diff suppressed because one or more lines are too long

183
data/raw/oshwhub/892dbc4ebca74227ac6269a1693380d8/source/manifest.json Normal file
View File

@@ -0,0 +1,183 @@
{
"project_uuid": "892dbc4ebca74227ac6269a1693380d8",
"fetched_at": "2026-04-28T11:48:23.363052+00:00",
"editor_version": "6.5.41",
"documents": [
{
"doc_uuid": "144f2e5178604ab8add093d7ffec0071",
"docType": 3,
"master": "784bdf243ee64af5ac6f003506591d18",
"path": "source/144f2e5178604ab8add093d7ffec0071.json",
"size": 917551,
"sha256": "6d3d48aa287efeaf18e092d80f8a965ba7a4e501170475759bdf81f82e8d774f"
},
{
"doc_uuid": "975ab458f0ab46b4a7e9152399ffbf57",
"docType": 1,
"master": "f607e8d2bc69470fb3117da9b65d45ab",
"path": "source/975ab458f0ab46b4a7e9152399ffbf57.json",
"size": 258763,
"sha256": "f1008ce2f4812dc55212cdb34c4920b5a60bbb4f9af167f5f7dd6e58fe400e03"
},
{
"doc_uuid": "b5f17a05abcc439491b9c11861fe0ee2",
"docType": 1,
"master": "72d1f0e29e384bcba077ca3e96920dd6",
"path": "source/b5f17a05abcc439491b9c11861fe0ee2.json",
"size": 258763,
"sha256": "f1008ce2f4812dc55212cdb34c4920b5a60bbb4f9af167f5f7dd6e58fe400e03"
},
{
"doc_uuid": "bbc0cb810edd408f9c9f34adf3e171d1",
"docType": 3,
"master": "0ce027b5537d44c6991d9ed7c3e4ac50",
"path": "source/bbc0cb810edd408f9c9f34adf3e171d1.json",
"size": 5256,
"sha256": "37ca6b546c4f1267e1780a2584e839c32a96622ba42992124f6b5620c181a122"
}
],
"upstream_version_documents": [
{
"uuid": "144f2e5178604ab8add093d7ffec0071",
"title": "PCB_ESP32_S3_WATCH_V22",
"description": "",
"docType": 3,
"master": "784bdf243ee64af5ac6f003506591d18",
"thumb": "//image.lceda.cn/histories/784bdf243ee64af5ac6f003506591d18.png",
"components": {
"19c77e72422b43b59c26b9fa67af5de4": 2,
"6f6335d68ae7446e8a357c20ae45d590": 31,
"ed690cb1431045bba32a67c49323a918": 7,
"2c1c812c188942e5b49999b55bf594c5": 24,
"b4560ae730a14c4fa4de0df5b641ef3e": 1,
"761557e59e0744adb7d07d8b0b86141d": 1,
"31d365bf281f4b80a939de54be9c2bdd": 1,
"3c7eafbdcf4a4bf8a52e4d58a4f0f666": 5,
"3c76518509c249e6a2444fa4139a5658": 1,
"5acad293e0b64918bd3acb58f1cf9066": 1,
"8d5788cbd01748978d7d28c3f170409f": 1,
"253417f9c8d54cf0894ef378394641cc": 1,
"eea6534d03824fadb1228aab224ae0e0": 1,
"0e1e27d30afa4c5d9ec8147cd951909f": 1,
"7a1de61d93c345aea87efd889ccfb62f": 1,
"936967cc102f48a08fcb86d0dfd18a80": 1,
"720869066b0446fe83631c47016641c0": 1,
"c026b6d376074775b663b33fd8389c4c": 1,
"7f5d3559cc414f77a212a6717ac89149": 1,
"d6fe490f32454c42b81a58cbb5b03888": 1,
"1185c8152efe486684bdcc6ff595ce64": 1,
"7f030c546df94d8ab670d35acba2549c": 1,
"653bfbcd65334c55867107231802b68b": 1,
"7e7eee2dc1794f33b913b55a9be5617f": 1,
"bfb7971950a742cfa83f17c9e30bd62c": 1,
"e8811ffbabd0409492a169948aab3f56": 1,
"81ac96664a514ff8ae27adbae413ae63": 1,
"813b44b10e1449ba9ed0f03c4cfdfd78": 1,
"912e94b8be154ba19e6f181b4dd2169d": 1
},
"updateTime": 1706799344,
"createTime": 1681304798,
"histories": [
"a72739c25d7c4ee4819e3a330179c4f8",
"34e61b41167c47d4820b6423c40d439b",
"5e367c69cfdb42d5b01e1e9e3d3bd547",
"a3e159535ba44456946eef6b5b94d28a",
"1aa878e9a1ae4babbd75d55df64e2879",
"cd6d21a38c3f4ada9746d2c7cbd512b7",
"c11c80a636ac444f8150bcf8ae98fe17",
"9d711ed44b05456f81c8f159e9ff421a",
"afb39b3620e247aaa7aa7bd3117bf9e9",
"784bdf243ee64af5ac6f003506591d18"
]
},
{
"uuid": "975ab458f0ab46b4a7e9152399ffbf57",
"title": "Sheet_V22",
"description": "",
"docType": 1,
"master": "f607e8d2bc69470fb3117da9b65d45ab",
"thumb": "//image.lceda.cn/histories/f607e8d2bc69470fb3117da9b65d45ab.png",
"components": {
"305675e0de1c4230a6c7a54d86bd3bc5": 1,
"f1506f54198446dc9f07443b8051b526": 1,
"9d140bbbd1904f83b41a8d5c8f27fc85": 1,
"4343bd47ba52400e8fa65c92be528fe6": 1,
"7d19d6f006684f60afb0c218c16ed1d9": 1,
"204c780ee7c54c0ab1adf33550b54cb5": 1,
"d0130f565b994b6ea8408ab84584129b": 1,
"3f6a9774b5f24ac298465fd10c2cf2a9": 33,
"e2d5588fc85c4705bac23f944ec3ae9d": 1,
"42d59bcfb32943a0aea2f5a79f439da3": 18,
"2d3bf49add9445a7a9ed38e7eead1728": 1,
"8d74e835115147068b0d314bb16ae99d": 1,
"76597433371a444e9d2838968dc13417": 1,
"a17ec733dc1b4cc4873ea6f4e40a3545": 1,
"a60d3d904d914b53a8761af07bd9fc24": 6,
"779f6465d7614ee0bb031b848d08dfbd": 1,
"dc0fa062e09a41408c16dbb49b24c67b": 1,
"0e4ccd272ef6429e8d5c6e4009007e6c": 1,
"326b564427ed85053559e109589138a8": 3,
"b3d1e8f3127f466e8955b937213b9a6b": 1,
"76bd287548ed4f9fbcb7402ad173346f": 1,
"695e793433b84c4ab417ea3207cb91a1": 1,
"99348140ef4ce40e50fbbd53b72e434f": 1,
"e80c89622a0943dd89afc26babb051b1": 5,
"0b0a4a3699414b95a2dd2fa6e361762c": 1,
"e7f8d3e1844c40a49e89adc3f4c1d694": 2,
"b6d4887ccdb442f5a11ac4b50feb4a03": 1,
"a453d09b8221458a9a7dbe9b5a2b74e3": 1,
"6a76741940dc4999a3f240684c1427e3": 1,
"404aa9221ff94de597b8734806e7c74f": 2,
"d0875acd9ad6454aafc85eaa1ec17abc": 1
},
"updateTime": 1706798988,
"createTime": 1681304849,
"histories": [
"8db68c0df22b4b5d8d80c0ee226c5093",
"4d99cccb6a7f49589912f56ca730f86d",
"8d26f76f25114b54ba5b92780c639a31",
"6d6a36f5aec64c9aa7ed4538ddb80546",
"5c472cf39701437c82ebd9675f6aef70",
"f607e8d2bc69470fb3117da9b65d45ab"
]
},
{
"uuid": "b5f17a05abcc439491b9c11861fe0ee2",
"title": "PCB版本说明",
"description": "",
"docType": 1,
"master": "72d1f0e29e384bcba077ca3e96920dd6",
"thumb": "//image.lceda.cn/histories/72d1f0e29e384bcba077ca3e96920dd6.png",
"components": null,
"updateTime": 1686659738,
"createTime": 1681304923,
"histories": [
"7a1f3ca262794980846ac2571f32e621",
"756480855f9743cba0fc4cbd2a548cff",
"31d443aa57074a2ea6829a4893b5d0e8",
"55cb1b2389504cebab370d5617153f33",
"5a03936e79b542cda98c574947019ac1",
"a5ec27eace9245c1b710ff90c6252574",
"9384c1bbd9e0419595fef2d8a7ef0130",
"f548eaf920ab4b99b5fd3142c8dd4ea3",
"b6bee8486d5244968e87a548362a3c31",
"b40180691f794bc5aa1256ebd199b401",
"72d1f0e29e384bcba077ca3e96920dd6"
]
},
{
"uuid": "bbc0cb810edd408f9c9f34adf3e171d1",
"title": "V1.2外壳后盖板",
"description": "",
"docType": 3,
"master": "0ce027b5537d44c6991d9ed7c3e4ac50",
"thumb": "//image.lceda.cn/histories/0ce027b5537d44c6991d9ed7c3e4ac50.png",
"components": [],
"updateTime": 1686659650,
"createTime": 1686659650,
"histories": [
"0ce027b5537d44c6991d9ed7c3e4ac50"
]
}
]
}