Facere/FacereDataset
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity
Files
ed713fa5571e90b0b49fbf82e5d98f666a207d62
FacereDataset/data/raw/oshwhub/7360e73de5dd428e9f29e10573f2d8ac/source/33158668940f4e6f8fd14bec7c34d622.json
Knowit c6279bff08 Add EasyEDA Pro 2.x legacy source ingestion (5/5 batch closure) 
补齐前一批失败的 2 个 legacy Pro 项目（立创·泰山派 RK3566、立创·梁山派），
打通 Pro 2.x 旧版工程的源抓取链路。结合上一 commit 的 modern Pro 3.x
路径，本仓库 5/5 Pro 项目 EPRO2/dataStr 全部端到端打通。

Pro 2.x 与 Pro 3.x 是两个完全不同的存储模型：
- Pro 3.x：git-style branch + linear history chain，AES-128-GCM 加密的
  EPRO2 增量消息流，按 history 重放（已在前一 commit 打通）
- Pro 2.x：无 branch / 无 history。文档以 EasyEDA Std plaintext dataStr
  存储（同 ["DOCTYPE","SCH","1.1"] 格式），按 doc UUID 通过
  /api/v2/documents/lists 批量 GET，主体无加密，只组件库走 AES

Pro 2.x 抓取链由 HAR (tmp/prodownload3.har, 178 请求) 反推：

  GET  /api/v4/projects/<P>                     → boards: [{sch, pcb, name}]
  GET  /api/projects/<P>/ticket?uuid=&g_ticket=-1
                                                → 完整项目 manifest
  POST /api/schematic/lists {uuids:[<sch>]}     → sort: [{uuid:<sheet>}]
  POST /api/v2/documents/lists {uuids,docType:1} → schematic plaintext
  POST /api/v2/documents/lists {uuids,docType:3} → PCB plaintext
  POST /api/coppers/search {paths}              → 铺铜层
  POST /api/textpath/search {paths,project_uuid}→ 字体/文字
  POST /api/v2/resources/search {hash,project_uuid} → BLOB 图片

实现：
- crawlers/oshwhub/crawler.py:
  - fetch_pro_source() refactor 成 dispatcher，先 GET project meta
    检查 branch_uuid，null 即旧版走 _fetch_pro_legacy()，非空走
    _fetch_pro_modern()
  - _fetch_pro_legacy() 新增（按上面 9 步流程拉所有 doc + 辅助层）
  - _pro_post_json() POST helper（与 _pro_get_json 对称）
- schemas/project.schema.json: source_format enum 加 easyeda-pro-legacy
- docs/sources/easyeda_pro_source.md rev 4: §1.1 旧版 vs 新版判别表更新、
  §2.7 新增旧版抓取流程 + 实测数据

落盘约定（旧版）：
  source/ticket.json                     完整 manifest
  source/<sheet_uuid>.json               每张原理图（含 dataStr）
  source/pcb_<pcb_uuid>.json             每块 PCB
  source/coppers.json/textpath.json/blobs.json  辅助 PCB 层资源
  source/manifest.json                   索引

实测：
  立创·梁山派      editor=2.1.30, 2 sheets+1 pcb,    1.0 MB,  78 sym/191 fp/128 dev
  立创·泰山派 RK3566 editor=2.1.40, 29 sheets+1 pcb, 0.8 MB, 299 sym/524 fp/295 dev

旧版项目体量比新版小两个数量级（梁山派 1 MB vs RK3576 66 MB）—— 没有
增量 history，组件库走单独端点，本身就是当前快照。

5/5 Pro 项目终极汇总：
  X86 主板          easyeda-pro        3.2.15  7374 docs / 481 MB
  泰山派 RK3566     easyeda-pro-legacy 2.1.40    30 docs / 0.8 MB
  梁山派            easyeda-pro-legacy 2.1.30     3 docs / 1.0 MB
  220V 桌面电源     easyeda-pro        3.2.69   771 docs /  26 MB
  ESP-VoCat         easyeda-pro        3.2.91   278 docs / 7.5 MB

共 8456 docs / ~516 MB plain。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 21:59:25 +08:00

1 line
9.1 KiB
JSON
Raw Blame History

{"uuid":"33158668940f4e6f8fd14bec7c34d622","created_at":"2023-12-01 14:36:53","creator":{"uuid":"c644bf6717124cfe9ae4d22a5ba8d83a","username":"eda_nqhjnjnc","nickname":"15211163466_27"},"dataStr":"[\"DOCTYPE\",\"SCH\",\"1.1\"]\n[\"HEAD\",{\"originX\":0,\"originY\":0,\"version\":\"2.1.39.1e0cbcef\",\"maxId\":4909}]\n[\"COMPONENT\",\"e1\",\"\",0,0,0,0,{},0]\n[\"FONTSTYLE\",\"st1\",null,null,null,20,null,null,null,null,1,1]\n[\"ATTR\",\"e19\",\"e1\",\"Symbol\",\"a8ec2f850ff14c8d8484a10b472a8799\",0,0,2506,-116,0,\"st1\",0]\n[\"FONTSTYLE\",\"st2\",null,null,null,null,null,null,null,null,null,null]\n[\"ATTR\",\"e34\",\"e1\",\"Footprint\",\"\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e3\",\"e1\",\"Company\",\"嘉立创EDA\",0,1,998,30,0,\"st1\",0]\n[\"FONTSTYLE\",\"st3\",null,null,null,15,null,null,null,null,1,0]\n[\"ATTR\",\"e4\",\"e1\",\"Drawed\",\"\",0,1,558,120,0,\"st3\",0]\n[\"ATTR\",\"e5\",\"e1\",\"Reviewed\",\"\",0,1,558,100,0,\"st3\",0]\n[\"FONTSTYLE\",\"st4\",null,null,null,15,null,null,null,null,1,1]\n[\"ATTR\",\"e6\",\"e1\",\"Version\",\"V1.0\",0,1,718,30,0,\"st4\",0]\n[\"ATTR\",\"e7\",\"e1\",\"Page Size\",\"A4\",0,1,800,30,0,\"st4\",0]\n[\"ATTR\",\"e8\",\"e1\",\"@Project Name\",\"立创·泰山派开发板\",0,1,920,100,0,\"st1\",0]\n[\"ATTR\",\"e9\",\"e1\",\"@Page Count\",\"29\",0,1,1102,61,0,\"st4\",0]\n[\"ATTR\",\"e10\",\"e1\",\"@Update Date\",\"2023-12-01\",0,0,1010,180,0,\"st3\",0]\n[\"ATTR\",\"e11\",\"e1\",\"@Create Date\",\"2023-12-01\",0,0,1010,160,0,\"st3\",0]\n[\"ATTR\",\"e12\",\"e1\",\"@Schematic Name\",\"schematic\",0,1,730,170,0,\"st1\",0]\n[\"ATTR\",\"e13\",\"e1\",\"Part Number\",\"\",0,1,1010,140,0,\"st3\",0]\n[\"ATTR\",\"e14\",\"e1\",\"@Page No\",\"20\",0,1,985,61,0,\"st4\",0]\n[\"ATTR\",\"e15\",\"e1\",\"@Page Name\",\"DEBUG\",0,1,730,140,0,\"st4\",0]\n[\"FONTSTYLE\",\"st5\",null,null,null,\"10\",null,null,null,null,null,null]\n[\"ATTR\",\"e18\",\"e1\",\"Device\",\"3147b110bcd447c49c6e9be850a52941\",0,0,null,null,0,\"st5\",0]\n[\"COMPONENT\",\"e537\",\"KLXES15AAA1.1\",640,440,90,0,{},0]\n[\"ATTR\",\"e3693\",\"e537\",\"Footprint\",\"a740c1cb3ade405790d761308d4a05d8\",0,0,null,null,0,\"st2\",0]\n[\"FONTSTYLE\",\"st6\",null,null,null,null,null,null,null,null,null,0]\n[\"ATTR\",\"e1678\",\"e537\",\"Symbol\",\"03d30bf8b909451a8ca2aea35c1b5028\",0,0,null,null,0,\"st6\",0]\n[\"ATTR\",\"e557\",\"e537\",\"Designator\",\"D6\",0,1,650,435,0,\"st2\",0]\n[\"ATTR\",\"e558\",\"e537\",\"Name\",\"={Manufacturer Part}\",0,0,650,435,0,\"st2\",0]\n[\"ATTR\",\"e559\",\"e537\",\"Device\",\"43c3af329db341c388014bbb245feeb7\",0,0,null,null,0,\"st5\",0]\n[\"ATTR\",\"e560\",\"e537\",\"Unique ID\",\"gge528\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e2058\",\"e537\",\"Reuse Block\",\"\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e2059\",\"e537\",\"Group ID\",\"\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e2060\",\"e537\",\"Channel ID\",\"\",0,0,null,null,0,\"st2\",0]\n[\"COMPONENT\",\"e567\",\"KLXES15AAA1.1\",605,440,90,0,{},0]\n[\"ATTR\",\"e3714\",\"e567\",\"Footprint\",\"a740c1cb3ade405790d761308d4a05d8\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e1699\",\"e567\",\"Symbol\",\"03d30bf8b909451a8ca2aea35c1b5028\",0,0,null,null,0,\"st6\",0]\n[\"ATTR\",\"e587\",\"e567\",\"Designator\",\"D5\",0,1,615,435,0,\"st2\",0]\n[\"ATTR\",\"e588\",\"e567\",\"Name\",\"={Manufacturer Part}\",0,0,615,435,0,\"st2\",0]\n[\"ATTR\",\"e589\",\"e567\",\"Device\",\"43c3af329db341c388014bbb245feeb7\",0,0,null,null,0,\"st5\",0]\n[\"ATTR\",\"e590\",\"e567\",\"Unique ID\",\"gge529\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e2061\",\"e567\",\"Reuse Block\",\"\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e2062\",\"e567\",\"Group ID\",\"\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e2063\",\"e567\",\"Channel ID\",\"\",0,0,null,null,0,\"st2\",0]\n[\"COMPONENT\",\"e670\",\"0402WGF1000TCE.1\",550,515,0,0,{},0]\n[\"ATTR\",\"e4555\",\"e670\",\"Supplier Footprint\",\"0402\",0,1,565,505,0,\"st2\",0]\n[\"ATTR\",\"e1727\",\"e670\",\"Symbol\",\"65b05b32b1ea4208a8288623150d615b\",0,0,null,null,0,\"st6\",0]\n[\"ATTR\",\"e687\",\"e670\",\"Designator\",\"R88\",0,1,565,515,0,\"st2\",0]\n[\"ATTR\",\"e688\",\"e670\",\"Name\",\"={Manufacturer Part}\",0,0,540,500,0,\"st2\",0]\n[\"ATTR\",\"e689\",\"e670\",\"Device\",\"e2e330b4ac6948d88ba461af27643ae3\",0,0,null,null,0,\"st5\",0]\n[\"ATTR\",\"e690\",\"e670\",\"Unique ID\",\"gge448\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e2064\",\"e670\",\"Reuse Block\",\"\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e2065\",\"e670\",\"Group ID\",\"\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e2066\",\"e670\",\"Channel ID\",\"\",0,0,null,null,0,\"st2\",0]\n[\"COMPONENT\",\"e697\",\"0402WGF1000TCE.1\",550,475,0,0,{},0]\n[\"ATTR\",\"e4573\",\"e697\",\"Supplier Footprint\",\"0402\",0,1,565,465,0,\"st2\",0]\n[\"ATTR\",\"e1745\",\"e697\",\"Symbol\",\"65b05b32b1ea4208a8288623150d615b\",0,0,null,null,0,\"st6\",0]\n[\"ATTR\",\"e714\",\"e697\",\"Designator\",\"R89\",0,1,565,475,0,\"st2\",0]\n[\"ATTR\",\"e715\",\"e697\",\"Name\",\"={Manufacturer Part}\",0,0,540,460,0,\"st2\",0]\n[\"ATTR\",\"e716\",\"e697\",\"Device\",\"e2e330b4ac6948d88ba461af27643ae3\",0,0,null,null,0,\"st5\",0]\n[\"ATTR\",\"e717\",\"e697\",\"Unique ID\",\"gge449\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e2067\",\"e697\",\"Reuse Block\",\"\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e2068\",\"e697\",\"Group ID\",\"\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e2069\",\"e697\",\"Channel ID\",\"\",0,0,null,null,0,\"st2\",0]\n[\"LINESTYLE\",\"st7\",null,null,\"none\",null]\n[\"WIRE\",\"e724\",[[605,485,605,455],[695,485,605,485],[605,515,570,515],[605,515,605,485]],\"st7\",0]\n[\"ATTR\",\"e4218\",\"e724\",\"NET\",\"TX\",0,1,675,485,0,\"st2\",0]\n[\"WIRE\",\"e725\",[[640,475,640,455],[695,475,640,475],[570,475,640,475]],\"st7\",0]\n[\"ATTR\",\"e4216\",\"e725\",\"NET\",\"RX\",0,1,675,475,0,\"st2\",0]\n[\"WIRE\",\"e728\",[[420,515,530,515]],\"st7\",0]\n[\"ATTR\",\"e729\",\"e728\",\"NET\",\"UART2_TX_M0_DEBUG\",0,1,420,515,0,\"st2\",0]\n[\"WIRE\",\"e730\",[[420,475,530,475]],\"st7\",0]\n[\"ATTR\",\"e731\",\"e730\",\"NET\",\"UART2_RX_M0_DEBUG\",0,1,420,475,0,\"st2\",0]\n[\"COMPONENT\",\"e2628\",\"GH125-S04DCA-00.1\",715,480,90,0,{},0]\n[\"ATTR\",\"e2629\",\"e2628\",\"Symbol\",\"c04ba3c9b095491baa80d10ea1beb640\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e2630\",\"e2628\",\"Designator\",\"CN2\",0,1,740,505,0,\"st2\",0]\n[\"ATTR\",\"e2655\",\"e2628\",\"Name\",\"={Manufacturer Part}\",0,1,740,495,0,\"st2\",0]\n[\"ATTR\",\"e2657\",\"e2628\",\"Device\",\"5e2cdac770f24de3a153d196b06a670c\",0,0,null,null,0,\"st5\",0]\n[\"ATTR\",\"e2658\",\"e2628\",\"Reuse Block\",\"\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e2659\",\"e2628\",\"Group ID\",\"\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e2660\",\"e2628\",\"Channel ID\",\"\",0,0,null,null,0,\"st2\",0]\n[\"ATTR\",\"e2661\",\"e2628\",\"Unique ID\",\"gge450\",0,0,null,null,0,\"st2\",0]\n[\"COMPONENT\",\"e2807\",\"\",660,525,0,0,{},0]\n[\"FONTSTYLE\",\"st8\",null,null,null,null,null,null,null,null,2,0]\n[\"ATTR\",\"e2809\",\"e2807\",\"Symbol\",\"b0656b6b4a5744e5adb7bdf36c0103ee\",0,0,660,495,0,\"st8\",0]\n[\"FONTSTYLE\",\"st9\",null,\"#9900FF\",null,null,null,null,null,null,2,1]\n[\"ATTR\",\"e2810\",\"e2807\",\"Global Net Name\",\"VCC5V0_SYS\",0,1,660,535,0,\"st9\",0]\n[\"FONTSTYLE\",\"st10\",null,null,null,\"10\",null,null,null,null,1,1]\n[\"ATTR\",\"e2811\",\"e2807\",\"Device\",\"66ae456a56584d099579688edf9ceea1\",0,0,660,525,0,\"st10\",0]\n[\"ATTR\",\"e2812\",\"e2807\",\"Name\",\"VCC5V0_SYS\",0,0,660,525,0,\"st2\",0]\n[\"FONTSTYLE\",\"st11\",null,\"#9933CC\",null,30,null,1,null,null,null,null]\n[\"TEXT\",\"e2955\",520,640,0,\"Debug\",\"st11\",0]\n[\"WIRE\",\"e3221\",[[720,415,720,400],[605,415,640,415],[605,415,605,425],[640,415,640,425],[720,415,760,415],[760,415,760,540],[720,525,720,540],[720,415,720,435],[640,415,720,415],[695,540,720,540],[695,495,695,540],[720,540,760,540]],\"st7\",0]\n[\"ATTR\",\"e4633\",\"e3221\",\"NET\",\"GND\",0,0,695,495,90,\"st2\",0]\n[\"LINESTYLE\",\"st12\",null,null,null,null]\n[\"WIRE\",\"e3257\",[[695,465,660,465],[660,465,660,525]],\"st12\",0]\n[\"ATTR\",\"e3259\",\"e3257\",\"NET\",\"VCC5V0_SYS\",0,0,660,500,90,\"st2\",0]\n[\"FONTSTYLE\",\"st13\",null,\"#00FF00\",null,20,null,0,null,null,null,null]\n[\"TEXT\",\"e4616\",300,490,0,\"default 3.3V\",\"st13\",0]\n[\"LINESTYLE\",\"st14\",null,3,null,3]\n[\"RECT\",\"e4618\",235,720,890,300,30,30,0,\"st14\",0]\n[\"RECT\",\"e4619\",465,690,650,615,30,30,0,\"st14\",0]\n[\"COMPONENT\",\"e4621\",\"\",720,400,0,0,{},0]\n[\"FONTSTYLE\",\"st15\",null,null,null,10,null,null,null,null,2,0]\n[\"ATTR\",\"e4622\",\"e4621\",\"Symbol\",\"3e2301b8470549f8ae3f43ee012cb390\",0,0,720,370,0,\"st15\",0]\n[\"FONTSTYLE\",\"st16\",null,null,null,10,null,null,null,null,1,1]\n[\"ATTR\",\"e4623\",\"e4621\",\"Global Net Name\",\"GND\",0,1,720,375,0,\"st16\",0]\n[\"ATTR\",\"e4629\",\"e4621\",\"Device\",\"29564a22c8d84bcaa93b6cbb648974a7\",0,0,null,null,0,\"st5\",0]","description":"","display_title":"DEBUG","docType":1,"modifier":{"uuid":"c644bf6717124cfe9ae4d22a5ba8d83a","username":"eda_nqhjnjnc","nickname":"15211163466_27"},"project_uuid":"7360e73de5dd428e9f29e10573f2d8ac","schematic_uuid":"6394f41146404ead95ba248444940c6c","sheet_id":29,"ticket":4,"title":"debug","updateTime":1701414519,"thumb":"https://image-pro.lceda.cn/pullimages/8657c75ce1a347949683aa98bbf929ed.webp","guest":false,"writable":false}
Reference in New Issue View Git Blame Copy Permalink