255 lines
4.1 KiB
Markdown
255 lines
4.1 KiB
Markdown
# 阿里云 APS Node 同步命令清单
|
||
|
||
## 1. 进入项目目录
|
||
|
||
```powershell
|
||
cd D:\project\python\aliyun-sync\aliyun-aps-sync
|
||
```
|
||
|
||
## 2. 安装依赖
|
||
|
||
```powershell
|
||
npm install
|
||
```
|
||
|
||
## 3. 登录
|
||
|
||
```powershell
|
||
npm run login
|
||
```
|
||
|
||
作用:
|
||
|
||
- 打开浏览器
|
||
- 手动完成阿里云 / RAM 登录
|
||
- 自动验证我的客户和账单查询
|
||
- 保存登录态
|
||
|
||
## 4. 全量同步
|
||
|
||
```powershell
|
||
npm run sync
|
||
```
|
||
|
||
如果要从已有 checkpoint 继续全量流程(当前主要覆盖 orders + bills):
|
||
|
||
```powershell
|
||
npm run sync -- --resume
|
||
```
|
||
|
||
默认包含:
|
||
|
||
- customers
|
||
- customerDetails
|
||
- orders
|
||
- orderDetails
|
||
- bills
|
||
- messages
|
||
|
||
并在同步过程中直接写入数据库。
|
||
|
||
## 5. 日增量同步
|
||
|
||
```powershell
|
||
npm run incremental
|
||
```
|
||
|
||
默认包含:
|
||
|
||
- orders
|
||
- orderDetails
|
||
- bills
|
||
- messages
|
||
|
||
说明:
|
||
|
||
- 不抓 customer
|
||
- 按数据库 watermark + overlap 抓取
|
||
|
||
## 6. 单独抓账单
|
||
|
||
```powershell
|
||
npm run bills
|
||
```
|
||
|
||
## 7. 单独抓消息
|
||
|
||
```powershell
|
||
npm run messages
|
||
```
|
||
|
||
## 8. 单独抓订单
|
||
|
||
```powershell
|
||
npm run orders
|
||
```
|
||
|
||
说明:会同时抓取订单列表与订单详情。
|
||
|
||
订单增量:
|
||
|
||
```powershell
|
||
npm run orders -- --incremental
|
||
```
|
||
|
||
订单断点继续:
|
||
|
||
```powershell
|
||
npm run orders -- --resume
|
||
```
|
||
|
||
## 9. 从最新 checkpoint 继续抓账单
|
||
|
||
```powershell
|
||
npm run bills -- --resume
|
||
```
|
||
|
||
## 10. 定时任务
|
||
|
||
```powershell
|
||
npm run schedule
|
||
```
|
||
|
||
默认读取:
|
||
|
||
```env
|
||
ALIYUN_APS_SCHEDULE_MODE=incremental
|
||
```
|
||
|
||
## 11. 常用 `.env` 配置
|
||
|
||
```env
|
||
ALIYUN_APS_BASE_URL=https://aps.aliyun.com
|
||
ALIYUN_APS_HEADLESS=false
|
||
ALIYUN_APS_BROWSER_MODE=launch
|
||
ALIYUN_APS_BROWSER_CHANNEL=
|
||
ALIYUN_APS_BROWSER_EXECUTABLE_PATH=
|
||
ALIYUN_APS_CDP_URL=http://127.0.0.1:9222
|
||
ALIYUN_APS_TIMEZONE=Asia/Shanghai
|
||
ALIYUN_APS_CRON=0 6 * * *
|
||
ALIYUN_APS_SCHEDULE_MODE=incremental
|
||
ALIYUN_APS_CLOSE_BROWSER=true
|
||
|
||
ALIYUN_APS_FULL_SYNC=true
|
||
ALIYUN_APS_ORDER_START_DATE=2024-01-01
|
||
ALIYUN_APS_INCREMENTAL_ORDER_START_DATE=
|
||
ALIYUN_APS_BILL_START_MONTH=2024-01
|
||
|
||
ALIYUN_APS_ORDER_INCREMENTAL_OVERLAP_DAYS=2
|
||
ALIYUN_APS_BILL_INCREMENTAL_OVERLAP_DAYS=7
|
||
ALIYUN_APS_MESSAGE_INCREMENTAL_OVERLAP_DAYS=7
|
||
|
||
ALIYUN_APS_DB_HOST=
|
||
ALIYUN_APS_DB_PORT=3306
|
||
ALIYUN_APS_DB_USER=
|
||
ALIYUN_APS_DB_PASSWORD=
|
||
ALIYUN_APS_DB_NAME=
|
||
ALIYUN_APS_DB_CHARSET=utf8mb4
|
||
ALIYUN_APS_DB_CONNECTION_LIMIT=5
|
||
|
||
ALIYUN_APS_SMTP_HOST=
|
||
ALIYUN_APS_SMTP_PORT=465
|
||
ALIYUN_APS_SMTP_SECURE=true
|
||
ALIYUN_APS_SMTP_USER=
|
||
ALIYUN_APS_SMTP_PASS=
|
||
ALIYUN_APS_NOTIFY_EMAIL=
|
||
|
||
ALIYUN_APS_CLOSE_BROWSER=true
|
||
```
|
||
|
||
浏览器选择规则:
|
||
|
||
- `ALIYUN_APS_BROWSER_MODE=launch`:Playwright 自己启动浏览器
|
||
- `ALIYUN_APS_BROWSER_MODE=cdp`:附着到你手动打开的浏览器
|
||
- 两项都留空:使用 Playwright 自带 Chromium
|
||
- `ALIYUN_APS_BROWSER_CHANNEL=chrome`:使用 Chrome
|
||
- `ALIYUN_APS_BROWSER_CHANNEL=msedge`:使用 Edge
|
||
- `ALIYUN_APS_BROWSER_EXECUTABLE_PATH=...`:使用指定浏览器路径
|
||
|
||
如果要自己手动打开 Chrome 并过验证码,可先执行:
|
||
|
||
```powershell
|
||
chrome.exe --remote-debugging-port=9222 --user-data-dir="C:\temp\aps-manual-profile"
|
||
```
|
||
|
||
然后 `.env` 配置:
|
||
|
||
```env
|
||
ALIYUN_APS_BROWSER_MODE=cdp
|
||
ALIYUN_APS_CDP_URL=http://127.0.0.1:9222
|
||
```
|
||
|
||
## 12. 推荐执行顺序
|
||
|
||
### 首次初始化
|
||
|
||
```powershell
|
||
cd D:\project\python\aliyun-sync\aliyun-aps-sync
|
||
npm install
|
||
npm run login
|
||
npm run sync
|
||
```
|
||
|
||
### 日常增量
|
||
|
||
```powershell
|
||
cd D:\project\python\aliyun-sync\aliyun-aps-sync
|
||
npm run incremental
|
||
```
|
||
|
||
### 单独同步消息
|
||
|
||
```powershell
|
||
cd D:\project\python\aliyun-sync\aliyun-aps-sync
|
||
npm run messages
|
||
```
|
||
|
||
### 单独同步订单
|
||
|
||
```powershell
|
||
cd D:\project\python\aliyun-sync\aliyun-aps-sync
|
||
npm run orders
|
||
```
|
||
|
||
### 账单长任务恢复
|
||
|
||
```powershell
|
||
cd D:\project\python\aliyun-sync\aliyun-aps-sync
|
||
npm run login
|
||
npm run bills -- --resume
|
||
```
|
||
|
||
### 常驻定时任务
|
||
|
||
```powershell
|
||
cd D:\project\python\aliyun-sync\aliyun-aps-sync
|
||
npm run schedule
|
||
```
|
||
|
||
## 13. 错误文件
|
||
|
||
运行异常时会保存:
|
||
|
||
```text
|
||
data/errors/<dataset>/
|
||
```
|
||
|
||
## 14. 运行时热键
|
||
|
||
| 按键 | 功能 |
|
||
| --- | --- |
|
||
| F7 | 暂停 |
|
||
| F8 | 继续 |
|
||
| F9 | 终止 |
|
||
|
||
## 15. 本地数据目录
|
||
|
||
```text
|
||
data/current/
|
||
data/history/
|
||
data/delta/
|
||
data/checkpoints/
|
||
data/runs/
|
||
data/errors/
|
||
```
|