比较提交
7
次代码提交
| 作者 | SHA1 | 提交日期 | |
|---|---|---|---|
|
|
72d678fa03 | ||
|
|
ae3ed9c3d7 | ||
|
|
1827628a93 | ||
|
|
12df14ebf0 | ||
|
|
58769c431f | ||
|
|
1f598007d1 | ||
|
|
604fce4fec |
没有与搜索条件相匹配的文件
+31
-35
@@ -1,46 +1,42 @@
|
||||
# 交接:飞牛 APFS 只读访问
|
||||
# 交接:大鱼硬盘拷贝
|
||||
|
||||
更新时间:2026-11-11(Asia/Shanghai)。
|
||||
更新时间:2026-11-11,Asia/Shanghai。
|
||||
|
||||
## 当前目标
|
||||
## 当前目标与进展
|
||||
|
||||
基础 APFS 只读挂载已验证。下一步用普通测试文件验证素材读取及导入,再确定应用封装方案。
|
||||
- 用户指定的“火影秋决”和“LCQ荒野乱斗”已完成 NAS 本机复制及核验,目标 `/vol5/1000/剪辑盘备份`,没有向 APFS 源分区写入。
|
||||
- 原生飞牛应用已实现 APFS / NTFS 识别、只读访问、目录勾选、目标目录/新建目录、后台复制、进度和校验、失败/中断任务重试。当前候选版本 0.1.4,待最终安装核对、发行包 Release 保存及清理。
|
||||
- 用户已要求停止逐条播报拷贝速度;仅告知完成或异常。
|
||||
- 用户已明确同意飞牛针对本应用的第三方免责提示并授权继续更新;同一提示无需重复询问,不适用于其他软件或不同条款。
|
||||
|
||||
## 已确认
|
||||
## 工程与正式位置
|
||||
|
||||
- SSH 已登录 `192.168.31.21:22`,管理员提权已验证;网页管理端口为 5668。
|
||||
- NAS:Intel i5-12400T / x86_64;Debian 12;内核 `6.18.18.c1107-trim`。
|
||||
- 有 `/dev/fuse`、FUSE 3.14、Docker;未检测到 apfs-fuse 或本机编译器。
|
||||
- 测试卡为约 29.7 GiB USB SD 卡;APFS 分区当前为 `/dev/sde2`,分区 GUID 类型 `7c3457ef-0000-11aa-aa11-00306543ecac`,4096 字节块大小,NXSB 标识正确。
|
||||
- APFS PARTUUID:`5b255495-2ba8-4e49-8774-227d8dd8878f`。每次操作仍需核实其设备身份、USB 传输及挂载状态,不能盲用 sde2。
|
||||
- 读卡器以 USB 5000M 连接。飞牛图形界面显示“未知文件系统”;独立 FUSE 挂载已成功,但尚未集成系统界面。
|
||||
- 本轮 SSH 曾返回 `Exceeded MaxStartups`,等待后恢复。未修改 SSH 服务。
|
||||
- 本机:`/Users/mac/Documents/dayu-git/repositories/fnos-apfs`;私有远程 `https://dayu.media/git/dayu/fnos-apfs.git`,分支 main,仓库 ID 61。
|
||||
- 固定 appname `dayu-diskcopy`,显示名“大鱼硬盘拷贝”;唯一安装入口 `/var/apps/dayu-diskcopy`,实际 target 在 `/vol3/@appcenter/dayu-diskcopy`,不是两份安装。
|
||||
- 访问:飞牛桌面图标,或 `https://192.168.31.21:5668/app/dayu-diskcopy`。仅管理员,经飞牛官方网关登录态,不增加 TCP 端口。
|
||||
- NAS:fnOS 1.2.0701 / Debian 12 / x86_64 / i5-12400T;Python 3、FUSE、rsync、ntfs-3g 为已有运行组件。
|
||||
- Web 服务实际为专用用户 UID 985;受限设备服务为 root。网关与设备两个 Unix Socket 都为 0660。源设备仅枚举 USB APFS/NTFS,路径/链接越界拒绝。
|
||||
- 核心代码 `application/`;FPK 定义 `packaging/fpk/`;构建入口 `scripts/build-fpk.py`。完整上游源码与 LZFSE 已含在 vendor 和 FPK;许可证及版本见 `UPSTREAM.json`。
|
||||
|
||||
## 工程与恢复
|
||||
## 当前验证
|
||||
|
||||
- 本机:`/Users/mac/Documents/dayu-git/repositories/fnos-apfs`。
|
||||
- 私有远程:`https://dayu.media/git/dayu/fnos-apfs.git`,分支 main;首个基线 `08e7da7f237c1a21aedad17b7a259726b8182060` 已推送并回读核验。
|
||||
- `vendor/apfs-fuse` 保存完整源码快照,含 LZFSE 子模块内容;固定版本见 `UPSTREAM.json`。apfs-fuse `66b86bd525e8cb90f9012543be89b1f092b75cf3`,LZFSE `e634ca58b4821d9f3d560cdc6df5dec02ffc93fd`。
|
||||
- `packaging/Dockerfile` 在固定 Debian 12 基础镜像中编译 Release;`scripts/build-driver.sh` 导出驱动;`scripts/validate-card.py` 按 PARTUUID + USB 序列号核对设备,临时设置内核只读,测试后卸载并恢复原标志。
|
||||
- 验证程序检查挂载只读、样本复制 SHA-256、APFS 分区写计数和首 MiB 内容;这不是对整盘所有文件或全部 APFS 特性的兼容性保证。
|
||||
- 首次代码检查点已从远程独立克隆核验,LZFSE 源码完整。实际验证代码为 `d24400b492a90275d8432e63db78c61ff0ddfa09`;本阶段恢复标签为 `poc-apfs-readonly-20261111`,详见 [RECOVERY.md](RECOVERY.md)。
|
||||
- NAS 验证工程路径为 `/var/tmp/dayu-apfs-validation`,驱动只放在该目录 `build/bin`,未安装至系统路径。保留该目录和专用构建镜像供后续开发。
|
||||
- 首次构建因 Debian 官方源下载缓慢主动取消,改用容器内中科大镜像后 Release 构建成功,仍保留 Debian 原有签名校验;NAS 宿主机软件源未修改。
|
||||
- 九项权限、网关校验、路径边界、文件系统识别和准备中任务可见性测试通过。Python/生命周期脚本语法及官方 fnpack 格式检查通过。
|
||||
- 0.1.3 已实机从界面新建目标目录、选 APFS 目录并复制成功;预置 100000 字节不完整文件,rsync 实际复用 99400 字节,完整结果 SHA-256 一致。已存在且 mtime 不同的文件也正确保留并通过内容校验。见 `verification/2026-11-11-app-ui.json`。
|
||||
- 0.1.4 仅收尾选择框状态、准备中任务可见性和有任务时禁止卸载;正在更新后核对。
|
||||
- NTFS:独立 64 MiB 镜像只读挂载,1 MiB 文件复制 SHA-256 一致;没有实体 NTFS USB 盘验证。见 `verification/2026-11-11-ntfs-image.json`。
|
||||
- APFS SD 基础验证见 `verification/2026-11-11-apfs-sd.json`。首版不承诺加密卷或全部 APFS 特性兼容。
|
||||
- 原始 macOS 扩展属性作为私有存档保存在拷贝目标,不盲目应用机器相关安全属性。真实文件名清单、数据、私有日志和凭据绝不上传 Git。
|
||||
|
||||
## 验证结果与边界
|
||||
## 已完成的比赛目录备份
|
||||
|
||||
- Python/Bash 语法检查通过;实际不匹配 USB 身份测试被拒绝。
|
||||
- NAS 本机 Release 构建成功,运行库均可解析;源码的上游编译警告保留,未修改上游实现。
|
||||
- APFS 分区内核只读标志设为 1 后,以 `ro,nodev,nosuid,noexec` 挂载成功。
|
||||
- 读取并复制三个 64 KiB 系统隐藏文件,合计 196608 字节,目标 SHA-256 与读取流一致。
|
||||
- APFS 分区写入计数测试前后均为 0,首 MiB 哈希不变;测试后已卸载、删除样本副本、恢复原内核只读标志。
|
||||
- 未找到 512 MiB 以内的普通文件可作样本;尚未验证普通素材导入、独立 macOS 参考哈希、大文件速度、加密及其他 APFS 特性。64 KiB 隐藏文件的速度没有代表性,不据此承诺提速。
|
||||
- 脱敏证据见 [verification/2026-11-11-apfs-sd.json](verification/2026-11-11-apfs-sd.json),编译依赖见同目录清单。证据不包含源文件内容、文件名或凭据。
|
||||
- 不使用当前聊天原工作目录“大鱼调色”,不修改 NAS 网站生产源。
|
||||
- 凭据在用户指定的本机安全位置,不进入工程。
|
||||
- 新增 501332211263 字节;复制用时 2197.872 秒;已有 64 个文件完整校验无差异,目录核对无差异,目标已 syncfs,源写计数为零,源已卸载并恢复原只读标志。见 `verification/2026-11-11-competition-copy.json`。
|
||||
- 复制时源盘 Tech / DD56419883A4F,APFS PARTUUID `d6bd791c-7bee-486b-8fef-6cf584381a55`;不能将设备名 sdg2 当永久身份。
|
||||
- 原型任务记录暂在 `/var/tmp/dayu-apfs-validation/jobs/copy-20261111-223613`;将迁移至正式应用 var/jobs 后清理原型目录。正在运行的测试任务不可移动;本轮实际任务均已完成。
|
||||
- 目标隐藏目录 `.dayu-apfs-metadata-20261111-223613` 为对应元数据存档,约 1.7 MB,7374 条记录;保留,不清理。
|
||||
|
||||
## 待完成
|
||||
## 安装与后续
|
||||
|
||||
1. 等待用户确认普通测试文件情况;建议 100–500 MB 视频,若可提供 macOS 的 SHA-256 则做独立比对。拔卡前从飞牛系统推出整张卡。
|
||||
2. 继续使用 PARTUUID 和 USB 身份核对,不假设重新插入后仍为 sde2;重新格式化后 PARTUUID 也会变化。
|
||||
3. 应用界面、FPK、自动识别、系统文件管理集成及真实 SSD 性能尚未实现,本阶段不是正式应用交付。
|
||||
- 当前设备 CLI install-fpk 对已有应用只提示 installed,不能宣称升级。更新要走网页应用中心“手动安装”,先点“从电脑上传”,再选择 FPK;窗口加载是异步的。
|
||||
- 原生网关改写 upstream Host,不能直接严格比较 Origin/Host;现用私有页面随机校验值、跨站请求限制和管理员 Header。匿名、非管理员和缺少校验值的写请求都被拒绝。
|
||||
- 用户界面测试目录 `/vol3/1000/大鱼拷贝功能验证-20261111` 及应用中的本轮测试任务记录在验证后清理;不要删除真实比赛目录或对应元数据存档。
|
||||
- 待完成:核对 0.1.4 的界面复制/选择框、迁移真实任务记录、清理临时挂载与生成副本,更新恢复说明,推送检查点/版本标签;将最终 FPK 上传对应 Release 并完整下载核对 SHA-256。
|
||||
@@ -1,38 +1,41 @@
|
||||
# 飞牛 APFS 只读访问验证
|
||||
# 大鱼硬盘拷贝
|
||||
|
||||
目标:将 APFS 移动存储直接连接 fnOS,在 NAS 本机读取并导入文件,绕过千兆网络。
|
||||
飞牛 fnOS 原生桌面应用,在 NAS 本机读取 APFS / NTFS 外接硬盘并复制目录,不经过电脑或 SMB 中转。
|
||||
|
||||
当前阶段为只读驱动兼容性验证,尚未制作飞牛应用安装包,也未集成系统“外接存储”界面。
|
||||
## 使用
|
||||
|
||||
2026-11-11 实机结果:APFS SD 卡只读挂载成功,三个系统隐藏文件复制校验通过,APFS 分区写计数为零。普通素材导入与大文件速度待测试;详见交接及 `verification/`。
|
||||
支持已测试的 fnOS 1.2.0701 x86_64。手动安装 FPK 后,在飞牛桌面打开“大鱼硬盘拷贝”,或访问 `/app/dayu-diskcopy`。使用管理员登录态,不增加独立网络端口。
|
||||
|
||||
## 安全边界
|
||||
1. 选择外接硬盘,勾选同一层级中的源目录。
|
||||
2. 选择自己的 NAS 存储空间和目标目录,可新建文件夹。
|
||||
3. 开始拷贝,在任务中查看进度和校验结果。关闭窗口不会终止任务。
|
||||
|
||||
- 只对核实身份的 USB APFS 测试分区执行只读访问;不格式化、不修复、不写源盘。
|
||||
- 不安装内核模块,不改飞牛存储服务、启动配置、系统软件包或现有 Docker 服务。
|
||||
- 编译与测试使用独立目录及容器;不使用 NAS 网站生产目录。
|
||||
- 账号密码只从用户指定的本机文件读取到内存,不进入源码、参数或记录。
|
||||
- 默认不启用 apfs-fuse 的 lax 模式,不记录加密调试数据。
|
||||
源盘不格式化、不修复;新挂载的卷使用只读挂载。已被飞牛挂载的 NTFS 优先复用,应用只读取文件,不修改系统挂载方式。已存在的文件先保留并校验,不同版本不自动覆盖。失败或中断的应用任务可重试,已完成文件会复用,部分文件可继续传输。
|
||||
|
||||
## 工程入口
|
||||
普通文件保留名称、时间和链接;APFS 原始扩展属性另外保存到目标中的隐藏存档,不盲目应用机器相关安全属性。存档可能含私人元数据,不上传源码仓库。首版不承诺所有 APFS 特性或加密卷兼容,不改写剪辑工程内部引用。
|
||||
|
||||
- [HANDOFF.md](HANDOFF.md):当前状态与下一步。
|
||||
- 驱动来源:[sgan81/apfs-fuse](https://github.com/sgan81/apfs-fuse),只读 FUSE 实现;固定源码版本及许可证见下文。
|
||||
## 验证
|
||||
|
||||
## 构建与验证
|
||||
- APFS:真实 4 TB USB 盘两目录本机复制,新增约 501 GB,既有 64 个文件校验一致,源分区写计数为零;见 `verification/2026-11-11-competition-copy.json`。
|
||||
- NTFS:只读镜像挂载和复制 SHA-256 比对通过,尚无实体 NTFS USB 盘验证;见 `verification/2026-11-11-ntfs-image.json`。
|
||||
- 网关身份、页面私有校验、跨站请求、路径/链接边界及文件系统识别测试见 `tests/`。
|
||||
- 最终实机交付状态以 [HANDOFF.md](HANDOFF.md) 为准。
|
||||
|
||||
完整驱动快照及 LZFSE 源码位于 `vendor/`,版本见 `UPSTREAM.json`,许可证保留在各自目录内;不需要从上游重新下载源码。
|
||||
## 开发与构建
|
||||
|
||||
在 x86_64 Linux + Docker 中,于工程根执行 `bash scripts/build-driver.sh`。仅容器内安装编译依赖;输出 `build/bin/apfs-fuse`、`apfsutil` 及依赖版本清单。NAS 本机需有 libfuse3、libstdc++、zlib、libbz2 的运行库。
|
||||
- `application/`:界面、降权 Web 服务、受限设备服务、生命周期控制。
|
||||
- `packaging/fpk/`:固定 appname `dayu-diskcopy` 的安装定义。
|
||||
- `scripts/copy-directories.py`:后台复制、内容核验与挂载回收。
|
||||
- `scripts/save-xattrs.py`:私有扩展属性存档。
|
||||
- `vendor/apfs-fuse/`:包含 LZFSE 的完整源码快照及许可证,版本见 `UPSTREAM.json`,FPK 内也附对应源码。
|
||||
|
||||
验证入口为 `scripts/validate-card.py --partuuid <APFS分区PARTUUID> --serial <USB设备序列号> --workdir <工程根目录>`,需管理员运行。它会暂时挂载、读取最多三个 512 MiB 以内样本、将副本写入工程测试目录并比对 SHA-256,然后删除副本及卸载。不会尝试写 APFS 源盘。结果保存在工作目录 `result.json`。
|
||||
Linux + Docker 中执行 `bash scripts/build-driver.sh` 构建 Release 驱动。验证后将两个可执行文件放入 `build/linux-bin/`。打包脚本核对本轮已验证驱动的固定 SHA-256;工具链变化后的二进制需重新验证,不能盲目绕过检查。
|
||||
|
||||
SD 卡只适合验证基本功能,不能代表外置 SSD 的吞吐;隐藏索引文件的读取也不能当作素材导入测试。
|
||||
```sh
|
||||
PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tests -v
|
||||
python3 scripts/build-fpk.py /path/to/official/fnpack
|
||||
```
|
||||
|
||||
## 恢复
|
||||
使用官方 fnpack 1.2.3,产物位于 `dist/`,不保存为 Git 对象或 LFS。更新通过网页应用中心“手动安装”;当前设备的 CLI install-fpk 对已安装应用只提示 installed,不能据此宣称更新成功。
|
||||
|
||||
目标远程:`https://dayu.media/git/dayu/fnos-apfs.git`,分支 `main`。
|
||||
|
||||
克隆后查看交接及固定驱动版本。凭据、磁盘数据、编译输出和容器镜像不纳入 Git;本机账号需独立配置。
|
||||
|
||||
完整恢复入口见 [RECOVERY.md](RECOVERY.md)。
|
||||
恢复入口见 [RECOVERY.md](RECOVERY.md),最新状态见交接。凭据、真实文件名清单、源文件内容和私有日志不进入 Git。
|
||||
@@ -0,0 +1,332 @@
|
||||
"""Privileged local-only disk operations; no TCP listener and no shell execution."""
|
||||
import fcntl
|
||||
import grp
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import pwd
|
||||
import re
|
||||
import signal
|
||||
import socket
|
||||
import socketserver
|
||||
import stat
|
||||
import struct
|
||||
import subprocess
|
||||
import time
|
||||
import uuid
|
||||
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
VAR = Path(os.environ['TRIM_PKGVAR'])
|
||||
APP_UID = int(os.environ['TRIM_UID'])
|
||||
APP_GID = int(os.environ['TRIM_GID'])
|
||||
LEGACY = Path('/var/tmp/dayu-apfs-validation')
|
||||
TERMINAL = {'complete', 'failed', 'needs_attention', 'cleanup_failed', 'cancelled'}
|
||||
HISTORY = {}
|
||||
|
||||
|
||||
def command(args, timeout=30):
|
||||
result = subprocess.run(args, capture_output=True, text=True, timeout=timeout, env={**os.environ, 'LC_ALL': 'C'})
|
||||
if result.returncode:
|
||||
raise ValueError((result.stderr or result.stdout).strip()[-600:] or '操作失败')
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
def beneath(path, root, must_exist=True):
|
||||
path = Path(path).resolve(strict=must_exist)
|
||||
root = Path(root).resolve(strict=True)
|
||||
if not path.is_relative_to(root):
|
||||
raise ValueError('所选路径超出可访问目录')
|
||||
return path
|
||||
|
||||
|
||||
def atomic(path, data):
|
||||
temporary = path.with_suffix('.tmp')
|
||||
temporary.write_text(json.dumps(data, ensure_ascii=False, indent=2))
|
||||
temporary.replace(path)
|
||||
|
||||
|
||||
def disks():
|
||||
tree = json.loads(command(['lsblk', '-J', '-b', '-o',
|
||||
'NAME,PATH,TYPE,SIZE,FSTYPE,UUID,PARTUUID,TRAN,MODEL,SERIAL,MOUNTPOINTS']))
|
||||
result = []
|
||||
for disk in tree['blockdevices']:
|
||||
if disk.get('tran') != 'usb' or not disk.get('size'):
|
||||
continue
|
||||
for part in [disk, *disk.get('children', [])]:
|
||||
filesystem = (part.get('fstype') or '').lower()
|
||||
if filesystem not in ('apfs', 'ntfs', 'ntfs3'):
|
||||
continue
|
||||
identity = part.get('partuuid') or part.get('uuid')
|
||||
if not identity or not disk.get('serial'):
|
||||
continue
|
||||
result.append({'id': identity, 'partuuid': part.get('partuuid'),
|
||||
'uuid': part.get('uuid'), 'device': part['path'], 'filesystem': filesystem,
|
||||
'serial': disk['serial'], 'model': (disk.get('model') or 'USB 硬盘').strip(),
|
||||
'size': part['size'], 'mountpoints': [x for x in part.get('mountpoints', []) if x]})
|
||||
return result
|
||||
|
||||
|
||||
def mounts():
|
||||
path = VAR / 'mounts.json'
|
||||
return json.loads(path.read_text()) if path.exists() else {}
|
||||
|
||||
|
||||
def jobs(uid=None):
|
||||
result = []
|
||||
for parent, legacy in ((VAR / 'jobs', False), (LEGACY / 'jobs', True)):
|
||||
if not parent.exists():
|
||||
continue
|
||||
for folder in sorted(parent.iterdir(), reverse=True):
|
||||
try:
|
||||
config = json.loads((folder / 'config.json').read_text())
|
||||
state = json.loads((folder / 'status.json').read_text())
|
||||
owner = config.get('uid', 1000)
|
||||
if uid is not None and uid != owner:
|
||||
continue
|
||||
log_path = folder / 'copy.log'
|
||||
data = log_path.read_bytes()[-16000:].decode(errors='replace') if log_path.exists() else ''
|
||||
progress = list(re.finditer(r'([\d,]+)\s+(\d+)%\s+([\d.]+)([kMGT]?)B/s\s+', data))
|
||||
copied = int(progress[-1][1].replace(',', '')) if progress else 0
|
||||
phase = state['phase']
|
||||
paused = (folder / 'paused').exists()
|
||||
if paused:
|
||||
phase = 'paused'
|
||||
alive = Path('/proc/' + str(state.get('pid', -1))).exists()
|
||||
if not alive and phase not in TERMINAL and time.time() - state.get('started', 0) > 10:
|
||||
phase = 'interrupted'
|
||||
if phase == 'complete' and 'source_write_counters_after' not in state:
|
||||
phase = 'finalizing'
|
||||
speed = 0
|
||||
if phase == 'copying':
|
||||
now = time.time()
|
||||
samples = HISTORY.setdefault(str(folder), [])
|
||||
samples.append((now, copied))
|
||||
samples[:] = [sample for sample in samples if sample[0] >= now - 90]
|
||||
elapsed = now - samples[0][0]
|
||||
speed = (copied - samples[0][1]) / elapsed if elapsed >= 15 else copied / max(1, now - state.get('copy_started', now))
|
||||
total = config.get('planned_copy_bytes', 0)
|
||||
if phase == 'complete':
|
||||
copied = total
|
||||
result.append({'id': ('legacy:' if legacy else '') + folder.name,
|
||||
'directories': config['directories'], 'destination': config['destination'],
|
||||
'phase': phase, 'bytes': copied, 'total': total, 'speed': speed,
|
||||
'eta': max(0, total - copied) / speed if speed else None,
|
||||
'started': state.get('started'), 'finished': state.get('finished'),
|
||||
'error': state.get('error') or state.get('reason') or state.get('cleanup_error'),
|
||||
'device_id': config.get('partuuid') or config.get('fs_uuid'),
|
||||
'legacy': legacy, 'uid': owner, '_folder': str(folder), '_config': config,
|
||||
'source_writes_unchanged': state.get('source_writes_unchanged')})
|
||||
except (OSError, ValueError, KeyError):
|
||||
continue
|
||||
return sorted(result, key=lambda item: item.get('started') or 0, reverse=True)
|
||||
|
||||
|
||||
def active():
|
||||
return [job for job in jobs() if job['phase'] not in TERMINAL and job['phase'] != 'interrupted']
|
||||
|
||||
|
||||
def mount_disk(identity):
|
||||
choices = [item for item in disks() if item['id'] == identity]
|
||||
if len(choices) != 1:
|
||||
raise ValueError('硬盘已移除或身份发生变化,请刷新')
|
||||
disk = choices[0]
|
||||
# Adopt the current authorized copy without remounting or moving its data.
|
||||
for job in active():
|
||||
config = job['_config']
|
||||
if job['device_id'] == identity and os.path.ismount(config['mount']):
|
||||
return {**disk, 'mount': config['mount'], 'base': str(Path(config['mount']) / config['source_root']),
|
||||
'managed': False, 'busy': True}
|
||||
known = mounts()
|
||||
if identity in known and os.path.ismount(known[identity]['mount']):
|
||||
return {**disk, **known[identity]}
|
||||
if disk['filesystem'].startswith('ntfs') and disk['mountpoints']:
|
||||
mount = Path(disk['mountpoints'][0])
|
||||
return {**disk, 'mount': str(mount), 'base': str(mount), 'managed': False, 'busy': False}
|
||||
if disk['mountpoints']:
|
||||
raise ValueError('该硬盘已被其他程序挂载,请先停止使用')
|
||||
device = Path(disk['device'])
|
||||
target = VAR / 'mounts' / re.sub(r'[^a-zA-Z0-9_-]', '_', identity)
|
||||
target.mkdir(parents=True, exist_ok=True)
|
||||
if any(target.iterdir()):
|
||||
raise ValueError('挂载目录不为空,已停止以保护数据')
|
||||
original = command(['blockdev', '--getro', str(device)])
|
||||
command(['blockdev', '--setro', str(device)])
|
||||
try:
|
||||
if disk['filesystem'] == 'apfs':
|
||||
command([str(ROOT / 'bin/apfs-fuse'), '-o', 'ro,nodev,nosuid,noexec', str(device), str(target)])
|
||||
else:
|
||||
command(['ntfs-3g', '-o', 'ro,nodev,nosuid,noexec', str(device), str(target)])
|
||||
if not os.path.ismount(target):
|
||||
raise ValueError('无法只读挂载;加密卷或此文件系统特性可能暂不支持')
|
||||
except BaseException:
|
||||
if os.path.ismount(target):
|
||||
command(['umount', str(target)])
|
||||
if original == '0':
|
||||
command(['blockdev', '--setrw', str(device)])
|
||||
raise
|
||||
base = target / 'root' if (target / 'root').is_dir() and (target / 'private-dir').is_dir() else target
|
||||
known[identity] = {'mount': str(target), 'base': str(base), 'managed': True,
|
||||
'device': str(device), 'original_ro': original, 'serial': disk['serial']}
|
||||
atomic(VAR / 'mounts.json', known)
|
||||
return {**disk, **known[identity]}
|
||||
|
||||
|
||||
def destination_roots(uid):
|
||||
return [path for path in Path('/').glob('vol[0-9]*')
|
||||
if (path / str(uid)).is_dir() and os.path.ismount(path)]
|
||||
|
||||
|
||||
def target_path(path, uid, must_exist=True):
|
||||
resolved = Path(path).resolve(strict=must_exist)
|
||||
if not any(resolved.is_relative_to(root / str(uid)) for root in destination_roots(uid)):
|
||||
raise ValueError('请选择当前用户存储空间中的目标目录')
|
||||
return resolved
|
||||
|
||||
|
||||
def listing(root, relative=''):
|
||||
root = root.resolve(strict=True)
|
||||
path = beneath(root / relative, root)
|
||||
entries = []
|
||||
for entry in path.iterdir():
|
||||
if entry.name.startswith('.') or entry.is_symlink() or not entry.is_dir():
|
||||
continue
|
||||
entries.append({'name': entry.name, 'path': str(entry.relative_to(root))})
|
||||
return {'path': '' if path == root else str(path.relative_to(root)), 'entries': sorted(entries, key=lambda item: item['name'].casefold())}
|
||||
|
||||
|
||||
def dispatch(request):
|
||||
uid = int(request['uid'])
|
||||
if uid < 1000:
|
||||
raise ValueError('请使用飞牛管理员账号')
|
||||
action = request['action']
|
||||
args = request.get('args', {})
|
||||
if action == 'status':
|
||||
return {'disks': disks(), 'jobs': [{k: v for k, v in job.items() if not k.startswith('_')}
|
||||
for job in jobs(uid)], 'version': os.environ.get('TRIM_APPVER', '0.1.0')}
|
||||
if action == 'source':
|
||||
disk = mount_disk(args['device'])
|
||||
return {**listing(Path(disk['base']), args.get('path', '')), 'device': disk['id']}
|
||||
if action == 'destinations':
|
||||
if not args.get('path'):
|
||||
return {'path': '', 'entries': [{'name': root.name + ' · 我的文件', 'path': str(root / str(uid))}
|
||||
for root in destination_roots(uid)]}
|
||||
path = target_path(args['path'], uid)
|
||||
return {'path': str(path), 'entries': [{'name': item.name, 'path': str(item)}
|
||||
for item in sorted(path.iterdir()) if item.is_dir() and not item.is_symlink() and not item.name.startswith('.')]}
|
||||
if action == 'mkdir':
|
||||
parent = target_path(args['path'], uid)
|
||||
name = args['name'].strip()
|
||||
if not name or name in ('.', '..') or '/' in name or '\0' in name or name.startswith('.'):
|
||||
raise ValueError('文件夹名称无效')
|
||||
new = parent / name
|
||||
new.mkdir()
|
||||
os.chown(new, uid, pwd.getpwuid(uid).pw_gid)
|
||||
return {'path': str(new)}
|
||||
if action == 'start':
|
||||
if active():
|
||||
raise ValueError('已有拷贝任务运行中,请等待完成')
|
||||
disk = mount_disk(args['device'])
|
||||
base = Path(disk['base'])
|
||||
paths = args.get('paths', [])
|
||||
if not paths or len(paths) > 100:
|
||||
raise ValueError('请选择 1–100 个目录')
|
||||
folders = [beneath(base / name, base) for name in paths]
|
||||
if any(not folder.is_dir() or folder == base for folder in folders):
|
||||
raise ValueError('请选择硬盘中的具体目录')
|
||||
if len({folder.name for folder in folders}) != len(folders):
|
||||
raise ValueError('所选目录存在同名项,请分开拷贝')
|
||||
parents = {folder.parent for folder in folders}
|
||||
if len(parents) != 1:
|
||||
raise ValueError('请选择同一层级下的目录')
|
||||
destination = target_path(args['destination'], uid)
|
||||
command(['runuser', '-u', pwd.getpwuid(uid).pw_name, '--', 'test', '-w', str(destination)])
|
||||
preview = command(['rsync', '-rltHn', '--ignore-existing', '--stats', '--protect-args',
|
||||
*map(str, folders), str(destination) + '/'], timeout=120)
|
||||
match = re.search(r'Total transferred file size: ([\d,]+)', preview)
|
||||
if not match:
|
||||
raise ValueError('无法计算拷贝大小,请稍后重试')
|
||||
total = int(match[1].replace(',', ''))
|
||||
available = os.statvfs(destination)
|
||||
if total + 1024**3 > available.f_bavail * available.f_frsize:
|
||||
raise ValueError('目标空间不足')
|
||||
job = VAR / 'jobs' / (time.strftime('%Y%m%d-%H%M%S') + '-' + uuid.uuid4().hex[:6])
|
||||
job.mkdir(parents=True)
|
||||
config = {'mount': disk['mount'], 'source_root': str(next(iter(parents)).relative_to(disk['mount'])),
|
||||
'destination': str(destination), 'partuuid': disk.get('partuuid'), 'fs_uuid': disk.get('uuid'),
|
||||
'device_path': disk['device'], 'serial': disk['serial'], 'uid': uid, 'gid': pwd.getpwuid(uid).pw_gid,
|
||||
'directories': [folder.name for folder in folders], 'planned_copy_bytes': total,
|
||||
'original_ro': disk.get('original_ro', '0'), 'managed_mount': disk['managed'],
|
||||
'filesystem': disk['filesystem'], 'selection_paths': paths}
|
||||
atomic(job / 'config.json', config)
|
||||
atomic(job / 'status.json', {'phase': 'preflight', 'started': time.time()})
|
||||
with (job / 'worker.log').open('ab') as log:
|
||||
subprocess.Popen(['/usr/bin/python3', str(ROOT / 'copy_worker.py'), str(job / 'config.json')],
|
||||
stdin=subprocess.DEVNULL, stdout=log, stderr=subprocess.STDOUT, start_new_session=True)
|
||||
return {'job': job.name}
|
||||
if action == 'resume':
|
||||
selected = next((item for item in jobs(uid) if item['id'] == args['job']), None)
|
||||
if not selected or selected['legacy'] or selected['phase'] not in ('failed', 'interrupted'):
|
||||
raise ValueError('此任务当前不能重试')
|
||||
config = selected['_config']
|
||||
return dispatch({'uid': uid, 'action': 'start', 'args': {
|
||||
'device': selected['device_id'], 'paths': config['selection_paths'], 'destination': config['destination']}})
|
||||
if action == 'eject':
|
||||
if any(job['device_id'] == args['device'] for job in active()):
|
||||
raise ValueError('硬盘仍有拷贝任务,完成后才能释放')
|
||||
known = mounts()
|
||||
item = known.get(args['device'])
|
||||
if not item or not item['managed']:
|
||||
raise ValueError('此盘由飞牛系统挂载,请在系统“外接存储”中推出')
|
||||
if os.path.ismount(item['mount']):
|
||||
command(['umount', item['mount']])
|
||||
if item['original_ro'] == '0':
|
||||
command(['blockdev', '--setrw', item['device']])
|
||||
del known[args['device']]
|
||||
atomic(VAR / 'mounts.json', known)
|
||||
return {'released': True}
|
||||
raise ValueError('未知操作')
|
||||
|
||||
|
||||
class Handler(socketserver.StreamRequestHandler):
|
||||
def handle(self):
|
||||
try:
|
||||
_, uid, _ = struct.unpack('3i', self.request.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, 12))
|
||||
if uid not in (0, APP_UID):
|
||||
raise ValueError('拒绝未授权本机客户端')
|
||||
raw = self.rfile.readline(262145)
|
||||
if len(raw) > 262144:
|
||||
raise ValueError('请求过大')
|
||||
request = json.loads(raw)
|
||||
result = {'ok': True, 'data': dispatch(request)}
|
||||
except Exception as error:
|
||||
result = {'ok': False, 'error': str(error)}
|
||||
self.wfile.write((json.dumps(result, ensure_ascii=False) + '\n').encode())
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
VAR.mkdir(exist_ok=True)
|
||||
path = VAR / 'broker.sock'
|
||||
path.unlink(missing_ok=True)
|
||||
server = socketserver.UnixStreamServer(str(path), Handler)
|
||||
os.chown(path, 0, APP_GID)
|
||||
path.chmod(0o660)
|
||||
signal.signal(signal.SIGTERM, lambda *_: (_ for _ in ()).throw(KeyboardInterrupt()))
|
||||
try:
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
finally:
|
||||
server.server_close()
|
||||
path.unlink(missing_ok=True)
|
||||
in_use = {job['device_id'] for job in active()}
|
||||
for identity, item in mounts().items():
|
||||
if identity in in_use or not item.get('managed'):
|
||||
continue
|
||||
try:
|
||||
if os.path.ismount(item['mount']):
|
||||
command(['umount', item['mount']])
|
||||
properties = command(['udevadm', 'info', '-q', 'property', '-n', item['device']])
|
||||
if ('ID_SERIAL_SHORT=' + item['serial']) in properties.splitlines() and item['original_ro'] == '0':
|
||||
command(['blockdev', '--setrw', item['device']])
|
||||
except Exception:
|
||||
pass
|
||||
@@ -0,0 +1,92 @@
|
||||
"""Unprivileged HTTP UI, reachable only through the authenticated fnOS gateway."""
|
||||
import http.server
|
||||
import grp
|
||||
import hmac
|
||||
import secrets
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import socket
|
||||
import socketserver
|
||||
import urllib.parse
|
||||
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
PREFIX = '/app/dayu-diskcopy'
|
||||
VAR = Path(os.environ['TRIM_PKGVAR'])
|
||||
CSRF_TOKEN = secrets.token_urlsafe(32)
|
||||
|
||||
|
||||
class Handler(http.server.BaseHTTPRequestHandler):
|
||||
def log_message(self, *args):
|
||||
pass
|
||||
|
||||
def reply(self, status, data, mime='application/json; charset=utf-8'):
|
||||
payload = data if isinstance(data, bytes) else json.dumps(data, ensure_ascii=False).encode()
|
||||
self.send_response(status)
|
||||
self.send_header('Content-Type', mime)
|
||||
self.send_header('Content-Length', str(len(payload)))
|
||||
self.send_header('Cache-Control', 'no-store')
|
||||
self.send_header('X-Content-Type-Options', 'nosniff')
|
||||
self.end_headers()
|
||||
self.wfile.write(payload)
|
||||
|
||||
def handle_request(self, post=False):
|
||||
try:
|
||||
uid = self.headers.get('X-Trim-Userid', '')
|
||||
if self.headers.get('X-Trim-Isadmin', '').lower() != 'true' or not uid.isdigit() or int(uid) < 1000:
|
||||
return self.reply(403, {'error': '请从飞牛桌面使用管理员账号打开此应用'})
|
||||
path = urllib.parse.urlsplit(self.path).path
|
||||
if path in (PREFIX, PREFIX + '/') and not post:
|
||||
page = (ROOT / 'index.html').read_bytes().replace(b'__DAYU_CSRF__', CSRF_TOKEN.encode())
|
||||
return self.reply(200, page, 'text/html; charset=utf-8')
|
||||
if not path.startswith(PREFIX + '/api/'):
|
||||
return self.reply(404, {'error': '页面不存在'})
|
||||
action = path.removeprefix(PREFIX + '/api/')
|
||||
writes = {'start', 'mkdir', 'eject', 'resume'}
|
||||
if post != (action in writes):
|
||||
return self.reply(405, {'error': '请求方法无效'})
|
||||
if post:
|
||||
if self.headers.get('X-Dayu-Request') != '1':
|
||||
return self.reply(403, {'error': '无效请求来源'})
|
||||
# fnOS rewrites upstream Host; validate the private page nonce instead.
|
||||
if not hmac.compare_digest(self.headers.get('X-Dayu-Token', ''), CSRF_TOKEN):
|
||||
return self.reply(403, {'error': '页面验证已失效,请刷新应用后重试'})
|
||||
if self.headers.get('Sec-Fetch-Site', '') not in ('', 'none', 'same-origin'):
|
||||
return self.reply(403, {'error': '请求来源不匹配'})
|
||||
length = int(self.headers.get('Content-Length', '0'))
|
||||
if not 0 <= length <= 262144:
|
||||
return self.reply(413, {'error': '请求过大'})
|
||||
args = json.loads(self.rfile.read(length) or '{}')
|
||||
else:
|
||||
args = {key: value[0] for key, value in urllib.parse.parse_qs(urllib.parse.urlsplit(self.path).query).items()}
|
||||
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as client:
|
||||
client.settimeout(180)
|
||||
client.connect(str(VAR / 'broker.sock'))
|
||||
client.sendall((json.dumps({'uid': int(uid), 'action': action, 'args': args}) + '\n').encode())
|
||||
with client.makefile('rb') as response:
|
||||
data = json.loads(response.readline(4 * 1024 * 1024))
|
||||
self.reply(200 if data['ok'] else 400, data.get('data') if data['ok'] else {'error': data['error']})
|
||||
except Exception:
|
||||
self.reply(500, {'error': '服务暂时不可用,请稍后刷新或在应用中心重启应用'})
|
||||
|
||||
def do_GET(self):
|
||||
self.handle_request()
|
||||
|
||||
def do_POST(self):
|
||||
self.handle_request(True)
|
||||
|
||||
|
||||
class Server(socketserver.ThreadingMixIn, socketserver.UnixStreamServer):
|
||||
daemon_threads = True
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
path = ROOT / 'app.sock'
|
||||
path.unlink(missing_ok=True)
|
||||
server = Server(str(path), Handler)
|
||||
os.chown(path, int(os.environ['TRIM_UID']), grp.getgrnam('Administrators').gr_gid)
|
||||
path.chmod(0o660)
|
||||
os.setgroups([])
|
||||
os.setgid(int(os.environ['TRIM_GID']))
|
||||
os.setuid(int(os.environ['TRIM_UID']))
|
||||
server.serve_forever()
|
||||
@@ -0,0 +1,33 @@
|
||||
<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
|
||||
<title>大鱼硬盘拷贝</title>
|
||||
<style>
|
||||
:root{font:14px -apple-system,BlinkMacSystemFont,"PingFang SC",sans-serif;color:#172c39;background:#f2f5f6;--accent:#167666;--line:#e0e7e8}*{box-sizing:border-box}body{margin:0}button,select,input{font:inherit}button{cursor:pointer;border:1px solid var(--line);border-radius:9px;padding:9px 15px;background:white;color:inherit}button:hover{background:#edf5f2;border-color:#bad4ce}button:disabled{opacity:.45;cursor:default}button.primary{background:var(--accent);color:white;border-color:var(--accent);font-weight:600}button:focus-visible,input:focus-visible,select:focus-visible{outline:3px solid #4eaea1;outline-offset:2px}header{padding:29px 34px 22px;display:flex;align-items:center;gap:14px}header svg{width:44px;height:44px;flex:none}h1{font-size:22px;margin:0 0 5px;letter-spacing:.5px}header p{margin:0;color:#6c7d85;font-size:13px}.badge{margin-left:auto;font-size:12px;padding:7px 11px;border-radius:20px;background:#e3efeb;color:var(--accent);white-space:nowrap}main{max-width:1180px;margin:auto;padding:0 30px 35px}.steps{display:flex;gap:20px;margin-bottom:18px;color:#7b8a91;font-size:12px}.steps span{display:flex;align-items:center;gap:7px}.steps b{display:inline-grid;place-items:center;width:21px;height:21px;border:1px solid #c8d5d6;border-radius:50%;font-weight:500}.steps span:first-child{color:var(--accent)}.columns{display:grid;grid-template-columns:1fr 1fr;gap:18px}.card{background:#fff;border:1px solid var(--line);border-radius:14px;overflow:hidden}.cardhead{padding:20px 21px 15px;border-bottom:1px solid #edf0f1}.cardhead h2{font-size:15px;margin:0 0 6px}.muted{color:#7c8b92;font-size:12px}.cardbody{padding:16px 20px}.disk{display:flex;gap:10px;align-items:center;padding:12px;border:1px solid var(--line);border-radius:10px;margin-bottom:14px}.disk select{width:100%;border:0;background:transparent;color:#213e4b;padding:4px 0;outline:none}.disk .fs{font-size:11px;background:#edf4f2;color:var(--accent);padding:5px 7px;border-radius:5px;min-width:43px;text-align:center}.tools{display:flex;align-items:center;gap:7px;margin-bottom:10px}.tools button{font-size:12px;padding:6px 10px}.path{font-size:12px;color:#6a8088;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;flex:1;direction:rtl;text-align:left}.folders{height:280px;overflow:auto;border:1px solid #e9eeee;border-radius:9px}.row{display:flex;align-items:center;gap:8px;padding:10px 12px;border-bottom:1px solid #f0f3f3;min-height:43px}.row:last-child{border:0}.row:hover{background:#f5f9f8}.row input{accent-color:var(--accent);width:16px;height:16px}.foldericon{width:20px;color:#9cbaaf;flex:none}.row button{border:0;padding:0;background:none;text-align:left;flex:1;overflow-wrap:anywhere;line-height:1.5}.row .arrow{color:#9cabb0}.empty{padding:45px 16px;text-align:center;color:#8a999e;font-size:13px;line-height:1.8}.destination{padding:12px;margin-top:12px;background:#f3f7f6;border-radius:8px;font-size:12px;line-height:1.7;overflow-wrap:anywhere}.actions{display:flex;align-items:center;gap:12px;padding:18px 0 22px}.actions .note{font-size:12px;color:#7c8d93;flex:1}.actions button.primary{padding:11px 24px}.tasks h2{font-size:15px;margin:0}.taskhead{padding:18px 21px;display:flex;justify-content:space-between;align-items:center}.task{padding:0 21px 20px}.task+.task{border-top:1px solid var(--line);padding-top:18px}.tasktitle{display:flex;align-items:center;gap:10px;justify-content:space-between;margin-bottom:8px;font-weight:600}.state{font-size:12px;font-weight:400;color:var(--accent);white-space:nowrap}.track{height:7px;border-radius:6px;background:#eef2f2;margin:13px 0 10px;overflow:hidden}.bar{height:100%;border-radius:6px;background:#258c77;transition:width .5s}.metrics{display:flex;gap:22px;font-size:12px;color:#657f89}.taskpath{font-size:12px;color:#94a1a5;overflow-wrap:anywhere;margin-top:10px}.err{background:#fff0eb;color:#a74c30;padding:12px 15px;border-radius:9px;line-height:1.6;margin-bottom:15px;display:none}footer{color:#89999f;font-size:12px;padding-top:18px;display:flex;justify-content:space-between;gap:10px}dialog{border:0;border-radius:14px;padding:25px;box-shadow:0 20px 70px #172c3930;max-width:460px;width:90%}dialog::backdrop{background:#172c3950}dialog input{width:100%;padding:12px;border:1px solid var(--line);border-radius:8px;margin:8px 0 20px}.dialogbuttons{display:flex;justify-content:flex-end;gap:10px}@media(max-width:720px){header{padding:20px}main{padding:0 16px 25px}.columns{grid-template-columns:1fr}.badge{display:none}.folders{height:200px}.steps{gap:10px;font-size:11px}.actions{flex-wrap:wrap}.actions .note{flex-basis:100%}.metrics{gap:12px;flex-wrap:wrap}footer{display:block;line-height:1.8}}
|
||||
</style>
|
||||
<header><svg viewBox="0 0 48 48" aria-hidden="true"><rect width="48" height="48" rx="13" fill="#167666"/><rect x="10" y="11" width="28" height="25" rx="5" fill="none" stroke="white" stroke-width="2"/><path d="M10 27h28M16 31h2" stroke="white" stroke-width="2"/><path d="m21 18 4 4 7-7" fill="none" stroke="#9ee5c7" stroke-width="2.5"/></svg><div><h1>大鱼硬盘拷贝</h1><p>外接硬盘直连导入,拷贝在 NAS 本机完成</p></div><span class="badge">APFS · NTFS · 只读访问源盘</span></header>
|
||||
<main><div class="steps"><span><b>1</b>选择源目录</span><span><b>2</b>选择存储位置</span><span><b>3</b>后台拷贝与校验</span></div><div id="error" class="err" role="alert"></div>
|
||||
<div class="columns"><section class="card"><div class="cardhead"><h2>从外接硬盘拷贝</h2><div class="muted">勾选同一层级中要拷贝的目录</div></div><div class="cardbody"><div class="disk"><span id="fs" class="fs">USB</span><select id="disk" aria-label="选择外接硬盘"></select><button id="refresh" title="刷新硬盘">刷新</button></div><div class="tools"><button id="sourceUp">上一级</button><span id="sourcePath" class="path">/</span><button id="release">释放挂载</button></div><div id="sources" class="folders"><div class="empty">请选择 APFS 或 NTFS 硬盘</div></div></div></section>
|
||||
<section class="card"><div class="cardhead"><h2>拷贝到 NAS</h2><div class="muted">选择你的存储空间和目标目录</div></div><div class="cardbody"><div class="disk"><span class="fs">NAS</span><div style="flex:1">我的存储空间<div class="muted" style="margin-top:4px">文件直接写入此处</div></div><button id="newFolder">新建文件夹</button></div><div class="tools"><button id="destUp">上一级</button><span id="destPath" class="path">存储空间</span></div><div id="destinations" class="folders"></div><div class="destination">当前目标:<span id="chosenDest">请选择一个目录</span></div></div></section></div>
|
||||
<div class="actions"><span id="selection" class="note">尚未选择源目录。同名文件会先保留,并在复制后校验。</span><button id="start" class="primary" disabled>开始拷贝</button></div>
|
||||
<section class="card tasks"><div class="taskhead"><h2>拷贝任务</h2><span class="muted">关闭窗口后仍继续运行</span></div><div id="tasks"><div class="empty">暂无拷贝任务</div></div></section>
|
||||
<footer><span>不格式化、不修复源盘;保留目录、文件名和链接。</span><span id="version">大鱼硬盘拷贝</span></footer></main>
|
||||
<dialog id="newDialog"><h3>新建目标文件夹</h3><form method="dialog" id="newForm"><input id="folderName" placeholder="文件夹名称" aria-label="新文件夹名称" required><div class="dialogbuttons"><button value="cancel" formnovalidate>取消</button><button class="primary" value="create">创建</button></div></form></dialog>
|
||||
<script>
|
||||
const API='/app/dayu-diskcopy/api/',$=id=>document.getElementById(id);
|
||||
let device='',sourcePath='',destPath='',selected=new Set(),lastDisks='',busy=false,loading=false;
|
||||
const states={preflight:'准备中',copying:'正在拷贝',verifying_existing:'校验已有文件',verifying_tree:'核对目录',saving_metadata:'保存扩展属性',finalizing:'正在完成',complete:'已完成',failed:'拷贝失败',needs_attention:'发现不同版本,请保留核对',cleanup_failed:'需要检查挂载状态',interrupted:'任务已中断'};
|
||||
function bytes(n){if(!n)return'0 B';const u=['B','KB','MB','GB','TB'],i=Math.min(4,Math.floor(Math.log(n)/Math.log(1000)));return(n/1000**i).toFixed(i>1?1:0)+' '+u[i]}
|
||||
function duration(s){if(s==null)return'估算中';if(s<60)return'不足 1 分钟';if(s<3600)return Math.ceil(s/60)+' 分钟';return Math.floor(s/3600)+' 小时 '+Math.ceil(s%3600/60)+' 分钟'}
|
||||
function error(message){$('error').textContent=message||'';$('error').style.display=message?'block':'none'}
|
||||
async function api(action,args={},post=false){const url=API+action+(post?'':'?'+new URLSearchParams(args));const r=await fetch(url,{method:post?'POST':'GET',headers:post?{'Content-Type':'application/json','X-Dayu-Request':'1','X-Dayu-Token':'__DAYU_CSRF__'}:{},body:post?JSON.stringify(args):undefined});const d=await r.json();if(!r.ok)throw Error(d.error||'操作失败');return d}
|
||||
function button(name,fn){const b=document.createElement('button');b.textContent=name;b.onclick=fn;return b}
|
||||
function folderRows(container,entries,source){container.replaceChildren();if(!entries.length){const e=document.createElement('div');e.className='empty';e.textContent=source?'当前目录下没有可选择的文件夹':'当前目录下没有子文件夹,可以直接选作目标';container.append(e);return}for(const entry of entries){const row=document.createElement('div');row.className='row';if(source){const check=document.createElement('input');check.type='checkbox';check.setAttribute('aria-label','选择 '+entry.name);check.checked=selected.has(entry.path);check.onchange=()=>{check.checked?selected.add(entry.path):selected.delete(entry.path);updateSelection()};row.append(check)}const icon=document.createElement('span');icon.className='foldericon';icon.textContent='▰';row.append(icon,button(entry.name,()=>source?browseSource(entry.path):browseDest(entry.path)));const arrow=document.createElement('span');arrow.className='arrow';arrow.textContent='›';row.append(arrow);container.append(row)}}
|
||||
function updateSelection(){$('selection').textContent=selected.size?'已选 '+selected.size+' 个目录。同名文件会先保留,并在复制后校验。':'尚未选择源目录。同名文件会先保留,并在复制后校验。';$('start').disabled=!selected.size||!destPath||busy||loading}
|
||||
async function browseSource(path=''){if(!device)return;try{loading=true;updateSelection();const d=await api('source',{device,path});sourcePath=d.path;selected.clear();$('sourcePath').textContent='/'+sourcePath;$('sourceUp').disabled=!sourcePath;folderRows($('sources'),d.entries,true);error()}catch(e){error(e.message)}finally{loading=false;updateSelection()}}
|
||||
async function browseDest(path=''){try{const d=await api('destinations',{path});destPath=d.path;$('destPath').textContent=destPath||'存储空间';$('chosenDest').textContent=destPath||'请选择一个目录';folderRows($('destinations'),d.entries,false);updateSelection();error()}catch(e){error(e.message)}}
|
||||
function renderJobs(jobs){busy=jobs.some(j=>!['complete','failed','needs_attention','cleanup_failed','interrupted','cancelled'].includes(j.phase));$('tasks').replaceChildren();if(!jobs.length){const e=document.createElement('div');e.className='empty';e.textContent='暂无拷贝任务';$('tasks').append(e)}for(const job of jobs.slice(0,10)){const box=document.createElement('div');box.className='task';const title=document.createElement('div');title.className='tasktitle';const name=document.createElement('span');name.textContent=job.directories.join('、');const state=document.createElement('span');state.className='state';state.textContent=states[job.phase]||job.phase;title.append(name,state);const track=document.createElement('div');track.className='track';const bar=document.createElement('div');bar.className='bar';bar.style.width=(job.phase==='complete'?100:Math.min(100,job.total?job.bytes/job.total*100:0))+'%';track.append(bar);const metrics=document.createElement('div');metrics.className='metrics';for(const value of [bytes(job.bytes)+' / '+bytes(job.total),job.phase==='copying'?bytes(job.speed)+'/s':'',job.phase==='copying'?'预计剩余 '+duration(job.eta):job.phase==='complete'?'文件与目录核验通过':'']){if(value){const e=document.createElement('span');e.textContent=value;metrics.append(e)}}const path=document.createElement('div');path.className='taskpath';path.textContent=job.destination;box.append(title,track,metrics,path);if(job.error){const e=document.createElement('div');e.className='taskpath';e.style.color='#a74c30';e.textContent=job.error;box.append(e)}if(!job.legacy&&['failed','interrupted'].includes(job.phase))box.append(button('继续此任务',async()=>{try{await api('resume',{job:job.id},true);await refresh()}catch(e){error(e.message)}}));$('tasks').append(box)}updateSelection()}
|
||||
async function refresh(){try{const d=await api('status');$('version').textContent='大鱼硬盘拷贝 '+d.version;const signature=JSON.stringify(d.disks.map(x=>[x.id,x.filesystem]));if(signature!==lastDisks){lastDisks=signature;$('disk').replaceChildren();for(const disk of d.disks){const option=document.createElement('option');option.value=disk.id;option.textContent=disk.model+' · '+bytes(disk.size)+' · '+disk.filesystem.toUpperCase();$('disk').append(option)}if(!d.disks.length){const o=document.createElement('option');o.textContent='未发现 APFS 或 NTFS 外接硬盘';$('disk').append(o);device=''}else if(!d.disks.some(x=>x.id===device)){device=d.disks[0].id;$('disk').value=device;await browseSource()}$('fs').textContent=(d.disks.find(x=>x.id===device)?.filesystem||'USB').toUpperCase()}renderJobs(d.jobs)}catch(e){error(e.message)}}
|
||||
$('disk').onchange=()=>{device=$('disk').value;$('fs').textContent=$('disk').selectedOptions[0]?.textContent.split(' · ').at(-1)||'USB';browseSource()};$('refresh').onclick=refresh;$('sourceUp').onclick=()=>browseSource(sourcePath.split('/').slice(0,-1).join('/'));$('destUp').onclick=()=>{const up=destPath.split('/').slice(0,-1).join('/');browseDest(/^\/vol\d+\/\d+/.test(up)?up:'')};$('start').onclick=async()=>{try{loading=true;updateSelection();await api('start',{device,paths:[...selected],destination:destPath},true);selected.clear();for(const c of document.querySelectorAll('#sources input'))c.checked=false;error();await refresh()}catch(e){error(e.message)}finally{loading=false;updateSelection()}};$('release').onclick=async()=>{try{await api('eject',{device},true);error('已释放应用挂载;拔盘前仍请在飞牛系统中推出整块硬盘');await refresh()}catch(e){error(e.message)}};$('newFolder').onclick=()=>{if(!destPath)return error('请先选择一个存储空间或目录');$('folderName').value='';$('newDialog').showModal()};$('newForm').onsubmit=async e=>{if(e.submitter?.value!=='create')return;e.preventDefault();try{const d=await api('mkdir',{path:destPath,name:$('folderName').value},true);$('newDialog').close();await browseDest(d.path)}catch(err){error(err.message)}};
|
||||
browseDest();refresh();setInterval(refresh,2500);
|
||||
</script></html>
|
||||
@@ -0,0 +1,58 @@
|
||||
"""fnOS lifecycle controller. Stops only the two service processes, not copy jobs."""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import signal
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
ROOT = Path(__file__).resolve().parent
|
||||
VAR = Path(os.environ['TRIM_PKGVAR'])
|
||||
STATE = VAR / 'services.json'
|
||||
|
||||
|
||||
def alive(pid, script):
|
||||
try:
|
||||
return str(ROOT / script).encode() in (Path('/proc') / str(pid) / 'cmdline').read_bytes()
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def main():
|
||||
action = sys.argv[1]
|
||||
VAR.mkdir(parents=True, exist_ok=True)
|
||||
os.chown(VAR, 0, int(os.environ['TRIM_GID']))
|
||||
VAR.chmod(0o750)
|
||||
records = json.loads(STATE.read_text()) if STATE.exists() else {}
|
||||
if action == 'status':
|
||||
raise SystemExit(0 if all(alive(records.get(name, -1), name) for name in ('broker.py', 'front.py')) else 3)
|
||||
if action == 'stop':
|
||||
for script, pid in records.items():
|
||||
if alive(pid, script):
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
for _ in range(30):
|
||||
if not any(alive(pid, script) for script, pid in records.items()):
|
||||
break
|
||||
time.sleep(.1)
|
||||
STATE.unlink(missing_ok=True)
|
||||
return
|
||||
if action != 'start':
|
||||
raise SystemExit(1)
|
||||
for script in ('broker.py', 'front.py'):
|
||||
if alive(records.get(script, -1), script):
|
||||
continue
|
||||
with (VAR / 'service.log').open('ab') as log:
|
||||
process = subprocess.Popen(['/usr/bin/python3', str(ROOT / script)],
|
||||
stdin=subprocess.DEVNULL, stdout=log, stderr=subprocess.STDOUT, start_new_session=True)
|
||||
records[script] = process.pid
|
||||
STATE.write_text(json.dumps(records))
|
||||
for _ in range(50):
|
||||
if (ROOT / 'app.sock').exists() and (VAR / 'broker.sock').exists():
|
||||
return
|
||||
time.sleep(.1)
|
||||
raise SystemExit('服务启动失败,请查看应用日志')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1 @@
|
||||
{".url":{"dayu-diskcopy.main":{"title":"大鱼硬盘拷贝","icon":"images/icon_{0}.png","type":"iframe","protocol":"","gatewayPrefix":"/app/dayu-diskcopy","gatewaySocket":"app.sock","url":"/app/dayu-diskcopy","allUsers":false,"control":{"accessPerm":"readonly"}}}}
|
||||
@@ -0,0 +1,3 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
exit 0
|
||||
@@ -0,0 +1,3 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
exit 0
|
||||
@@ -0,0 +1,3 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
exit 0
|
||||
@@ -0,0 +1,9 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
for program in python3 rsync ntfs-3g; do
|
||||
if ! command -v "$program" >/dev/null; then
|
||||
printf '系统缺少运行组件:%s\n' "$program" > "$TRIM_TEMP_LOGFILE"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
exit 0
|
||||
@@ -0,0 +1,3 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
exec /usr/bin/python3 "$TRIM_APPDEST/service.py" "$1"
|
||||
@@ -0,0 +1,3 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
exit 0
|
||||
@@ -0,0 +1,10 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
if test -f "$TRIM_APPDEST/service.py"; then
|
||||
if ! /usr/bin/python3 -c 'import sys; sys.path.insert(0, sys.argv[1]); import broker; sys.exit(1 if broker.active() else 0)' "$TRIM_APPDEST"; then
|
||||
printf '仍有拷贝任务正在运行,请等待完成后再卸载。\n' > "$TRIM_TEMP_LOGFILE"
|
||||
exit 1
|
||||
fi
|
||||
/usr/bin/python3 "$TRIM_APPDEST/service.py" stop
|
||||
fi
|
||||
exit 0
|
||||
@@ -0,0 +1,3 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
exit 0
|
||||
@@ -0,0 +1,3 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
exit 0
|
||||
@@ -0,0 +1 @@
|
||||
{"defaults":{"run-as":"root"},"username":"dayu_diskcopy","groupname":"dayu_diskcopy"}
|
||||
@@ -0,0 +1 @@
|
||||
{}
|
||||
@@ -0,0 +1,16 @@
|
||||
appname=dayu-diskcopy
|
||||
version=0.1.4
|
||||
display_name=大鱼硬盘拷贝
|
||||
desc=直接读取 APFS、NTFS 外接硬盘,在飞牛 NAS 本机后台拷贝目录、查看进度和校验结果。
|
||||
source=thirdparty
|
||||
platform=x86
|
||||
maintainer=大鱼
|
||||
maintainer_url=https://dayu.media
|
||||
distributor=大鱼
|
||||
os_min_version=1.2.0701
|
||||
desktop_uidir=ui
|
||||
desktop_applaunchname=dayu-diskcopy.main
|
||||
ctl_stop=true
|
||||
checkport=false
|
||||
disable_authorization_path=false
|
||||
changelog=首版:APFS 与 NTFS 只读访问、目录选择、后台复制、进度与校验。
|
||||
@@ -0,0 +1,80 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build a self-contained native fnOS FPK with the validated Linux driver."""
|
||||
from pathlib import Path
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import struct
|
||||
import subprocess
|
||||
import sys
|
||||
import zlib
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
stage = ROOT / 'build/fpk'
|
||||
if stage.exists():
|
||||
shutil.rmtree(stage)
|
||||
shutil.copytree(ROOT / 'packaging/fpk', stage)
|
||||
(stage / 'wizard').mkdir(exist_ok=True)
|
||||
(stage / 'wizard/.gitkeep').touch()
|
||||
app = stage / 'app'
|
||||
for path in (ROOT / 'application').iterdir():
|
||||
if path.is_file():
|
||||
shutil.copy2(path, app / path.name)
|
||||
shutil.copy2(ROOT / 'scripts/copy-directories.py', app / 'copy_worker.py')
|
||||
shutil.copy2(ROOT / 'scripts/save-xattrs.py', app / 'save_xattrs.py')
|
||||
(app / 'bin').mkdir()
|
||||
for name, expected in [('apfs-fuse','89a7083c42fb498b021d9f7fabbc5b754eaae503649500e4926965fdcfbcb11e'),
|
||||
('apfsutil','692bbd961678c5db1a48e00cb2bc96c50e82dd77e39d87cfe11087608b4658ae')]:
|
||||
binary = ROOT / 'build/linux-bin' / name
|
||||
if hashlib.sha256(binary.read_bytes()).hexdigest() != expected:
|
||||
raise SystemExit('Driver hash mismatch: ' + name)
|
||||
shutil.copy2(binary, app / 'bin' / name)
|
||||
(app / 'bin' / name).chmod(0o755)
|
||||
shutil.copytree(ROOT / 'vendor/apfs-fuse', app / 'source/apfs-fuse')
|
||||
shutil.copy2(ROOT / 'UPSTREAM.json', app / 'source/UPSTREAM.json')
|
||||
shutil.copy2(ROOT / 'packaging/Dockerfile', app / 'source/Dockerfile')
|
||||
for path in (stage / 'cmd').iterdir():
|
||||
path.chmod(0o755)
|
||||
|
||||
|
||||
def icon(size):
|
||||
def chunk(kind, payload):
|
||||
return struct.pack('>I', len(payload)) + kind + payload + struct.pack('>I', zlib.crc32(kind + payload) & 0xffffffff)
|
||||
rows = []
|
||||
for y in range(size):
|
||||
row = bytearray()
|
||||
for x in range(size):
|
||||
px, py = x * 256 / size, y * 256 / size
|
||||
radius = 50
|
||||
dx, dy = max(abs(px - 128) - 78, 0), max(abs(py - 128) - 78, 0)
|
||||
inside = dx * dx + dy * dy <= radius * radius
|
||||
color = (22, 118, 102, 255) if inside else (0, 0, 0, 0)
|
||||
if 53 <= px <= 203 and 62 <= py <= 192 and (px < 63 or px > 193 or py < 72 or py > 182 or 144 < py < 153):
|
||||
color = (255, 255, 255, 255)
|
||||
if 78 <= px <= 90 and 164 <= py <= 172:
|
||||
color = (255, 255, 255, 255)
|
||||
if 100 <= px <= 126 and abs(py - (px - 3)) < 5 or 122 <= px <= 165 and abs(py - (249 - px)) < 5:
|
||||
color = (158, 229, 199, 255)
|
||||
row.extend(color)
|
||||
rows.append(b'\0' + row)
|
||||
return b'\x89PNG\r\n\x1a\n' + chunk(b'IHDR', struct.pack('>IIBBBBB', size, size, 8, 6, 0, 0, 0)) + chunk(b'IDAT', zlib.compress(b''.join(rows))) + chunk(b'IEND', b'')
|
||||
|
||||
|
||||
images = app / 'ui/images'
|
||||
images.mkdir(exist_ok=True)
|
||||
for size in (64, 256):
|
||||
payload = icon(size)
|
||||
(images / ('icon_' + str(size) + '.png')).write_bytes(payload)
|
||||
(stage / ('ICON.PNG' if size == 64 else 'ICON_256.PNG')).write_bytes(payload)
|
||||
subprocess.run([sys.argv[1], 'build', '--directory', str(stage)], cwd=stage, check=True)
|
||||
output = ROOT / 'dist'
|
||||
output.mkdir(exist_ok=True)
|
||||
packages = list(stage.glob('*.fpk'))
|
||||
if len(packages) != 1:
|
||||
raise SystemExit('Expected one FPK package')
|
||||
version = next(line.split('=',1)[1].strip() for line in (stage / 'manifest').read_text().splitlines() if line.startswith('version='))
|
||||
destination = output / ('dayu-diskcopy-' + version + '-x86.fpk')
|
||||
shutil.copy2(packages[0], destination)
|
||||
print(destination)
|
||||
print('sha256:', hashlib.sha256(destination.read_bytes()).hexdigest())
|
||||
@@ -0,0 +1,131 @@
|
||||
#!/usr/bin/env python3
|
||||
"""NAS-local copy job with read-only source, resumable rsync and status records.
|
||||
|
||||
Config and file-name manifests stay on the NAS; never commit user data to Git.
|
||||
Existing files are retained during copying and checksum-checked afterwards.
|
||||
"""
|
||||
import fcntl
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import stat
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import traceback
|
||||
|
||||
|
||||
def main():
|
||||
config_path = Path(sys.argv[1]).resolve(strict=True)
|
||||
job = config_path.parent
|
||||
config = json.loads(config_path.read_text())
|
||||
mount = Path(config['mount'])
|
||||
source = mount / config['source_root']
|
||||
destination = Path(config['destination']).resolve(strict=True)
|
||||
device = Path(config['device_path']).resolve(strict=True) if config.get('device_path') else (Path('/dev/disk/by-partuuid') / config['partuuid']).resolve(strict=True)
|
||||
managed = config.get('managed_mount', True)
|
||||
os.environ['LC_ALL'] = 'C'
|
||||
lock = (job.parent.parent / 'transfer.lock').open('a')
|
||||
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
state = {'pid': os.getpid(), 'started': time.time(), 'phase': 'preflight',
|
||||
'planned_copy_bytes': config['planned_copy_bytes'], 'directories': config['directories']}
|
||||
|
||||
def save(phase, **extra):
|
||||
state.update(phase=phase, updated=time.time(), **extra)
|
||||
temporary = job / 'status.tmp'
|
||||
temporary.write_text(json.dumps(state, ensure_ascii=False, indent=2) + '\n')
|
||||
temporary.replace(job / 'status.json')
|
||||
|
||||
def run_to_log(args, log_name):
|
||||
with (job / log_name).open('wb') as log:
|
||||
return subprocess.run(args, stdout=log, stderr=subprocess.STDOUT).returncode
|
||||
|
||||
save('preflight')
|
||||
try:
|
||||
properties = dict(line.split('=', 1) for line in subprocess.check_output(
|
||||
['udevadm', 'info', '-q', 'property', '-n', str(device)], text=True).splitlines() if '=' in line)
|
||||
if properties.get('ID_SERIAL_SHORT') != config['serial'] or properties.get('ID_BUS') != 'usb':
|
||||
raise RuntimeError('USB device identity changed.')
|
||||
if not os.path.ismount(mount):
|
||||
raise RuntimeError('Source is not mounted.')
|
||||
mounted = json.loads(subprocess.check_output(
|
||||
['findmnt', '-J', '-M', str(mount), '-o', 'SOURCE,OPTIONS'], text=True))['filesystems'][0]
|
||||
if Path(mounted['source']).resolve() != device or (managed and 'ro' not in mounted['options'].split(',')):
|
||||
raise RuntimeError('Unexpected source mount or source is not read-only.')
|
||||
if managed and subprocess.check_output(['blockdev', '--getro', str(device)], text=True).strip() != '1':
|
||||
raise RuntimeError('Block device is not read-only.')
|
||||
writes_before = (Path('/sys/class/block') / device.name / 'stat').read_text().split()[4:8:2]
|
||||
state['source_write_counters_before'] = writes_before
|
||||
available = os.statvfs(destination)
|
||||
if available.f_bavail * available.f_frsize < config['planned_copy_bytes'] + 1024**3:
|
||||
raise RuntimeError('Insufficient destination free space.')
|
||||
existing = []
|
||||
for name in config['directories']:
|
||||
folder = source / name
|
||||
if folder.is_symlink() or not folder.is_dir():
|
||||
raise RuntimeError('Expected source directory missing or linked.')
|
||||
for base, dirs, files in os.walk(folder, followlinks=False, onerror=lambda e: (_ for _ in ()).throw(e)):
|
||||
for filename in files:
|
||||
entry = Path(base) / filename
|
||||
relative = entry.relative_to(source)
|
||||
target = destination / relative
|
||||
if os.path.lexists(target):
|
||||
existing.append(os.fsencode(relative))
|
||||
(job / 'existing-files.list').write_bytes(b'\0'.join(existing) + (b'\0' if existing else b''))
|
||||
save('copying', existing_files_to_verify=len(existing), copy_started=time.time())
|
||||
source_args = [str(source / name) for name in config['directories']]
|
||||
command = ['rsync', '-rltH', '--chown=' + str(config.get('uid', 1000)) + ':' + str(config.get('gid', 1001)), '--ignore-existing', '--partial', '--no-whole-file',
|
||||
'--partial-dir=.dayu-apfs-partial', '--info=progress2,stats2', '--outbuf=L',
|
||||
'--stats', '--no-inc-recursive', '--protect-args', *source_args, str(destination) + '/']
|
||||
result = run_to_log(command, 'copy.log')
|
||||
if result:
|
||||
raise RuntimeError('rsync copy exited with status ' + str(result))
|
||||
save('verifying_existing', copy_finished=time.time())
|
||||
if existing:
|
||||
result = run_to_log(['rsync', '-rcnli', '--from0',
|
||||
'--files-from=' + str(job / 'existing-files.list'),
|
||||
'--out-format=%i %n%L', '--protect-args',
|
||||
str(source) + '/', str(destination) + '/'], 'existing-check.log')
|
||||
if result:
|
||||
raise RuntimeError('Existing-file checksum check failed: ' + str(result))
|
||||
changes = (job / 'existing-check.log').read_text().strip()
|
||||
if changes:
|
||||
save('needs_attention', reason='Existing destination files differ; retained unchanged.',
|
||||
mismatch_log=str(job / 'existing-check.log'))
|
||||
return
|
||||
save('verifying_tree')
|
||||
# Existing files keep their original metadata. Their contents were fully
|
||||
# checksum-checked above, so a different mtime alone is not a copy failure.
|
||||
result = run_to_log(['rsync', '-rlHni', '--size-only', '--out-format=%i %n%L', '--protect-args',
|
||||
*source_args, str(destination) + '/'], 'tree-check.log')
|
||||
if result or (job / 'tree-check.log').read_text().strip():
|
||||
raise RuntimeError('Final directory comparison found differences; see tree-check.log.')
|
||||
subprocess.run(['sync', '-f', str(destination)], check=True)
|
||||
metadata_helper = Path(__file__).with_name('save_xattrs.py')
|
||||
if config.get('filesystem') == 'apfs' and metadata_helper.exists():
|
||||
save('saving_metadata')
|
||||
metadata_target = destination / ('.dayu-diskcopy-metadata-' + job.name)
|
||||
result = run_to_log(['/usr/bin/python3', str(metadata_helper), str(config_path), str(metadata_target)], 'metadata-summary.json')
|
||||
if result:
|
||||
raise RuntimeError('File copy passed but extended attribute archive failed.')
|
||||
save('complete', finished=time.time(), existing_files_checksum_match=True,
|
||||
new_files_verified_by_rsync_transfer_checksums=True)
|
||||
except BaseException as error:
|
||||
save('failed', error=str(error), finished=time.time())
|
||||
traceback.print_exc()
|
||||
finally:
|
||||
try:
|
||||
if managed and os.path.ismount(mount):
|
||||
subprocess.run(['umount', str(mount)], check=True, timeout=60)
|
||||
if managed and config['original_ro'] == '0':
|
||||
subprocess.run(['blockdev', '--setrw', str(device)], check=True)
|
||||
writes_after = (Path('/sys/class/block') / device.name / 'stat').read_text().split()[4:8:2]
|
||||
save(state['phase'], unmounted=managed and not os.path.ismount(mount), source_write_counters_after=writes_after,
|
||||
source_writes_unchanged=writes_after == state.get('source_write_counters_before'))
|
||||
except BaseException as error:
|
||||
save('cleanup_failed', prior_phase=state['phase'], cleanup_error=str(error))
|
||||
fcntl.flock(lock, fcntl.LOCK_UN)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,57 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Archive macOS extended attributes beside a NAS backup, without applying them.
|
||||
|
||||
Attribute values may contain private user information. Output stays on the NAS.
|
||||
"""
|
||||
import base64
|
||||
import collections
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
|
||||
def main():
|
||||
config = json.loads(Path(sys.argv[1]).read_text())
|
||||
uid, gid = config.get('uid', 1000), config.get('gid', 1001)
|
||||
source = Path(config['mount']) / config['source_root']
|
||||
destination = Path(sys.argv[2])
|
||||
destination.mkdir(mode=0o700)
|
||||
os.chown(destination, uid, gid)
|
||||
count = collections.Counter()
|
||||
entries = 0
|
||||
output = destination / 'extended-attributes.jsonl'
|
||||
fd = os.open(output, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o600)
|
||||
with os.fdopen(fd, 'w') as stream:
|
||||
for name in config['directories']:
|
||||
folder = source / name
|
||||
paths = [folder]
|
||||
for base, dirs, files in os.walk(folder, followlinks=False,
|
||||
onerror=lambda e: (_ for _ in ()).throw(e)):
|
||||
paths.extend(Path(base) / item for item in dirs + files)
|
||||
for path in paths:
|
||||
attributes = {}
|
||||
for key in os.listxattr(path, follow_symlinks=False):
|
||||
value = os.getxattr(path, key, follow_symlinks=False)
|
||||
attributes[key] = base64.b64encode(value).decode('ascii')
|
||||
count[key] += 1
|
||||
if attributes:
|
||||
stream.write(json.dumps({'path': str(path.relative_to(source)),
|
||||
'attributes_base64': attributes}, ensure_ascii=False) + '\n')
|
||||
entries += 1
|
||||
stream.flush()
|
||||
os.fsync(stream.fileno())
|
||||
os.chown(output, uid, gid)
|
||||
summary = {'entries': entries, 'attribute_names': dict(count), 'bytes': output.stat().st_size,
|
||||
'resource_fork_attributes': count['com.apple.ResourceFork']}
|
||||
(destination / 'README.txt').write_text(
|
||||
'这是 APFS 源目录的 macOS 扩展属性原始存档,不包含普通文件的数据内容。\n'
|
||||
'extended-attributes.jsonl 每行记录相对路径与 Base64 编码的原始属性。\n'
|
||||
'包含用户标签、Finder 信息和机器相关安全元数据,请随剪辑备份私下保管,不上传源码仓库。\n'
|
||||
'普通文件已由 rsync 复制;还原至 Mac 时应按需使用这些属性,不要盲目重放机器相关属性。\n')
|
||||
os.chown(destination / 'README.txt', uid, gid)
|
||||
print(json.dumps(summary, ensure_ascii=False))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,143 @@
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import socket
|
||||
import socketserver
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
def module(name):
|
||||
os.environ.setdefault('TRIM_PKGVAR', '/tmp/dayu-diskcopy-test')
|
||||
os.environ.setdefault('TRIM_UID', '99999')
|
||||
os.environ.setdefault('TRIM_GID', '99999')
|
||||
spec = importlib.util.spec_from_file_location(name, ROOT / 'application' / (name + '.py'))
|
||||
result = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(result)
|
||||
return result
|
||||
|
||||
|
||||
class PathBoundaryTests(unittest.TestCase):
|
||||
def test_path_traversal_and_symlink_escape_are_rejected(self):
|
||||
broker = module('broker')
|
||||
with tempfile.TemporaryDirectory() as folder:
|
||||
root = Path(folder) / 'root'
|
||||
root.mkdir()
|
||||
outside = Path(folder) / 'outside'
|
||||
outside.mkdir()
|
||||
(root / 'shortcut').symlink_to(outside)
|
||||
self.assertEqual(broker.beneath(root, root), root.resolve())
|
||||
for path in (root / '../outside', root / 'shortcut'):
|
||||
with self.assertRaises(ValueError):
|
||||
broker.beneath(path, root)
|
||||
|
||||
def test_system_identity_rejected_before_any_disk_operation(self):
|
||||
broker = module('broker')
|
||||
with self.assertRaises(ValueError):
|
||||
broker.dispatch({'uid': 0, 'action': 'start', 'args': {}})
|
||||
|
||||
def test_usb_apfs_ntfs_identified_and_internal_disks_excluded(self):
|
||||
broker = module('broker')
|
||||
tree = {'blockdevices': [
|
||||
{'tran': 'usb', 'size': 1000, 'serial': 'usb1', 'model': 'Test', 'children': [
|
||||
{'path': '/dev/test1', 'size': 900, 'fstype': 'ntfs', 'partuuid': 'abc', 'mountpoints': []},
|
||||
{'path': '/dev/test2', 'size': 100, 'fstype': 'apfs', 'partuuid': 'def', 'mountpoints': []}]},
|
||||
{'tran': 'sata', 'size': 1000, 'serial': 'system', 'fstype': 'ntfs'}]}
|
||||
with patch.object(broker, 'command', return_value=json.dumps(tree)):
|
||||
detected = broker.disks()
|
||||
self.assertEqual([item['filesystem'] for item in detected], ['ntfs', 'apfs'])
|
||||
|
||||
def test_root_listing_uses_empty_relative_path(self):
|
||||
broker = module('broker')
|
||||
with tempfile.TemporaryDirectory() as folder:
|
||||
self.assertEqual(broker.listing(Path(folder))['path'], '')
|
||||
|
||||
def test_preparing_job_visible_before_copy_log_is_created(self):
|
||||
broker = module('broker')
|
||||
with tempfile.TemporaryDirectory() as folder:
|
||||
root = Path(folder)
|
||||
job = root / 'jobs/test'
|
||||
job.mkdir(parents=True)
|
||||
(job / 'config.json').write_text(json.dumps({'uid':1000,'directories':['sample'],'destination':'/vol3/1000'}))
|
||||
(job / 'status.json').write_text(json.dumps({'phase':'preflight','started':time.time()}))
|
||||
with patch.object(broker, 'VAR', root), patch.object(broker, 'LEGACY', root / 'absent'):
|
||||
data = broker.jobs(1000)
|
||||
self.assertEqual(len(broker.active()), 1)
|
||||
self.assertEqual(len(data), 1)
|
||||
self.assertEqual(data[0]['phase'], 'preflight')
|
||||
|
||||
|
||||
class GatewayTests(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.front = module('front')
|
||||
cls.temp = tempfile.TemporaryDirectory(dir='/tmp', prefix='dayu-ui-')
|
||||
cls.path = str(Path(cls.temp.name) / 'web.sock')
|
||||
cls.server = cls.front.Server(cls.path, cls.front.Handler)
|
||||
cls.thread = threading.Thread(target=cls.server.serve_forever, daemon=True)
|
||||
cls.thread.start()
|
||||
|
||||
@classmethod
|
||||
def tearDownClass(cls):
|
||||
cls.server.shutdown()
|
||||
cls.server.server_close()
|
||||
cls.thread.join()
|
||||
cls.temp.cleanup()
|
||||
|
||||
def request(self, method, path, headers=''):
|
||||
with socket.socket(socket.AF_UNIX) as client:
|
||||
client.connect(self.path)
|
||||
client.sendall((method + ' ' + path + ' HTTP/1.0\r\nHost: nas.local\r\n' + headers + '\r\n').encode())
|
||||
response = b''
|
||||
while data := client.recv(65536):
|
||||
response += data
|
||||
return int(response.split(b' ')[1]), response
|
||||
|
||||
def test_anonymous_and_non_admin_are_rejected(self):
|
||||
self.assertEqual(self.request('GET', '/app/dayu-diskcopy')[0], 403)
|
||||
self.assertEqual(self.request('GET', '/app/dayu-diskcopy', 'X-Trim-Userid: 1000\r\nX-Trim-Isadmin: false\r\n')[0], 403)
|
||||
|
||||
def test_admin_can_load_interface(self):
|
||||
code, data = self.request('GET', '/app/dayu-diskcopy', 'X-Trim-Userid: 1000\r\nX-Trim-Isadmin: true\r\n')
|
||||
self.assertEqual(code, 200)
|
||||
self.assertIn('大鱼硬盘拷贝'.encode(), data)
|
||||
self.assertNotIn(b'__DAYU_CSRF__', data)
|
||||
self.assertIn(self.front.CSRF_TOKEN.encode(), data)
|
||||
|
||||
def test_writes_require_private_page_nonce_and_no_cross_site_context(self):
|
||||
headers = 'X-Trim-Userid: 1000\r\nX-Trim-Isadmin: true\r\nContent-Length: 0\r\n'
|
||||
self.assertEqual(self.request('POST', '/app/dayu-diskcopy/api/start', headers)[0], 403)
|
||||
self.assertEqual(self.request('POST', '/app/dayu-diskcopy/api/start', headers + 'X-Dayu-Request: 1\r\n')[0], 403)
|
||||
token = 'X-Dayu-Token: ' + self.front.CSRF_TOKEN + '\r\n'
|
||||
self.assertEqual(self.request('POST', '/app/dayu-diskcopy/api/start', headers + 'X-Dayu-Request: 1\r\n' + token + 'Sec-Fetch-Site: cross-site\r\n')[0], 403)
|
||||
|
||||
def test_valid_nonce_supports_gateway_rewritten_host(self):
|
||||
class Reply(socketserver.StreamRequestHandler):
|
||||
def handle(self):
|
||||
self.rfile.readline()
|
||||
self.wfile.write(b'{"ok":true,"data":{"accepted":true}}\n')
|
||||
broker_path = str(Path(self.temp.name) / 'broker.sock')
|
||||
server = socketserver.UnixStreamServer(broker_path, Reply)
|
||||
thread = threading.Thread(target=server.handle_request, daemon=True)
|
||||
thread.start()
|
||||
headers = ('X-Trim-Userid: 1000\r\nX-Trim-Isadmin: true\r\nContent-Length: 0\r\n'
|
||||
'X-Dayu-Request: 1\r\nSec-Fetch-Site: same-origin\r\n'
|
||||
'Origin: https://nas.local:5668\r\nX-Dayu-Token: ' + self.front.CSRF_TOKEN + '\r\n')
|
||||
try:
|
||||
with patch.object(self.front, 'VAR', Path(self.temp.name)):
|
||||
code, data = self.request('POST', '/app/dayu-diskcopy/api/mkdir', headers)
|
||||
self.assertEqual(code, 200)
|
||||
self.assertIn(b'accepted', data)
|
||||
finally:
|
||||
thread.join(timeout=2)
|
||||
server.server_close()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"date": "2026-11-11",
|
||||
"tested_app_version": "0.1.3",
|
||||
"native_gateway_admin_login": true,
|
||||
"ui_create_destination_folder": true,
|
||||
"ui_start_copy": true,
|
||||
"file_bytes": 404309,
|
||||
"partial_basis_matched_bytes": 99400,
|
||||
"file_sha256_matches_reference": true,
|
||||
"existing_file_different_mtime_preserved_and_checksum_passed": true,
|
||||
"all_jobs_complete": true,
|
||||
"all_jobs_source_writes_unchanged": true,
|
||||
"all_jobs_unmounted": true
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"date": "2026-11-11",
|
||||
"directories": [
|
||||
"火影秋决",
|
||||
"LCQ荒野乱斗"
|
||||
],
|
||||
"destination": "/vol5/1000/剪辑盘备份",
|
||||
"status": "complete",
|
||||
"logical_source_file_bytes": 548102817612,
|
||||
"newly_copied_bytes": 501332211263,
|
||||
"copy_seconds": 2197.872,
|
||||
"average_copy_MB_per_second": 228.099,
|
||||
"existing_files_checksum_verified": 64,
|
||||
"existing_checksum_differences": 0,
|
||||
"directory_differences": 0,
|
||||
"new_files_verified_by_rsync_stream_checksums": true,
|
||||
"independent_full_new_file_readback_performed": false,
|
||||
"source_writes_unchanged": true,
|
||||
"source_unmounted": true,
|
||||
"original_readonly_flag_restored": true,
|
||||
"destination_syncfs_passed": true,
|
||||
"destination_user_access_checked": true,
|
||||
"metadata_archive_entries": 7374,
|
||||
"metadata_archive_bytes": 1690603,
|
||||
"source_worker_commit": "604fce4fecc5d4a8ea5f830d653ced830bcc37d7"
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"ntfs_image_readonly_mount": true,
|
||||
"copied_bytes": 1048576,
|
||||
"source_and_copy_sha256_match": true,
|
||||
"sha256": "fbbab289f7f94b25736c58be46a994c441fd02552cc6022352e3d86d2fab7c83",
|
||||
"hardware_ntfs_usb_tested": false
|
||||
}
|
||||
在新工单中引用
屏蔽一个用户