# super_http_transfer

**Repository Path**: mazingan/super_http_transfer

## Basic Information

- **Project Name**: super_http_transfer
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-12-24
- **Last Updated**: 2025-12-24

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Super HTTP Transfer (SHT)

[English](#english) | [中文](#中文)

---

## English

### Overview

Super HTTP Transfer is a high-performance, chunked file transfer tool designed to replace SCP for large file transfers over HTTP. It supports resumable transfers, concurrent uploads/downloads, and works in firewall-restricted environments.

**Key Features:**
- ✓ Chunked file transfer (10MB default chunk size)
- ✓ Resumable uploads and downloads
- ✓ Concurrent chunk transfers (5 workers by default)
- ✓ SHA-256 hash verification
- ✓ HTTP Basic Authentication
- ✓ Absolute system path support (like scp)
- ✓ Directory recursive transfer
- ✓ Cross-platform support (Linux, macOS, Windows)
- ✓ Zero external dependencies (Go stdlib only)

### Building

#### Prerequisites
- Go 1.16 or higher

#### Build Commands

```bash
# Build for current platform
make build

# Build server only
make server

# Build client only
make client

# Cross-compile for Linux AMD64
make build-linux

# Cross-compile for macOS AMD64
make build-darwin

# Cross-compile for Linux ARM64
make build-linux-arm64

# Cross-compile for macOS ARM64 (Apple Silicon)
make build-darwin-arm64

# Build for all platforms
make build-all
```

### Server Setup

#### Starting the Server

```bash
# Basic startup (default port 8080)
./build/cmd/server

# With custom options
./build/cmd/server --port 9000 --data-dir /var/data --username admin --password secret

# Options:
#   --port PORT           Server port (default: 8080)
#   --data-dir DIR        Data storage directory (default: ./data)
#   --username USER       HTTP Basic Auth username
#   --password PASS       HTTP Basic Auth password
```

#### Server Directory Structure

```
{data-dir}/
├── uploads/           # Temporary upload chunks
│   └── {fileId}/
│       ├── meta.json
│       └── chunks/
│           ├── chunk_0.dat
│           ├── chunk_1.dat
│           └── ...
└── files/            # Completed files
    └── {user-path}/
        └── {filename}
```

### Client Usage

#### Basic Syntax

```bash
# Upload file
./build/cmd/client <local-file> <username:password@host:port/remote-path/>

# Download file
./build/cmd/client <username:password@host:port/remote-path/file> <local-path/>

# Upload directory (recursive)
./build/cmd/client -r <local-dir> <username:password@host:port/remote-path/>

# Download directory (recursive)
./build/cmd/client -r <username:password@host:port/remote-path/> <local-dir>
```

#### Upload Examples

```bash
# Upload single file to absolute path
./build/cmd/client ./large-file.tar.gz admin:secret@localhost:8080/var/backups/

# Upload directory recursively
./build/cmd/client -r ./my-project admin:secret@localhost:8080/backups/

# Upload with custom chunk size (20MB)
./build/cmd/client --chunk-size 20M ./file.iso admin:secret@localhost:8080/data/

# Resume interrupted upload
./build/cmd/client --resume ./file.tar.gz admin:secret@localhost:8080/backups/
```

#### Download Examples

```bash
# Download single file
./build/cmd/client admin:secret@localhost:8080/var/backups/file.tar.gz ./downloads/

# Download directory recursively
./build/cmd/client -r admin:secret@localhost:8080/backups/my-project/ ./restore/

# Download with custom concurrency (10 workers)
./build/cmd/client --concurrent 10 admin:secret@localhost:8080/data/file.iso ./
```

#### Client Options

```bash
# Flags:
#   -r, --recursive       Recursive transfer for directories
#   --resume              Resume interrupted transfer
#   --quiet               Suppress progress output
#   --verbose             Show detailed output
#   --chunk-size SIZE     Chunk size (e.g., 10M, 20M, 1G)
#   --concurrent NUM      Number of concurrent workers
#   --config FILE         Config file path
#   --username USER       Override config username
#   --password PASS       Override config password
```

#### Output Example

```
✓ Chunk 1/10 uploaded (10.00 MB)
✓ Chunk 2/10 uploaded (10.00 MB)
✓ Chunk 3/10 uploaded (10.00 MB)
...
✓ Chunk 10/10 uploaded (5.23 MB)

============================================================
✓ Upload completed successfully!
  File: large-file.tar.gz
  Size: 95.23 MB
  Chunks: 10
  Time: 12.5s
  Speed: 7.62 MB/s
  Remote: /var/backups/large-file.tar.gz
============================================================
```

### Testing

```bash
# Run all tests
make test

# Run specific package tests
go test ./pkg/server
go test ./pkg/client
```

### Cleanup

```bash
# Remove built binaries
make clean
```

---

## 中文

### 概述

Super HTTP Transfer 是一个高性能的分块文件传输工具，设计用于替代 SCP 进行大文件 HTTP 传输。它支持可恢复传输、并发上传/下载，并在防火墙受限的环境中工作。

**主要特性：**
- ✓ 分块文件传输（默认 10MB 块大小）
- ✓ 可恢复的上传和下载
- ✓ 并发块传输（默认 5 个工作线程）
- ✓ SHA-256 哈希验证
- ✓ HTTP 基本身份验证
- ✓ 绝对系统路径支持（类似 scp）
- ✓ 目录递归传输
- ✓ 跨平台支持（Linux、macOS、Windows）
- ✓ 零外部依赖（仅使用 Go 标准库）

### 编译

#### 前置要求
- Go 1.16 或更高版本

#### 编译命令

```bash
# 为当前平台编译
make build

# 仅编译服务器
make server

# 仅编译客户端
make client

# 交叉编译 Linux AMD64
make build-linux

# 交叉编译 macOS AMD64
make build-darwin

# 交叉编译 Linux ARM64
make build-linux-arm64

# 交叉编译 macOS ARM64（Apple Silicon）
make build-darwin-arm64

# 为所有平台编译
make build-all
```

### 服务器设置

#### 启动服务器

```bash
# 基本启动（默认端口 8080）
./build/cmd/server

# 自定义选项
./build/cmd/server --port 9000 --data-dir /var/data --username admin --password secret

# 选项：
#   --port PORT           服务器端口（默认：8080）
#   --data-dir DIR        数据存储目录（默认：./data）
#   --username USER       HTTP 基本认证用户名
#   --password PASS       HTTP 基本认证密码
```

#### 服务器目录结构

```
{data-dir}/
├── uploads/           # 临时上传块
│   └── {fileId}/
│       ├── meta.json
│       └── chunks/
│           ├── chunk_0.dat
│           ├── chunk_1.dat
│           └── ...
└── files/            # 已完成的文件
    └── {user-path}/
        └── {filename}
```

### 客户端使用

#### 基本语法

```bash
# 上传文件
./build/cmd/client <本地文件> <用户名:密码@主机:端口/远程路径/>

# 下载文件
./build/cmd/client <用户名:密码@主机:端口/远程路径/文件> <本地路径/>

# 上传目录（递归）
./build/cmd/client -r <本地目录> <用户名:密码@主机:端口/远程路径/>

# 下载目录（递归）
./build/cmd/client -r <用户名:密码@主机:端口/远程路径/> <本地目录>
```

#### 上传示例

```bash
# 上传单个文件到绝对路径
./build/cmd/client ./large-file.tar.gz admin:secret@localhost:8080/var/backups/

# 递归上传目录
./build/cmd/client -r ./my-project admin:secret@localhost:8080/backups/

# 使用自定义块大小上传（20MB）
./build/cmd/client --chunk-size 20M ./file.iso admin:secret@localhost:8080/data/

# 恢复中断的上传
./build/cmd/client --resume ./file.tar.gz admin:secret@localhost:8080/backups/
```

#### 下载示例

```bash
# 下载单个文件
./build/cmd/client admin:secret@localhost:8080/var/backups/file.tar.gz ./downloads/

# 递归下载目录
./build/cmd/client -r admin:secret@localhost:8080/backups/my-project/ ./restore/

# 使用自定义并发数下载（10 个工作线程）
./build/cmd/client --concurrent 10 admin:secret@localhost:8080/data/file.iso ./
```

#### 客户端选项

```bash
# 标志：
#   -r, --recursive       递归传输目录
#   --resume              恢复中断的传输
#   --quiet               抑制进度输出
#   --verbose             显示详细输出
#   --chunk-size SIZE     块大小（例如：10M、20M、1G）
#   --concurrent NUM      并发工作线程数
#   --config FILE         配置文件路径
#   --username USER       覆盖配置中的用户名
#   --password PASS       覆盖配置中的密码
```

#### 输出示例

```
✓ Chunk 1/10 uploaded (10.00 MB)
✓ Chunk 2/10 uploaded (10.00 MB)
✓ Chunk 3/10 uploaded (10.00 MB)
...
✓ Chunk 10/10 uploaded (5.23 MB)

============================================================
✓ Upload completed successfully!
  File: large-file.tar.gz
  Size: 95.23 MB
  Chunks: 10
  Time: 12.5s
  Speed: 7.62 MB/s
  Remote: /var/backups/large-file.tar.gz
============================================================
```

### 测试

```bash
# 运行所有测试
make test

# 运行特定包的测试
go test ./pkg/server
go test ./pkg/client
```

### 清理

```bash
# 删除编译的二进制文件
make clean
```

---

## Architecture / 架构

### Server Components / 服务器组件
- `pkg/server/server.go` - HTTP 服务器实现
- `pkg/server/handler.go` - API 端点处理器
- `pkg/server/task_manager.go` - 上传任务生命周期管理
- `pkg/server/file_manager.go` - 文件操作和块管理
- `pkg/server/cleaner.go` - 过期上传清理

### Client Components / 客户端组件
- `pkg/client/cli.go` - 命令行界面和路由
- `pkg/client/uploader.go` - 分块上传实现
- `pkg/client/downloader.go` - 分块下载实现
- `pkg/client/worker_pool.go` - 并发工作线程池
- `pkg/client/progress.go` - 进度显示
- `pkg/client/cache.go` - 可恢复传输缓存

### Shared Components / 共享组件
- `pkg/auth/` - HTTP 基本认证
- `pkg/config/` - 配置管理
- `pkg/model/` - 数据模型
- `pkg/utils/` - 工具函数（哈希、路径安全、URL 解析、重试逻辑）

---

## License

MIT License