fix(file)!: stream uploads through staged storage

- fix: replace multipart form parsing with streaming multipart reads and
  apply request body limits when max_upload_size is configured.
- refactor: route uploads through staging paths before promotion to
  long-term data paths, keeping incomplete uploads out of durable
  storage records.
- test: cover oversized uploads, unlimited uploads, staged cleanup, and
  local storage promotion boundaries.
- docs: document the staged upload model and multipart parent_id query
  parameter.
This commit is contained in:
2026-07-05 17:18:19 +08:00
parent bfeb4b26e4
commit b988b4b15e
10 changed files with 457 additions and 50 deletions
+3 -3
View File
@@ -15,7 +15,7 @@ Repository (GORM data access) Storage (file I/O)
Rules:
- Handler has no business logic — parse request, call service, write response.
- Service has no HTTP awareness — operates on domain models and interfaces.
- Repository abstracts the database; Storage abstracts where bytes live.
- Repository abstracts the database; Storage abstracts where bytes live, including staged upload promotion.
- `internal/server` is the composition root — wires all dependencies together.
## Package Map
@@ -34,7 +34,7 @@ Rules:
| **Business** | `internal/service` | Business logic: `AuthService` (register, login, refresh, logout, passkey CRUD) | ✅ |
| | `internal/model` | Domain types (User, File, Credential, Session), error codes | ✅ |
| **Data** | `internal/repository` | Repository interfaces + GORM implementations (User, Session, File, Credential) | ✅ |
| | `internal/storage` | Storage backend interface + local disk impl | 🛠 WIP |
| | `internal/storage` | Storage backend interface + local disk impl, with staging and promotion | 🛠 WIP |
| **Util** | `internal/auth` | JWT sign/verify (HS256), token type discrimination (access/refresh), password hashing (bcrypt), app passkey tokens | ✅ |
| | `internal/api` | Unified JSON error response helpers | ✅ |
@@ -54,7 +54,7 @@ POST /api/v1/account/passkeys
DELETE /api/v1/account/passkeys/:id
GET /api/v1/files
POST /api/v1/files
POST /api/v1/files # JSON creates directories; multipart uploads files. File upload parent_id is a query parameter.
GET /api/v1/files/:id
GET /api/v1/files/:id/content
PUT /api/v1/files/:id
+19
View File
@@ -67,3 +67,22 @@
- Refresh tokens cannot authenticate API requests; access tokens cannot be used to issue new token pairs.
- The placeholder JWT secret is safe for local startup, but tokens signed with it are invalidated on restart.
- New duration config fields require zero boilerplate — declare as `time.Duration` in the struct.
## 2026-07-05: Staged File Uploads
**Context**: Multipart uploads previously used `ParseMultipartForm`, which parses the request before service-level size checks and may spill oversized requests to temporary disk. The file service also wrote directly to the long-term storage path and then attempted compensating deletes on upload failure.
**Decisions**:
| Decision | Guidance |
|----------|----------|
| Stream multipart requests | Handlers use `Request.MultipartReader()` instead of `ParseMultipartForm`/`FormFile`, so uploads are streamed into the service. |
| Optional upload limits | `storage.max_upload_size = 0` means unlimited. Positive values enable both HTTP body limiting and service-level file content limiting. |
| Staging before promotion | Storage backends write upload bytes to a staging path first, then promote the object to the long-term data path only after validation succeeds. |
| Promote before DB create | The service promotes the object before creating the active file record, preventing visible DB rows from pointing at missing objects. If DB creation fails after promotion, the service best-effort deletes the promoted object. |
| Upload parent location | Multipart upload `parent_id` is passed as a query parameter, keeping the multipart body focused on the file stream. |
**Consequences**:
- Interrupted, malformed, and oversized uploads leave only staging objects, which are safe to clean by path prefix and age.
- Local storage can promote with `os.Rename`; future S3 storage can implement promotion with copy/delete while keeping business visibility controlled by the DB row.
- A DB failure after promotion can still leave a long-term orphan object, but it is not visible through the file API and can be cleaned independently.
+1 -1
View File
@@ -20,7 +20,7 @@ Package-level implementation order (each task includes unit tests):
3. `internal/model` — domain types, error codes ✅
4. `internal/api` — error response helpers ✅
5. `internal/auth` — JWT utils ✅
6. `internal/storage` — backend interface + local fs
6. `internal/storage` — backend interface + local fs with staged upload promotion
7. `internal/repository` — interfaces + GORM/SQLite impl ✅
8. `internal/service` — auth, file, admin services ✅ (auth done)
9. `internal/middleware` — logger, cors, auth ✅ (auth done)