fix(file)!: stream uploads through staged storage
- fix: replace multipart form parsing with streaming multipart reads and apply request body limits when max_upload_size is configured. - refactor: route uploads through staging paths before promotion to long-term data paths, keeping incomplete uploads out of durable storage records. - test: cover oversized uploads, unlimited uploads, staged cleanup, and local storage promotion boundaries. - docs: document the staged upload model and multipart parent_id query parameter.
This commit is contained in:
@@ -15,7 +15,7 @@ Repository (GORM data access) Storage (file I/O)
|
||||
Rules:
|
||||
- Handler has no business logic — parse request, call service, write response.
|
||||
- Service has no HTTP awareness — operates on domain models and interfaces.
|
||||
- Repository abstracts the database; Storage abstracts where bytes live.
|
||||
- Repository abstracts the database; Storage abstracts where bytes live, including staged upload promotion.
|
||||
- `internal/server` is the composition root — wires all dependencies together.
|
||||
|
||||
## Package Map
|
||||
@@ -34,7 +34,7 @@ Rules:
|
||||
| **Business** | `internal/service` | Business logic: `AuthService` (register, login, refresh, logout, passkey CRUD) | ✅ |
|
||||
| | `internal/model` | Domain types (User, File, Credential, Session), error codes | ✅ |
|
||||
| **Data** | `internal/repository` | Repository interfaces + GORM implementations (User, Session, File, Credential) | ✅ |
|
||||
| | `internal/storage` | Storage backend interface + local disk impl | 🛠 WIP |
|
||||
| | `internal/storage` | Storage backend interface + local disk impl, with staging and promotion | 🛠 WIP |
|
||||
| **Util** | `internal/auth` | JWT sign/verify (HS256), token type discrimination (access/refresh), password hashing (bcrypt), app passkey tokens | ✅ |
|
||||
| | `internal/api` | Unified JSON error response helpers | ✅ |
|
||||
|
||||
@@ -54,7 +54,7 @@ POST /api/v1/account/passkeys
|
||||
DELETE /api/v1/account/passkeys/:id
|
||||
|
||||
GET /api/v1/files
|
||||
POST /api/v1/files
|
||||
POST /api/v1/files # JSON creates directories; multipart uploads files. File upload parent_id is a query parameter.
|
||||
GET /api/v1/files/:id
|
||||
GET /api/v1/files/:id/content
|
||||
PUT /api/v1/files/:id
|
||||
|
||||
@@ -67,3 +67,22 @@
|
||||
- Refresh tokens cannot authenticate API requests; access tokens cannot be used to issue new token pairs.
|
||||
- The placeholder JWT secret is safe for local startup, but tokens signed with it are invalidated on restart.
|
||||
- New duration config fields require zero boilerplate — declare as `time.Duration` in the struct.
|
||||
|
||||
## 2026-07-05: Staged File Uploads
|
||||
|
||||
**Context**: Multipart uploads previously used `ParseMultipartForm`, which parses the request before service-level size checks and may spill oversized requests to temporary disk. The file service also wrote directly to the long-term storage path and then attempted compensating deletes on upload failure.
|
||||
|
||||
**Decisions**:
|
||||
|
||||
| Decision | Guidance |
|
||||
|----------|----------|
|
||||
| Stream multipart requests | Handlers use `Request.MultipartReader()` instead of `ParseMultipartForm`/`FormFile`, so uploads are streamed into the service. |
|
||||
| Optional upload limits | `storage.max_upload_size = 0` means unlimited. Positive values enable both HTTP body limiting and service-level file content limiting. |
|
||||
| Staging before promotion | Storage backends write upload bytes to a staging path first, then promote the object to the long-term data path only after validation succeeds. |
|
||||
| Promote before DB create | The service promotes the object before creating the active file record, preventing visible DB rows from pointing at missing objects. If DB creation fails after promotion, the service best-effort deletes the promoted object. |
|
||||
| Upload parent location | Multipart upload `parent_id` is passed as a query parameter, keeping the multipart body focused on the file stream. |
|
||||
|
||||
**Consequences**:
|
||||
- Interrupted, malformed, and oversized uploads leave only staging objects, which are safe to clean by path prefix and age.
|
||||
- Local storage can promote with `os.Rename`; future S3 storage can implement promotion with copy/delete while keeping business visibility controlled by the DB row.
|
||||
- A DB failure after promotion can still leave a long-term orphan object, but it is not visible through the file API and can be cleaned independently.
|
||||
|
||||
+1
-1
@@ -20,7 +20,7 @@ Package-level implementation order (each task includes unit tests):
|
||||
3. `internal/model` — domain types, error codes ✅
|
||||
4. `internal/api` — error response helpers ✅
|
||||
5. `internal/auth` — JWT utils ✅
|
||||
6. `internal/storage` — backend interface + local fs
|
||||
6. `internal/storage` — backend interface + local fs with staged upload promotion
|
||||
7. `internal/repository` — interfaces + GORM/SQLite impl ✅
|
||||
8. `internal/service` — auth, file, admin services ✅ (auth done)
|
||||
9. `internal/middleware` — logger, cors, auth ✅ (auth done)
|
||||
|
||||
Reference in New Issue
Block a user