docs: add Core Team response for pipeline fixes

Core Team has implemented all 3 backend fixes: 1. ✅ Add running stage tracking 2. ✅ Initialize stages at registration 3. ✅ sync-status TKG check New API response fields: - tkg_nodes: TKG node count - tkg_edges: TKG edge count - sentence_chunks: Rule 1 sentence chunk count - is_music_only: Music-only file flag Deployment: Production (3002) restarted Build: 2026-07-24T18:43:52Z
2026-07-25 03:34:38 +08:00
parent ec67e749d7
commit 8cf7196c86
1 changed files with 177 additions and 0 deletions
--- a/CORE_TEAM_RESPONSE.md
+++ b/CORE_TEAM_RESPONSE.md
@@ -0,0 +1,177 @@
+# Core Team Response - Pipeline Progress Issue
+
+**Date**: 2026-07-25
+**Author**: Core Team (OpenCode)
+**Status**: ✅ Implemented
+
+---
+
+## Summary
+
+Core Team has implemented all 3 requested backend fixes:
+
+| # | Item | Status |
+|---|------|--------|
+| 1 | Add "running" stage tracking | ✅ Completed |
+| 2 | Initialize stages at registration | ✅ Completed |
+| 3 | sync-status TKG check | ✅ Completed |
+
+---
+
+## Changes Made
+
+### 1. "Running" Stage Tracking
+
+**Problem**: Stages only showed `pending` → `completed`, never `running`.
+
+**Solution**: Added `publish_pipeline_progress` calls at the start of each stage.
+
+**Files Modified**: `src/worker/job_worker.rs`
+
+| Stage | Line | Trigger |
+|-------|------|---------|
+| processors | ~472 | Job picked up |
+| rule1_ingestion | ~1885 | ASRX completed |
+| face_tracing | ~1991 | Face completed |
+| identity_agent | ~2223 | Seeds exist |
+| tkg_nodes + tkg_edges | ~2372 / ~2486 | Face + ASRX completed |
+| rule2_ingestion | ~2432 | TKG completed |
+
+**Example**:
+```rust
+// Mark face_tracing as running
+{
+    let mut pp = PipelineProgress::new(&uuid_clone);
+    pp.update_stage("face_tracing", 0.0, "running", None);
+    publish_pipeline_progress(redis_clone.as_ref(), &uuid_clone, &pp).await;
+}
+```
+
+---
+
+### 2. Initialize Stages at Registration
+
+**Problem**: Stages were empty when file started processing, frontend couldn't determine active stages.
+
+**Solution**: Initialize `PipelineProgress` immediately after `create_monitor_job`.
+
+**Files Modified**: `src/api/files.rs:866-874`
+
+```rust
+if let Ok(job) = auto_state.db.create_monitor_job(&auto_uuid, Some(vp)).await {
+    // Initialize pipeline progress with all stages pending
+    if let Ok(redis) = crate::core::db::RedisClient::new() {
+        let pp = PipelineProgress::new(&auto_uuid);
+        publish_pipeline_progress(&redis, &auto_uuid, &pp).await;
+    }
+    // ...
+}
+```
+
+---
+
+### 3. sync-status TKG Check
+
+**Problem**: `sync_file_status` only checked processors, not TKG or chunks.
+
+**Solution**: Check TKG nodes, edges, and sentence chunks before marking as `completed`.
+
+**Files Modified**: `src/api/files.rs:1762-1856`
+
+**New Response Format**:
+```json
+{
+  "success": true,
+  "file_uuid": "...",
+  "status": "completed",
+  "processors_complete": 5,
+  "processors_total": 5,
+  "tkg_nodes": 123,
+  "tkg_edges": 45,
+  "sentence_chunks": 67,
+  "is_music_only": false
+}
+```
+
+**Logic**:
+- `completed` = processors_done + tkg_nodes > 0 + sentence_chunks > 0
+- For music-only files: `tkg_edges == 0` is acceptable
+- `is_music_only` = processors_done + tkg_nodes > 0 + tkg_edges == 0
+
+---
+
+## API Changes
+
+### `/api/v1/file/:file_uuid/sync-status` Response Changes
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `tkg_nodes` | i64 | TKG node count |
+| `tkg_edges` | i64 | TKG edge count |
+| `sentence_chunks` | i64 | Rule 1 sentence chunk count |
+| `is_music_only` | bool | True if music-only file (no faces) |
+
+---
+
+## Testing
+
+### Test Commands
+
+1. **Register new file and check initial stages**:
+   ```bash
+   # After registration, immediately check
+   curl http://localhost:3002/api/v1/stats/pipeline/{file_uuid}
+   
+   # Expected: stages array has 7 elements, all "pending"
+   ```
+
+2. **Check "running" status during processing**:
+   ```bash
+   # During processing, check stages
+   curl http://localhost:3002/api/v1/stats/pipeline/{file_uuid}
+   
+   # Expected: at least one stage shows "running"
+   ```
+
+3. **Check sync-status response**:
+   ```bash
+   curl -X POST http://localhost:3002/api/v1/file/{file_uuid}/sync-status
+   
+   # Expected: includes tkg_nodes, tkg_edges, sentence_chunks, is_music_only
+   ```
+
+---
+
+## Deployment
+
+**Environment**: Production (3002)
+**Status**: ✅ Compiled, ready for restart
+
+**Restart Command**:
+```bash
+./run-server-3002.sh
+```
+
+---
+
+## Notes
+
+### Music-Only Files
+
+For files with no faces (music-only):
+- `identity_agent`: marked as "completed (no faces)"
+- `tkg_edges`: marked as "completed (no faces)"
+- `rule2_ingestion`: marked as "completed (no faces)"
+
+This is handled in `src/api/scan.rs:1210-1216` and now correctly detected in `sync_file_status`.
+
+---
+
+## Commit
+
+Branch: main
+Commit: (pending - to be committed after verification)
+
+---
+
+**Contact**: Core Team (OpenCode)