# Core Team Response - Pipeline Progress Issue

**Date**: 2026-07-25
**Author**: Core Team (OpenCode)
**Status**: ✅ Implemented

---

## Summary

Core Team has implemented all 3 requested backend fixes:

| # | Item | Status |
|---|------|--------|
| 1 | Add "running" stage tracking | ✅ Completed |
| 2 | Initialize stages at registration | ✅ Completed |
| 3 | sync-status TKG check | ✅ Completed |

---

## Changes Made

### 1. "Running" Stage Tracking

**Problem**: Stages only showed `pending` → `completed`, never `running`.

**Solution**: Added `publish_pipeline_progress` calls at the start of each stage.

**Files Modified**: `src/worker/job_worker.rs`

| Stage | Line | Trigger |
|-------|------|---------|
| processors | ~472 | Job picked up |
| rule1_ingestion | ~1885 | ASRX completed |
| face_tracing | ~1991 | Face completed |
| identity_agent | ~2223 | Seeds exist |
| tkg_nodes + tkg_edges | ~2372 / ~2486 | Face + ASRX completed |
| rule2_ingestion | ~2432 | TKG completed |

**Example**:
```rust
// Mark face_tracing as running
{
    let mut pp = PipelineProgress::new(&uuid_clone);
    pp.update_stage("face_tracing", 0.0, "running", None);
    publish_pipeline_progress(redis_clone.as_ref(), &uuid_clone, &pp).await;
}
```

---

### 2. Initialize Stages at Registration

**Problem**: Stages were empty when file started processing, frontend couldn't determine active stages.

**Solution**: Initialize `PipelineProgress` immediately after `create_monitor_job`.

**Files Modified**: `src/api/files.rs:866-874`

```rust
if let Ok(job) = auto_state.db.create_monitor_job(&auto_uuid, Some(vp)).await {
    // Initialize pipeline progress with all stages pending
    if let Ok(redis) = crate::core::db::RedisClient::new() {
        let pp = PipelineProgress::new(&auto_uuid);
        publish_pipeline_progress(&redis, &auto_uuid, &pp).await;
    }
    // ...
}
```

---

### 3. sync-status TKG Check

**Problem**: `sync_file_status` only checked processors, not TKG or chunks.

**Solution**: Check TKG nodes, edges, and sentence chunks before marking as `completed`.

**Files Modified**: `src/api/files.rs:1762-1856`

**New Response Format**:
```json
{
  "success": true,
  "file_uuid": "...",
  "status": "completed",
  "processors_complete": 5,
  "processors_total": 5,
  "tkg_nodes": 123,
  "tkg_edges": 45,
  "sentence_chunks": 67,
  "is_music_only": false
}
```

**Logic**:
- `completed` = processors_done + tkg_nodes > 0 + sentence_chunks > 0
- For music-only files: `tkg_edges == 0` is acceptable
- `is_music_only` = processors_done + tkg_nodes > 0 + tkg_edges == 0

---

## API Changes

### `/api/v1/file/:file_uuid/sync-status` Response Changes

| Field | Type | Description |
|-------|------|-------------|
| `tkg_nodes` | i64 | TKG node count |
| `tkg_edges` | i64 | TKG edge count |
| `sentence_chunks` | i64 | Rule 1 sentence chunk count |
| `is_music_only` | bool | True if music-only file (no faces) |

---

## Testing

### Test Commands

1. **Register new file and check initial stages**:
   ```bash
   # After registration, immediately check
   curl http://localhost:3002/api/v1/stats/pipeline/{file_uuid}
   
   # Expected: stages array has 7 elements, all "pending"
   ```

2. **Check "running" status during processing**:
   ```bash
   # During processing, check stages
   curl http://localhost:3002/api/v1/stats/pipeline/{file_uuid}
   
   # Expected: at least one stage shows "running"
   ```

3. **Check sync-status response**:
   ```bash
   curl -X POST http://localhost:3002/api/v1/file/{file_uuid}/sync-status
   
   # Expected: includes tkg_nodes, tkg_edges, sentence_chunks, is_music_only
   ```

---

## Deployment

**Environment**: Production (3002)
**Status**: ✅ Deployed, running

**Implemented Files**:
- `src/worker/job_worker.rs`: 6 处 "running" 状态更新
- `src/api/files.rs:866`: 注册时初始化 stages
- `src/api/files.rs:1762`: sync-status 检查 TKG + chunks
- `src/api/files.rs:16`: 新增 imports

**Build**: 2026-07-24T18:43:52Z

---

## Notes

### Music-Only Files

For files with no faces (music-only):
- `identity_agent`: marked as "completed (no faces)"
- `tkg_edges`: marked as "completed (no faces)"
- `rule2_ingestion`: marked as "completed (no faces)"

This is handled in `src/api/scan.rs:1210-1216` and now correctly detected in `sync_file_status`.

---

## Commit

Branch: main
Commit: (pending - to be committed after verification)

---

**Contact**: Core Team (OpenCode)