Compare commits
2 Commits
a096bb645c
...
8cf7196c86
| Author | SHA1 | Date | |
|---|---|---|---|
| 8cf7196c86 | |||
| ec67e749d7 |
177
CORE_TEAM_RESPONSE.md
Normal file
177
CORE_TEAM_RESPONSE.md
Normal file
@@ -0,0 +1,177 @@
|
||||
# Core Team Response - Pipeline Progress Issue
|
||||
|
||||
**Date**: 2026-07-25
|
||||
**Author**: Core Team (OpenCode)
|
||||
**Status**: ✅ Implemented
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Core Team has implemented all 3 requested backend fixes:
|
||||
|
||||
| # | Item | Status |
|
||||
|---|------|--------|
|
||||
| 1 | Add "running" stage tracking | ✅ Completed |
|
||||
| 2 | Initialize stages at registration | ✅ Completed |
|
||||
| 3 | sync-status TKG check | ✅ Completed |
|
||||
|
||||
---
|
||||
|
||||
## Changes Made
|
||||
|
||||
### 1. "Running" Stage Tracking
|
||||
|
||||
**Problem**: Stages only showed `pending` → `completed`, never `running`.
|
||||
|
||||
**Solution**: Added `publish_pipeline_progress` calls at the start of each stage.
|
||||
|
||||
**Files Modified**: `src/worker/job_worker.rs`
|
||||
|
||||
| Stage | Line | Trigger |
|
||||
|-------|------|---------|
|
||||
| processors | ~472 | Job picked up |
|
||||
| rule1_ingestion | ~1885 | ASRX completed |
|
||||
| face_tracing | ~1991 | Face completed |
|
||||
| identity_agent | ~2223 | Seeds exist |
|
||||
| tkg_nodes + tkg_edges | ~2372 / ~2486 | Face + ASRX completed |
|
||||
| rule2_ingestion | ~2432 | TKG completed |
|
||||
|
||||
**Example**:
|
||||
```rust
|
||||
// Mark face_tracing as running
|
||||
{
|
||||
let mut pp = PipelineProgress::new(&uuid_clone);
|
||||
pp.update_stage("face_tracing", 0.0, "running", None);
|
||||
publish_pipeline_progress(redis_clone.as_ref(), &uuid_clone, &pp).await;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. Initialize Stages at Registration
|
||||
|
||||
**Problem**: Stages were empty when file started processing, frontend couldn't determine active stages.
|
||||
|
||||
**Solution**: Initialize `PipelineProgress` immediately after `create_monitor_job`.
|
||||
|
||||
**Files Modified**: `src/api/files.rs:866-874`
|
||||
|
||||
```rust
|
||||
if let Ok(job) = auto_state.db.create_monitor_job(&auto_uuid, Some(vp)).await {
|
||||
// Initialize pipeline progress with all stages pending
|
||||
if let Ok(redis) = crate::core::db::RedisClient::new() {
|
||||
let pp = PipelineProgress::new(&auto_uuid);
|
||||
publish_pipeline_progress(&redis, &auto_uuid, &pp).await;
|
||||
}
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. sync-status TKG Check
|
||||
|
||||
**Problem**: `sync_file_status` only checked processors, not TKG or chunks.
|
||||
|
||||
**Solution**: Check TKG nodes, edges, and sentence chunks before marking as `completed`.
|
||||
|
||||
**Files Modified**: `src/api/files.rs:1762-1856`
|
||||
|
||||
**New Response Format**:
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"file_uuid": "...",
|
||||
"status": "completed",
|
||||
"processors_complete": 5,
|
||||
"processors_total": 5,
|
||||
"tkg_nodes": 123,
|
||||
"tkg_edges": 45,
|
||||
"sentence_chunks": 67,
|
||||
"is_music_only": false
|
||||
}
|
||||
```
|
||||
|
||||
**Logic**:
|
||||
- `completed` = processors_done + tkg_nodes > 0 + sentence_chunks > 0
|
||||
- For music-only files: `tkg_edges == 0` is acceptable
|
||||
- `is_music_only` = processors_done + tkg_nodes > 0 + tkg_edges == 0
|
||||
|
||||
---
|
||||
|
||||
## API Changes
|
||||
|
||||
### `/api/v1/file/:file_uuid/sync-status` Response Changes
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `tkg_nodes` | i64 | TKG node count |
|
||||
| `tkg_edges` | i64 | TKG edge count |
|
||||
| `sentence_chunks` | i64 | Rule 1 sentence chunk count |
|
||||
| `is_music_only` | bool | True if music-only file (no faces) |
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
### Test Commands
|
||||
|
||||
1. **Register new file and check initial stages**:
|
||||
```bash
|
||||
# After registration, immediately check
|
||||
curl http://localhost:3002/api/v1/stats/pipeline/{file_uuid}
|
||||
|
||||
# Expected: stages array has 7 elements, all "pending"
|
||||
```
|
||||
|
||||
2. **Check "running" status during processing**:
|
||||
```bash
|
||||
# During processing, check stages
|
||||
curl http://localhost:3002/api/v1/stats/pipeline/{file_uuid}
|
||||
|
||||
# Expected: at least one stage shows "running"
|
||||
```
|
||||
|
||||
3. **Check sync-status response**:
|
||||
```bash
|
||||
curl -X POST http://localhost:3002/api/v1/file/{file_uuid}/sync-status
|
||||
|
||||
# Expected: includes tkg_nodes, tkg_edges, sentence_chunks, is_music_only
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Deployment
|
||||
|
||||
**Environment**: Production (3002)
|
||||
**Status**: ✅ Compiled, ready for restart
|
||||
|
||||
**Restart Command**:
|
||||
```bash
|
||||
./run-server-3002.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
### Music-Only Files
|
||||
|
||||
For files with no faces (music-only):
|
||||
- `identity_agent`: marked as "completed (no faces)"
|
||||
- `tkg_edges`: marked as "completed (no faces)"
|
||||
- `rule2_ingestion`: marked as "completed (no faces)"
|
||||
|
||||
This is handled in `src/api/scan.rs:1210-1216` and now correctly detected in `sync_file_status`.
|
||||
|
||||
---
|
||||
|
||||
## Commit
|
||||
|
||||
Branch: main
|
||||
Commit: (pending - to be committed after verification)
|
||||
|
||||
---
|
||||
|
||||
**Contact**: Core Team (OpenCode)
|
||||
11
src/store.ts
11
src/store.ts
@@ -916,20 +916,19 @@ export async function loadPipelineStats(fileUuid: string, force = false): Promis
|
||||
try {
|
||||
const data: any = await apiCall('get_pipeline_stats', { fileUuid })
|
||||
|
||||
// 过滤掉identity_agent阶段(独立于pipeline)
|
||||
const filteredStages = (data.stages || []).filter((s: any) => s.name !== 'identity_agent')
|
||||
const stages = data.stages || []
|
||||
|
||||
// 覆盖权重为最终设计值(Core API返回的权重可能不匹配)
|
||||
const WEIGHT_MAP: Record<string, number> = {
|
||||
processors: 0.30,
|
||||
rule1_ingestion: 0.10,
|
||||
face_tracing: 0.10,
|
||||
rule1_ingestion: 0.05,
|
||||
face_tracing: 0.05,
|
||||
identity_agent: 0.10,
|
||||
tkg_nodes: 0.20,
|
||||
tkg_edges: 0.15,
|
||||
rule2_ingestion: 0.15
|
||||
}
|
||||
|
||||
const mappedStages = filteredStages.map((s: any) => ({
|
||||
const mappedStages = stages.map((s: any) => ({
|
||||
name: s.name,
|
||||
weight: WEIGHT_MAP[s.name] ?? s.weight ?? 0,
|
||||
progress: s.progress || 0,
|
||||
|
||||
Reference in New Issue
Block a user