fix: deterministic physical item motion in 3D demo
Add physicalItemMotion as the single source of item pose, synchronize belt/item speed at 1 m/s, route items through chutes, keep C/D items settled inside roll-cages, render historical settled items, and use STL models with explicit fallbacks.
46
docs/ITERATION_VISUAL_QA_REPORT.md
Normal file
@@ -0,0 +1,46 @@
|
||||
# Iteration Visual QA Report
|
||||
|
||||
Date: 2026-07-09
|
||||
Branch: `dan_branch`
|
||||
Scope: one limited 3D demo improvement cycle for `https://arhipovdan.ru/`.
|
||||
|
||||
## Before / After Screenshots
|
||||
|
||||
Screenshots saved in `docs/visual_qa_screenshots/`:
|
||||
|
||||
- `before_initial_production.png` — production initial state before playback.
|
||||
- `after_running_item_on_belt.png` — production after deploy, item moving on belt.
|
||||
- `after_cv_overlay_no_overlap.png` — production after deploy, CV overlay visible without HUD overlap.
|
||||
- `after_demo_complete_8_cases.png` — production after deploy, all 8 cases completed.
|
||||
- `after_details_page.png` — `/details` production check.
|
||||
|
||||
## Top 3 Issues Fixed
|
||||
|
||||
1. Conveyor/item timing did not read as 1 m/s: movement phases were retimed to physical distances, and belt animation now runs only while the item is moving.
|
||||
2. Items looked like route-colored cubes: STL/procedural item materials now use product-like surfaces with subtle route-colored outlines instead of category-color body fill.
|
||||
3. CV overlay could overlap the top-right HUD: the measurement overlay now lives on the left side with bounded height.
|
||||
|
||||
## Verification
|
||||
|
||||
- `npm run build` — passed.
|
||||
- `npm run test` — passed, 11 test files / 113 tests.
|
||||
- `docker compose -p owl -f docker-compose.server.yml up -d --build` — passed, `owl-web-1` recreated and started.
|
||||
- Playwright production QA — passed with `NO_CONSOLE_ERRORS`.
|
||||
|
||||
## Acceptance Checklist
|
||||
|
||||
- [x] No console errors.
|
||||
- [x] Item does not fly.
|
||||
- [x] Item does not fall through the belt.
|
||||
- [x] STL/procedural items are visible and no longer read as route-colored cubes.
|
||||
- [x] Belt and item movement are synchronized during movement phases.
|
||||
- [x] HUD and CV overlay do not overlap.
|
||||
- [x] Play starts the demo.
|
||||
- [x] `/details` works.
|
||||
- [x] Before/after screenshots exist.
|
||||
|
||||
## Notes
|
||||
|
||||
- No changes were made to `/details`, nginx, Dockerfile, classifier logic, or scenarios.
|
||||
- No physics engine was added.
|
||||
- No commit or push was made.
|
||||
35
docs/PHYSICAL_MOTION_FIX_REPORT.md
Normal file
@@ -0,0 +1,35 @@
|
||||
# Physical Motion Fix Report
|
||||
|
||||
## Причины предыдущего хаоса
|
||||
Ранее движение товара рассчитывалось независимо по осям с использованием интерполяции между хардкодными точками, что приводило к телепортации или "зависанию" объектов (в том числе над корзиной). Также отсутствовала система хранения объектов после завершения их этапа маршрутизации. Указанные в `ModelAssets` STL-модели загружались, но отображались некорректно. Из-за отсутствия независимой физической модели каждый фрейм пытался обновлять позиции локально.
|
||||
|
||||
## Решение
|
||||
1. **Создана `physicalItemMotion.ts`** — единая физико-кинематическая модель `getPhysicalItemPose`. Она гарантирует детерминированное вычисление `[x,y,z]` и вращения на основе `elapsedMs`, отсчитанного от момента спавна товара, а также `CONVEYOR_SPEED_MPS` (1 м/с).
|
||||
2. **Синхронизация ленты и товара**: Время и расстояние на ленте точно соответствуют $d = 1.0 \times t$. Движущаяся текстура ленты использует эту же скорость.
|
||||
3. **Хранение (settled) элементов**: В `SorterDigitalTwinContinuous.tsx` теперь рендерятся все созданные (spawned) объекты за последние 20 циклов. Объекты попадают в корзину `C` или `D` по желобу (`chute`), меняя высоту, и остаются там (isSettled).
|
||||
4. **STL модели**: Добавлен единый компонент `PhysicalPlaybackItem.tsx`, который грузит `STLGeometry` или использует `FallbackPrimitive` в случае их отсутствия, применяя при этом корректные материалы с реалистичным `roughness` и `metalness`.
|
||||
|
||||
## Подтверждение
|
||||
- Скорость товара 1 м/с подтверждена тестами в `physicalItemMotion.test.ts`.
|
||||
- Объекты остаются внутри cage благодаря смещению на базе `slotIndex`.
|
||||
- Fallback применён только к отсутствующим или "heavy" STL. (Короб 300, 400, Тарелка, Бутылка, Цилиндр, ЛанчБокс — STL).
|
||||
|
||||
## Результаты QA и сборка
|
||||
- `npm run build` — Успешно (0 ошибок).
|
||||
- `npm run test` — Успешно (117 тестов пройдено).
|
||||
- `docker-compose` — Контейнеры пересобраны и запущены.
|
||||
- Production QA Script завершен с 0 ошибок консоли (`NO_CONSOLE_ERRORS`).
|
||||
|
||||
Скриншоты лежат в папке `docs/physical_motion_fix_screenshots/`:
|
||||
- `stl_models_visible.png`
|
||||
- `belt_sync_t0.png`
|
||||
- `belt_sync_t1.png`
|
||||
- `item_on_belt_no_flying.png`
|
||||
- `route_to_c_chute.png`
|
||||
- `item_settled_in_c_cage.png`
|
||||
- `route_to_d_chute.png`
|
||||
- `item_settled_in_d_cage.png`
|
||||
- `no_overlay_overlap.png`
|
||||
- `demo_complete_items_not_chaotic.png`
|
||||
|
||||
Ограничения: `slotIndex` использует простую матричную сетку 3xN, что при сильном переполнении корзины может вызвать небольшие наложения (clipping) STL-моделей друг на друга, но это визуально приемлемо и объекты остаются строго внутри корзины.
|
||||
BIN
docs/physical_motion_fix_screenshots/belt_sync_t0.png
Normal file
|
After Width: | Height: | Size: 160 KiB |
BIN
docs/physical_motion_fix_screenshots/belt_sync_t1.png
Normal file
|
After Width: | Height: | Size: 190 KiB |
|
After Width: | Height: | Size: 193 KiB |
BIN
docs/physical_motion_fix_screenshots/item_on_belt_no_flying.png
Normal file
|
After Width: | Height: | Size: 206 KiB |
BIN
docs/physical_motion_fix_screenshots/item_settled_in_c_cage.png
Normal file
|
After Width: | Height: | Size: 208 KiB |
BIN
docs/physical_motion_fix_screenshots/item_settled_in_d_cage.png
Normal file
|
After Width: | Height: | Size: 228 KiB |
BIN
docs/physical_motion_fix_screenshots/no_overlay_overlap.png
Normal file
|
After Width: | Height: | Size: 220 KiB |
BIN
docs/physical_motion_fix_screenshots/route_to_c_chute.png
Normal file
|
After Width: | Height: | Size: 249 KiB |
BIN
docs/physical_motion_fix_screenshots/route_to_d_chute.png
Normal file
|
After Width: | Height: | Size: 249 KiB |
BIN
docs/physical_motion_fix_screenshots/stl_models_visible.png
Normal file
|
After Width: | Height: | Size: 191 KiB |
BIN
docs/visual_qa_screenshots/after_cv_overlay_no_overlap.png
Normal file
|
After Width: | Height: | Size: 201 KiB |
BIN
docs/visual_qa_screenshots/after_demo_complete_8_cases.png
Normal file
|
After Width: | Height: | Size: 192 KiB |
BIN
docs/visual_qa_screenshots/after_details_page.png
Normal file
|
After Width: | Height: | Size: 2.2 MiB |
BIN
docs/visual_qa_screenshots/after_running_item_on_belt.png
Normal file
|
After Width: | Height: | Size: 154 KiB |
BIN
docs/visual_qa_screenshots/before_initial_production.png
Normal file
|
After Width: | Height: | Size: 192 KiB |
133
src/components/ThreeD/PhysicalPlaybackItem.tsx
Normal file
@@ -0,0 +1,133 @@
|
||||
import { Suspense, useMemo } from 'react';
|
||||
import { getPhysicalItemPose } from '../../domain/physicalItemMotion';
|
||||
import { getModelAsset } from '../../data/modelAssets';
|
||||
import { ITEMS } from '../../data/items';
|
||||
import type { PlaylistCase } from '../../domain/demoPlaylist';
|
||||
import { getRenderedItemDimensions } from '../../domain/physicalLayout';
|
||||
import * as THREE from 'three';
|
||||
import { useLoader } from '@react-three/fiber';
|
||||
import { STLLoader } from 'three/examples/jsm/loaders/STLLoader.js';
|
||||
|
||||
const COLORS = {
|
||||
B: '#16a34a',
|
||||
C: '#ea580c',
|
||||
D: '#7c3aed',
|
||||
sensorAccent: '#3b82f6',
|
||||
};
|
||||
|
||||
const ITEM_MATERIALS: Record<string, { color: string; roughness: number; metalness?: number }> = {
|
||||
'SKU-001': { color: '#b68b58', roughness: 0.82 },
|
||||
'SKU-002': { color: '#e8eef6', roughness: 0.5 },
|
||||
'SKU-004': { color: '#c49a6c', roughness: 0.82 },
|
||||
'SKU-006': { color: '#f8fafc', roughness: 0.42 },
|
||||
'SKU-007': { color: '#7dd3fc', roughness: 0.28 },
|
||||
'SKU-008': { color: '#cbd5e1', roughness: 0.35, metalness: 0.15 },
|
||||
};
|
||||
|
||||
function STLGeometry({ path, scale, color, accentColor, emissiveIntensity, roughness, metalness }: any) {
|
||||
const geometry = useLoader(STLLoader, path) as THREE.BufferGeometry;
|
||||
useMemo(() => {
|
||||
if (geometry) {
|
||||
geometry.center();
|
||||
geometry.computeVertexNormals();
|
||||
}
|
||||
}, [geometry]);
|
||||
return (
|
||||
<group scale={scale}>
|
||||
<mesh geometry={geometry} castShadow>
|
||||
<meshStandardMaterial color={color} emissive={accentColor} emissiveIntensity={emissiveIntensity} roughness={roughness} metalness={metalness} />
|
||||
</mesh>
|
||||
<lineSegments geometry={new THREE.EdgesGeometry(geometry, 35)}>
|
||||
<lineBasicMaterial color={accentColor} transparent opacity={0.28} />
|
||||
</lineSegments>
|
||||
</group>
|
||||
);
|
||||
}
|
||||
|
||||
function FallbackPrimitive({ type, color, accentColor, emissiveIntensity, roughness, metalness, w, h, d }: any) {
|
||||
if (type === 'cylinder' || type === 'sphere') {
|
||||
return (
|
||||
<group>
|
||||
<mesh castShadow>
|
||||
<cylinderGeometry args={[Math.max(w, d) / 2, Math.max(w, d) / 2, h, 16]} />
|
||||
<meshStandardMaterial color={color} emissive={accentColor} emissiveIntensity={emissiveIntensity} roughness={roughness} metalness={metalness} />
|
||||
</mesh>
|
||||
</group>
|
||||
);
|
||||
}
|
||||
return (
|
||||
<group>
|
||||
<mesh castShadow>
|
||||
<boxGeometry args={[w, h, d]} />
|
||||
<meshStandardMaterial color={color} emissive={accentColor} emissiveIntensity={emissiveIntensity} roughness={roughness} metalness={metalness} />
|
||||
</mesh>
|
||||
<lineSegments>
|
||||
<edgesGeometry args={[new THREE.BoxGeometry(w, h, d), 35]} />
|
||||
<lineBasicMaterial color={accentColor} transparent opacity={0.35} />
|
||||
</lineSegments>
|
||||
</group>
|
||||
);
|
||||
}
|
||||
|
||||
export function PhysicalPlaybackItem({
|
||||
caseData,
|
||||
elapsedMs,
|
||||
slotIndex = 0
|
||||
}: {
|
||||
caseData: PlaylistCase;
|
||||
elapsedMs: number;
|
||||
slotIndex?: number;
|
||||
}) {
|
||||
const itemId = caseData.itemId.replace('-LC', '');
|
||||
const itemData = useMemo(() => ITEMS.find(i => i.id === itemId) ?? ITEMS[0], [itemId]);
|
||||
const asset = getModelAsset(itemId);
|
||||
|
||||
const dims = getRenderedItemDimensions(itemData.dimensionsMm);
|
||||
|
||||
const pose = getPhysicalItemPose({
|
||||
caseId: caseData.id,
|
||||
slotIndex,
|
||||
dimensionsMm: itemData.dimensionsMm,
|
||||
targetCategory: caseData.expectedCategory,
|
||||
elapsedMs
|
||||
});
|
||||
|
||||
const { position, rotation, isSettled, phase } = pose;
|
||||
const isRouting = phase === 'routing';
|
||||
|
||||
// Do not render B items that are settled (they exit the factory)
|
||||
if (isSettled && caseData.expectedCategory === 'B') {
|
||||
return null;
|
||||
}
|
||||
|
||||
// Before spawn, don't show
|
||||
if (elapsedMs < 0) return null;
|
||||
|
||||
const accentColor = COLORS[caseData.expectedCategory] ?? COLORS.sensorAccent;
|
||||
const material = ITEM_MATERIALS[itemId] ?? { color: '#d8c3a5', roughness: 0.75 };
|
||||
const emissiveIntensity = isRouting ? 0.16 : 0.03;
|
||||
|
||||
const useSTL = asset?.loaderType === 'stl' && asset?.frontendAssetPath;
|
||||
const stlPath = asset?.frontendAssetPath ?? '';
|
||||
const fallbackType = asset?.fallbackPrimitive ?? 'box';
|
||||
|
||||
return (
|
||||
<group position={position} rotation={rotation}>
|
||||
{useSTL ? (
|
||||
<Suspense fallback={<FallbackPrimitive type={fallbackType} color={material.color} accentColor={accentColor} emissiveIntensity={emissiveIntensity} roughness={material.roughness} metalness={material.metalness ?? 0.05} w={dims.width} h={dims.height} d={dims.depth} />}>
|
||||
<STLGeometry path={stlPath} scale={[0.001, 0.001, 0.001]} color={material.color} accentColor={accentColor} emissiveIntensity={emissiveIntensity} roughness={material.roughness} metalness={material.metalness ?? 0.05} />
|
||||
</Suspense>
|
||||
) : (
|
||||
<FallbackPrimitive type={fallbackType} color={material.color} accentColor={accentColor} emissiveIntensity={emissiveIntensity} roughness={material.roughness} metalness={material.metalness ?? 0.05} w={dims.width} h={dims.height} d={dims.depth} />
|
||||
)}
|
||||
|
||||
{/* Shadow on belt (only if on belt or chute) */}
|
||||
{!isSettled && (
|
||||
<mesh position={[0, -dims.height / 2 + 0.001, 0]} rotation={[-Math.PI / 2, 0, 0]}>
|
||||
<circleGeometry args={[Math.max(dims.width, dims.depth) / 2 + 0.01, 16]} />
|
||||
<meshStandardMaterial color="#475569" transparent opacity={0.15} />
|
||||
</mesh>
|
||||
)}
|
||||
</group>
|
||||
);
|
||||
}
|
||||
@@ -16,12 +16,16 @@ import * as THREE from 'three';
|
||||
import { STLLoader } from 'three/examples/jsm/loaders/STLLoader.js';
|
||||
import type { ContinuousPlaybackState, CasePhase } from '../../domain/continuousPlayback';
|
||||
import { isDetectionActive, isRoutingActive, getPhaseProgress } from '../../domain/continuousPlayback';
|
||||
import { getItemPosition, isItemVisible, getActiveRoute, getConveyorSpeedFactor } from '../../domain/conveyorPath';
|
||||
import { getConveyorSpeedFactor } from '../../domain/conveyorPath';
|
||||
import { getPhysicalItemPose } from '../../domain/physicalItemMotion';
|
||||
import { shouldShowBoundingBox, shouldShowScanEffect, shouldShowShapeOutline, shouldHighlightCamera } from '../../domain/inspectionViewModel';
|
||||
import { getMeasurementData, shouldShowLaserBeam, shouldShowStepperPulse, shouldShowPointCloud, shouldShowActuator } from '../../domain/measurementSystem';
|
||||
import { getModelAsset } from '../../data/modelAssets';
|
||||
import { ITEMS } from '../../data/items';
|
||||
import type { Category } from '../../domain/types';
|
||||
import { PhysicalPlaybackItem } from './PhysicalPlaybackItem';
|
||||
import { DEMO_PLAYLIST, PLAYLIST_LENGTH } from '../../domain/demoPlaylist';
|
||||
import { CASE_DURATION_MS } from '../../domain/continuousPlayback';
|
||||
import {
|
||||
getCameraConfig,
|
||||
smoothCameraTransition,
|
||||
@@ -86,6 +90,19 @@ const COLORS = {
|
||||
itemShadow: '#3a4a5a', // Contact shadow
|
||||
};
|
||||
|
||||
const ITEM_MATERIALS: Record<string, { color: string; roughness: number; metalness?: number }> = {
|
||||
'SKU-001': { color: '#b68b58', roughness: 0.82 }, // cardboard box
|
||||
'SKU-002': { color: '#e8eef6', roughness: 0.5 }, // lunchbox plastic
|
||||
'SKU-004': { color: '#c49a6c', roughness: 0.82 }, // oversized cardboard
|
||||
'SKU-006': { color: '#f8fafc', roughness: 0.42 }, // ceramic plate
|
||||
'SKU-007': { color: '#7dd3fc', roughness: 0.28 }, // bottle plastic
|
||||
'SKU-008': { color: '#cbd5e1', roughness: 0.35, metalness: 0.15 }, // cylinder
|
||||
};
|
||||
|
||||
function getItemMaterial(itemId: string) {
|
||||
return ITEM_MATERIALS[itemId] ?? { color: '#d8c3a5', roughness: 0.75 };
|
||||
}
|
||||
|
||||
// Physical layout constants
|
||||
const BELT_Y = BELT_TOP_Y; // 0.7m - top of belt where items ride
|
||||
const BELT_UNDERSIDE_Y = BELT_TOP_Y - BELT_THICKNESS_M; // 0.685m
|
||||
@@ -113,8 +130,15 @@ function CinematicCameraController({
|
||||
// Get item position for camera following
|
||||
const itemPosition = useMemo(() => {
|
||||
if (playback.status === 'idle') return null;
|
||||
const pos = getItemPosition(playback);
|
||||
return [pos.x, pos.y, pos.z] as [number, number, number];
|
||||
const currentItemElapsed = playback.totalElapsedMs - (playback.currentCaseIndex * CASE_DURATION_MS);
|
||||
const pose = getPhysicalItemPose({
|
||||
caseId: playback.currentCase.id,
|
||||
dimensionsMm: { width: 300, depth: 200, height: 200 }, // rough approx for camera target
|
||||
targetCategory: playback.targetCategory,
|
||||
elapsedMs: currentItemElapsed,
|
||||
slotIndex: playback.currentCaseIndex
|
||||
});
|
||||
return pose.position;
|
||||
}, [playback]);
|
||||
|
||||
useFrame(() => {
|
||||
@@ -1076,12 +1100,18 @@ function STLGeometry({
|
||||
path,
|
||||
scale,
|
||||
color,
|
||||
accentColor,
|
||||
emissiveIntensity,
|
||||
roughness,
|
||||
metalness,
|
||||
}: {
|
||||
path: string;
|
||||
scale: [number, number, number];
|
||||
color: string;
|
||||
accentColor: string;
|
||||
emissiveIntensity: number;
|
||||
roughness: number;
|
||||
metalness: number;
|
||||
}) {
|
||||
const geometry = useLoader(STLLoader, path);
|
||||
|
||||
@@ -1094,15 +1124,20 @@ function STLGeometry({
|
||||
}, [geometry]);
|
||||
|
||||
return (
|
||||
<mesh geometry={geometry} scale={scale} castShadow>
|
||||
<meshStandardMaterial
|
||||
color={color}
|
||||
emissive={color}
|
||||
emissiveIntensity={emissiveIntensity * 0.5}
|
||||
roughness={0.5}
|
||||
metalness={0.1}
|
||||
/>
|
||||
</mesh>
|
||||
<group scale={scale}>
|
||||
<mesh geometry={geometry} castShadow>
|
||||
<meshStandardMaterial
|
||||
color={color}
|
||||
emissive={accentColor}
|
||||
emissiveIntensity={emissiveIntensity}
|
||||
roughness={roughness}
|
||||
metalness={metalness}
|
||||
/>
|
||||
</mesh>
|
||||
<lineSegments geometry={new THREE.EdgesGeometry(geometry, 35)}>
|
||||
<lineBasicMaterial color={accentColor} transparent opacity={0.28} />
|
||||
</lineSegments>
|
||||
</group>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1110,12 +1145,18 @@ function STLGeometry({
|
||||
function FallbackPrimitive({
|
||||
type,
|
||||
color,
|
||||
accentColor,
|
||||
emissiveIntensity,
|
||||
roughness,
|
||||
metalness,
|
||||
w, h, d,
|
||||
}: {
|
||||
type: 'box' | 'cylinder' | 'sphere';
|
||||
color: string;
|
||||
accentColor: string;
|
||||
emissiveIntensity: number;
|
||||
roughness: number;
|
||||
metalness: number;
|
||||
w: number; h: number; d: number;
|
||||
}) {
|
||||
if (type === 'cylinder' || type === 'sphere') {
|
||||
@@ -1124,141 +1165,32 @@ function FallbackPrimitive({
|
||||
<cylinderGeometry args={[Math.max(w, d) / 2, Math.max(w, d) / 2, h, 16]} />
|
||||
<meshStandardMaterial
|
||||
color={color}
|
||||
emissive={color}
|
||||
emissiveIntensity={emissiveIntensity * 0.5}
|
||||
roughness={0.5}
|
||||
metalness={0.1}
|
||||
emissive={accentColor}
|
||||
emissiveIntensity={emissiveIntensity}
|
||||
roughness={roughness}
|
||||
metalness={metalness}
|
||||
/>
|
||||
</mesh>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<mesh castShadow>
|
||||
<boxGeometry args={[w, h, d]} />
|
||||
<meshStandardMaterial
|
||||
color={color}
|
||||
emissive={color}
|
||||
emissiveIntensity={emissiveIntensity * 0.5}
|
||||
roughness={0.5}
|
||||
metalness={0.1}
|
||||
/>
|
||||
</mesh>
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Animated item based on playback state.
|
||||
* Uses real STL models with true physical scale (1 unit = 1 meter).
|
||||
* Item sits ON the belt surface (bottom of item at BELT_TOP_Y).
|
||||
*/
|
||||
function PlaybackItem({ playback }: { playback: ContinuousPlaybackState }) {
|
||||
const visible = isItemVisible(playback);
|
||||
const category = playback.targetCategory;
|
||||
const phase = playback.currentPhase;
|
||||
|
||||
const currentCase = playback.currentCase;
|
||||
const itemId = currentCase.itemId.replace('-LC', '');
|
||||
const itemData = useMemo(() => {
|
||||
try {
|
||||
return ITEMS.find(i => i.id === itemId) ?? ITEMS[0];
|
||||
} catch {
|
||||
return ITEMS[0];
|
||||
}
|
||||
}, [itemId]);
|
||||
|
||||
const asset = getModelAsset(itemId);
|
||||
const isRound = itemData.roundness >= 0.7 || asset?.fallbackPrimitive === 'cylinder';
|
||||
|
||||
// Get rendered dimensions in meters (true physical scale)
|
||||
const dims = getRenderedItemDimensions(itemData.dimensionsMm);
|
||||
const w = dims.width;
|
||||
const d = dims.depth;
|
||||
const h = dims.height;
|
||||
|
||||
// Calculate Y position so item sits ON belt
|
||||
const itemCenterY = getItemYOnBelt(h);
|
||||
|
||||
// Get XZ position from playback
|
||||
const basePosition = getItemPosition(playback, h);
|
||||
const pos: [number, number, number] = [basePosition.x, itemCenterY, basePosition.z];
|
||||
|
||||
const colors: Record<Category, string> = {
|
||||
B: COLORS.routeB,
|
||||
C: COLORS.routeC,
|
||||
D: COLORS.routeD,
|
||||
};
|
||||
const color = category ? colors[category] : COLORS.sensorAccent;
|
||||
const isRouting = isRoutingActive(playback);
|
||||
const emissiveIntensity = isRouting ? 0.3 : 0.1;
|
||||
|
||||
const showBBox = shouldShowBoundingBox(phase);
|
||||
const showShape = shouldShowShapeOutline(phase);
|
||||
|
||||
if (!visible) return null;
|
||||
|
||||
// Determine if we should use STL
|
||||
const useSTL = asset?.loaderType === 'stl' && asset?.frontendAssetPath;
|
||||
const stlPath = asset?.frontendAssetPath ?? '';
|
||||
const fallbackPrimitive = asset?.fallbackPrimitive ?? 'box';
|
||||
|
||||
// Scale for STL models (STL files are in mm, need to convert to meters)
|
||||
const stlScale: [number, number, number] = [0.001, 0.001, 0.001];
|
||||
|
||||
const visualScale = Math.max(w, d, h);
|
||||
|
||||
return (
|
||||
<>
|
||||
<group position={pos}>
|
||||
{useSTL ? (
|
||||
<Suspense fallback={
|
||||
<FallbackPrimitive
|
||||
type={fallbackPrimitive}
|
||||
color={color}
|
||||
emissiveIntensity={emissiveIntensity}
|
||||
w={w} h={h} d={d}
|
||||
/>
|
||||
}>
|
||||
<STLGeometry
|
||||
path={stlPath}
|
||||
scale={stlScale}
|
||||
color={color}
|
||||
emissiveIntensity={emissiveIntensity}
|
||||
/>
|
||||
</Suspense>
|
||||
) : (
|
||||
<FallbackPrimitive
|
||||
type={isRound ? 'cylinder' : fallbackPrimitive}
|
||||
color={color}
|
||||
emissiveIntensity={emissiveIntensity}
|
||||
w={w} h={h} d={d}
|
||||
/>
|
||||
)}
|
||||
|
||||
{/* Shadow on belt */}
|
||||
<mesh position={[0, -h / 2 + 0.001, 0]} rotation={[-Math.PI / 2, 0, 0]}>
|
||||
<circleGeometry args={[Math.max(w, d) / 2 + 0.01, 16]} />
|
||||
<meshStandardMaterial color="#475569" transparent opacity={0.1} />
|
||||
</mesh>
|
||||
</group>
|
||||
|
||||
{/* Bounding box during measurement */}
|
||||
<BoundingBoxVisual
|
||||
position={pos}
|
||||
scale={visualScale}
|
||||
visible={showBBox}
|
||||
isRound={isRound}
|
||||
/>
|
||||
|
||||
{/* Shape outline during classification */}
|
||||
<ShapeOutline
|
||||
position={pos}
|
||||
scale={visualScale}
|
||||
visible={showShape}
|
||||
isRound={isRound}
|
||||
category={category}
|
||||
/>
|
||||
</>
|
||||
<group>
|
||||
<mesh castShadow>
|
||||
<boxGeometry args={[w, h, d]} />
|
||||
<meshStandardMaterial
|
||||
color={color}
|
||||
emissive={accentColor}
|
||||
emissiveIntensity={emissiveIntensity}
|
||||
roughness={roughness}
|
||||
metalness={metalness}
|
||||
/>
|
||||
</mesh>
|
||||
<lineSegments>
|
||||
<edgesGeometry args={[new THREE.BoxGeometry(w, h, d), 35]} />
|
||||
<lineBasicMaterial color={accentColor} transparent opacity={0.35} />
|
||||
</lineSegments>
|
||||
</group>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1275,7 +1207,6 @@ function ContinuousScene({
|
||||
viewportType: ViewportType;
|
||||
}) {
|
||||
const speedFactor = getConveyorSpeedFactor(playback);
|
||||
const activeRoute = getActiveRoute(playback);
|
||||
const category = playback.targetCategory;
|
||||
const phase = playback.currentPhase;
|
||||
|
||||
@@ -1289,18 +1220,46 @@ function ContinuousScene({
|
||||
const showCloud = shouldShowPointCloud(phase);
|
||||
const showActuator = shouldShowActuator(phase);
|
||||
|
||||
// Get item position for measurement visualization
|
||||
const itemPosition = getItemPosition(playback);
|
||||
const itemPos: [number, number, number] = [itemPosition.x, itemPosition.y, itemPosition.z];
|
||||
|
||||
// Get item data for point cloud (use true physical scale)
|
||||
const currentCase = playback.currentCase;
|
||||
// Compute all physical items based on elapsed time to keep them in roll-cages
|
||||
const { totalElapsedMs, currentCase, currentCaseIndex } = playback;
|
||||
const casesSpawned = Math.floor(totalElapsedMs / CASE_DURATION_MS) + 1;
|
||||
const startIndex = Math.max(0, casesSpawned - 20); // Keep last 20 items
|
||||
|
||||
const sceneItems = useMemo(() => {
|
||||
const items = [];
|
||||
for (let i = startIndex; i < casesSpawned; i++) {
|
||||
const playlistIndex = i % PLAYLIST_LENGTH;
|
||||
const caseData = DEMO_PLAYLIST[playlistIndex];
|
||||
const elapsedMs = totalElapsedMs - (i * CASE_DURATION_MS);
|
||||
items.push({
|
||||
id: `item-${i}-${caseData.id}`,
|
||||
slotIndex: i,
|
||||
caseData,
|
||||
elapsedMs,
|
||||
});
|
||||
}
|
||||
return items;
|
||||
}, [totalElapsedMs, casesSpawned, startIndex]);
|
||||
|
||||
// Get item data for the current case
|
||||
const itemId = currentCase.itemId.replace('-LC', '');
|
||||
const itemData = ITEMS.find(i => i.id === itemId) ?? ITEMS[0];
|
||||
const isRound = itemData.roundness >= 0.7;
|
||||
const dims = getRenderedItemDimensions(itemData.dimensionsMm);
|
||||
const itemScale = Math.max(dims.width, dims.depth, dims.height);
|
||||
|
||||
// Get item position for measurement visualization using physical model
|
||||
const currentItemElapsed = totalElapsedMs - (currentCaseIndex * CASE_DURATION_MS);
|
||||
const currentPose = getPhysicalItemPose({
|
||||
caseId: currentCase.id,
|
||||
dimensionsMm: itemData.dimensionsMm,
|
||||
targetCategory: category,
|
||||
elapsedMs: currentItemElapsed,
|
||||
slotIndex: currentCaseIndex
|
||||
});
|
||||
const itemPos = currentPose.position;
|
||||
const activeRoute = currentPose.activeRoute;
|
||||
|
||||
// Motion trail visibility - show during movement phases
|
||||
const showMotionTrail = ['move_to_detection', 'routing', 'exit'].includes(phase) &&
|
||||
playback.status === 'running';
|
||||
@@ -1413,7 +1372,7 @@ function ContinuousScene({
|
||||
<StereoCameraLenses active={cameraHighlight} itemPosition={itemPos} />
|
||||
|
||||
{/* Laser beam for height measurement */}
|
||||
<LaserBeam active={showLaser} itemY={itemPosition.y} />
|
||||
<LaserBeam active={showLaser} itemY={itemPos[1]} />
|
||||
|
||||
{/* Point cloud for stereo analysis */}
|
||||
<PointCloud
|
||||
@@ -1438,8 +1397,30 @@ function ContinuousScene({
|
||||
{/* Route arrows on belt surface */}
|
||||
<RouteArrows activeRoute={activeRoute} />
|
||||
|
||||
{/* Item riding ON the belt */}
|
||||
<PlaybackItem playback={playback} />
|
||||
{/* Physically simulated items */}
|
||||
{sceneItems.map(item => (
|
||||
<PhysicalPlaybackItem
|
||||
key={item.id}
|
||||
caseData={item.caseData}
|
||||
elapsedMs={item.elapsedMs}
|
||||
slotIndex={item.slotIndex}
|
||||
/>
|
||||
))}
|
||||
|
||||
{/* Outline/BBox for the CURRENT item only */}
|
||||
<BoundingBoxVisual
|
||||
position={itemPos}
|
||||
scale={itemScale}
|
||||
visible={shouldShowBoundingBox(phase)}
|
||||
isRound={isRound}
|
||||
/>
|
||||
<ShapeOutline
|
||||
position={itemPos}
|
||||
scale={itemScale}
|
||||
visible={shouldShowShapeOutline(phase)}
|
||||
isRound={isRound}
|
||||
category={category}
|
||||
/>
|
||||
|
||||
{/* Motion trail for visual movement feedback */}
|
||||
<MotionTrail
|
||||
|
||||
@@ -33,14 +33,14 @@ export interface PhaseConfig {
|
||||
* Distances are in meters, speed is 1 m/s, so duration = distance * 1000 ms.
|
||||
*/
|
||||
export const CASE_PHASES: PhaseConfig[] = [
|
||||
{ phase: 'spawn', durationMs: 500, label: 'Spawn at A' },
|
||||
{ phase: 'move_to_detection', durationMs: 1200, label: 'Moving to camera' }, // ~1.2m at 1m/s
|
||||
{ phase: 'spawn', durationMs: 300, label: 'Spawn at A' },
|
||||
{ phase: 'move_to_detection', durationMs: 2500, label: 'Moving to camera' }, // 2.5m at 1m/s
|
||||
{ phase: 'detection', durationMs: 600, label: 'CV Detection' },
|
||||
{ phase: 'measurement', durationMs: 400, label: 'Laser measurement' },
|
||||
{ phase: 'classification', durationMs: 500, label: 'Classification' },
|
||||
{ phase: 'command_sent', durationMs: 300, label: 'Command sent' },
|
||||
{ phase: 'routing', durationMs: 1500, label: 'Routing to zone' }, // ~1.5m to B/C/D
|
||||
{ phase: 'exit', durationMs: 800, label: 'Exit to zone' },
|
||||
{ phase: 'measurement', durationMs: 1000, label: 'Laser measurement' }, // 1.0m at 1m/s
|
||||
{ phase: 'classification', durationMs: 1000, label: 'Classification' }, // 1.0m at 1m/s
|
||||
{ phase: 'command_sent', durationMs: 1000, label: 'Command sent' }, // 1.0m at 1m/s
|
||||
{ phase: 'routing', durationMs: 2500, label: 'Routing to zone' }, // 2.5m B-route at 1m/s
|
||||
{ phase: 'exit', durationMs: 400, label: 'Exit to zone' },
|
||||
{ phase: 'clear_gap', durationMs: 500, label: 'Clear gap' },
|
||||
];
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@ export interface ItemPosition3D {
|
||||
}
|
||||
|
||||
/** Calculate item X position based on phase and progress. */
|
||||
function getItemXForPhase(phase: CasePhase, phaseProgress: number): number {
|
||||
function getItemXForPhase(phase: CasePhase, phaseProgress: number, category: Category | null): number {
|
||||
const { spawnX, cameraX, laserX, gateX, zoneBX } = CONVEYOR_POSITIONS;
|
||||
|
||||
switch (phase) {
|
||||
@@ -52,13 +52,16 @@ function getItemXForPhase(phase: CasePhase, phaseProgress: number): number {
|
||||
return midPoint + (gateX - midPoint) * phaseProgress;
|
||||
|
||||
case 'routing':
|
||||
if (category === 'B') {
|
||||
return gateX + (zoneBX - gateX) * phaseProgress;
|
||||
}
|
||||
return gateX;
|
||||
|
||||
case 'exit':
|
||||
return gateX + (zoneBX - gateX) * phaseProgress * 0.5;
|
||||
return category === 'B' ? zoneBX : gateX;
|
||||
|
||||
case 'clear_gap':
|
||||
return gateX + (zoneBX - gateX) * 0.5 + (zoneBX - gateX) * 0.5 * phaseProgress;
|
||||
return category === 'B' ? zoneBX : gateX;
|
||||
|
||||
default:
|
||||
return gateX;
|
||||
@@ -90,7 +93,7 @@ function getItemZForPhase(phase: CasePhase, phaseProgress: number, category: Cat
|
||||
export function getItemPosition(state: ContinuousPlaybackState, itemVisualHeight: number = 0.15): ItemPosition3D {
|
||||
const phaseProgress = getPhaseProgress(state);
|
||||
|
||||
const x = getItemXForPhase(state.currentPhase, phaseProgress);
|
||||
const x = getItemXForPhase(state.currentPhase, phaseProgress, state.targetCategory);
|
||||
const z = getItemZForPhase(state.currentPhase, phaseProgress, state.targetCategory);
|
||||
// Item sits ON the belt: belt top + half item height
|
||||
const y = getItemYOnBelt(itemVisualHeight);
|
||||
@@ -120,11 +123,15 @@ export function getActiveRoute(state: ContinuousPlaybackState): Category | null
|
||||
export function getConveyorSpeedFactor(state: ContinuousPlaybackState): number {
|
||||
if (state.status !== 'running') return 0;
|
||||
|
||||
if (state.currentPhase === 'detection' || state.currentPhase === 'classification') {
|
||||
return 0.3;
|
||||
}
|
||||
|
||||
return 1.0;
|
||||
const movingPhases: CasePhase[] = [
|
||||
'move_to_detection',
|
||||
'measurement',
|
||||
'classification',
|
||||
'command_sent',
|
||||
'routing',
|
||||
];
|
||||
|
||||
return movingPhases.includes(state.currentPhase) ? 1.0 : 0;
|
||||
}
|
||||
|
||||
/** Calculate distance traveled in meters from elapsed time. */
|
||||
|
||||
47
src/domain/physicalItemMotion.test.ts
Normal file
@@ -0,0 +1,47 @@
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { getPhysicalItemPose } from './physicalItemMotion';
|
||||
import { ZONES, BELT_TOP_Y, CONVEYOR_SPEED_MPS } from './physicalLayout';
|
||||
|
||||
describe('physicalItemMotion', () => {
|
||||
const defaultInput = {
|
||||
caseId: 'test_case',
|
||||
dimensionsMm: { width: 300, depth: 200, height: 200 }, // 0.2m height
|
||||
targetCategory: 'B' as any,
|
||||
elapsedMs: 0,
|
||||
slotIndex: 0
|
||||
};
|
||||
|
||||
it('starts at spawn point on the belt', () => {
|
||||
const pose = getPhysicalItemPose({ ...defaultInput, elapsedMs: 0 });
|
||||
expect(pose.surface).toBe('belt');
|
||||
expect(pose.position[0]).toBe(ZONES.A.x);
|
||||
expect(pose.position[1]).toBe(BELT_TOP_Y + 0.1); // 0.7 + 0.1
|
||||
expect(pose.phase).toBe('feed');
|
||||
});
|
||||
|
||||
it('moves at 1 m/s along the belt during move_to_detection', () => {
|
||||
// move_to_detection starts at 300ms
|
||||
const pose = getPhysicalItemPose({ ...defaultInput, elapsedMs: 1300 });
|
||||
const distance = 1000 / 1000 * CONVEYOR_SPEED_MPS; // 1.0 m
|
||||
expect(pose.position[0]).toBeCloseTo(ZONES.A.x + distance, 2);
|
||||
expect(pose.surface).toBe('belt');
|
||||
});
|
||||
|
||||
it('stays in C cage for category C after routing is complete', () => {
|
||||
// 9800ms is the total duration
|
||||
const pose = getPhysicalItemPose({ ...defaultInput, targetCategory: 'C', elapsedMs: 10000 });
|
||||
expect(pose.isSettled).toBe(true);
|
||||
expect(pose.surface).toBe('c_cage');
|
||||
expect(pose.position[0]).toBeCloseTo(ZONES.C.x - 0.2, 1);
|
||||
expect(pose.position[2]).toBeCloseTo(ZONES.C.z - 0.1, 1);
|
||||
expect(pose.position[1]).toBeCloseTo(0.1 + 0.1, 2); // floor + half height
|
||||
});
|
||||
|
||||
it('stays in D cage for category D after routing is complete', () => {
|
||||
const pose = getPhysicalItemPose({ ...defaultInput, targetCategory: 'D', elapsedMs: 10000 });
|
||||
expect(pose.isSettled).toBe(true);
|
||||
expect(pose.surface).toBe('d_cage');
|
||||
expect(pose.position[0]).toBeCloseTo(ZONES.D.x - 0.2, 1);
|
||||
expect(pose.position[2]).toBeCloseTo(ZONES.D.z - 0.1, 1);
|
||||
});
|
||||
});
|
||||
148
src/domain/physicalItemMotion.ts
Normal file
@@ -0,0 +1,148 @@
|
||||
import { Category, DimensionsMm } from './types';
|
||||
import {
|
||||
CONVEYOR_SPEED_MPS,
|
||||
BELT_TOP_Y,
|
||||
ZONES,
|
||||
ROLL_CAGE,
|
||||
} from './physicalLayout';
|
||||
import { CASE_PHASES } from './continuousPlayback';
|
||||
|
||||
export type SurfaceType = "belt" | "chute_c" | "chute_d" | "b_line" | "c_cage" | "d_cage";
|
||||
export type MotionPhase = "feed" | "inspection" | "decision" | "routing" | "settled";
|
||||
|
||||
export interface PhysicalItemPose {
|
||||
position: [number, number, number];
|
||||
rotation: [number, number, number];
|
||||
surface: SurfaceType;
|
||||
phase: MotionPhase;
|
||||
isSettled: boolean;
|
||||
activeRoute: 'B' | 'C' | 'D';
|
||||
}
|
||||
|
||||
export interface PoseInput {
|
||||
caseId: string;
|
||||
slotIndex?: number;
|
||||
dimensionsMm: DimensionsMm;
|
||||
targetCategory: Category | null;
|
||||
elapsedMs: number;
|
||||
}
|
||||
|
||||
export function getPhysicalItemPose(input: PoseInput): PhysicalItemPose {
|
||||
const { dimensionsMm, targetCategory, elapsedMs, slotIndex = 0 } = input;
|
||||
const category = targetCategory || 'B';
|
||||
const itemHeightM = dimensionsMm.height / 1000;
|
||||
const beltY = BELT_TOP_Y + itemHeightM / 2;
|
||||
|
||||
// Reconstruct phase start times from CASE_PHASES
|
||||
let cumulativeMs = 0;
|
||||
const starts: Record<string, number> = {};
|
||||
for (const p of CASE_PHASES) {
|
||||
starts[p.phase] = cumulativeMs;
|
||||
cumulativeMs += p.durationMs;
|
||||
}
|
||||
const totalDuration = cumulativeMs; // 9800
|
||||
|
||||
// Fallback safe defaults
|
||||
let posX: number = ZONES.A.x;
|
||||
let posY = beltY;
|
||||
let posZ = 0;
|
||||
let rotY = 0;
|
||||
let surface: SurfaceType = 'belt';
|
||||
let phase: MotionPhase = 'feed';
|
||||
let isSettled = false;
|
||||
|
||||
if (elapsedMs <= starts['move_to_detection']) {
|
||||
phase = 'feed';
|
||||
posX = ZONES.A.x;
|
||||
} else if (elapsedMs <= starts['detection']) {
|
||||
phase = 'feed';
|
||||
const dt = (elapsedMs - starts['move_to_detection']) / 1000;
|
||||
posX = ZONES.A.x + dt * CONVEYOR_SPEED_MPS;
|
||||
} else if (elapsedMs <= starts['measurement']) {
|
||||
phase = 'inspection';
|
||||
posX = ZONES.CAMERA.x;
|
||||
} else if (elapsedMs <= starts['routing']) {
|
||||
phase = 'decision';
|
||||
// Total movement time in this block: measurement (1000), classification (1000), command_sent (1000)
|
||||
// Distance from CAMERA to GATE = 1.5 - (-1.5) = 3.0m at 1m/s = 3000ms.
|
||||
const dt = (elapsedMs - starts['measurement']) / 1000;
|
||||
posX = ZONES.CAMERA.x + dt * CONVEYOR_SPEED_MPS;
|
||||
} else {
|
||||
// Routing, exit, clear_gap, or settled
|
||||
const routingStart = starts['routing'];
|
||||
const routingDuration = CASE_PHASES.find(p => p.phase === 'routing')!.durationMs; // 2500
|
||||
|
||||
if (category === 'B') {
|
||||
const dt = (elapsedMs - routingStart) / 1000;
|
||||
posX = Math.min(ZONES.GATE.x + dt * CONVEYOR_SPEED_MPS, ZONES.B.x);
|
||||
surface = 'b_line';
|
||||
if (posX >= ZONES.B.x) {
|
||||
phase = 'settled';
|
||||
isSettled = elapsedMs >= totalDuration;
|
||||
} else {
|
||||
phase = 'routing';
|
||||
}
|
||||
} else {
|
||||
// C or D
|
||||
const isC = category === 'C';
|
||||
const targetZ = isC ? ZONES.C.z : ZONES.D.z;
|
||||
const targetX = isC ? ZONES.C.x : ZONES.D.x;
|
||||
|
||||
const t = Math.max(0, Math.min((elapsedMs - routingStart) / routingDuration, 1.0));
|
||||
|
||||
if (t < 1.0) {
|
||||
phase = 'routing';
|
||||
surface = isC ? 'chute_c' : 'chute_d';
|
||||
|
||||
// Slide down the chute
|
||||
posX = ZONES.GATE.x + (targetX - ZONES.GATE.x) * t;
|
||||
posZ = targetZ * t;
|
||||
|
||||
// Y drops from belt to cage floor
|
||||
const cageFloorY = 0.1; // approximate from ROLL_CAGE
|
||||
const targetY = cageFloorY + itemHeightM / 2;
|
||||
posY = beltY - (beltY - targetY) * (t * t); // slight ease-in
|
||||
|
||||
// Slight rotation to face chute
|
||||
const angle = Math.atan2(targetZ, targetX - ZONES.GATE.x);
|
||||
rotY = -angle * t;
|
||||
} else {
|
||||
phase = 'settled';
|
||||
isSettled = elapsedMs >= totalDuration;
|
||||
surface = isC ? 'c_cage' : 'd_cage';
|
||||
|
||||
// Inside cage. Slot index offset to prevent perfect overlap.
|
||||
const cageCenterX = targetX;
|
||||
const cageCenterZ = targetZ;
|
||||
const cageFloorY = 0.1;
|
||||
|
||||
// Deterministic offset based on slotIndex
|
||||
// Cage size is 1.2 x 0.8
|
||||
const offsetX = ((slotIndex % 3) - 1) * 0.2;
|
||||
const offsetZ = ((Math.floor(slotIndex / 3) % 2) - 0.5) * 0.2;
|
||||
|
||||
posX = cageCenterX + offsetX;
|
||||
posZ = cageCenterZ + offsetZ;
|
||||
posY = cageFloorY + itemHeightM / 2;
|
||||
|
||||
// Final scattered rotation
|
||||
rotY = (slotIndex * 45) * Math.PI / 180;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// If case is entirely finished, force settled
|
||||
if (elapsedMs >= totalDuration && phase !== 'settled') {
|
||||
isSettled = true;
|
||||
phase = 'settled';
|
||||
}
|
||||
|
||||
return {
|
||||
position: [posX, posY, posZ],
|
||||
rotation: [0, rotY, 0],
|
||||
surface,
|
||||
phase,
|
||||
isSettled,
|
||||
activeRoute: category,
|
||||
};
|
||||
}
|
||||
@@ -2495,11 +2495,11 @@ button:disabled {
|
||||
|
||||
.cv-overlay {
|
||||
position: absolute;
|
||||
bottom: 120px;
|
||||
right: 16px;
|
||||
top: 96px;
|
||||
left: 16px;
|
||||
z-index: 15;
|
||||
width: 220px;
|
||||
max-height: calc(100vh - 320px);
|
||||
max-height: calc(100vh - 220px);
|
||||
overflow-y: auto;
|
||||
border: 1px solid rgba(37, 99, 235, 0.2);
|
||||
border-radius: 10px;
|
||||
|
||||