Sentinel

A cooperative perception prototype that pairs dashcam and top-down footage, prompts a vision-language model for structured scene understanding, and routes hazard events through a graph memory that feeds a driver-facing warning surface.

Indian roads mix unstructured traffic, informal signage, and non-standard hazards. Off-the-shelf perception stacks trained on Western data miss the long tail. The prototype explores whether structured VLM prompting plus a graph memory can capture these edge cases in a reviewable way.

1. 01Pair front-facing dashcam clips with matching top-view segments across a shared timeline.
2. 02Prompt Qwen2.5-VL with a fixed schema to emit structured JSON per frame: actors, hazards, lane state, ambient signals.
3. 03Store detections in Neo4j as event nodes linked to scene, location, and actor nodes.
4. 04Surface high-severity events as driver warnings; capture reviewer feedback back into a verified dataset.

• Designed the JSON schema and prompt scaffolding for structured perception.
• Built the ingestion + FastAPI backend routing frames to the VLM and graph.
• Modeled the Neo4j graph for scenes, hazards, and driver events.
• Wired a minimal warning UI reading from the graph in near real time.

• → End-to-end prototype: video in, structured hazards out, graph-backed.
• → A repeatable review loop that grows a verified hazard dataset over time.