ForgeAR

This project is a real-time, Edge AI–powered AR assistant built during the Qualcomm EdgeAI Hackathon to explore how on-device intelligence can support field workers in complex, hands-on environments. The system combines computer vision, on-device LLM reasoning, and augmented reality overlays to guide users through step-by-step tasks without relying on cloud connectivity. A mobile client streams live camera frames to a backend over WebSockets, where YOLOv8 performs object and state detection. Detected visual context is then passed to a locally running Llama 3.2B model via AnythingLLM, executing entirely on the Snapdragon NPU for reasoning and step planning. The backend returns structured instructions and bounding box metadata in real time, which the frontend renders as AR overlays, providing visual guidance and contextual instructions directly on the camera feed. The MVP demonstrates a guided Instant Pot workflow, but the architecture is designed to generalize to industrial use cases such as assembly, inspection, maintenance, and diagnostics. The project emphasizes low-latency inference, on-device execution, and system robustness, showcasing how Edge AI can preserve institutional knowledge and support dynamic, real-world workflows in environments where cloud-based solutions are impractical.