AI Agent in VR Space
AI Agent in VR Space
Unity, C#, LLM APIs, RAG , Prompt Engineering, VR
Overview
Developed as a final 6th-semester group project, this VR application features a fully interactive AI agent designed to communicate with students and professors. The agent serves as a virtual guide and mentor, answering questions about the academic program and providing technical insights on VR, AR, and Unity development.
My Role & Core Systems As the dedicated LLM Developer for the team, I was responsible for the AI's technical functionality, architectural pipeline, and behavioral roleplay.
LLM Integration & Persona: Engineered the prompts and context windows to ensure the AI maintained a consistent, helpful persona that fits naturally into an academic VR environment.
RAG Architecture: Designed and implemented a Retrieval-Augmented Generation (RAG) system. This allowed the AI to pull precise, domain-specific data (university curriculum, XR documentation) in real-time without hallucinating facts.
Response Optimization: Streamlined the data pipeline to reduce API latency, ensuring that the communication between the user and the AI agent felt seamless and immersive in a real-time spatial setting.
The Challenge Bridging external AI pipelines with a real-time VR application poses significant latency and accuracy challenges. Implementing the RAG system was a critical technical hurdle: it required structuring a local knowledge base so the AI could rapidly retrieve complex technical information about XR technologies while maintaining a natural, uninterrupted conversational flow.
Overview
Developed as a final 6th-semester group project, this VR application features a fully interactive AI agent designed to communicate with students and professors. The agent serves as a virtual guide and mentor, answering questions about the academic program and providing technical insights on VR, AR, and Unity development.
My Role & Core Systems As the dedicated LLM Developer for the team, I was responsible for the AI's technical functionality, architectural pipeline, and behavioral roleplay.
LLM Integration & Persona: Engineered the prompts and context windows to ensure the AI maintained a consistent, helpful persona that fits naturally into an academic VR environment.
RAG Architecture: Designed and implemented a Retrieval-Augmented Generation (RAG) system. This allowed the AI to pull precise, domain-specific data (university curriculum, XR documentation) in real-time without hallucinating facts.
Response Optimization: Streamlined the data pipeline to reduce API latency, ensuring that the communication between the user and the AI agent felt seamless and immersive in a real-time spatial setting.
The Challenge Bridging external AI pipelines with a real-time VR application poses significant latency and accuracy challenges. Implementing the RAG system was a critical technical hurdle: it required structuring a local knowledge base so the AI could rapidly retrieve complex technical information about XR technologies while maintaining a natural, uninterrupted conversational flow.