VRSES:
Training students in essential communication skills with AI and VR.

Background

In 2023, the PUSH embarked on a new project inspired by the explosion of AI in society. Wanting to develop something to benefit the DePaul community, the VRSES project was born. Evolving out of PUSH Studio’s A Just Reality project, but with emphasis on student communication skills, the team began the long task of creating an interactive VR experience.

Challenge 

Soft skills training is an issue across industries, with employers stating that there has never been more need for employees with strong communication and interpersonal skills. Students who do not speak English as their first language, students with social anxiety or other forms of neurodivergence, and students whose learning was negatively affected by the COVID-19 pandemic are all key benefactors of this training. Focusing on the DePaul community, developing a scenario that would be relatable to as many students as possible was a key priority.

On the technical end, creating the pipeline to convert user’s spoken dialogue to a format the LLM can analyze was its own endeavor. Utilizing a variety of tools and emerging technologies, the PUSH team has been able to streamline this process into a user-friendly interaction.

Deliverables:
High-fidelity interactive prototypes

PUSH services:
UI/UX design, User Research, Technology Prototyping

Industry:
Virtual Reality, Training, High-Fidelity Prototypes

PUSH team: 

  • Mit Dave

  • Caroline Rumani

  • Humaira Tasneem

Early stage design: developing dialogue layout

Mid stage design: roleplaying conversation with professor

Approach

The VRSES project began with extensive research during the summer of 2023. A competitive analysis of leading VR training products was compiled, along with information on the analysis and assessment of communication skills. From this, the team began ideating on what this experience would look like. Where does it take place, who are the characters, what specific communication skills are being assessed, and so on.

Once the fall arrived, PUSH began working with interested DePaul professors on how to best align the project with the needs of its students. The decision was made to base the initial scenario on a rhetorical analysis assignment from WRD 103, a course many undergraduates take during their first year.

Continuous testing and demoing with professors and students has been vital to the development of the project.

Solution

The culmination of this project is a Unity-based prototype designed for a classroom setting, featuring a chatbot equipped with natural language processing capabilities and a distinct personality. The prototype has received numerous positive reviews, validating the effectiveness and appeal of the approach. The next steps for the project include finalizing the implementation of scoring and sentiment analysis features, with plans to commence official testing in DePaul University classrooms by fall 2024.


Example avatar from VR experience

Technical pipeline diagram