Connected virtual learning in physical space

 

Duration

Nov 2021 - Dec 2021 (1 mo)

Team

Chung-Han(Joanne) Huang, Laura Li

Advisor

Dr. Michael Nebeling, Audrey Labrie

Implementation

Lens Studio, Rhinoceros, Tinkcad, iMessage, Photoshop, Illustrator, Figma, Premiere Pro, After Effect

Keywords / Domain

AR, Augmented Reality, Remote learning, Connection, Network, User experience, User interface, XR Prototype, Virtual and physical space

Problem statement

Zoom meeting screenshot (image credit)

People need real interaction and connection in remote learning. Harvard Business Review shows that “the personal networks have shrunk by close to 16% during the pandemic,” A non-negligible demand of real connection is eager to find a solution.

How might we better connect students in an online learning environment but keep the flexibility of learning anywhere?

Persona

 
 

Digital prototype

Solution

  1. Marker-based virtual classroom layout

    -> see all your classmates and what they are doing at a time

  2. Spatial audio for a sense of intimacy

    ->join and hear inspiring conversations anytime

  3. Bridge to the physical environment and users’ familiar devices

    -> import 2D and 3D contents seamlessly

Why AR

We are providing the possibility for users to work in public spaces, so the user could join the class anywhere.

In addition, users might want to use their devices that they are familiar with for note-taking and looking up documents. 

We also want to provide the users the opportunity to engage with the real environment, and import content from the real world more easily.

Physical prototype & takeaway

Takeaway and improvement

  1. Users might be overwhelmed when people keep approaching to talk

    -> indicate user status in their profile tags

  2. Users might want to talk privately or one-on-one meeting

    -> create two kinds of rooms: lounge room and private room

  3. Users might want to import content from their surroundings and devices

    -> allow to import through photogrammetry, uploading files, and screen-sharing

Main interactions

User status

Add content with photogrammetry

Navigate with teleportation

Manipulate 3D content

Invite users to room

Add content with screen sharing

Privacy

We found that capturing fingerprints and palm prints is possible in our project, raising a serious privacy issue.

->

We decided to add a post effect on hands to turn them low-resolution, blurring the biometric identities.

Implementation

  1. 3D modeling: Rhinoceros, Tinkercad

  2. Avatar: iMessage, After Effects

  3. Marker: Photoshop, Illustrator

  4. Interface elements: Figma

  5. Video editing: Premiere Pro, After Effect

  6. AR filter: Lens Studio

  • Image tracking - project the virtual classroom layout

  • Hand-tracking - hand-menu

  • Behavior script - hand menus interaction

  • Tracking surface, Object Controller- manipulate the 3D content

  • Hand segmentation - add post effect on hands for privacy

Future work

  1. Let users customize room layouts and size to provide flexibility

  2. Let users turn into VR if they want to explore the room immersively

    Seeing spaces in third-person perspective provides more possibilities to interact with people.