XRTV - Extended Reality Television
An Interactive Learning System for Exploring Overlooked and Unfamiliar Forms of Interaction with Visual Media
Role
Product Designer
Collaborator
Sakdiphat Tanphiphatari - XR Developer
Timeline
March 2025 - May 2025
Problem
Television technology has evolved dramatically since the 1950s, from the first scanners to gesture-controlled displays , yet public understanding of this history hasn't kept pace. Visits to institutions like ACMI reveal that this history is largely presented through static text and images, leaving visitors with limited contextual understanding of how TV technology has evolved or where it's headed. This project addresses that gap by using XR to transform television history into an interactive, embodied learning experience, allowing users to engage with both legacy and emerging TV technologies through interaction rather than passive viewing.
Research & Discovery
We ran secondary research alongside a fieldwork study (7 semi-structured interviews, 24 surveys) to understand how people consume visual media today and how they relate to XR as a learning tool.
Key findings:
Simplicity beats novelty. 75% of participants preferred traditional remote controls over newer input methods, valuing familiarity and ease of use over gesture-based interaction.
XR is known but underused as a learning tool. ~46% of respondents were familiar with XR, and another ~42% were aware of it but had never tried it — awareness didn't translate to educational use, likely due to unclear applications and a steep learning curve.
Portable screens (not TVs) are now the default medium for visual media consumption, and most participants were unaware gesture-controlled TVs existed at all.
How Might We
How might we support people in discovering and interacting with overlooked or unfamiliar forms of visual media?
How might we make learning about interface evolution immersive, fun, and accessible even for XR beginners?
Concept & Design Process
Working from a double-diamond process, we sketched three early directions and converged on one centred on tangible + immersive interaction — letting users physically and virtually interact with TV models spanning past and present, rather than reading about them.
We built two personas (technical vs. non-technical backgrounds) and a storyboard to keep the design grounded in real user behaviour rather than internal assumptions.
The system took shape as two connected prototypes:
ShapesXR — used for UI design and onboarding: guided panels teaching users how retro TVs and gesture-controlled TVs work.
Unity — used for realistic interaction: a mixed-reality build combining a physical miniature retro TV (with a working dial, driven by an ESP32 + breadboard) with a virtual digital twin on Meta Quest 3, plus functioning hand-gesture control for the modern TV.
Notable iterations along the way:
Switched from a cluttered horizontal UI layout to a condensed vertical one after early testing showed it hurt engagement.
Cut a planned third "future TV" model due to time constraints.
Repeatedly resoldered and adjusted hardware after the physical dial kept detaching and lagging behind the virtual output.
Discovered ShapesXR couldn't support video playback or screen-sharing for evaluations — so we split tooling: ShapesXR for UI visualisation, Unity for functional testing.
Evaluation
We tested with 13 survey respondents at the Endeavour Exhibition and ran moderated usability testing with 12 participants using Think Aloud, focused on clarity of instructions, engagement, and user confidence.
What worked:
The physical retro TV's dial felt intuitive and nostalgic, and gesture control — while harder to learn — was seen as fun and highly engaging once understood.
Learning outcomes improved: engagement rose noticeably once users entered the "Learn" mode, and most participants felt hands-on learning was more effective than reading or lectures, especially for children.
The experience was accessible even to first-time XR users.
What didn't:
Text was too small and instructions lacked upfront overview, leaving some users unsure where to start.
The vertical UI layout — chosen to reduce clutter — caused neck discomfort for some users, contradicting our initial assumption.
Users struggled to distinguish physical from virtual controls, and gesture recognition was inconsistent (e.g. swipe vs. push overlap) with no visual feedback confirming a correct gesture.
Broader XR constraints — headset weight, cybersickness, and space requirements — surfaced as real barriers to comfortable use.
Reflection & Next Steps
This project reinforced that novel interaction alone doesn't guarantee usability — even highly engaging gesture controls need clear feedback loops and onboarding to feel confident rather than frustrating. It also validated the core hypothesis: interactive, embodied learning outperformed static formats for building contextual understanding, particularly for younger users.
Future iterations would focus on:
Adding real-time visual/audio feedback for gesture recognition
Expanding beyond two TV models to better represent the full history
Testing a horizontal layout and shorter text to reduce cognitive and physical strain
Validating findings further with education and XR experts via interviews and cognitive walkthroughs