The 6th issue of the Fresh Perspectives series takes you on a playful but serious journey along the issues at stake when designing a mixed reality experience in the context of theatre and performance. The publication features a selection of projects describing in detail the artistic design processes, as well as the challenges and opportunities brought about by the use of mixed reality technologies.Joris Weijdom, researcher and lecturer at the Professorship, wrote the key article and curated this publication.
DOCUMENT
This position paper presents a proposal for evaluating interaction with light in a mixed reality setup. Current processes of designing and testing new forms of user interaction (UI) for controlling lighting are long and end up being restricted in actually testing a small number of possible interactions. Apart from the apparent advantage of overcoming testing a small number of potential interactions, the advantages of a simulated environment lie in the fact that such an environment is fully controllable and adaptable to the researchers' needs. Finally, we sketch potential challenges of using a mixed reality setup for evaluating interaction with light.
LINK
One of the key challenges in the rapid technological advance of Virtual Reality (VR) and Mixed Reality (MR) concerns the design of collaborative experiences. VR systems do not readily support team collaboration because they tend to focus on individual experiences and do not easily facilitate naturalistic collaboration. MR environments provide solutions for collaborative experiences, but establishing smooth communication between hardware components and software modules faces a major hurdle. This paper presents the background to and main challenges of an ongoing project on collaboration in an MR lab, aiming to design a serious 'team collaboration' game. To this end, we utilized a common game engine to engineer a cost-effective solution that would make the game playable in a configuration operated by WorldViz and Volfoni equipment. Evaluation of various solutions in the development process found a Unity 3D Cluster Rendering Beta solution to be the most cost-effective and successful.
DOCUMENT
One of the key challenges in the rapid technological advance of Virtual Reality (VR) and Mixed Reality (MR) concerns the design of collaborative experiences. VR systems do not readily support team collaboration because they tend to focus on individual experiences and do not easily facilitate naturalistic collaboration. MR environments provide solutions for collaborative experiences, but establishing smooth communication between hardware components and software modules faces a major hurdle. This paper presents the background to and main challenges of an ongoing project on collaboration in an MR lab, aiming to design a serious 'team collaboration' game. To this end, we utilized a common game engine to engineer a cost-effective solution that would make the game playable in a configuration operated by WorldViz and Volfoni equipment. Evaluation of various solutions in the development process found a Unity 3D Cluster Rendering Beta solution to be the most cost-effective and successful.
DOCUMENT
The aim of this research is to explore the potential of Mixed Reality (MR) technologies for Operator Support in order to progress towards Industry 4.0 (I4.0) particularly for SMEs. Through a series of interventions and interviews conducted with local SMEs, potential use cases and their drawbacks have been identified. From this, insights were derived that serve as a starting point for conducting further experiments with MR technology in the smart manufacturing laboratory at the THUAS in Delft. The intervention consisted of a free form workshop in which the participants get ‘tinkering’ time to explore MR in their own work environment. The various levels of awareness were assessed in three stages: during an introductory interview, and after an instruction meeting and some ‘tinkering’. The study took place in the period from January 2022 to July 2022 with 10 local SMEs in the Netherlands. The results show that for all SMEs the awareness and understanding increased. The use cases identified by operators themselves concerned Quality Control, Diagnostics, Instruction, Specification and Improvement of Operations. Drawbacks foreseen related to Ergonomic Concerns, Resistance from operators, Technical considerations, Unavailability of MR device and an insufficient digital infrastructure to support MR in full extent. The use case most promising to the participants was further developed into a physical prototype for an ‘assisted assembly cell’ by which the aspects of ergonomics and the mentioned technical considerations could be analysed.
MULTIFILE
Collaborative Mixed Reality Environments (CMREs) enable designing Performative Mixed Reality Experiences (PMREs) to engage participants’ physical bodies, mixed reality environments, and technologies utilized. However, the physical body is rarely purposefully incorporated throughout such design processes, leaving designers seated behind their desks, relying on their previous know-how and assumptions. In contrast, embodied design techniques from HCI and performing arts afford direct corporeal feedback to verify and adapt experiential aesthetics within the design process. This paper proposes a performative prototyping method, which combines bodystorming methods with Wizard of Oz techniques with a puppeteering approach, using inside-out somaesthetic- and outside-in dramaturgical perspectives. In addition, it suggests an interdisciplinary vocabulary to share and evaluate PMRE experiences during and after its design collaboration. This method is exemplified and investigated by comparing two case studies of PMRE design projects in higher-art education using the existing Social VR platform NEOS VR adapted as a CMRE.
DOCUMENT
This paper shows an overview of design practices of the XR-lab at the Amsterdam University of Applied Sciences, The Netherlands. Over the course of six years, interdisciplinary teams of students have delivered 55+ prototypes in virtual, augmented, and mixed reality for a variety of 40+ clients. As human-computer interaction is entering a new evolutionary phase towards human-computer integration, new opportunities in extended reality (XR) have the potential to fundamentally alter human characteristics and abilities. Therefore, this paper begins with taking a philosophical stance on ‘being human’ and the anthropological concept of ‘liminality’ in XR-experiences. A further exploration of the concept of 'emotional rehearsal spaces' uses know-how from performance art, dance, architecture, and dramaturgy. Insights from tangible practices at the XR-lab show the cultural journey in XR-collaborations. This is made visible through a quick and dirty experiment on artistic thinking, design thinking, and system thinking, which shows how interdisciplinary collaborations are able to ignite new combinations of thought in design teams and individual professionals. Finally, we show an overview of specific design methods and tools that have been explored at the XR-lab over the years.
DOCUMENT
The use of immersive technologies has changed the consumption environment in which retailers provide services. We present findings from a study designed to investigate consumer responses toward a $17 million AI-embedded mixed reality (MR) exhibit in a retail/entertainment complex which combines advanced technology entertainment with retail shopping. Findings from our study demonstrate that the quality of AI (i.e., speech recognition and synthesis via machine learning) associated with an augmented object increases MR immersion associated with spatial immersion, MR enjoyment, and consumers’ perceptions of novel experiences. Collectively, these increase consumer engagement, and positively influence behavioral responses—specifically, purchase intentions and intentions to share experiences with social groups. Overall, findings from this study show that interactive AI and MR technology open new avenues to promote consumer engagement.
LINK
Persuasive games exist for a wide variety of objectives, from marketing, to healthcare and activism. Some of the more socially-aware ones cast players as members of disenfranchised minorities, such as migrants, prompting them to 'see what they see'. In parallel, a growing number of designers has recently started to leverage immersive technologies to enable the public to temporarily inhabit another person, to 'sense what they sense'. From these two converging perspectives, we hypothesize a still-uncharted space of opportunities at the crossroads of games, empathy, persuasion, and immersion. Following a Research through Design approach, we explored this space by designing A Breathtaking Journey, an embodied and multisensory mixed-reality game providing a first-person perspective of a refugee's journey. A qualitative study was conducted with a grounded theory/open coding methodology to tease out empathy-arousing characteristics, and to chart this novel game design space. As we elaborate on our analysis, we provide insights on empathic mixed-reality experiences, and conclude with offering three design opportunities: visceral engagement, reflective moments and affective appeals, to spur future research and design.
DOCUMENT
