The aim of this dissertation is to examine how adult learners with a spoken language background who are acquiring a signed language, learn how to use the space in front of the body to express grammatical and topographical relations. Moreover, it aims at investigating the effectiveness of different types of instruction, in particular instruction that focuses the learner's attention on the agreement verb paradigm. To that end, existing data from a learner corpus (Boers-Visker, Hammer, Deijn, Kielstra & Van den Bogaerde, 2016) were analyzed, and two novel experimental studies were designed and carried out. These studies are described in detail in Chapters 3–6. Each chapter has been submitted to a scientific journal, and accordingly, can be read independently.1 Yet, the order of the chapters follows the chronological order in which the studies were carried out, and the reader will notice that each study served as a basis to inform the next study. As such, some overlap in the sections describing the theoretical background of each study was unavoidable.
MULTIFILE
Communication between healthcare professionals and deaf patients has been particularly challenging during the COVID-19 pandemic. We have explored the possibility to automatically translate phrases that are frequently used in the diagnosis and treatment of hospital patients, in particular phrases related to COVID-19, from Dutch or English to Dutch Sign Language (NGT). The prototype system we developed displays translations either by means of pre-recorded videos featuring a deaf human signer (for a limited number of sentences) or by means of animations featuring a computer-generated signing avatar (for a larger, though still restricted number of sentences). We evaluated the comprehensibility of the signing avatar, as compared to the human signer. We found that, while individual signs are recognized correctly when signed by the avatar almost as frequently as when signed by a human, sentence comprehension rates and clarity scores for the avatar are substantially lower than for the human signer. We identify a number of concrete limitations of the JASigning avatar engine that underlies our system. Namely, the engine currently does not offer sufficient control over mouth shapes, the relative speed and intensity of signs in a sentence (prosody), and transitions between signs. These limitations need to be overcome in future work for the engine to become usable in practice.
This article examines the impact of the COVID-19 pandemic on the sign language interpreting profession drawing on data from a fourth and final survey conducted in June 2021 as part of a series of online “living surveys” during the pandemic. The survey, featuring 331 respondents, highlights significant changes in the occupational conditions and practices of sign language interpreters due to the sudden shift towards remote video-mediated interpreting. The findings reveal a range of challenges faced by interpreters, including the complexities of audience design, lack of backchanneling from deaf consumers, the need for heightened self-monitoring, nuanced conversation management, and team work. Moreover, the study highlights the physical and mental health concerns that have emerged among interpreters as a result of the shift in working conditions, and a need for interpreters to acquire new skills such as coping with the multimodal nature of online interpreting. While the blend of remote, hybrid, and on-site work has introduced certain advantages, it also poses new challenges encompassing workload management, online etiquette, and occupational health concerns. The survey’s findings underscore the resilience and adaptability of SLIs in navigating the shift to remote interpreting, suggesting a lasting transformation in the profession with implications for future practice, training, and research in the post-pandemic era.
Psychosocial problems related to social isolation are a growing issue for wellbeing and health and have become a significant societal problem. This is especially relevant for children and adults with chronic illnesses and disabilities, and those spending extended periods in hospitals or permanently living in assisted living facilities. A lack of social relationships, social connectivity, and the inability to travel freely leads to feelings of isolation and loneliness. Loneliness interventions often use mediated environments to improve the feeling of connectedness. It has been proven that the utilization of haptic technologies enhances realism and the sense of presence in both virtual environments and telepresence in physical places by allowing the user to experience interaction through the sense of touch. However, the technology application is mostly limited to the experiences of serious games in professional environments and for-entertainment-gaming. This project aims to explore how haptic technologies can support the storytelling of semi-scripted experiences in VR to improve participants’ sense of presence and, therefore, the feeling of connectedness. By designing and prototyping the experience, the project aims to obtain insights and offer a better understanding of designing haptic-technology-supported storytelling and its potential to improve connectedness and become a useful tool in isolation interventions. The project will be conducted through the process of participants’ co-creation.
Digital technologies in public spaces have become more prescient, capable, and invisible. As a result, the need to explain and mediate these technologies has become more urgent. Current processes for designing digital trust interaction protocols, visual languages, and interfaces for the urban environment have been dominated by governing actors: policing, government, and tech-companies. When communities are involved in the design process, their participation is limited to formats these organisations prescribe. By default, these designs exclude the lived technological experiences of communities that use the built environment. The outcome is a lopsided appraisal of digital trust, and designs that are insufficiently transparent and equitable– and as a result, not understood and embraced by the communities who must use them. This design-research aims to develop prototypes that include how urban interactive technologies are ‘lived’ in the spaces where they are implemented. These experiences will be teased-out through site-specific aesthetic and performative actions, which in turn inform the design process. The envisioned contribution includes ways of ’doing’ to the field of situated design, and concrete prototypes for alternative digital trust protocols, visual languages, and interfaces. By flipping the current approach on its head, this research argues that the practical and ethical departure points for addressing digital ‘trust deficits’ are already within the diverse communities who use the built environment.
