The creation of artifacts is one of the factors that make us human. Artifacts contribute to our continual adaptation to the world by permitting better knowledge and control of it. The focus of this chapter is on the role of one specific kind of artifact: sensors. In contrast to our immediate perception of the world from our senses, sensors provide large amount of reliable measurements of the physical world that enhance human cognitive capacities in overcoming our perceptual limitations. However, “raw” sensor data require interpretation that relies on different types of expertise and knowledge to provide relevant meaning for human (adaptive) purposes. We suggest that a cognitive approach to understanding the differences between the different types of knowledge provided by current sensors as artifacts and the human senses is of interest. This approach questions the conception of human cognition as an analytic system of processing information from the world rather than as one which interprets and gives meanings to the world. We posit that understanding the differences between human and artificial sensors can shape a new era of technological advancement that is uniquely collaborative insofar as it would rely on the partnership of scientists working in the Humanities and in the Natural Sciences. In this article we provide some data from cognitive research that outline the beginnings of a pluridisciplinary endeavor to conceive sensors which integrate performances of artifacts and the diversity and richness of human cognition, with the goal of transforming so-called “intelligent” devices into cognitive sensors.
LINK
Anticipatory force planning during grasping is based on visual cues about the object’s physical properties and sensorimotor memories of previous actions with grasped objects. Vision can be used to estimate object mass based on the object size to identify and recall sensorimotor memories of previously manipulated objects. It is not known whether subjects can use density cues to identify the object’s center of mass (CM) and create compensatory moments in an anticipatory fashion during initial object lifts to prevent tilt. We asked subjects (n=8) to estimate CM location of visually symmetric objects of uniform densities (plastic or brass, symmetric CM) and non-uniform densities (mixture of plastic and brass, asymmetric CM). We then asked whether subjects can use density cues to scale fingertip forces when lifting the visually symmetric objects of uniform and non-uniform densities. Subjects were able to accurately estimate an object’s center of mass based on visual density cues. When the mass distribution was uniform, subjects could scale their fingertip forces in an anticipatory fashion based on the estimation. However, despite their ability to explicitly estimate CM location when object density was non-uniform, subjects were unable to scale their fingertip forces to create a compensatory moment and prevent tilt on initial lifts. Hefting object parts in the hand before the experiment did not affect this ability. This suggests a dichotomy between the ability to accurately identify the object’s CM location for objects with non-uniform density cues and the ability to utilize this information to correctly scale their fingertip forces. These results are discussed in the context of possible neural mechanisms underlying sensorimotor integration linking visual cues and anticipatory control of grasping.
LINK
Different inputs from a multisensory object or event are often integrated into a coherent and unitary percept, despite differences in sensory formats, neural pathways, and processing times of the involved modalities. Presumably, multisensory integration occurs if the cross-modal inputs are presented within a certain window of temporal integration where inputs are perceived as being simultaneous. Here, we examine the role of ongoing neuronal alpha (i.e. 10-Hz) oscillations in multimodal synchrony perception. While EEG was measured, participants performed a simultaneity judgement task with visual stimuli preceding auditory ones. At stimulus onset asynchronies (SOA's) of 160–200 ms, simultaneity judgements were around 50%. For trials with these SOA's, occipital alpha power was smaller preceding correct judgements, and the individual alpha frequency was correlated with the size of the temporal window of integration. In addition, simultaneity judgements were modulated as a function of oscillatory phase at 12.5 Hz, but the latter effect was only marginally significant. These results support the notion that oscillatory neuronal activity in the alpha frequency range, which has been taken to shape perceptual cycles, is instrumental in multisensory perception.
LINK
This project develops a European network for transdisciplinary innovation in artistic engagement as a catalyst for societal transformation, focusing on immersive art. It responds to the professionals in the field’s call for research into immersive art’s unique capacity to ‘move’ people through its multisensory, technosocial qualities towards collective change. The project brings together experts leading state-of-the-art research and practice in related fields with an aim to develop trajectories for artistic, methodological, and conceptual innovation for societal transformation. The nascent field of immersive art, including its potential impact on society, has been identified as a priority research area on all local-to-EU levels, but often suffers from the common (mis)perception as being technological spectacle prioritising entertainment values. Many practitioners create immersive art to enable novel forms of creative engagement to address societal issues and enact change, but have difficulty gaining recognition and support for this endeavour. A critical challenge is the lack of knowledge about how their predominantly sensuous and aesthetic experience actually lead to collective change, which remains unrecognised in the current systems of impact evaluation predicated on quantitative analysis. Recent psychological insights on awe as a profoundly transformative emotion signals a possibility to address this challenge, offering a new way to make sense of the transformational effect of directly interacting with such affective qualities of immersive art. In parallel, there is a renewed interest in the practice of cultural mediation, which brings together different stakeholders to facilitate negotiation towards collective change in diverse domains of civic life, often through creative engagements. Our project forms strategic grounds for transdisciplinary research at the intersection between these two developments. We bring together experts in immersive art, psychology, cultural mediation, digital humanities, and design across Europe to explore: How can awe-experiences be enacted in immersive art and be extended towards societal transformation?
This project aims to develop a measurement tool to assess the inclusivity of experiences for people with varying challenges and capabilities on the auditory spectrum. In doing so, we performed an in-depth exploration of scientific literature and findings from previous projects by Joint Projects. Based on this, we developed an initial conceptual model that focuses on sensory perception, emotion, cognition, and e[ort in relation to hearing and fatigue. Within, this model a visitor attraction is seen as an “experienscape” with four key elements: content, medium, context, and individual. In co-creative interviews with experts by experience with varying challenges on the auditory spectrum, they provided valuable insights that led to a significant expansion of this initial model. This was a relevant step, as in the scientific and professional literature, little is known about the leisure experiences of people with troubled hearing. For example, personal factors such as a person’s attitude toward their own hearing loss and the social dynamics within their group turned out to greatly influence the experience. The revised model was then applied in a case study at Apenheul, focusing on studying differences in experience of their gorilla presentation amongst people with varying challenges on the auditory spectrum.Societal issueThe Netherlands is one of the countries in Europe with the highest density of visitor attractions. Despite this abundance, many visitor attractions are not fully accessible to everyone, particularly to visitors with disabilities who sometimes are not eligible to ride due to safety concerns, yet when eligible generally still encounter numerous barriers. Accessibility of visitor attractions can be approached in various ways. However, because the focus often lies on operational and technical aspects (e.g., reducing stimuli at certain times of the day by turning o[ music, o[ering alternative wheelchair entrances), strategic and community-focused approaches are often overlooked. More importantly, there is also a lack of attention to the experience of visitors with disabilities. This becomes apparent from several studies from Joint Projects, where visitor attractions are being visited together with experts by experience with various disabilities. Nevertheless, experience is often being regarded as the 'core product' of the leisure sector. The right to meet, discover, develop, relax and thus enjoy this core product is hindered for many people with disabilities due to a lack of knowledge, inaccessibility (physical, digital, social, communicative as well as financial) and discrimination in society. Additionally, recreation entrepreneurs still face a significant gap in reaching the potential market of guests with disabilities and their networks. Thus, despite the numerous initiatives in the leisure sector aimed at improving accessibility on technical and operational fronts, often people with disabilities are still not being able to experience the same kind of enjoyment as those without. These observations form the pressing impetus for initiating the current research project, tapping into the numerous opportunities for learning, development and growth on making leisure offer more inclusive.Benefit to societyIn total, the current project approach comes with a number of enrichments in terms of both knowledge and methodology: a mixed-methods approach that allows for comparing data from different sources to obtain a more complete picture of the experience; a methodological co-design process that honours the 'nothing about us without us' principle; and benchmarking for a group (i.e., people with challenges on the auditory spectrum) that despite the size of its population has thus far mostly been overlooked.
