Key to reinforcement learning in multi-agent systems is the ability to exploit the fact that agents only directly influence only a small subset of the other agents. Such loose couplings are often modelled using a graphical model: a coordination graph. Finding an (approximately) optimal joint action for a given coordination graph is therefore a central subroutine in cooperative multi-agent reinforcement learning (MARL). Much research in MARL focuses on how to gradually update the parameters of the coordination graph, whilst leaving the solving of the coordination graph up to a known typically exact and generic subroutine. However, exact methods { e.g., Variable Elimination { do not scale well, and generic methods do not exploit the MARL setting of gradually updating a coordination graph and recomputing the joint action to select. In this paper, we examine what happens if we use a heuristic method, i.e., local search, to select joint actions in MARL, and whether we can use outcome of this local search from a previous time-step to speed up and improve local search. We show empirically that by using local search, we can scale up to many agents and complex coordination graphs, and that by reusing joint actions from the previous time-step to initialise local search, we can both improve the quality of the joint actions found and the speed with which these joint actions are found.
LINK
Artificially intelligent agents increasingly collaborate with humans in human-agent teams. Timely proactive sharing of relevant information within the team contributes to the overall team performance. This paper presents a machine learning approach to proactive communication in AI-agents using contextual factors. Proactive communication was learned in two consecutive experimental steps: (a) multi-agent team simulations to learn effective communicative behaviors, and (b) human-agent team experiments to refine communication suitable for a human team member. Results consist of proactive communication policies for communicating both beliefs and goals within human-agent teams. Agents learned to use minimal communication to improve team performance in simulation, while they learned more specific socially desirable behaviors in the human-agent team experiment
DOCUMENT
Autonomously exploring and mapping is one of the open challenges of robotics and artificial intelligence. Especially when the environments are unknown, choosing the optimal navigation directive is not straightforward. In this paper, we propose a reinforcement learning framework for navigating, exploring, and mapping unknown environments. The reinforcement learning agent is in charge of selecting the commands for steering the mobile robot, while a SLAM algorithm estimates the robot pose and maps the environments. The agent, to select optimal actions, is trained to be curious about the world. This concept translates into the introduction of a curiosity-driven reward function that encourages the agent to steer the mobile robot towards unknown and unseen areas of the world and the map. We test our approach in explorations challenges in different indoor environments. The agent trained with the proposed reward function outperforms the agents trained with reward functions commonly used in the literature for solving such tasks.
MULTIFILE
Teachers have a crucial role in bringing about the extensive social changes that are needed in the building of a sustainable future. In the EduSTA project, we focus on sustainability competences of teachers. We strengthen the European dimension of teacher education via Digital Open Badges as means of performing, acknowledging, documenting, and transferring the competencies as micro-credentials. EduSTA starts by mapping the contextual possibilities and restrictions for transformative learning on sustainability and by operationalising skills. The development of competence-based learning modules and open digital badge-driven pathways will proceed hand in hand and will be realised as learning modules in the partnering Higher Education Institutes and badge applications open for all teachers in Europe.Societal Issue: Teachers’ capabilities to act as active facilitators of change in the ecological transition and to educate citizens and workforce to meet the future challenges is key to a profound transformation in the green transition.Teachers’ sustainability competences have been researched widely, but a gap remains between research and the teachers’ practise. There is a need to operationalise sustainability competences: to describe direct links with everyday tasks, such as curriculum development, pedagogical design, and assessment. This need calls for an urgent operationalisation of educators’ sustainability competences – to support the goals with sustainability actions and to transfer this understanding to their students.Benefit to society: EduSTA builds a community, “Academy of Educators for Sustainable Future”, and creates open digital badge-driven learning pathways for teachers’ sustainability competences supported by multimodal learning modules. The aim is to achieve close cooperation with training schools to actively engage in-service teachers.Our consortium is a catalyst for leading and empowering profound change in the present and for the future to educate teachers ready to meet the challenges and act as active change agents for sustainable future. Emphasizing teachers’ essential role as a part of the green transition also adds to the attractiveness of teachers’ work.
The Dutch main water systems face pressing environmental, economic and societal challenges due to climatic changes and increased human pressure. There is a growing awareness that nature-based solutions (NBS) provide cost-effective solutions that simultaneously provide environmental, social and economic benefits and help building resilience. In spite of being carefully designed and tested, many projects tend to fail along the way or never get implemented in the first place, wasting resources and undermining trust and confidence of practitioners in NBS. Why do so many projects lose momentum even after a proof of concept is delivered? Usually, failure can be attributed to a combination of eroding political will, societal opposition and economic uncertainties. While ecological and geological processes are often well understood, there is almost no understanding around societal and economic processes related to NBS. Therefore, there is an urgent need to carefully evaluate the societal, economic, and ecological impacts and to identify design principles fostering societal support and economic viability of NBS. We address these critical knowledge gaps in this research proposal, using the largest river restoration project of the Netherlands, the Border Meuse (Grensmaas), as a Living Lab. With a transdisciplinary consortium, stakeholders have a key role a recipient and provider of information, where the broader public is involved through citizen science. Our research is scientifically innovative by using mixed methods, combining novel qualitative methods (e.g. continuous participatory narrative inquiry) and quantitative methods (e.g. economic choice experiments to elicit tradeoffs and risk preferences, agent-based modeling). The ultimate aim is to create an integral learning environment (workbench) as a decision support tool for NBS. The workbench gathers data, prepares and verifies data sets, to help stakeholders (companies, government agencies, NGOs) to quantify impacts and visualize tradeoffs of decisions regarding NBS.
-Chatbots are being used at an increasing rate, for instance, for simple Q&A conversations, flight reservations, online shopping and news aggregation. However, users expect to be served as effective and reliable as they were with human-based systems and are unforgiving once the system fails to understand them, engage them or show them human empathy. This problem is more prominent when the technology is used in domains such as health care, where empathy and the ability to give emotional support are most essential during interaction with the person. Empathy, however, is a unique human skill, and conversational agents such as chatbots cannot yet express empathy in nuanced ways to account for its complex nature and quality. This project focuses on designing emotionally supportive conversational agents within the mental health domain. We take a user-centered co-creation approach to focus on the mental health problems of sexual assault victims. This group is chosen specifically, because of the high rate of the sexual assault incidents and its lifetime destructive effects on the victim and the fact that although early intervention and treatment is necessary to prevent future mental health problems, these incidents largely go unreported due to the stigma attached to sexual assault. On the other hand, research shows that people feel more comfortable talking to chatbots about intimate topics since they feel no fear of judgment. We think an emotionally supportive and empathic chatbot specifically designed to encourage self-disclosure among sexual assault victims could help those who remain silent in fear of negative evaluation and empower them to process their experience better and take the necessary steps towards treatment early on.
