Key to reinforcement learning in multi-agent systems is the ability to exploit the fact that agents only directly influence only a small subset of the other agents. Such loose couplings are often modelled using a graphical model: a coordination graph. Finding an (approximately) optimal joint action for a given coordination graph is therefore a central subroutine in cooperative multi-agent reinforcement learning (MARL). Much research in MARL focuses on how to gradually update the parameters of the coordination graph, whilst leaving the solving of the coordination graph up to a known typically exact and generic subroutine. However, exact methods { e.g., Variable Elimination { do not scale well, and generic methods do not exploit the MARL setting of gradually updating a coordination graph and recomputing the joint action to select. In this paper, we examine what happens if we use a heuristic method, i.e., local search, to select joint actions in MARL, and whether we can use outcome of this local search from a previous time-step to speed up and improve local search. We show empirically that by using local search, we can scale up to many agents and complex coordination graphs, and that by reusing joint actions from the previous time-step to initialise local search, we can both improve the quality of the joint actions found and the speed with which these joint actions are found.
LINK
This chapter presents the currently not established and identifies design requirements for new systems to address this challenge and provide directions for possible improvement. As a result, this chapter introduces the concept of SamenMarkt®, a participatory system in which multi-agent system technology enables distributed price negotiation, distribution and communication between producers, retailers and consumers.
LINK
Efficiency of city logistics activities suffers due to conflicting personal preferences and distributed decision making by multiple city logistics stakeholders. This is exacerbated by interdependency of city logistics activities, decision making with limited information and stakeholders’ preference for personal objectives over system efficiency. Accordingly, the key to understanding the causes of inefficiency in the city logistics domain is understanding the interaction between heterogeneous stakeholders of the system. With the capabilities of representing a system in a natural and flexible way, agent based modelling (ABM) is a promising alternative for the city logistics domain. This research focuses on developing a framework for the successful implementation of the ABM approach for the city logistics domain. The framework includes various elements – a multi-perspective semantic data model (i.e. ontology) and its validation, the development of an agent base model using this ontology, and a validation approach for the agent-based model. Conclusively, the framework shows that a rigorous course can be taken to successfully implement agent based modelling approach for the city logistics domain.
DOCUMENT
The demand for mobile agents in industrial environments to perform various tasks is growing tremendously in recent years. However, changing environments, security considerations and robustness against failure are major persistent challenges autonomous agents have to face when operating alongside other mobile agents. Currently, such problems remain largely unsolved. Collaborative multi-platform Cyber- Physical-Systems (CPSs) in which different agents flexibly contribute with their relative equipment and capabilities forming a symbiotic network solving multiple objectives simultaneously are highly desirable. Our proposed SMART-AGENTS platform will enable flexibility and modularity providing multi-objective solutions, demonstrated in two industrial domains: logistics (cycle-counting in warehouses) and agriculture (pest and disease identification in greenhouses). Aerial vehicles are limited in their computational power due to weight limitations but offer large mobility to provide access to otherwise unreachable places and an “eagle eye” to inform about terrain, obstacles by taking pictures and videos. Specialized autonomous agents carrying optical sensors will enable disease classification and product recognition improving green- and warehouse productivity. Newly developed micro-electromechanical systems (MEMS) sensor arrays will create 3D flow-based images of surroundings even in dark and hazy conditions contributing to the multi-sensor system, including cameras, wireless signatures and magnetic field information shared among the symbiotic fleet. Integration of mobile systems, such as smart phones, which are not explicitly controlled, will provide valuable information about human as well as equipment movement in the environment by generating data from relative positioning sensors, such as wireless and magnetic signatures. Newly developed algorithms will enable robust autonomous navigation and control of the fleet in dynamic environments incorporating the multi-sensor data generated by the variety of mobile actors. The proposed SMART-AGENTS platform will use real-time 5G communication and edge computing providing new organizational structures to cope with scalability and integration of multiple devices/agents. It will enable a symbiosis of the complementary CPSs using a combination of equipment yielding efficiency and versatility of operation.
