Industrial robot manipulators are widely used for repetitive applications that require high precision, like pick-and-place. In many cases, the movements of industrial robot manipulators are hard-coded or manually defined, and need to be adjusted if the objects being manipulated change position. To increase flexibility, an industrial robot should be able to adjust its configuration in order to grasp objects in variable/unknown positions. This can be achieved by off-the-shelf vision-based solutions, but most require prior knowledge about each object tobe manipulated. To address this issue, this work presents a ROS-based deep reinforcement learning solution to robotic grasping for a Collaborative Robot (Cobot) using a depth camera. The solution uses deep Q-learning to process the color and depth images and generate a greedy policy used to define the robot action. The Q-values are estimated using Convolutional Neural Network (CNN) based on pre-trained models for feature extraction. Experiments were carried out in a simulated environment to compare the performance of four different pre-trained CNNmodels (RexNext, MobileNet, MNASNet and DenseNet). Results showthat the best performance in our application was reached by MobileNet,with an average of 84 % accuracy after training in simulated environment.
DOCUMENT
Just-in-time adaptive intervention (JITAI) has gained attention recently and previous studies have indicated that it is an effective strategy in the field of mobile healthcare intervention. Identifying the right moment for the intervention is a crucial component. In this paper the reinforcement learning (RL) technique has been used in a smartphone exercise application to promote physical activity. This RL model determines the ‘right’ time to deliver a restricted number of notifications adaptively, with respect to users’ temporary context information (i.e., time and calendar). A four-week trial study was conducted to examine the feasibility of our model with real target users. JITAI reminders were sent by the RL model in the fourth week of the intervention, while the participants could only access the app’s other functionalities during the first 3 weeks. Eleven target users registered for this study, and the data from 7 participants using the application for 4 weeks and receiving the intervening reminders were analyzed. Not only were the reaction behaviors of users after receiving the reminders analyzed from the application data, but the user experience with the reminders was also explored in a questionnaire and exit interviews. The results show that 83.3% reminders sent at adaptive moments were able to elicit user reaction within 50 min, and 66.7% of physical activities in the intervention week were performed within 5 h of the delivery of a reminder. Our findings indicated the usability of the RL model, while the timing of the moments to deliver reminders can be further improved based on lessons learned.
DOCUMENT
The number of applications in which industrial robots share their working environment with people is increasing. Robots appropriate for such applications are equipped with safety systems according to ISO/TS 15066:2016 and are often referred to as collaborative robots (cobots). Due to the nature of human-robot collaboration, the working environment of cobots is subjected to unforeseeable modifications caused by people. Vision systems are often used to increase the adaptability of cobots, but they usually require knowledge of the objects to be manipulated. The application of machine learning techniques can increase the flexibility by enabling the control system of a cobot to continuously learn and adapt to unexpected changes in the working environment. In this paper we address this issue by investigating the use of Reinforcement Learning (RL) to control a cobot to perform pick-and-place tasks. We present the implementation of a control system that can adapt to changes in position and enables a cobot to grasp objects which were not part of the training. Our proposed system uses deep Q-learning to process color and depth images and generates an (Formula presented.) -greedy policy to define robot actions. The Q-values are estimated using Convolution Neural Networks (CNNs) based on pre-trained models for feature extraction. To reduce training time, we implement a simulation environment to first train the RL agent, then we apply the resulting system on a real cobot. System performance is compared when using the pre-trained CNN models ResNext, DenseNet, MobileNet, and MNASNet. Simulation and experimental results validate the proposed approach and show that our system reaches a grasping success rate of 89.9% when manipulating a never-seen object operating with the pre-trained CNN model MobileNet.
DOCUMENT
In het kader van het Hoogwaterbeschermingsprogramma (HWBP) neemt de vraag naar klei voor het versterken van dijken toe, echter is het aanbod beperkt. Dit voorstel richt zich op ontwikkelen van nieuwe duurzame en kosteneffectieve technieken die het mogelijk maken om zout sediment uit estuaria in te kunnen zetten voor de dijkversterkingsopgave. Gebiedseigen materiaal, met name het zoute slib, kan worden ingezet voor klei productie in lokale dijkverzwaring en draagt bij aan duurzaam grondstoffenverbruik, klimaatadaptatie en de ecologische kwaliteit van estuaria. Met het project “Ontzouten rijpend slib voor Deltabescherming” gaan het lectoraat Sustainable River Management van de HAN, Ecoshape, Netics in samenwerking met partijen verenigd in het interbestuurlijk project IBP-VLOED onderzoeken hoe zout slib (kosten)effectief kan worden ontdaan van het zout, zodat het gebruikt kan worden in de regionale dijkversterkingsopgave. In IBP-Vloed zijn alle relevante nationale en regionale (semi)overheden, kennisinstellingen en belangenorganisaties vertegenwoordigd die zich richten op hergebruik van slib uit het Eems-Dollard estuarium. Beoogd wordt om een geschikte kosteneffectieve en schaalbare ontzoutingsmethode (strategie) te ontwikkelen die rekening houdt met de samenhang van de governing parameters en de heterogeniteit in samenstelling en structuur van het zoute slib uit estuaria zoals het Eems-Dollard gebied. De resultaten worden gepresenteerd tijdens een workshop en gebundeld in de vorm van best practices.
