Abusive online behaviors occur at a large scale on all social media, and have dire consequences for their victims. Although the problem is largely acknowledged, technological solutions remain limited to detecting and hiding abusive comments. More can be done to address abusers themselves. We propose to investigate the potential of conversational technologies to dialogue with abusers. In this problem description paper, we outline directions for studying the effectiveness dialogue strategies, e.g., to educate or deter abusers, or keep them busy with chatbots thus limiting the time they spend perpetuating abuses.