This paper proposes and showcases a methodology to develop an observational behavior assessment instrument to assess psychological competencies of police officers. We outline a step-by-step methodology for police organizations to measure and evaluate behavior in a meaningful way to assess these competencies. We illustrate the proposed methodology with a practical example. We posit that direct behavioral observation can be key in measuring the expression of psychological competence in practice, and that psychological competence in practice is what police organizations should care about. We hope this paper offers police organizations a methodology to perform scientifically informed observational behavior assessment of their police officers’ psychological competencies and inspires additional research efforts into this important area.