Physical activity (PA) is a key strategy for improving symptoms in people with rheumatic and musculoskeletal diseases (RMDs). The aim of this study was to investigate and rank the importance of known barriers and facilitators for engaging in PA, from the perspective of people living with RMD. Five hundred thirty-three people with RMD responded to a survey (nine questions) disseminated by the People with Arthritis and Rheumatism (PARE) network of the European Alliance of Associations for Rheumatology (EULAR). The survey required participants to rank — based on their perceived importance — known PA barriers and facilitators from the literature, and specifically RMD symptoms as well as healthcare and community factors that may affect PA participation. Of the participants, 58% reported rheumatoid arthritis as their primary diagnosis, 89% were female, and 59% were between 51 and 70 years of age. Overall, participants reported fatigue (61.4%), pain (53.6%) and painful/swollen joints (50.6%) as the highest ranked barriers for engaging in PA. Conversely, less fatigue (66.8%) and pain (63.6%), and being able to do daily activities more easy (56.3%) were identified as the most important facilitators to PA. Three literature identified PA barriers, i.e., general health (78.8%), fitness (75.3%) and mental health (68.1%), were also ranked as being the most important for PA engagement. Symptoms of RMDs, such as pain and fatigue, seem to be considered the predominant barriers to PA by people with RMD; the same barriers are also the ones that they want to improve through increasing PA, suggesting a bi-directional relationship between these factors.