There is a dualism in current policy discourses on public health between neoliberal conceptions of health, and health as a product of social conditions. These conceptions also co-exist in the narratives of citizens. In this qualitative study carried out in the Netherlands, we analysed how people living in structural vulnerability perceive the potential tension of this dualism. Whereas the respondents’ narratives undoubtedly reflect the neoliberal view on health responsibility, their life stories revealed how health was a product of intergenerational transmission of poverty, violence and ill health. Neoliberal policies that expect citizens to minimise health risks were seen as unrealistic and raised feelings of apathy and fatalism. Behavioural messages were considered to oppose personal autonomy. Instead, the respondents appeared to envision health as a joint, balanced effort by themselves, their social network and public services. The tension between their own view on responsibility and the neoliberal view negatively affected their sense of self. These feelings were strengthened by the fact that the respondents felt abandoned by a government that calls upon self-sufficiency to deal with problems outside their control. On the other hand, they also stressed the importance of agency, particularly in breaking vicious transgenerational cycles. These findings indicate that the views of people living in disadvantaged neighbourhoods require further elucidation to develop a more nuanced and balanced view on responsibilities for health in public policies, and thus to prevent ignoring the problems faced by people who do not meet the expectations of current neoliberal policies.