Evaluation Methodologies

Although practitioners in the CCD field clearly see the value of arts practice as a tool for reducing health and social inequalities, the evidence to support this is still emerging. And while practitioners acknowledge evaluation as important, methods for gathering evidence vary greatly, and evaluation is not yet central to much community arts practice. Calls for more scientific approaches to evaluation point out limitations such as including reliance on anecdote, small sample size, limited hypothesis testing and a lack of longitudinal components.

From an arts perspective, there is some scepticism about whether the true value of art and the artistic process can be measured by empirical means, and about the risks of narrowly measuring the arts as only an instrument of intervention, which may overlook other unexpected or intangible outcomes.

In reality, most social science researchers use a combination of quantitative and qualitative methodology, often referred to as mixed-methods research. While quantitative methods can limit the complex realities of social phenomena by ignoring non-measurable indicators, the benefit of quantitative methods is that they are experimental in nature and have the potential to verify hypotheses in a positivist paradigm. This mixed-methods approach may be of some benefit to the arts and health conundrum about finding appropriate methods for evaluation.

Further Reading
Belfiore, E & Bennet, O 2010, 'Beyond the 'toolkit approach': arts impact evaluation research and the realities of cultural policy-making,' Journal for Cultural Research, vol. 14, no. 2, pp. 121–42.
Haseman, B 2006, 'A manifesto for performative research', Media International Australia Incorporating Culture and Policy: Quarterley Journal of Media Research and Resources, no. 118, pp. 98–106.
McQueen-Thompson, D, James, P & Zingaras, C 2002, Promoting mental health and wellbeing through community and cultural development: a review of literature focussing on community arts practice, VicHealth and the Globalism Institute.
Merli, P 2002, 'Evaluating the social impact of participation in arts activities', International Journal of Cultural Policy, vol. 8, no. 1, pp. 107–18.
Newman, T, Curtis, K & Stephens, J 2003, 'Do community-based arts projects results in social gains? a review of the literature,' Community Development Journal, vol. 38, no. 4, pp. 310–22.

Quantitative research methods

Quantitative research is the systematic investigation of quantitative phenomena and their relationships, and tests hypotheses, objectives or aims. Objective measurement and statistical analysis are central to quantitative research. While quantitative methods can limit the complex realities of social phenomena by ignoring non-measurable indicators, the benefit of quantitative methods is that they can verify hypotheses. The most common methods of data collection are surveys, direct measurement or observation, and analysis of secondary sources or data.

Further Reading
Anwar McHenry, J 2010, The arts and social wellbeing in Australian rural communities, Thesis Summary, The University of Western Australia.
Hacking, S, Secker, J, Spandler, H, Kent, L & Johnson, V 2009, Evaluating the impact of participatory arts projects for people with mental health needs, Health and Social Care in the Community, vol. 16, no. 6, pp. 638–48.
Kelaher, M, Dunt, D, Curry, S, Berman, N, Joubert, L & Johnson, V 2009, Evaluation of the community arts development scheme, Final Report.
Staricoff, RL 2004, Arts in health: a review of the medical literature, Arts Council of England.

Qualitative research methods

Qualitative research involves an interpretive approach to evaluation, and attempts to find the answer to research questions by interpreting phenomena according to the meanings people bring to them. The collection of data occurs in a natural setting and is sensitive to the people and places under study. Data analysis is inductive, establishing patterns or themes. The most common forms of data collection in qualitative research include in-depth interviews, focus groups and observation.

There are emerging qualitative evaluation research methods that are unique to cultural community development, offering creative processes and mediums that are meaningful to participants. For example, participatory action film making can be an effective way of reaching and articulating the views of traditionally powerless groups in the community development process.

Further Reading
Howells, V & Zelnik, T 2009, Making art: a qualitative study of personal and group transformations in a community arts studio.
Jermyn, H 2004, The art of inclusion, Research Report 35, Arts Council of England.
Stickely, T, Hui, A, Morgan, G & Bertram, G 2007, Experiences and constructions of art: a narrative-discourse analysis, Journal of Psychiatric and Mental Health Nursing, vol. 14, no. 8, pp. 783–90.
Van Lith, T, Fenner, P, Schofield, M, Pawson, Q & Morgan, M 2008, Creativity, the arts and arts therapy in mental health recovery: developing a research agenda, LaTrobe University.

Bookmark/Search this post with: