ARE YOU WONDERING IF I CAN HELP WITH EVALUATION AND HOW MUCH IT WILL COST? READ THIS BLOG POST...
evaluation-what-do-you-want-and-how-much-does-it-cost.html
evaluation-what-do-you-want-and-how-much-does-it-cost.html
EVALUATION
Many thanks for all the hard work you put in. It's been a pleasure working with you. I've learnt a lot and will be happy to vouch for the quality of your work any time. You'll certainly be at the front of my mind on any future projects that need evaluation. Your report is an impressive evaluation report and gives us an excellent foundation for planning an even better festival next year. |
We received a clear, detailed report which related well to the brief and its key aims and objectives. You were very communicative, flexible in process and scheduling to meet the demands of the project, and able to relate to a range of proejct partners and participants well. Evaluation of the project was clear and insightful, with clear understanding of the aims and objectives of the work, its opportunities and challenges, with constructive suggestions for development or improvement in the future. |
STRATEGY
Artist's Rooms, Gallery Oldham: Evaluation Framework Richard Long, Cornish Slate Ellipse 2009. Tate / National Galleries of Scotland.
© Richard Long
Gallery Oldham hosted artworks by Richard Long, as part of the Artist Rooms project created by National Galleries of Scotland & Tate. The team ran a four day residential for young people, combining time in the gallery with three days in a rural outdoor centre, where they explored their natural surroundings to make artworks in response to Richard Long's work. The co-ordinator wanted guidance developing a meaningful evaluation approach, beyond box-ticking and more appropriate for the tone of the project. The Artist Rooms standard monitoring form, and criteria of Arts Award Explore certification also needed to be included in the approach. The work involved clarifying the aims and objectives of the residential, creating a suitable framework with practical evaluation tools and approaches; and help organising, analysing and summarising the data and findings from the project.
|
Venture Arts: Evaluation Strategy Line drawing by Robert Dixon, learning disabled artist based at Venture Arts.
Venture Arts support learning disabled artists as equals in high profile galleries, museums and art fairs. With a long history of projects, all evaluated differently, they wanted a new ongoing, internally run evaluation strategy to track and investigate the power of their work in the long term; including their new status as an Arts Council England National Portfolio Organisation. After researching their business plan, previous evaluations, funding applications, and consulting with the artists they work with, a strategy was developed covering social, cultural, economic and environmental outcomes locally, regionally, nationally, and eventually internationally. Crucially, the approach needs to work for the tutors, volunteers, staff members, project partners and the artists themselves - learning disabled and otherwise. The strategy created includes everything from a logic model theory and story of change, evaluation framework with methodologies and templates; recommendations around networks and software; to identifying appropriate and meaningful qualitative approaches suitable for learning disabled artists to express what matters for them, including non-verbal methods.
|
UNIVERSITIES
Manchester Metropolitan University: Creating Our Future Histories
Working with the Institute of Humanities & Social Research at Manchester Metropolitan University, and Manchester Histories Festival to lead on the evaluation of their joint programme Creating Our Future Histories, which partnered up and coming researchers with cultural community groups. The action research programme aimed to develop real world research skills of the researchers in a way which contributes advice and practical support to community and cultural groups around Manchester. Advising the team on how to embed good evaluation practice with all the partners, working as critical friend / independent evaluator to the project mentors, providing a final overarching report of the programme, and running an evaluation workshop for all involved.
Image: courtesy of Morag Rose, GM CVO Manchester Beacon for Public EngagementThe national Beacon programme of Public Engagement, encourages universities to become more open to a wider diversity of people by improving public understanding of university resources and assetts, and finding new ways to hold conversations with all types of public groups and individuals. Sally worked as Evaluation Consultant to the Manchester Beacon (Manchester University, Manchester Metropolitan University, Salford University and MOSI [Museum of Science & Industry] and Manchester, Knowledge Capital). The role included mentoring the Director and Project Managers on project planning and evaluation cycles; providing support, training and resources for evaluation methodologies; and advising on evaluation of cultural and community work.
Creative & Cultural Industries Exchange, University of Leeds: Identifying Impact
The Creative and Cultural Industries Exchange is University of Leeds' home for partnering academics and researchers with industry organisations in creative and cultural fields. Thanks to funding from the Higher Education Innovation Fund, over 40 projects have been created through such partnerships. The work involved exploring their impact to create a series of case studies showcasing the diversity and value of their activity, and identifying some of the ingredients for success, to help inform future programmes and development.
|
Manchester Metropolitan University & Creative Partnerships
Working for CUE Creative, part of the Institute of Education at Manchester Metropolitan University, evaluating several school enquiry based learning projects as part of the final tranche of Creative Partnerships. The folio of projects with them ranged from early years photographs being used for consultation about Foundation Stage children's outdoor environment and reflection on how Foundation Stage children learn; through to changing the way an inner-city secondary school thinks by using live performance art - and so many themes in between.
The Passions of Youth: Evaluation Consultant
Working as Evaluation Consultant on The Passions of Youth was an action research project developed by Manchester Metropolitan University, involving artists, youth workers, and heritage organisations working with young men in groups across Manchester, to help them explore the pastimes that fuel their life, such as football, music, and speedway racing through media training, heritage exploration and interviewing older men who paved the way for them. The evaluation briefwas to identify the successes and challenges of the project, the outcomes for the young men involved, to what extent the project meets its aims, and how the programme could influence the work of other heritage based youth and community engagement projects in the future.
|
PROFESSIONAL DEVELOPMENT
Manchester Museums & Galleries Partnership: Islamic Collections
Curators from Manchester Museum, The Whitworth Art Gallery, and Manchester Art Gallery worked in partnership to develop their expertise around the Islamic items in their collections. The evaluation identified the impact of a series of activities including visiting other collections; talking to experts from the UK and beyond, sharing practice and discussion with colleagues in the sector; and digging deep into their collections to research more about the links to Islam. It also resulted in a set of recommendations to ensure the new learning can reach its maximum potential in the medium and long term.
Image: Pillow End, 1700-1800 © Manchester City Galleries
Image: Pillow End, 1700-1800 © Manchester City Galleries
National Heritage Training Group: Skills for the Future Evaluator
The National Heritage Training Group delivered a Heritage Lottery Fund Skills for the Future programme, training people in traditional building skills to help preserve historic buildings and structures in the UK. Some took part in short courses, while others joined placements of 1 or 2 years. Most also completed qualifications. The evaluation explored the processes , outputs and outcomes of the programme, as well as identifying what the enablers and challenges were that tipped experiences one way or the other. The evaluation covered the experiences of trainees, those taking up placements, the companies who offer and host the placements, and the project management team.
FESTIVALS
Open House Families
Included in Open House's celebration and exploration of all things architectural across London, is their family programme. For 2019 that catered for over 6000 people visiting over 50 events across 15 locations in just two days. The evaluation included analysis of information about families who attend a range of venues, tours and accessible events. New to their programme this year are tours by families, for families; and special events for families of children with special needs, conditions and disabilities. The resulting evaluation report is helping the team report back to funders and partners, as well as learning more about the continual development and improvement of their work. You can read or download the report here .
Manchester Histories Festival
Evaluating the 2016 Manchester Histories Festival, with a particular emphasis on the social impact of their activity, as well as the more usual quantitative demographic and attendance information. Working together with the organisation's director, we agreed on a framework of priority outcomes taken from a range of established meaningful frameworks spanning cultural and social sectors. Sense of place, local pride, learning, increased togetherness and subjective wellbeing were the basis for our exploration. Process evaluation of the festival partnerships and volunteers programme was also included. The evaluation framework and plan covered a wide partnership and programme and relied on the goodwill and contributions from over 200 festival partners, event organisers and volunteers.
Image: Liz Ackerley |
Manchester Science Festival
Evaluation Consultant for the 2008 to 2012 Manchester Science Festivals. The festival includes over 30 partners and over 200 events. Evaluation each year tests how effectively the festival delivers its intended aims and objectives, with a special focus which changes each year and has so far included:
- Understanding who visitors are and what types of learning takes place - Partnerships with higher education institutes - Community and outreach work - What motivates people to come to the festival Evaluation includes quantitative and creative qualitative information gathering, including creative consultation and feedback opportunities integrated into the festival's programming. Manchester International Festival
Evaluating the Sacred Sounds and Biospheric Project programme strands for Manchester International Festival. This work was an impact study reporting against the objectives for both projects, looking at how they make a difference locally and nationally, and their potential to influence change internationally.
Image: Sacred Sounds Choir, copyright Robert Martin. |
HERITAGE
Jewish Museum London: Handling Collection
Working over two years on the evaluation of a pilot resource and it's final version created by Jewish Museum London. The travelling handling collection is designed to respond to the needs of Religious Studies GCSE exam syllabi, to introduce children and young people to a rounded understanding of the diversity of Jewish culture from a contemporary perspective, and to help the Jewish Museum's collections and themes reach a broader spread of education audiences unlikely to make it to the museum site in person. As well as evaluating the effectiveness of the resource from the perspectives of teachers and pupils, the work also involved mentoring the museum team in how to structure formative and summative evaluation lines of enquiry and methodologies; as well as developing a framework for continuous monitoring of the impact of the resource.
Manchester Pride: Out! Evaluation Consultant
Evaluation of Out!, a project updating and bringing together established but disparate LGBT histories of Manchester into one place, offering a coherent body of information, combining city centre trails, online resources, and activities by groups and organisations around Manchester. The project involved volunteers, workshops and a series of showcase, sharing and oral history collecting events to encourage people to engage with the city as a place of revealed histories. The evaluation role was to help the project co-ordinator monitor participation, identify successes and challenges, and ensure their evaluation reflects Heritage Lottery Fund principles. Additionally, the project was selected as a case study for Heritage Lottery Fund's own national evaluation, and the role included working alongside their national evaluators to ensure data collection and reporting fit with the national needs, meaning our frameworks and needs had to be co-ordinated so as not to duplicate, or to overwhelm participants with evaluation fatigue! The Manchester Metrics system (later to become the Culture Metrics or Quality Metrics) were embedded into the process as one aspect of the data collection.
Image: Courtesy of Manchester Pride National Maritime Museum Interactive Development
Working as an associate of Frankly, Green + Webb, helping the National Maritime Museum develop and evaluate a new interactive area of their museum. Together we explored how maps and digital technologies can help encourage family interaction and increase the personal relevance of the museum and it's stories of maritime heritage for families and other visitors.
Image: James O'Kenkins |
The Dukes: Port Stories Evaluation Consultant
Having received Heritage Lottery Fund investment, The Dukes, as part of a Lancaster-wide consortium, commissioned an evaluation of the processes, outputs and outcomes of Port Stories. Port Stories was a public and community engagement project aiming to increase the knowledge of local people and tourists regarding the city's Georgian maritime past, whilst also making links with the city's contemporary role in migration. A complex series of partnerships, art commissions, community historian / artist volunteer involvement, and production of digital tours of the city were all researched, leading to a detailed report identifying the success, challenges and recommendations of the activity. The report was structured to meet Heritage Lottery Fund evaluation guidelines, and followed best practice regarding social impact evaluation.
Lancashire Witches 400
Supporting the team at Green Close Studios on the large scale Lancashire Witches 400 project, funded by Arts Council England and Heritage Lottery Fund. Evaluating the programme of high profile public and interactive art commissions, the development of a heritage walking trail across the Pennines, projects with local schools, the CPD programme and associated resources for teachers, a workshop programme for local communities, and a series of talks and special interest events for local artists and historians. The evaluation also focussed on the impact of this large-scale project on Green Close's business development and local partnerships.
|
STEM
Seeking Perfection
STEM evaluation working with Nowgen and Manchester Science Festival evaluating the partnership, experiences of young people, and audiences of this project which explores the subject of human enhancement. The team worked together creating a drama production investigating some of the facts, myths, benefits and controversies of this subject. The performance they created was shown during the festival, accompanied by a public debate. A fascinating example of creative public engagement and science communication. More here and here
|
Looking Forward: Arts & Health
Nowgen use arts and creative activity to increase public understanding of genetics. I worked with them on their STEM evaluation to investigate their Looking Forward project, in which artists worked with young cystic fibrosis patients to create a new online resource exploring gene therapy as a potential treatment for the condition. It's a very complex and therefore fascinating project which you can read more about *here*
|
CHILDREN & YOUNG PEOPLE
Grow Big: Sense
Sense was Grow Big's first project, funded by Calderdale Council and the Big Lottery Fund. The evaluation identified the effectiveness of their sensory play training for volunteers and early years practitioners and the impact of a specially designed sensory play programme inside a unique inflatable dome, for 3-4 years for children from areas of high multiple deprivation. A specific focus was to see if the project could increase development in personal, social and emotional skills. The evaluation showed that it achieved not only this, but also developed communication skills to the same extent.
Manchester City Council: Cultural EntitlementManchester City Council contracted Sally to evaluate their two year pilot cultural change pilot programme. The programme aimed to test models of what 'cultural entitlement' might be for children and young people aged 0 to 19. The evaluation explored the transferability and sustainability of children and young people's activity and associated capacity building and professional networks; the accessibility of the programmes for their target participants; relevance of the programme to the Council's Local Authority Agreements and Children & Young People's Plan; and made recommendations with regard to rolling the programme out across the city.
Find Your Talent, LiverpoolMusic education evaluation for the national Find Your Talent programme, working with Liverpool pathfinders through Royal Philharmonic Orchestra: Evaluating Key Stage 3 pupils' understanding of live music within their Merseyside localities; the progression of their learning throughout the project; and measuring their change in opinions of the music national curriculum. Also evaluating the project's capacity to impact on teaching staff's knowledge of how and where to commission live music for schools, and interpret live music as a resource for school based learning.
|
Hayward Gallery: See ThroughProducing the final evaluation report summarising Hayward Gallery's three year young people's programme See Through. Over 8000 young people took part and the Hayward and its many partners learned a great deal about how to build and sustain relationships with young people, and how to work cross-departmentally, using their exhibitions programme as a starting point. The report provided a concise round-up of findings and achievements for the proejct's funder Paul Hamlyn Foundation; and for internal and external advocacy purposes.
Earlyarts & Youth Music
Earlyarts worked with Youth Music on the Spotlighting programme delivering professional development for those working in early years and creative sectors in order to enhance the skills, confidence and resources for their work with young children. The evaluation reported on skills and attitudes the practitioners developed, and the potential impact on the children they work with, as a result of the sessions which covered music, story telling, and outdoor learning in woodland and beach environments.
Shout: St Helen's Children's ServicesSally worked with Isaacs UK cultural consultancy as Evaluation Consultant for a programme of creative interventions in children's centres and schools across the NW borough of St Helens. The project was co-ordinated by local authority children's services as part of their wider Give Us a Shout strategy to improve communication in children under 5 at early years settings throughout the borough. The project involved a range of artists and therapists working across eight settings over a full school year.
Liverpool Capital of Culture: Little AcornsContracted as Early Years Evaluation Consultant for part of Liverpool Capital of Culture to explore a programme aiming to identify good practice in artists of all types, working in early years settings across Merseyside. The project invited the artists, staff and children to find ways of exploring the central themes of Liverpool, community, heritage, identity and family.
|
EXHIBITIONS
New North & South: Manchester Art Gallery / Manchester Museum / The Whitworth / The Tetley / Liverpool Biennial / The British Council / Colombo Biennale / Dhaka Arts Summit / Karachi Muziris Biennale / Kochi Biennale.
New North and South was a network of eleven partners across the North of England and South Asia collaborating on a three year programme of co-commissions, exhibitions and critical exchange to explore shared heritage and develop artistic talent. The evaluation involved audiences, curators, engagement and communication professionals, artists, volunteers and schools to find out about the impact and legacies of the network.
Image: Neha Choksi Untitled, 2017 (production still) Courtesy the artist and Project 88 Wonder Materials: Museum of Science & Industry
Providing holistic overarching evaluation of the museum's new, first internally curated touring exhibition and the public programme around it. The focus was to identify how successfully they communicated a complicated, abstract concept about an invisible subject, creatively, informatively and inspiringly to a range of visitors; and what they needed to learn from this experience for future internally curated touring shows. The evaluation explores the processes and outcomes, breadth and depth of experiences from young and older adults, school groups, families, the design, layout and content of the exhibition.
Art & SuperCityExhibition and education project evaluation report for Art, a contemporary street art participation project and exhibition by Urbis exhibitions centre; street art and youth culture company Spearfish; and St Ambrose Barlow School in Salford. The project developed young people's work on the theme of the urban environment. This was then integrated into a major architecture and public realm exhibition of the work of Will Alsop, entitled SuperCity. The evaluation gathered and analysed qualitative and quantitative information from artists, project managers, teachers, students and visitors, to determine the successes and challenges of the project and make recommendations for the future.
|
Wonderlab: National Science & Media Museum
The museum launched its first fully interactive exhibition exploring light and sound, as part of a remit to explore science, technology and media. The evaluation brief was to explore how well Wonderlab achieves learning and engagement particularly in children and young people - i.e. families and Key 2 / Key Stage 3 pupils; and working with the Explainers on the gallery floor to understand what it takes to staff and support visitors in a fully hands-on exhibition environment. The results have helped the development of other exhibitions at the museum, and across the wider Science Museum group.
Wasted - The Trouble with Rubbish: Museum of Science & IndustryExhibition evaluation for MOSI (Museum of Science & Industry). User testing the interactives and exhibition layout through creative consultation with children aged 7 to 11, to explore the exhibition's success in being suitable for a target audience of families and school groups. The exhibition intends to challenge preconceptions of waste disposal mechanisms; relay key messages about environmental choices; and identify the project's impact on professional practice to inform other exhibition development.
|
EVALUATION CPD
"I really enjoyed your workshop and learned loads - thanks!"
"I have more confidence to frame and analyse qualitative feedback usefully"
"I learned we don't have to cover all our aims with every single thing, we can look at them across a bigger picture"
"You've managed to clarify and simplify the complicated thing we've all been struggling with"
"I realised there's a lot of expertise and agreement between us, and being able to express why we make the decisions we do will be hugely helpful"
"Thinking about different definitions of impact has made a massive difference to be understanding that other people's starting points can be different to mine, it's really shone a light on things for me"
Feedback from University of Manchester / John Rylands Library colleagues.
"I have more confidence to frame and analyse qualitative feedback usefully"
"I learned we don't have to cover all our aims with every single thing, we can look at them across a bigger picture"
"You've managed to clarify and simplify the complicated thing we've all been struggling with"
"I realised there's a lot of expertise and agreement between us, and being able to express why we make the decisions we do will be hugely helpful"
"Thinking about different definitions of impact has made a massive difference to be understanding that other people's starting points can be different to mine, it's really shone a light on things for me"
Feedback from University of Manchester / John Rylands Library colleagues.
UNIVERSITY OF MANCHESTER
The Institute for Cultural Practices at the University of Manchester asked Sally to run a seminar for arts and culture undergraduates exploring the basics of evaluation. The session included a jargon buster, some basic principles, significant national evaluation frameworks the students might be likely to encounter in future employment, and practical activities designed to help them unpick the risk of bias, and to design their own evaluation framework.
The Institute for Cultural Practices at the University of Manchester asked Sally to run a seminar for arts and culture undergraduates exploring the basics of evaluation. The session included a jargon buster, some basic principles, significant national evaluation frameworks the students might be likely to encounter in future employment, and practical activities designed to help them unpick the risk of bias, and to design their own evaluation framework.
MANCHESTER HISTORIES
Manchester Histories contracted Sally to run two evaluation workshops, and write a toolkit, for their community partners as part of the Hidden Histories project, upskilling local archive and heritage collectors / keepers / researchers to develop their work using professional practice. Workshops One was for evaluation beginners and looked at terminology, and how to measure outcomes and analyse data effectively. Workshop Two was for intermediate evaluators who have some experience but wanted to refine their approaches to be more robust, reliable and applicable to national standards. A free-standing toolkit to complement the workshops and be used by others is currently in production.
Manchester Histories contracted Sally to run two evaluation workshops, and write a toolkit, for their community partners as part of the Hidden Histories project, upskilling local archive and heritage collectors / keepers / researchers to develop their work using professional practice. Workshops One was for evaluation beginners and looked at terminology, and how to measure outcomes and analyse data effectively. Workshop Two was for intermediate evaluators who have some experience but wanted to refine their approaches to be more robust, reliable and applicable to national standards. A free-standing toolkit to complement the workshops and be used by others is currently in production.
MANCHESTER METROPOLITAN UNIVERSITY
The Humanities Research Centre commissioned Sally to run a half day evaluation workshop for faculty staff, post graduate researchers, and community partners who were working together on community engagement and research activity. The session covered the basics of outcomes, outputs, ethical considerations, and how to create and reliably use engaging, accessible, creative consultation.
The Humanities Research Centre commissioned Sally to run a half day evaluation workshop for faculty staff, post graduate researchers, and community partners who were working together on community engagement and research activity. The session covered the basics of outcomes, outputs, ethical considerations, and how to create and reliably use engaging, accessible, creative consultation.
MANCHESTER BEACON FOR PUBLIC ENGAGEMENT
Manchester Beacon, a Research Councils UK Initiative, commissioned Sally to write a toolkit for those receiving its action research funding, across University of Manchester, Manchester Metropolitan University, University of Salford, and Museum of Science & Industry. Much of the guidance included is transferable. The toolkit is freely available here.
Manchester Beacon, a Research Councils UK Initiative, commissioned Sally to write a toolkit for those receiving its action research funding, across University of Manchester, Manchester Metropolitan University, University of Salford, and Museum of Science & Industry. Much of the guidance included is transferable. The toolkit is freely available here.
ENGAGE EVALUATION TOOLKIT
Engage (the National Association of Visual Arts & Gallery Education) contracted Sally to write a toolkit on Evaluation & Advocacy, as part of its Making Connections professional development programme for gallery education and art education practitioners including artists and teachers. The toolkit is available from Engage.
Engage (the National Association of Visual Arts & Gallery Education) contracted Sally to write a toolkit on Evaluation & Advocacy, as part of its Making Connections professional development programme for gallery education and art education practitioners including artists and teachers. The toolkit is available from Engage.