How we measure our impact
Our first stage of gathering impact evidence has helped us give an account of why we exist and develop a unique scale to measure the public’s ‘economic experience’ which is also our ‘Theory of Change’.
We’ve gathered evidence which helps tell the story of why public economic communication needs to change, and which raises the voices of the people we want to impact in the form of powerful moments of personal impact. Through our interventions such as our research interviewing, people have reported a profound effect on their feeling of empowerment around discussing economics. Through our research interviews, we often evoke answers like the one below from Priya, 27, South London:
"The economy is the place we live. And we don't know anything about it. We know about 5% of the things and the other 95% is not talked about so you can understand it. But you can see, from just a short amount of time, anyone can understand it and feel a lot more informed. I want to know a lot more and do a lot more. Normally no-one asks you about this."
Read more in The Case for Understandable Economics.
What we are currently working towards - NESTA’s Impact Evidence Level 2:
Economy’s next step as a young charity is to achieve funding to help us carry out a full evaluation of our impact at Level 2 NESTA Standards of Evidence. Achieving this will mean that we will be able to show the correlation of our impact with our interventions, on our unique economic experience scale. To achieve Level 2 NESTA Standard, we are implementing:
- Evaluation metrics - knowing what to measure
- Pre- and post-survey
- Cohort study
- Regular surveys
- Interviews (before and after intervention)
We are carrying this out in our first full programme of our Act mission in delivering our Discovering the Economy toolkit as a programme in schools. Our ambition is to scale the impact of understandable economics throughout communication of economics in the public sphere by providing full guidance and training which means our impact can be replicated by new economic communicators.
Evaluating our 'Theory of Change'
We take our audience on a journey from 'I'm not part of the conversation' through to 'Identity', then 'Understanding' and finally 'Empowerment'. A brief description defining these stages is below:
| Stage | Description |
|---|---|
| I'm not part of the conversation | Economics is nothing to do with me, boring, just money or stress, something I can't understand and not something I feel able to discuss, question or shape. |
| Identity | Highlighting the economics in everyday life, for those who feel it has nothing to do with them |
| Understanding | Demystifying jargon and breaking economics down in simple terms, for those who feel it’s unnecessarily complicated |
| Empowerment | Giving people ways to question and shape economics, for those who feel they can’t change it |
Our current evaluation metrics for each stage are as follows:
| Stage | What this looks like |
|---|---|
| I’m not part of the conversation | Economics is a fearful topic of conversation and something I would avoid - "Economics, oh I'm too stupid" |
| Identity | - Identifies that economics is a part of their life - Perceives economics to be to do with them as an individual - Views economics as connected to things he or she already cares about - Reports the relevance of learning more about economics to their daily life |
| Understanding | - Reports a sense of clarity around key concepts of economics - Reports an increased awareness and understanding of how economics relates to specific areas of their day to day life - Reports an increased level of understanding of the definitions of and rationale behind key economic concepts in the news cycle, such as trade agreements, quantitative easing, budget deficit - Identifies as being better able to understand media discussions on economic topics |
| Empowerment | - Identifies as feeling able to engage in conversations about the economy to the level they would like to - Reports that learning more about economics is something they do/will actively undertake - Identifies as having a feeling of affect on economics in specific spheres of influence - Identifies that they know how to access economics and when it might be of use to them - Identifies economics is a useful way to solve problems they have - Reports they feel able to make informed decisions when voting about a policy which would affect the economy - Identifies as desiring to shape the economy - Reports that they feel more able to apply economic knowledge to improve their quality of life in terms of personal finances and voting choices |