GoodMeasure
Results
Prepared for
McKenzie Centre
24th July 2024
+
McKenzie Centre is dedicated to enabling children with disabilities and their whānau to have great lives through realising their child's potential.
How ImpactLab defines social value
Social value is the estimated social impact in dollar terms that a programme achieves for participants over their lifetime.
Throughout our lives, different events occur that impact our overall wellbeing journey. ImpactLab measures the impact on an individual’s wellbeing across multiple domains when they’re supported by a programme to make positive changes in their life.
We measure this impact in terms of both positive benefits (such as increased income) and avoided costs to government.
To calculate social value, we combine these impact values with:
- evidence from global literature about how effective a programme can be;
- the size of the opportunity for the people a programme serves to achieve more positive outcomes; and
- the number of people supported.
By combining these inputs, the social value calculation helps us understand how a programme or intervention helps change lives for the better. We combine the social value with cost information to calculate a programme’s social return on investment.
The four key components to measure social value
Outcomes
What positive long-term changes in participants’ lives does McKenzie to create?
Effectiveness
What academic evidence is there about how effective a programme like McKenzie Centre can be at achieving those changes?
Population
How many people does McKenzie Centre reach, and how many engage long enough to meaningfully benefit?
Opportunity
Who does McKenzie Centre serve, and what is the opportunity to make a difference for those people?
= Social Value
Scope - McKenzie Centre GoodMeasure
This GoodMeasure estimates the social value created by McKenzie Centre during the period of 1st January 2023 - 31st December 2023.
McKenzie Centre is an early intervention centre that serves young children (ages 0 - 5) with disabilities and their whānau. The Centre adopts a transdisciplinary approach that brings together the knowledge and expertise of both whānau and specialists to design and implement bespoke development plans for each child.
Children and whānau enrolled into the programme attend weekly early intervention play-based sessions, facilitated by an early intervention team who specialise in various disciplines. These sessions continue to inform the child's development plan until they are ready to attend school.
McKenzie Centre invests in its community by hosting parent focused workshops, hangout sessions, and other events that build parent capability and confidence whilst growing their connection to the community.
For children and whānau on the waiting list, weekly early intervention playgroup sessions are available. These sessions are facilitated by 2 staff and a Tuakana who are available to provide support in the interim.
Population: McKenzie Centre
How many people do you reach
and how many engage enough to meaningfully benefit?
Groups | Total starting | Total engaging | Engagement rate |
Children with disabilities | 127 | 112 | 88% |
Whānau/Family members (Adults) | 161 | 146 | 91% |
Siblings | 23 | 23 | 100% |
Total | 311 | 281 | 90% |
Note: Social value is only counted per meaningfully engaged participant, while cost is per participant enrolled with McKenzie Centre during the period in scope.
Starting definition: Child with a disability or developmental delay and their whānau who were enrolled at McKenzie Centre during the period in scope.
Meaningfully Engagement definition: Children and their whānau who attended McKenzie Centre for 10 or more weeks.
Assumption: All children and their whānau enrolled during the period in scope have been included in this measure.
GoodMeasure Metrics: McKenzieCentre
$3,003,096
Total social value
Social Value: The estimated lifetime social value that this programme generated for participants during the measurement period.
$10,687
Social value per person
Social Value per person: When we divide your total social value by the total number of people that meaningfully engage in the programme, we can derive a social value per person.
$1: $1.80
SROI
SROI: When we divide your total social value by your total operational costs the result is your SROI - the measurable social value as a proportion of programme cost.
GoodMeasure Metrics: McKenzieCentre
$20,508
Social Value per child
Social Value per person: When we divide your total social value by the total number of people that meaningfully engage in the programme, we can derive a social value per person.
$4,515
Social value per adult
Social Value per person: When we divide your total social value by the total number of people that meaningfully engage in the programme, we can derive a social value per person.
$2,046
Social value per sibling
Social Value per person: When we divide your total social value by the total number of people that meaningfully engage in the programme, we can derive a social value per person.
Outcomes: McKenzie Centre
What positive long-term changes in peoples’ lives does McKenzie Centre help to create?
GoodMeasure Outcomes | Additional Outcomes | |
Children with disabilities | All | |
Improve mental health | Improved developmental trajectories | |
Improve physical health | Improved family functioning | |
Increase academic achievement (NCEA 3) | Improved social skills | |
Increase employment | Increased confidence | |
Reduce risky behaviour | Improved community connection | |
Reduce offending | Increased independence | |
Reduce addiction | Increased family empowerment | |
Increase communication abilities | ||
Whānau/Family level outcomes | Enhanced emotional regulation | |
Reduce family violence | Reduce stress | |
Reduce victimisation (family violence) | Reduce anxiety | |
Reduce child placement | Reduce need for intensive services | |
Improve infant safety | ||
Whānau/Family members (Adults) and Siblings | ||
Improve mental health | ||
Reduce emergency benefit | ||
Reduce risky behaviour | ||
Reduce offending | ||
Reduce addiction |
McKenzie Centre (MC) is considered to be highly effective across a range of outcomes due to the breadth and depth of its service offerings for children with disabilities and their whānau. The Centre's early intervention, transdisciplinary approach focuses on the whole whānau, helping them to be informed and better equipped to lead flourishing lives and address their unique challenges with confidence.
According to a rapid literature search, MCs' method is highly effective at assessing the unique needs of participants and connecting them with a well-oiled, highly trained team of professionals who can service specific needs. This support is long-lasting and comprehensive, including parental programmes, community-building activities, sibling involvement, and more, resulting in outcomes that range from improvements in mental and physical well-being to academic achievement and behavioural outcomes.
Improve mental health: This outcome reflects the comprehensive range of supports offered by MC for all members of the whānau. Research indicates that children with disabilities and their whānau are at higher risk of poor mental well-being as compared with the general population. However, this risk is alleviated by early intervention programmes that engage parents early on, foster skill development among children, nurture positive familial relationships, and promote empowerment through trustworthy, personalised support.
Improve physical health: This outcome was applied to children attending the physiotherapist and/or occupational therapist as a means to reflect the impact of this mahi on the child's long-term physical well-being.
Increase academic achievement (NCEA 3): This outcome reflects the importance of early intervention programmes in addressing challenges relating to children with disabilities functioning productively within the mainstream school system. Research connects the following aspects of MC with increased academic achievement:
- Early intervention support and its impact on brain development
- Transition support for children as they enter the schooling system
- Play therapy and socialisation
- Speech and language therapy
- Whānau capacity and knowledge building
Reduce family violence and Reduce victimisation (family violence): These outcomes were included due to the prevalence of family violence within families who have one or more children with a disability. This violence is often attributed to parental stress and the sense of helplessness they may feel as they find themselves unable to cope with the demands of their child's unique needs. Based on a rapid literature review, ImpactLab considers MC's early intervention and family-centered approach to be an effective means of growing whānau confidence, understanding, and ability whilst providing a positive community for them to be involved in. Overall, these aspects of MC enhance family functioning and reduce the prevalence of family violence.
Reduce child placement This outcome is included as a consequence of increases in family functioning as a result of MC's family-centered approach.
Increase employment: This outcome is a flow-on effect of the child's improved attendance and success within the mainstream schooling system. Although this outcome was included at a low rate, ImpactLab believes it is important to show the long-term employability impacts of early intervention.
Reduce offending - addiction - risky behaviour: These outcomes capture the impact of MC's wide-ranging support on the long-term behavioral outcomes of children with disabilities and their whānau. Reduced parental stress, healthy sibling relationships, and children with disabilities reaching their developmental potential are associated with decreased risks in risk behaviours such as substance abuse.
Demographics: McKenzie Centre
Who do you serve?
Children with disabilities
Age
100% of all Children were between the ages of 0 - 6.
Gender
73% of all Children were male and 27% female.
Ethnicity
The largest ethnicity group was Māori, followed by European (incl. Pākeha).
Whānau/Family members (Adults) + Siblings
Age
All adult participants were aged between 19 - 65 and sibling participants between 0 - 6.
- 73% of all children participants were male.
- 40% of all children participants were of Māori ethnicity.
- 100% of all sibling participants were also between the ages of 0 - 6.
Opportunity: McKenzie Centre
Who do you serve, and what is the opportunity to make a difference for those people?
Children with disabilities
Experienced Family Violence
The sample data received for children with disabilities showed that 15% of children had experienced family violence.
This data informed the opportunity rates for the following outcomes:
- Reduce family violence
- Reduce victimisation (family violence)
Oranga Tamariki (OT) Involvement
The sample data received for children with disabilities showed that 12% of children had experienced at some point, OT (Oranga Tamariki) involvement.
This data informed the opportunity rates for the Reduce child placement outcome.
Disabilities
The most common diagnosis among all children was Autism Spectrum Disorder (ASD), followed by multiple disabilities and intellectual disabilities.
32% of all children were NOT formally diagnosed but exhibited signs of a potential diagnosis.
Disability data was used to inform opportunity rates for sub-populations relating to Improve mental health and Increase academic achievement outcomes.
Whānau/Family members (Adults)
Education, Employment and Training Factors
The sample data received for whānau/family members (adults) showed that:
- 26% of whānau/family members (adults) were unemployed
- 26% of whānau/family members (adults) were receiving a hardship benefit
This data informed the opportunity rate for the Reduce hardship benefit outcome, as well as the unemployed sub-population rate for this outcome.
The opportunity input seeks to reflect the proportion of participants who have the opportunity to achieve a particular outcome as a result of engaging in the programme.
Further information - Children with disabilities:
- Improve physical health: ImpactLab used the number of children attending Physiotherapy and/or Occupational Therapy as the opportunity rate for the Improve physical health outcome (68%).
Further information - Whānau/Family members (adults):
- Mental health: ImpactLab used a sub-population rate for the opportunity rate that reflects the higher percentage of carers whom suffer from poor mental health as compared with the general population.
Disability Sector - Data availability: Public research and data are limited in reflecting the spectrum of disability and lifetime outcomes. There is also limited caring responsibilities, costs or impacts. For these reasons, ImpactLab is limited in the degree to which it can reflect disability specific opportunity rates within Social Value calculations.
Note: In setting opportunity rates for MC, ImpactLab used a sample of 34 (the number of children who started during the period in scope to reflect the percentages of the wider population of those enrolled during this same period.
Costs: McKenzie Centre
$1,647,753
Total investment
Total investment: The sum of the programme's direct and indirect costs.
$1,238,382
Direct costs
Direct costs: The expenses involved with running the prorgamme.
$409,371
Indirect costs
Indirect costs: Expenses that don't come from directly running the programme but are still needed as part of it.
$5,298
Total cost per starting participant
Total cost per starting participant: The total investment divided by the number of starting participants.
The above costs stated are for the period in scope, 1st January 2023 - 31st December 2023. These costs are assumed to be representative of the investment required to achieve programme outcomes.
Future opportunities
What might you do to enhance your view of the social value McKenzie Centre creates?
Options to improve programme effectiveness data
Progress
- If feasible, track participant physical and mental well-being pre-engagement and once per year during their engagement period.
- Track the experience of children as they enter school . How well do they adjust to this environment?
- Track outward referrals. What percentage of referrals are taken up? What is the result of referral take up?
Options to improve opportunity and population data
- Collect separate ethnicity, gender and age data on whānau/family members (adults) and siblings
- If feasible, collate data on Oranga Tamariki involvement, family violence incidents, and hardship benefit usage into a centralised spreadsheet.
- If feasible, collate information about the substance use behaviours of whānau/family members (adults).
Next Steps
What you could do with your results
Share your GoodMeasure report with funders and supporters
- Funders will appreciate your commitment to impact measurement as it demonstrates transparency and a desire to do good, better. Invite input from both existing and prospective funders. They may have helpful insights and advice – or additional funding – to help you further increase impact.
Review your GoodMeasure report as a team
- Discuss key insights from the Report with your leadership, programme and finance team. How could you increase your impact even further?
Celebrate with your team!
- You’ve worked hard to achieve the SROI figure captured in your GoodMeasure Report, it’s important to pause every so often to reflect on the impact you’re achieving together.
Book a GoodMeasure Renewal
- ImpactLab offers a GoodMeasure Renewal, allowing you to understand how the changes you've implemented have impacted your SROI. Ask your Lead Analyst for more details.
How we utilise research
Research informs the Outcome and Effectiveness components of our social value equation. We conduct a rapid literature review to:
- Identify what long-term outcomes your programme is having on participants (by substantiating the link between the activities of the programme and outcomes for participants).
- Estimate the effectiveness of your programme in achieving each outcome by checking how closely it follows proven methods for the participants in the programme.
- Understand the people you work with and your Theory of Change
- Using your Theory of Change, we build an Intervention Logic to substantiate quantifiable impacts
- Test your programme against domestic and international academic literature
Summary of Inputs and Assumptions
The inputs and assumptions used to calculate your SROI
Inputs to this SROI | Key Assumptions | |
Outcomes |
|
|
Effectiveness |
|
|
Population |
|
|
Opportunity |
| How people were counted
Inclusions/exclusions
|
Cost |
|
|
Key References
This is a selection of the evidence we used to calculate programme effectiveness:
Aktan, Osman, Şenol Orakcı, and Mehmet Durnalı. "Investigation of the relationship between burnout, life satisfaction and quality of life in parents of children with disabilities." European Journal of Special Needs Education 35.5 (2020): 679-695.
Dawson, Geraldine. "Early behavioral intervention, brain plasticity, and the prevention of autism spectrum disorder." Development and psychopathology 20.3 (2008): 775-803.
Fisman, Sandra, et al. "A longitudinal study of siblings of children with chronic disabilities." The Canadian Journal of Psychiatry 45.4 (2000): 369-375.
Guralnick, Michael J. "Why early intervention works: A systems perspective." Infants & young children 24.1 (2011): 6-28.
Kresak, Karen, Peggy Gallagher, and Cheryl Rhodes. "Siblings of infants and toddlers with disabilities in early intervention." Topics in Early Childhood Special Education 29.3 (2009): 143-154.
Lamsal, Ramesh, and Wendy J. Ungar. "Impact of growing up with a sibling with a neurodevelopmental disorder on the quality of life of an unaffected sibling: a scoping review." Disability and rehabilitation 43.4 (2021): 586-594.
Lifter, Karin, et al. "Overview of play: Its uses and importance in early intervention/early childhood special education." Infants & Young Children 24.3 (2011): 225-245.
Meltzer, Ariella. "What is ‘sibling support’? Defining the social support sector serving siblings of people with disability." Social Science & Medicine 291 (2021): 114466.
Nelson, Helen Jean, et al. "Models, theoretical design and formal evaluation of integrated specialist community health service provision for the first 2000 days: a scoping review." BMJ open 13.5 (2023): e070042.
Smythe, Tracey, et al. "Early intervention for children with developmental disabilities in low and middle-income countries–the case for action." International health 13.3 (2021): 222-231.
Stancheva-Popkostadinova, Vaska, and Tatjana Zorcec. "14 Play in Early Intervention for Children with Disabilities." Play development in children with disabilties (2017): 9783110522143-016.
Staunton, Emma, Claire Kehoe, and Louise Sharkey. "Families under pressure: Stress and quality of life in parents of children with an intellectual disability." Irish journal of psychological medicine 40.2 (2023): 192-199.
Definitions
Intervention - An intentional process through which a defined group of people have the opportunity to create a positive change in their life trajectory.
Intervention type - A categorisation to group similar interventions based on their activities (i.e. how resources are used). These categories have been developed by ImpactLab based on academic literature and the input of organisations participating in the SROI process.
Organisation - The organisation delivering the programmes measured.
Programme - The unit of measurement of an SROI which consists of one or more interventions.
Participant - A person or group of people for whom a programme exists to make a positive difference.
Sector - The part of the charity or social sector within which the organisation primarily operates. This is an organisation-level categorisation.
Social value - The social impact in dollar terms that the amount invested achieves for participants over their lifetime. The social value is calculated by combining outcome values with a service delivery quality score, the size of the opportunity to support a population, and the number of people supported.
Limitations
- The themes analysed in this report are based on observed correlations and provide broad conclusions rather than tight causative claims.
- Programme intervention practices are determined via narrative and operational data provided by an organisation. It does not include direct observation of programmes, and as such social value forecasts do not capture variation in programme practice e.g., in workforce skills or programme fidelity across locations.
- Comparisons should be considered indicative only, as metrics can be influenced by a variety of factors, including differences in data quality, scoping decisions, improvements to methodology over time and limitations in the available academic literature.
- Many aspects of social impact cannot appropriately be quantified in dollar terms, and SROI findings should be considered alongside other important sources of information such as participant feedback and more bespoke forms of evaluation.
- GoodMeasure is a standardised measurement model — different interventions are treated as consistently as possible to enable comparability, which means the uniqueness of each intervention is not fully reflected.
- Cost and participant data inputs are provided by the organisation. Responsibility sits with each organisation to ensure their data is accurate and genuinely reflects the programme.
- Estimates have varying confidence levels due to differing quality and availability of data inputs. The GoodMeasure methodology takes the approach of using the data that is available in order to support ongoing data improvement.
- The lifetime (dollar) value of an outcome is conservatively valued over a 5-year period. This is aligned with New Zealand Treasury’s approach of measuring impact within a contained period.
Disclaimer
This disclaimer sets out important information about the scope of ImpactLab Limited’s services. ImpactLab endeavours to ensure that all material and information used for and presented in any GoodMeasure, including ROI calculations and impact numbers, is accurate and reliable (information). However, the information is based on various sources, including information organisations provide to ImpactLab which is not independently verified. ImpactLab does not make any representations or warranties in respect of information it uses or presents in relation to any GoodMeasure or this report. This includes any representation or warranty relating to the accuracy, adequacy, availability or completeness of information, or that it is suitable for its intended use. ImpactLab does not provide advice or make recommendations for any decisions made by any person, financial or otherwise, either in relation to any GoodMeasure, or this report. Data and percentages stated in this report may have been rounded.
Aggregated data stated in this report is based upon data provided to ImpactLab pursuant to its privacy policy and terms and conditions. Data ImpactLab uses except in exceptional circumstances must be aggregated and anonymised so that no participant in any programme ImpactLab analyses can be identified within data Impactlab uses or produces. Where ImpactLab uses the New Zealand Integrated Data Infrastructure (IDI), it does so subject to the conditions for access set by Stats NZ for IDI data users.
Attribution
Some data and information used in the Social ROI calculations is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) Licence. It is attributed to the NZ Treasury.
ImpactLab - Do good, better.
ImpactLab’s vision is to help create a world where investment works for communities, so that people can live the lives they choose.
We work with the best available data and evidence to estimate social value in dollar terms – a metric that is measurable, consistent, and comparable.
We build close relationships with our customers to understand them, and the needs of their communities, supporting them to do good, better.
Contact:
04 391 2755
[email protected]
LinkedIn | Facebook
Level 1, 2-8 Chancery Street, Auckland, 1010
Level 8, 187 Featherston Street, Wellington 6011
© ImpactLab Ltd, 2024. All rights reserved