GoodMeasure
Results
Prepared for
Home & Family Charitable Trust
Home & Family are dedicated to make an impact within communities by supporting parent’s behaviour to be more suitable to the needs of their children.
The GoodMeasure Objective
Through the GoodMeasure, Home & Family aims to:
Assess the effectiveness and social value of the Kōmanawa Parenting programme, and to consider potential for wider implementation beyond its current reach in Canterbury.
How ImpactLab defines social value
Social value is the estimated social impact in dollar terms that a programme achieves for participants over their lifetime.
Throughout our lives, different events occur that impact our overall wellbeing journey. ImpactLab measures the impact on an individual’s wellbeing across multiple domains when they’re supported by a programme to make positive changes in their life.
We measure this impact in terms of both positive benefits (such as increased income) and avoided costs to government.
To calculate social value, we combine these impact values with:
- evidence from global literature about how effective a programme can be;
- the size of the opportunity for the people a programme serves to achieve more positive outcomes; and
- the number of people supported.
By combining these inputs, the social value calculation helps us understand how a programme or intervention helps change lives for the better. We combine the social value with cost information to calculate a programme’s social return on investment.
The four key components to measure social value
Outcomes
What positive long-term changes in participants’ lives does help to create?
Effectiveness
What academic evidence is there about how effective a programme like can be at achieving those changes?
Population
How many people does reach, and how many engage long enough to meaningfully benefit?
Opportunity
Who does serve, and what is the opportunity to make a difference for those people?
= Social Value
Scope: 1/10/2023 - 30/09/2024
SafeCare: This branch of Kōmanawa parenting is a 1:1 parent training programme for parents and caregivers of tamariki aged 0-5. It uses a skill-based curriculum that builds on the knowledge, experience and existing strengths of a parent/caregiver. SafeCare comprises of three modules; (1) Parent-infant/child interaction that targets risk factors associated with neglect and physical abuse, (2) Health that targets risk factors for medical neglect, and (3) Home Safety that targets risk factors for environmental neglect and unintentional injury. While there is a core structure to the delivery of the SafeCare content, Home & Family also provide wraparound support and social work services to whānau who need it, operating on a case-by-case basis, tailoring each intervention to the parent and children’s needs.
Brief Intervention: This branch of Kōmanawa parenting is for parents and caregivers of children aged 6-18. It assists parents in setting goals to enhance their understanding of child development and expectations. Also known as Intentional Parenting, it explores the impact of parental behaviours, choices, and characteristics on the child and their behaviour. By fostering introspection, this programme supports parents in analysing and implementing strategies to get the best response from their children, who are typically displaying concerning behaviours within family-system relationships.
Excluded: All other Home & Family Services
GoodMeasure Metrics: Kōmanawa Parenting
Total Social Value
$679,794
Social Value: The estimated lifetime social value that this programme generated for participants during the measurement period.
Social Value Per Person
Total: $4,962
SafeCare Parents: $3,565
SafeCare Kids: $17,229
Brief Intervention Parents: $2,431
Brief Intervention Kids: $1,250
Social Value per person: When we divide your total social value by the total number of people that meaningfully engage in the programme, we can derive a social value per person.
These figures are separated out into each participant group, since there are parents and kids across both programmes who can benefit from Kōmanawa Parenting.
SROI
$1: $2.20
SROI: When we divide your total social value by your total operational costs the result is your SROI - the measurable social value as a proportion of programme cost.
Outcomes:
What positive long-term changes in peoples’ lives does help to create?
SafeCare
GoodMeasure Outcomes | Additional Outcomes |
These outcomes contribute directly to your SROI | These outcomes contribute indirectly to your SROI |
Increase academic achievement | Increase health literacy |
Improve housing | Reduce child maltreatment |
Improve infant safety | Improve home safety |
Improve mental health | Increase access to social services |
Improve physical health | Reduce stress |
Reduce addiction | Improve parent-child relationship |
Reduce child placement | Improve parenting skills |
Reduce family violence | Reduce family separation |
Reduce offending | Improve behaviour |
Reduce risky behaviour | |
Reduce victimisation (family violence) |
Brief Intervention
GoodMeasure Outcomes | Additional Outcomes |
These outcomes contribute directly to your SROI | These outcomes contribute indirectly to your SROI |
Improve mental health | Reduce child maltreatment |
Reduce addiction | Improve home safety |
Reduce child placement | Improve behaviour |
Reduce family violence | Improve parent-child relationship |
Reduce offending | Improve parenting skills |
Reduce risky behaviour | |
Reduce victimisation (family violence) |
Increase academic achievement: Measures increased income and government savings associated with different levels of academic attainment
Improve housing: Measures cost savings associated with increased housing quality
Improve infant safety: Measures hospital costs associated with infant safety
Improve mental health: Intrinsic measurement of an improvement in mental health
Improve physical health: Intrinsic measurement of an improvement in physical health
Reduce addiction: Measures government costs associated with addiction
Reduce child placement: Measures government costs associated with child placement
Reduce family violence: Measures government costs associated with family violence
Reduce offending: Measures increased income and government savings associated with reduced offending
Reduce risky behaviour: Measures government costs associated with risky behaviour
Reduce victimisation (family violence): Measures hospitalisation and emergency accommodation costs associated with family violence
Increase academic achievement
- Early intervention through parenting programmes is linked with better academic outcomes. This is seen through increased executive functioning, advanced language skills, improved classroom behaviour, greater math skills, higher academic achievement and engagement in education in general, and better school adjustment (Sandler et al., 2011).
Improve housing
- This outcome was applied due to referrals which are made by Home & Family social workers to Kainga Ora.
Improve infant safety
- Improving infant safety is a strong component of the SafeCare programme. Home safety and Infant and Child healthcare are both core programme components which are designed to limit home hazards and improve health-related behaviour and knowledge.
- In a review paper, it was found that parenting programmes can help reduce the likelihood of child injuries or ingestion of hazardous substances (Sandler et al., 2011).
- Further evidence suggests that SafeCare can help reduce the number of hazards in the home to create a safer home environment for infants and toddlers (Gershater-Molko et al., 2003).
Improve mental health
- Kids: Parent training in behaviour management has been found to be one of the proven interventions for the prevention of behavioural disorders (Kieling et al., 2011). SafeCare is likely to improve the long-term mental health of children, given that research has found it to be associated with small to moderate reductions in externalising behaviours (Burke Lefever et al., 2017), including conduct problems (Dretzke et al., 2009), and lower risk of diagnosis of externalising disorders (Sandler et al., 2011. A range of studies also highlight how parenting programmes in general can help improve emotion regulation skills and reduce internalising problems (Sandler et al., 2011). These effects are found to be sustained for at least two years after the programme has ended.
- Parents: SafeCare has been found to be associated with small to moderate reductions in stress (Culbreth et al., 2024). Given that stress is a strong predictor of poor mental health, it is reasonable to suggest that this SafeCare can help improve mental health in general as well. There is some evidence to suggest that anxious/depressed symptoms are significantly reduced for caregivers following the SafeCare intervention (Romano et al., 2020). There is also evidence from other papers (Silovsky et al., 2023; La Barrie et al., 2023) which highlight more specifically that SafeCare can help improve levels of depression and PTSD
Improve physical health
- While there is a lack of evidence demonstrating the direct relationship between SafeCare and improved health for children, there is strong evidence that SafeCare can help improve a parent’s health knowledge and behaviour (Gershater-Molko et al., 2003; Oppenheim-Weller et al., 2020), meaning their child is likely to receive a better health response from them when needed.
Reduce addiction, Reduce offending, and Reduce risky behaviour
- Parenting programmes are associated with long-term benefits for kids, such as less substance use, lesser intentions to use illegal substances, and actual drug use itself (Sandler et al., 2011).
- In a study by Silovsky and colleagues (2023), SafeCare was found to have no significant effect on alcohol or drug use in parents.
- The literature suggests that preventative parenting programmes are one of the most effective interventions at reducing offending outcomes later in life (Prinz & Sandler, 2007; Farrington et al., 2022). Findings from a review paper suggest children whose parents are involved in these programmes are demonstrate less delinquency and involvement with the legal system than comparison groups (Sandler et al., 2011).
- There is evidence that parenting programmes can reduce levels of aggression for parents (Romano et al., 2020).
Reduce child placement and Reduce Family Violence
- SafeCare has been found to have a moderate effect on positive parental behaviour (e.g., praising child), proactive parenting, dysfunctional parent-child interactions, and parental distress (Whitaker et al., 2020), a small effect on harmful behaviours such as neglect, physical punishment, and psychological aggression (Gallitto et al., 2021), as well as intimate partner violence victimisation (Silovsky et al., 2023), and a strong effect on parents’ perception of their child’s behavioural problems, and their ability to cope with them (Arruabarrena et al., 2019).
- More directly, SafeCare has been suggested to be more effective at reducing Child protection service recidivism than services as usual or the counterfactual of no intervention (Chaffin et al., 2012).
- There is evidence that the duration of parenting programmes may have a moderating effect on maltreatment outcomes, whereby shorter or longer interventions appear to be more effective than medium length interventions (Euser et al., 2015).
Effectiveness:
What academic evidence is there about how effective a programme like Kōmanawa parenting can be at achieving long-term changes?
Intervention Logic
Opportunity:
Who do you serve, and what is the opportunity to make a difference for those people?
Population:
Who do you serve, how many people do you reach,
and how many engage enough to meaningfully benefit?
Groups | Total starting | Total engaging | Engagement rate |
SafeCare Parents | 21 | 18 | 86% |
SafeCare Kids | 33 | 27 | 82% |
Brief Intervention Parents | 50 | 30 | 60% |
Brief Intervention Kids | 99 | 62 | 63% |
Total | 203 | 137 | 67% |
Note: Social value is only counted per engaged participant, while cost is per starting participant.
💡Insight: The engagement rate is much higher for SafeCare than it is for the brief intervention programme
SafeCare
Starting definition: Parent/Caregiver completes baseline assessment
Meaningful engagement definition: Parent/Caregiver attends at least 4 weeks
Brief Intervention
Starting definition: Parent/Caregiver makes initial contact
Meaningful engagement definition: Parent/Caregiver attends at least 2 weeks
Costs: Home & Family
Total Investment
$306,090
Total investment: The sum of the programme's direct and indirect costs.
Direct Costs
$270,596
Direct costs: The expenses involved with running the prorgamme.
Indirect Costs
$35,494
Indirect costs: Expenses that don't come from directly running the programme but are still needed as part of it.
Total Cost Per Starting Participant
$1,508
Total cost per starting participant: The total investment divided by the number of starting participants.
Costs only represent the cost of the Kōmanawa Parenting programme, and are stated for the period in scope: 1 Oct 2023 - 30 Sep 2024
Increasing Social Value
What possibilities you have to do even more good
Options to improve programme effectiveness data
Long-term effectiveness
- Keep tracking child placement events in the families you work with, particularly after you finish working with them.
- Collect data on other post-programme outcomes, especially for mental and physical health.
- Conduct participant surveys mid-way through the programme, as well as post-programme.
- Build a case study to inform long term participant impact (for adults and children).
- Identify if there is potential to build on referral networks to provide further support where needed.
Options to improve population data
Tracking engagement
- Work on improving the engagement rate for the Brief Intervention programme. Identify barriers to successful completion and focus on which of these could be addressed.
- Create a system where you can identify early who is at risk of disengaging and implement strategies to prevent this from happening.
- Identify if there is an opportunity to reach families sooner.
Options to improve opportunity data
Opportunity data
- Participants within this scope were identified as experiencing high levels of risk for family violence and child placement, but numbers for mental health rates are not as high as might have been expected. Ensure data collection for these factors is accurate.
- Collect opportunity data at individual level for kids in the brief intervention programme.
- Identify if there is potential to further reach the targeted population.
- Identify potential partnerships with external agencies to collect data that could further help identify the level of risk families face.
Next Steps
What you could do with your results
Share your GoodMeasure report with funders and supporters
- Funders will appreciate your commitment to impact measurement as it demonstrates transparency and a desire to do good, better. Invite input from both existing and prospective funders. They may have helpful insights and advice – or additional funding – to help you further increase impact.
Review your GoodMeasure report as a team
- Discuss key insights from the Report with your leadership, programme and finance team. How could you increase your impact even further?
Celebrate with your team!
- You’ve worked hard to achieve the SROI figure captured in your GoodMeasure Report, it’s important to pause every so often to reflect on the impact you’re achieving together.
Book a GoodMeasure Renewal
- ImpactLab offers a GoodMeasure Renewal, allowing you to understand how the changes you've implemented have impacted your SROI. Ask your Lead Analyst for more details.
How we utilise research
Research informs the Outcome and Effectiveness components of our social value equation. We conduct a rapid literature review to:
- Identify what long-term outcomes your programme is having on participants (by substantiating the link between the activities of the programme and outcomes for participants).
- Estimate the effectiveness of your programme in achieving each outcome by checking how closely it follows proven methods for the participants in the programme.
- Understand the people you work with and your Theory of Change
- Using your Theory of Change, we build an Intervention Logic to substantiate quantifiable impacts
- Test your programme against domestic and international academic literature
Summary of Inputs and Assumptions
The inputs and assumptions used to calculate your SROI
Inputs to this SROI | Key Assumptions | |
Outcomes |
|
|
Effectiveness |
|
|
Population |
| How people were counted
|
Opportunity |
|
|
Cost |
|
|
Key References
This is a selection of the evidence we used to calculate programme effectiveness:
Anand, L., et al. "Mindful parenting: A meta-analytic review of intrapersonal and interpersonal parental outcomes." Current Psychology, 42.10 (2023): 8367-8383.
Arruabarrena, I., et al. "Implementation of an early preventive intervention programme for child neglect: Safecare." Psicothema, 31.4 (2019): 443-449.
Burke Lefever, J. E., et al. "Long-term impact of a cell phone–enhanced parenting intervention." Child maltreatment, 22.4 (2017): 305-314.
Chaffin, M., et al. "A statewide trial of the SafeCare home-based services model with parents in Child Protective Services." Pediatrics, 129.3 (2012): 509-515.
Culbreth, R., et al. "Adaptation of SafeCare, an evidence-based parenting program, for caregivers of infants in the neonatal intensive care unit." Applied Nursing Research, 78 (2024): 151817.
Dretzke, J., et al. "The clinical effectiveness of different parenting programmes for children with conduct problems: a systematic review of randomised controlled trials." Child and adolescent psychiatry and mental health, 3 (2009): 1-10.
Euser, S., et al. "A gloomy picture: A meta-analysis of randomized controlled trials reveals disappointing effectiveness of programs aiming at preventing child maltreatment." BMC public health, 15 (2015): 1-14.
Farrington, D. P., et al. "Effectiveness of 12 types of interventions in reducing juvenile offending and antisocial behaviour." Canadian journal of criminology and criminal justice, 64.4 (2022): 47-68.
Gallitto, E., et al. "Investigating the impact of the SafeCare program on parenting behaviours in child welfare-involved families." Child and Adolescent Social Work Journal, 38.1 (2021): 115-126.
Gershater-Molko, R. M., et al. "Project SafeCare: Improving health, safety, and parenting skills in families reported for, and at-risk for child maltreatment." Journal of family violence, 18 (2003): 377-386.
Kieling, C., et al. "Child and adolescent mental health worldwide: evidence for action." The lancet, 378.9801 (2011): 1515-1525.
La Barrie, D. L., et al. "The Initial Outcomes of SafeCare® on the Physiological and Behavioral Outcomes of Black Mothers Who Have Experienced Significant Trauma." Child & Family Behavior Therapy, 45.4 (2023): 366-393.
McCann, D. “Experiences of Education for Children in Care: Part 2: Review of New Zealand Government Data.” Oranga Tamariki Voices of Children and Young People Team (2019).
Oppenheim‐Weller, S., et al. "Evaluating SafeCare® in Israel: Benefits for the families." Child & Family Social Work, 25.3 (2020): 665-673.
Prinz, R. J., and Sanders, M. R. "Adopting a population-level approach to parenting and family support interventions." Clinical psychology review, 27.6 (2007): 739-749.
Rogers-Brown, J. S., et al. "Behavior change across implementations of the SafeCare model in real world settings." Children and Youth Services Review,117 (2020): 105284.
Romano, E., et al. "Does the SafeCare parenting program impact caregiver mental health?" Journal of Child and Family Studies, 29 (2020): 2653-2665.
Sandler, I. N., et al. "Long-term impact of prevention programs to promote effective parenting: Lasting effects but uncertain processes." Annual review of psychology, 62.1 (2011): 299-329.
Silovsky, J., et al. "Risk and protective factors associated with adverse childhood experiences in vulnerable families: Results of a randomized clinical trial of SafeCare®." Child maltreatment, 28.2 (2023): 384-395.
Townshend, K., et al. "The effectiveness of mindful parenting programs in promoting parents’ and children's wellbeing: a systematic review." JBI Evidence Synthesis, 14.3 (2016): 139-180.
Von Borczyskowski, A., et al. "Alcohol and drug abuse among young adults who grew up in substitute care—Findings from a Swedish national cohort study." Children and Youth Services Review, 35.12 (2013): 1954-1961.
Whitaker, D. J., et al. "Effect of the SafeCare© intervention on parenting outcomes among parents in child welfare systems: A cluster randomized trial." Preventive Medicine, 138 (2020): 106167.
Definitions
Intervention - An intentional process through which a defined group of people have the opportunity to create a positive change in their life trajectory.
Intervention type - A categorisation to group similar interventions based on their activities (i.e. how resources are used). These categories have been developed by ImpactLab based on academic literature and the input of organisations participating in the SROI process.
Organisation - The organisation delivering the programmes measured.
Programme - The unit of measurement of an SROI which consists of one or more interventions.
Participant - A person or group of people for whom a programme exists to make a positive difference.
Sector - The part of the charity or social sector within which the organisation primarily operates. This is an organisation-level categorisation.
Social value - The social impact in dollar terms that the amount invested achieves for participants over their lifetime. The social value is calculated by combining outcome values with a service delivery quality score, the size of the opportunity to support a population, and the number of people supported.
Limitations
- The themes analysed in this report are based on observed correlations and provide broad conclusions rather than tight causative claims.
- Programme intervention practices are determined via narrative and operational data provided by an organisation. It does not include direct observation of programmes, and as such social value forecasts do not capture variation in programme practice e.g., in workforce skills or programme fidelity across locations.
- Comparisons should be considered indicative only, as metrics can be influenced by a variety of factors, including differences in data quality, scoping decisions, improvements to methodology over time and limitations in the available academic literature.
- Many aspects of social impact cannot appropriately be quantified in dollar terms, and SROI findings should be considered alongside other important sources of information such as participant feedback and more bespoke forms of evaluation.
- GoodMeasure is a standardised measurement model — different interventions are treated as consistently as possible to enable comparability, which means the uniqueness of each intervention is not fully reflected.
- Cost and participant data inputs are provided by the organisation. Responsibility sits with each organisation to ensure their data is accurate and genuinely reflects the programme.
- Estimates have varying confidence levels due to differing quality and availability of data inputs. The GoodMeasure methodology takes the approach of using the data that is available in order to support ongoing data improvement.
- The lifetime (dollar) value of an outcome is conservatively valued over a 5-year period. This is aligned with New Zealand Treasury’s approach of measuring impact within a contained period.
Disclaimer
This disclaimer sets out important information about the scope of ImpactLab Limited’s services. ImpactLab endeavours to ensure that all material and information used for and presented in any GoodMeasure, including ROI calculations and impact numbers, is accurate and reliable (information). However, the information is based on various sources, including information organisations provide to ImpactLab which is not independently verified. ImpactLab does not make any representations or warranties in respect of information it uses or presents in relation to any GoodMeasure or this report. This includes any representation or warranty relating to the accuracy, adequacy, availability or completeness of information, or that it is suitable for its intended use. ImpactLab does not provide advice or make recommendations for any decisions made by any person, financial or otherwise, either in relation to any GoodMeasure, or this report. Data and percentages stated in this report may have been rounded.
Aggregated data stated in this report is based upon data provided to ImpactLab pursuant to its privacy policy and terms and conditions. Data ImpactLab uses except in exceptional circumstances must be aggregated and anonymised so that no participant in any programme ImpactLab analyses can be identified within data Impactlab uses or produces. Where ImpactLab uses the New Zealand Integrated Data Infrastructure (IDI), it does so subject to the conditions for access set by Stats NZ for IDI data users.
Attribution
Some data and information used in the Social ROI calculations is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) Licence. It is attributed to the NZ Treasury.
ImpactLab - Do good, better.
ImpactLab’s vision is to help create a world where investment works for communities, so that people can live the lives they choose.
We work with the best available data and evidence to estimate social value in dollar terms – a metric that is measurable, consistent, and comparable.
We build close relationships with our customers to understand them, and the needs of their communities, supporting them to do good, better.
Contact:
04 391 2755
info@impactlab.co.nz
LinkedIn | Facebook
Level 1, 2-8 Chancery Street, Auckland, 1010
Level 8, 187 Featherston Street, Wellington 6011
© ImpactLab Ltd, 2024. All rights reserved