aes26 International Evaluation Conference

Authors: Phillip Belling, NSW Department Of Education Jessica de Jesus, NSW Department Of Education, Rowan Savage, Aboriginal Education & Communities, NSW Department of Education

Presenter note: Rowan has been an important contributor to this work and was originally hoped to join the presentation in person. As the opportunity to present at aes26 was only confirmed in the past two days, the short timeframe unfortunately made it difficult to arrange for him to travel to Garrmalang | Darwin. We’re sorry Rowan can’t be here in person and acknowledge the perspective and contribution he has brought to the work being shared.

Government strategies to improve Aboriginal children’s access to culturally safe, high-quality early childhood education and care (ECEC) are increasingly ambitious, and also increasingly measured. Yet many mainstream outcome frameworks privilege what is easy to count (enrolments, attendance, completions) over what Aboriginal families and communities say really matters (cultural safety, belonging, relationships, self-determination, and thriving). And while there is a growing number of evaluations of pilots, projects and small scale programs, there are few examples of culturally responsive evaluations of large-scale, multi-program and strategy-level initiatives.

This session shares an approach being used in the NSW Department of Education to build culturally meaningful process indicators and outcome measures for a statewide ECEC strategy. Drawing on lessons from recent evaluations of state-level ECEC programs and evaluation designs, we present a practical method for moving from policy intent to measurable outcomes without collapsing cultural concepts into deficit metrics.

The approach combines: (1) outcome-domain mapping that starts with Aboriginal-led commitments and community-informed constructs; (2) a set of core indicators that deliberately pairs quantitative indicators with qualitative evidence (yarning, narrative case examples, workforce and service reflections) so that relationships and cultural safety are visible; and (3) governance and data sensemaking processes that keep Aboriginal authority central in interpreting evidence and deciding what “good” looks like across diverse local contexts, while remaining accountable to statewide commitments.

We also discuss why “measuring what matters” is a safeguarding task. In recent evaluation work we observed that indicators can unintentionally shift attention, create compliance behaviours, redistribute decision rights, and generate cultural load. In this session we outline design choices that mitigate these risks, including co-design checkpoints, minimum evidence standards, and using indicators as prompts for shared interpretation rather than as league tables.

Participants will leave with an adaptable template for developing culturally meaningful indicators and outcome measures, and examples they can translate to their own settings and evaluation partnerships.