aes26 International Evaluation Conference

Authors: Emma Weaver, Menzies, Shiree Mack, Menzies, Caroline Miller, Menzies, Louise Maple-Brown, Menzies
The Merne Mwerre Artweye Areye-ke (MMAA) diabetes prevention program was developed for Aboriginal and Torres Strait Islander children aged 6-11 years and their caregivers in Central Australia in response to community concerns about the increasingly high rate of youth obesity and type 2 diabetes in the region. The program is being delivered and evaluated in partnership between Central Australian Aboriginal Congress, the Aboriginal community-controlled health service, and Menzies School of Health Research across ten communities. While grounded in community leadership and co-facilitation by Aboriginal staff, the program is being delivered through a randomised controlled trial (RCT) to facilitate evaluation of clinical effectiveness.

This creates an ethical tension; RCTs privilege pre-specified outcomes, standardisation, and methodological control, whereas community-led initiatives require relational accountability, flexibility, and responsiveness to local priorities. The evaluation therefore confronts questions of power - whose knowledge counts, who defines success, and how competing accountabilities are balanced.

To navigate these tensions, the evaluation framework integrates adaptive qualitative inquiry alongside quantitative measures. Iterative feedback loops, reflective field notes, and ongoing dialogue with local leaders have supported transparency and ethical responsiveness. Program adaptations have included prioritising relationship-building, embedding local language and strengths-based framing, providing practical supports for participation, and reframing outcomes to reflect change valued by families and communities. The evaluation has also shifted from conventional semi-structured interviews to culturally grounded yarning approaches, recognising Indigenous ways of knowing as critical forms of evidence.

This presentation, delivered as a dialogue between an Aboriginal facilitator and a non-Indigenous evaluator, will critically reflect on trade-offs, missed assumptions, and lessons learned. It will explore how evaluators can uphold methodological rigour while prioritising differing voices, acknowledging power, and remaining accountable to community-defined values. Transferable strategies will be shared for ethically navigating complex, culturally grounded evaluations.

Authors: Nicholas Hill Place Australia, Eve Millar, Place Australia
The use of place-based initiatives (PBIs) to address complex and entrenched disadvantage is expanding across Australia. These initiatives typically go beyond the delivery of single programs and involve cross-sector partnerships that place communities at the centre of efforts to address local problems. While a growing number of initiatives are demonstrating impact, the diversity of approaches, frameworks, and indicators used contributes to a fragmented evidence base. Inconsistencies in how impact is conceptualised and reported limit opportunities for shared learning and present a barrier to the growth and sustainability of the place-based ecosystem.

PLACE Australia is working collaboratively with stakeholders across the ecosystem—including government, philanthropy, not-for-profits, and community organisations—to develop a shared impact framework with a set of flexible indicators that more consistently demonstrate the impact of PBIs, support ongoing learning, and strengthen the sector. As the framework moves from development to implementation, a number of practical challenges arise. These include how shared indicators can be applied flexibly across diverse initiatives, how to balance consistency with local adaptation, how frameworks can support learning rather than compliance, and how Indigenous knowledge and community voice can be embedded in practice.

This roundtable brings together evaluators and practitioners to explore these challenges and identify practical pathways for implementation. Through facilitated discussion, participants will share their practice insights on implementation opportunities, risk and design considerations. The discussion will inform the next phase of testing and implementation of the shared impact framework.

Participants will be invited to reflect on the following questions:

1.How can shared indicators be consistently applied across diverse place-based initiatives while remaining meaningful to local contexts?

2.How can shared impact approaches support learning and improvement without becoming compliance-driven reporting requirements?

3.What risks and opportunities should be considered when implementing shared impact approaches across the place-based ecosystem?

4.What is needed to support the implementation and uptake of the shared impact framework across the sector?

Authors: Lucy Walker, Nation Partners, Justine Smith, Nation Partners
Participatory evaluation is widely regarded as best practice for producing useful, ethical, and contextually grounded insights, particularly in community-based programs. Yet in practice, participatory approaches sometimes unfold in ways that are more complex, unpredictable, and sometimes unsuccessful than anticipated.

This presentation explores what happens when participatory evaluation does not go to plan, and what we can learn from these experiences. The objective of this session is to have a frank conversation about when participation doesn’t go as planned and to inspire more realistic, context-sensitive approaches to designing and implementing participatory evaluation.

The presenters (two) will draw on their experiences and examples to propose three key messages.

- Participation can’t always be assumed or expected- it is shaped by interests, relationships, perceived value and capacity with competing priorities.
- Misalignment between stakeholders (e.g. partners, community members, project funders, and evaluators) can present central challenges.
- When things don’t go according to plan it presents an opportunity for valuable insight and reflection and shouldn’t be avoided.

Through exploring cases where stakeholders are disengaged, where expectations diverge and methods don’t go to plan, we will surface the barriers and enablers to applying participatory methods in practice.

The presenters will share their experiences and insights with embedded survey via QR code to capture attendees reflections along the way. We will also explore methods and strategies to better understand stakeholder context, identify and plan for risks, and adapt approaches in the midst of delivery.

Author: Annabel Prescott (Traction for Young People), Samantha Garbutt (Traction for Young People)
Youth program evaluation faces a critical ethical question: Who is evaluation data for?

Drawing on Feedback Informed Treatment principles and youth empowerment frameworks, this presentation argues that when young people engage with their own data to understand and celebrate their growth, evaluation becomes a tool for agency rather than extraction.

Using TRACTION, a Queensland youth mentoring organisation, as a case study, we examine how internationally validated screening tools serve young people first and organisational learning second. Young people complete visual assessments at program entry, engage with their progress through facilitated conversations during the program, and review before-and-after results at completion. They keep their own copies and use their data in conversations with family, teachers, and others. This process makes visible the work young people have done, enabling them to name and claim their own progress.

The presentation explores three critical tensions:

1. How do we design evaluation that acknowledges young people as active participants in their own change process, not passive subjects of measurement?
2. What shifts when we position self-awareness and celebration as primary evaluation outcomes, with strategic metrics as secondary?
3. How do we ensure data collection practices honour young people's agency rather than extracting information purely for organisational purposes?

We present real examples of facilitated data conversations, visual assessment tools, and moments when young people recognise their own growth through evidence. When evaluation is grounded in participant empowerment, where people understand, acknowledge, and celebrate their own growth, it creates a foundation for ethical strategic data use.

This approach transfers to evaluation with any structurally marginalised population: First Nations communities, people experiencing disadvantage, or those marginalised by traditional service systems. Attendees will leave with critical questions for examining whether their own evaluation practices serve participant agency or organisational needs first

Culturally Grounded Evaluation: Innovative Methods from the Champions4Change Workshop
Author: Sunet Jordaan
The Champions4Change program is a First Nations–led initiative supporting people with lived experience of acute rheumatic fever (ARF) and rheumatic heart disease (RHD) to deliver culturally safe education and advocacy in their communities. In 2025, the Heart Foundation delivered a workshop to strengthen Champions’ knowledge, confidence and leadership skills using culturally grounded and accessible evaluation methods.
This presentation shares the evaluation approach used to assess knowledge gained, confidence built and participants’ experiences. Instead of surveys, evaluators used Yarning circles, storytelling and visual self-assessment tools to create a culturally safe environment aligned with community ways of knowing, being and doing. First Nations facilitators with established relationships with Champions led the activities.
Champions used a visual “road journey” to represent changes in confidence and reflected on their understanding of ARF and RHD through yarning conversations and storytelling discussions. The session highlights how culturally grounded, visual and narrative methods can improve accessibility, participation and lived experience leadership while generating richer insights than traditional surveys.

Agile Evaluation Approaches to Combatting Antisemitism in Australia
Authors: Linda Gyorki, Milo McKay
This presentation explores agile evaluation through a real-world evaluation of place-based initiatives aimed at combating antisemitism and strengthening social cohesion. As global political developments, social polarisation and misinformation reshape communities, evaluators working in sensitive environments can no longer rely on fixed designs. This presentation demonstrates how evaluation can adapt without sacrificing rigour or credibility. Three key lessons highlight agile evaluation in practice. First, polarisation affects participation and engagement, making trust-building and relationship management essential evaluation skills. Second, iterative data collection enables evaluators to identify emerging issues, adjust methods and remain responsive to changing community dynamics. Third, responsiveness itself builds credibility. When evaluation processes visibly adapt to local events and stakeholder realities, findings become more useful and trusted by communities.
Aligning with the conference theme, “Making space, valuing place,” the presentation examines how evaluation can remain grounded in community experience while responding to broader global forces. The session will share practical lessons for evaluators navigating sensitivity, external events and methodological adaptation.

Co-design and evaluation as bridges: An adaptive approach to delivering technology for coral reef conservation
Author: Emily Maher
Delivering multilateral conservation projects requires working across disciplinary, cultural and institutional boundaries. In Southeast Asia, coral reef monitoring and management are further challenged by limited resources, capacity and coordination. This presentation shares the adaptive project management and evaluation approaches used by the Australian Institute of Marine Science to support coral reef monitoring through knowledge, technology and expertise exchange. A deliberate “year zero” planning phase aligned expectations, identified local needs and strengthened collaboration. Guided by co-design and adaptive management principles, evaluative processes structured dialogue between scientists, policymakers, managers and practitioners while integrating diverse priorities and cultural considerations. Evaluation functioned as connective infrastructure rather than simply monitoring progress. Regular reflection on partner feedback and evolving implementation needs supported timely adaptation and negotiation of trade-offs. This collaborative approach fostered strong local ownership and sustained government support, contributing to outcomes including a national coral reef monitoring plan in Brunei Darussalam and agreed monitoring standards in the Philippines and Vietnam. The presentation reflects on roadblocks, unexpected outcomes and lessons for complex projects operating across policy, technology, science and implementation.

Effects of community water fluoridation on child dental caries in remote Northern Territory, Australia
Author: Ramakrishna Chondur
Community water fluoridation (CWF) is a cost-effective intervention for reducing dental caries at a population level. This Northern Territory (NT) study used a difference-in-difference (DiD) analysis to examine dental caries outcomes among children exposed to CWF across 50 remote NT communities. Methods: Oral health data from the NT Department of Health (2008–2020) included 24,546 children aged 1–17 years. Drinking water fluoride data from the Power and Water Corporation were linked to the oral health dataset. The DiD analysis compared a treatment group with two control groups to assess the impact of CWF on dental caries outcomes using the decayed, missing and filled teeth (dmft/DMFT) index. Results: Dental caries significantly decreased among children in the treatment group following implementation of CWF, with greater reductions than both control groups over the same period.
Conclusion: CWF produced population-level reductions in dental caries among children in remote NT communities, supporting longstanding NT Department of Health policy and demonstrating improved oral health outcomes.

Enshittification: A new category for your next MEL framework?
Author: Duncan Rintoul
The term enshittification (or enpoopification for delicate ears) describes how digital services often begin as useful tools but degrade in quality and usability over time. This decline is commonly linked to increasing prioritisation of profit, resulting in cluttered interfaces, manipulated content, more advertising, rising costs and poorer user experience.
Coined by Canadian writer Cory Doctorow, the term was named “word of the year” by the American Dialect Society in 2023 and Australia’s Macquarie Dictionary in 2024. Drawing on Doctorow’s 2025 book Why Everything Suddenly Got Worse and What To Do About It, this Ignite presentation argues that enshittification deserves a place in risk matrices and MEL frameworks across multiple sectors. The presentation also argues for the value of longitudinal process evaluation, not only longitudinal outcome evaluation, to better understand how systems and user experiences deteriorate over time.

From Admin to Evidence: Transforming Data Quality Through Culture and Capability
Author: Cath Cooper
This Ignite presentation will offer practical insights into how redefining purpose, building capability, and embedding supportive systems can transform data from a burden into a meaningful asset for evaluation.
Key points: 1.Cultural Shift: we reframed “admin tasks” as “Evidence of Impact”, shifting the narrative to honour the value of data as a foundation for learning and accountability 2.System Redesign: A collaboratively built CMS, supported by guides, workshops and induction, strengthened capability and consistency. 3.Sustained Improvement: Monthly monitoring, mentoring and rapid feedback loops led to significantly improved data accuracy and reduced correction time, enabling reliable reporting and decision making.

Authors: Anne Stephens, Ethos of Engagement, Jill Thomas
This presentation explores the design of a systemic Monitoring and Evaluation (M&E) system for counter-trafficking in persons (CTIP) in the ASEAN region. Developed in 2023, the system supports the adoption and implementation of three victim-centred and gender-sensitive guides for counter trafficking in persons, in use across ASEAN member states. The presentation focuses on how systemic, participatory approaches to evaluation design can enhance uptake of evaluation and the role of a well-designed framework to support the capacity of individuals to monitor and evaluate their work.  

The objective of this presentation is to present the process used to develop a simple to use M&E system within a complex setting; show the public facing guidance documents and tools used to support novice and highly skilled evaluators to use the system; and describe our challenges and learnings.

This presentation offers a timely and practice-grounded contribution to the evaluation field by demonstrating how evaluators can design for relevance, capacity development and impact in complex, real-world settings. It provides actionable insights for practitioners seeking to strengthen the value and use of evaluation in increasingly uncertain and contested environments.

Authors Fergus Bailey, ARTD, Jade Maloney, ARTD
Australia’s Disability Strategy 2021-2031 emphasises the importance of inclusive practices across all sectors, including evaluation. We all have a role to play in achieving the vision of an inclusive Australian society that ensures people with disability can fulfil their potential, as equal members of the community.

At the 2025 AES conference, Melinda Nicholls, Amanda Charles, and Jane Spring AM introduced a practical guide to facilitate meaningful inclusion of people with disability in evaluation, embedding lived experience and promoting equitable participation.

ARTD Consultants is conducting the mid-term evaluation of Australia’s Disability Strategy 2021-2031, using the guide and their own lived experience evaluation framework for a lived experience centred approach. This a significant test of the guide in action.

In this panel, DHDA staff, Jade Maloney (CEO, ARTD Consultants), Fergus Bailey (Senior Consultant, ARTD Consultants), and lived experience representatives will discuss how the perspectives of people with lived experience were centred throughout the evaluation - in key evaluation team roles (including ARTD staff and a lived experience team), governance roles (e.g. Steering Committee), and consultation with people with disability across Australia.

They will provide perspectives as commissioners, evaluators, and people with lived experience on how the guide was operationalised for this process and the benefits to the evaluation.

They will share practical advice for how evaluators can effectively engage people with disability in leading and contributing to evaluations, which are applicable to engaging people with lived experience more broadly in evaluation governance, design and delivery roles, including tailored approaches for diverse communities.

After a presentation and discussion, Jade Maloney will facilitate questions as an experienced AES presenter and moderator, prompting around barriers, enablers, risk management, how challenges accommodated, and other considerations for this approach. Following the session the team will share learnings for the future with broader AES members.

Authors: Lucy Spanswick, Wungening Aboriginal Corporation, Shenae Parremore, Wungening Aboriginal Corporation
 Evaluations of community programs often privilege measurable outputs while overlooking the relational and cultural work that enables meaningful change. This presentation shares insights from the evaluation of an Aboriginal-led alcohol and other drug healing program that sought to make visible the work that is frequently unseen within Aboriginal Community Controlled Organisations. It explores how an Aboriginal led evaluation shines a light on Aboriginal ways of working as a response to systemic barriers facing communities. It also considers how evaluation can better recognise and value this work.

Drawing on insights from Elders, staff, participants and leadership, the evaluation adopted an Aboriginal-led, strengths-based approach. A co-research group provided cultural and relational leadership, ensuring accountability to community and shaping both the design and interpretation of the evaluation. Culturally responsive methods centred participant voices and experiences of healing, connection and change.

The presentation focuses on three key insights. First, it demonstrates how Aboriginal-led evaluation approaches can make visible the relational and often unseen work that underpins meaningful outcomes, including trust, connection and safety. Second, it highlights the value of centring lived experience and community voice in both design and interpretation, showing how methods such as yarning and co-research can strengthen the depth and integrity of evaluation findings. Third, it positions evaluation as an active part of the change process, not just a tool for measurement, showing how relational approaches can contribute to healing, learning and continuous improvement in complex service contexts.

Through practical examples and short project videos, participants will be invited to reflect on what may be invisible in their own evaluation practice. The session offers insights relevant across sectors, encouraging evaluators to rethink how impact is measured and how evaluation can contribute to self-determination and community wellbeing.

Author: Annabel Prescott, Traction For Young People
Youth program evaluation faces a critical ethical question: Who is evaluation data for?

Drawing on Feedback Informed Treatment principles and youth empowerment frameworks, this presentation argues that when young people engage with their own data to understand and celebrate their growth, evaluation becomes a tool for agency rather than extraction.

Using TRACTION, a Queensland youth mentoring organisation, as a case study, we examine how internationally validated screening tools serve young people first and organisational learning second. Young people complete visual assessments at program entry, engage with their progress through facilitated conversations during the program, and review before-and-after results at completion. They keep their own copies and use their data in conversations with family, teachers, and others. This process makes visible the work young people have done, enabling them to name and claim their own progress.

The presentation explores three critical tensions:

1. How do we design evaluation that acknowledges young people as active participants in their own change process, not passive subjects of measurement?
2. What shifts when we position self-awareness and celebration as primary evaluation outcomes, with strategic metrics as secondary?
3. How do we ensure data collection practices honour young people's agency rather than extracting information purely for organisational purposes?

We present real examples of facilitated data conversations, visual assessment tools, and moments when young people recognise their own growth through evidence. When evaluation is grounded in participant empowerment, where people understand, acknowledge, and celebrate their own growth, it creates a foundation for ethical strategic data use.

This approach transfers to evaluation with any structurally marginalised population: First Nations communities, people experiencing disadvantage, or those marginalised by traditional service systems. Attendees will leave with critical questions for examining whether their own evaluation practices serve participant agency or organisational needs first.

Authors: Natasha Freeman, Menzies School of Health Research, Edwina Murphy, Menzies School of Health Research
This presentation explores the importance of respecting context and culture when working with First Nations participants and colleagues, rethinking conventional data collection methods and outcome indicators to elicit important learnings.

We will share the experiences of our culturally and professionally diverse team delivering and evaluating Doing It Together, a peer-support and peer-education program for First Nations youth living with type 2 diabetes. The project required a thoughtful and responsive approach to ensure that local First Nations voices were prioritised and processes culturally grounded.

Strengthening capacity across the team through genuine two-way learning was central, recognising that concepts such as “research” and “evaluation” required ongoing discussion to collectively explore their meaning and purpose. This was particularly relevant for team members new to professional roles and, sometimes, in first time employment. Creating this space enabled all team members, particularly youth peer facilitators, to feel comfortable with the project’s intent and direction. It prompted important shifts in data collection and defining impact, more meaningfully reflecting First Nations ways of knowing, being and valuing.

Strategies included supported reflection, discussion and two-way learning, building on established relationships, and adopting a Developmental Evaluation approach to support intentional reflection on processes and outcomes, enabling timely adaptation. We were open to what constitutes meaningful data and valued context-specific insights.

We practised data collection as a skill throughout—learning to elicit and honour stories and lived experience, administer surveys with purpose, and reflect critically on what “impact” and “success” could look like. Data analysis was similarly approached with care, prioritising First Nations youth voices, particularly where differing world views shaped interpretation.

The presentation will be co-facilitated by the Evaluation Officer and an Aboriginal Peer Facilitator. Modelling the project’s relational approach, we will showcase activities used for connecting project participants, inviting audience movement and interaction. We welcome questions throughout.