aes26 International Evaluation Conference

Authors: Nicholas Hill Place Australia, Eve Millar, Place Australia
The use of place-based initiatives (PBIs) to address complex and entrenched disadvantage is expanding across Australia. These initiatives typically go beyond the delivery of single programs and involve cross-sector partnerships that place communities at the centre of efforts to address local problems. While a growing number of initiatives are demonstrating impact, the diversity of approaches, frameworks, and indicators used contributes to a fragmented evidence base. Inconsistencies in how impact is conceptualised and reported limit opportunities for shared learning and present a barrier to the growth and sustainability of the place-based ecosystem.

PLACE Australia is working collaboratively with stakeholders across the ecosystem—including government, philanthropy, not-for-profits, and community organisations—to develop a shared impact framework with a set of flexible indicators that more consistently demonstrate the impact of PBIs, support ongoing learning, and strengthen the sector. As the framework moves from development to implementation, a number of practical challenges arise. These include how shared indicators can be applied flexibly across diverse initiatives, how to balance consistency with local adaptation, how frameworks can support learning rather than compliance, and how Indigenous knowledge and community voice can be embedded in practice.

This roundtable brings together evaluators and practitioners to explore these challenges and identify practical pathways for implementation. Through facilitated discussion, participants will share their practice insights on implementation opportunities, risk and design considerations. The discussion will inform the next phase of testing and implementation of the shared impact framework.

Participants will be invited to reflect on the following questions:

1.How can shared indicators be consistently applied across diverse place-based initiatives while remaining meaningful to local contexts?

2.How can shared impact approaches support learning and improvement without becoming compliance-driven reporting requirements?

3.What risks and opportunities should be considered when implementing shared impact approaches across the place-based ecosystem?

4.What is needed to support the implementation and uptake of the shared impact framework across the sector?

Authors: Lucy Walker, Nation Partners, Justine Smith, Nation Partners
Participatory evaluation is widely regarded as best practice for producing useful, ethical, and contextually grounded insights, particularly in community-based programs. Yet in practice, participatory approaches sometimes unfold in ways that are more complex, unpredictable, and sometimes unsuccessful than anticipated.

This presentation explores what happens when participatory evaluation does not go to plan, and what we can learn from these experiences. The objective of this session is to have a frank conversation about when participation doesn’t go as planned and to inspire more realistic, context-sensitive approaches to designing and implementing participatory evaluation.

The presenters (two) will draw on their experiences and examples to propose three key messages.

- Participation can’t always be assumed or expected- it is shaped by interests, relationships, perceived value and capacity with competing priorities.
- Misalignment between stakeholders (e.g. partners, community members, project funders, and evaluators) can present central challenges.
- When things don’t go according to plan it presents an opportunity for valuable insight and reflection and shouldn’t be avoided.

Through exploring cases where stakeholders are disengaged, where expectations diverge and methods don’t go to plan, we will surface the barriers and enablers to applying participatory methods in practice.

The presenters will share their experiences and insights with embedded survey via QR code to capture attendees reflections along the way. We will also explore methods and strategies to better understand stakeholder context, identify and plan for risks, and adapt approaches in the midst of delivery.

Culturally Grounded Evaluation: Innovative Methods from the Champions4Change Workshop
Author: Sunet Jordaan
The Champions4Change program is a First Nations–led initiative supporting people with lived experience of acute rheumatic fever (ARF) and rheumatic heart disease (RHD) to deliver culturally safe education and advocacy in their communities. In 2025, the Heart Foundation delivered a workshop to strengthen Champions’ knowledge, confidence and leadership skills using culturally grounded and accessible evaluation methods.
This presentation shares the evaluation approach used to assess knowledge gained, confidence built and participants’ experiences. Instead of surveys, evaluators used Yarning circles, storytelling and visual self-assessment tools to create a culturally safe environment aligned with community ways of knowing, being and doing. First Nations facilitators with established relationships with Champions led the activities.
Champions used a visual “road journey” to represent changes in confidence and reflected on their understanding of ARF and RHD through yarning conversations and storytelling discussions. The session highlights how culturally grounded, visual and narrative methods can improve accessibility, participation and lived experience leadership while generating richer insights than traditional surveys.

Agile Evaluation Approaches to Combatting Antisemitism in Australia
Authors: Linda Gyorki, Milo McKay
This presentation explores agile evaluation through a real-world evaluation of place-based initiatives aimed at combating antisemitism and strengthening social cohesion. As global political developments, social polarisation and misinformation reshape communities, evaluators working in sensitive environments can no longer rely on fixed designs. This presentation demonstrates how evaluation can adapt without sacrificing rigour or credibility. Three key lessons highlight agile evaluation in practice. First, polarisation affects participation and engagement, making trust-building and relationship management essential evaluation skills. Second, iterative data collection enables evaluators to identify emerging issues, adjust methods and remain responsive to changing community dynamics. Third, responsiveness itself builds credibility. When evaluation processes visibly adapt to local events and stakeholder realities, findings become more useful and trusted by communities.
Aligning with the conference theme, “Making space, valuing place,” the presentation examines how evaluation can remain grounded in community experience while responding to broader global forces. The session will share practical lessons for evaluators navigating sensitivity, external events and methodological adaptation.

Co-design and evaluation as bridges: An adaptive approach to delivering technology for coral reef conservation
Author: Emily Maher
Delivering multilateral conservation projects requires working across disciplinary, cultural and institutional boundaries. In Southeast Asia, coral reef monitoring and management are further challenged by limited resources, capacity and coordination. This presentation shares the adaptive project management and evaluation approaches used by the Australian Institute of Marine Science to support coral reef monitoring through knowledge, technology and expertise exchange. A deliberate “year zero” planning phase aligned expectations, identified local needs and strengthened collaboration. Guided by co-design and adaptive management principles, evaluative processes structured dialogue between scientists, policymakers, managers and practitioners while integrating diverse priorities and cultural considerations. Evaluation functioned as connective infrastructure rather than simply monitoring progress. Regular reflection on partner feedback and evolving implementation needs supported timely adaptation and negotiation of trade-offs. This collaborative approach fostered strong local ownership and sustained government support, contributing to outcomes including a national coral reef monitoring plan in Brunei Darussalam and agreed monitoring standards in the Philippines and Vietnam. The presentation reflects on roadblocks, unexpected outcomes and lessons for complex projects operating across policy, technology, science and implementation.

Effects of community water fluoridation on child dental caries in remote Northern Territory, Australia
Author: Ramakrishna Chondur
Community water fluoridation (CWF) is a cost-effective intervention for reducing dental caries at a population level. This Northern Territory (NT) study used a difference-in-difference (DiD) analysis to examine dental caries outcomes among children exposed to CWF across 50 remote NT communities. Methods: Oral health data from the NT Department of Health (2008–2020) included 24,546 children aged 1–17 years. Drinking water fluoride data from the Power and Water Corporation were linked to the oral health dataset. The DiD analysis compared a treatment group with two control groups to assess the impact of CWF on dental caries outcomes using the decayed, missing and filled teeth (dmft/DMFT) index. Results: Dental caries significantly decreased among children in the treatment group following implementation of CWF, with greater reductions than both control groups over the same period.
Conclusion: CWF produced population-level reductions in dental caries among children in remote NT communities, supporting longstanding NT Department of Health policy and demonstrating improved oral health outcomes.

Enshittification: A new category for your next MEL framework?
Author: Duncan Rintoul
The term enshittification (or enpoopification for delicate ears) describes how digital services often begin as useful tools but degrade in quality and usability over time. This decline is commonly linked to increasing prioritisation of profit, resulting in cluttered interfaces, manipulated content, more advertising, rising costs and poorer user experience.
Coined by Canadian writer Cory Doctorow, the term was named “word of the year” by the American Dialect Society in 2023 and Australia’s Macquarie Dictionary in 2024. Drawing on Doctorow’s 2025 book Why Everything Suddenly Got Worse and What To Do About It, this Ignite presentation argues that enshittification deserves a place in risk matrices and MEL frameworks across multiple sectors. The presentation also argues for the value of longitudinal process evaluation, not only longitudinal outcome evaluation, to better understand how systems and user experiences deteriorate over time.

From Admin to Evidence: Transforming Data Quality Through Culture and Capability
Author: Cath Cooper
This Ignite presentation will offer practical insights into how redefining purpose, building capability, and embedding supportive systems can transform data from a burden into a meaningful asset for evaluation.
Key points: 1.Cultural Shift: we reframed “admin tasks” as “Evidence of Impact”, shifting the narrative to honour the value of data as a foundation for learning and accountability 2.System Redesign: A collaboratively built CMS, supported by guides, workshops and induction, strengthened capability and consistency. 3.Sustained Improvement: Monthly monitoring, mentoring and rapid feedback loops led to significantly improved data accuracy and reduced correction time, enabling reliable reporting and decision making.

Authors: Virgina Thomas, Bioeconomy Science Institute Maiangi Taiao, Stewart Graham, Bioeconomy Science Institute Maiangi Taiao, Helen Percy, Bioeconomy Science Institute Maiangi Taiao
A culture of impact, empowered by evaluation, encourages accountability and learning, supporting projects to focus on outcomes, demonstrate efficacy and make improvements.  Fostering such an environment promotes success, yet can be challenging to achieve, requiring leadership, communication, adaptability, capacity and capability building, and resources.

Our paper discusses the (co)creation of impact culture, and evaluation systems and tools, in the Bioeconomy Science Institute Maiangi Taiao, a new public research organisation created by merging four of Aotearoa New Zealand’s Crown Research Institutes (CRIs). Establishing an impact culture in this new organisation requires building bridges and blurring boundaries between the organisations and systems that have been merged.  

While the four CRIs all had systems and tools to plan for and evaluate impact, the approaches were unique to each organisation, as were their impact cultures. Since the merger, colleagues have collaborated to bring the previously separate systems and tools together into a new evaluation ecosystem.

In our paper we focus on creating a cohesive impact culture, empowered by evaluation, through i. inter-organisational dialogue, ii. building an enabling environment for evaluation that is adaptable to different needs and functions, and iii. disseminating evaluation tools and resources and promoting evaluative thinking across organisations through collegial networks.

Using our experience as a case study, we will inspire our audience to reconsider the impact and evaluation culture, systems and tools in their own organisations using real time polling (e.g., Mentimeter) to encourage reflection and participation.

We draw on the work of Blundo and Canto (2019) and Ferre (2025) on building an evaluative culture including: creating a system that is adaptable to the different needs and functions of the legacy organisations, fostering dialogue across and beyond the evaluation sector to include fields such as economics and social science, and building capacity, capability and resources for evaluation.

Author: Matthew Armstrong, CGIAR
This presentation provides practical insights on how and when to use economics in evaluation. Economic ideas shape public policy, funding, and program design. Economics promises efficiency, cost effectiveness, value for money and optimisation. Many evaluation theories, for example Stufflebeam’s CIPP model (2007), Scriven’s (2007) consumer orientated model, and Patton’s (2003) utilisation-focused model are biased towards the social concerns of evaluation.

Yet, evaluators are often tasked with incorporating economic concepts, methods and language into evaluations. But how do we make them fit-for-purpose, credible and culturally appropriate? This case study examines when and how economic concepts should be integrated into evaluation.

CGIAR, a global agricultural research network, is introducing tools to apply comparative advantage in portfolio design and evaluation. Comparative advantage suggests organisations should specialise in areas where they are relatively strongest and partner elsewhere to maximise impact. The theory is compelling: comparative advantage offers reduced duplication and deliberate partnerships at lower costs.

However, how do we credibly measure and make evaluative judgements about comparative advantage? This study found considerable variation in how comparative advantage was applied across the organisation and consternation in response to the roll-out that hindered its usefulness. This presentation describes an interdisciplinary negotiation undertaken to develop guidance for rigorous process and performance evaluations of comparative advantage within CGIAR. It involves a structured process to identify synergies and tensions between existing evaluation and economic practice relating to comparative advantage, a step-by-step approach to undertaking evaluations with comparative advantage criteria, and recommendations for developing an organisation-wide approach to comparative advantage to support consistent, high-quality. Furthermore, this case study illustrates a crucial requirement to reflect upon how economics is integrated into evaluation, including when it should be adapted or challenged.

In the interactive component, participants will reflect on how economics shapes evaluation in their organisations.

Author: Themis Antony, Beyond Blue
This session will provide an overview of a practical approach utilised by Beyond Blue to develop an engaging, streamlined and systematic approach to capturing evaluation-ready outcomes data.

Objective and key messages:
The objective of the session is to share how traditional and adapted approaches were used to develop a core outcomes dataset that heavily engaged staff in the process, while factoring in the inherent challenges of defining, collecting and using outcomes data within the multi-faceted operating environment in which Beyond Blue works.

The key element of the approach was its deliberate focus on making space for, and meaningfully engaging, staff across the organisation to shape the outcomes development process. Rather than being imposed, the approach was co-developed with staff and embedded in day-to-day practice. This resulted in a network of interrelated program logic models that are owned by individual program areas.
The data generated through this approach is also owned by program areas and used to inform evaluations and is actively owned by staff at all levels. Insights are shared widely, including with community members, government and corporate funders, donors and sector colleagues, supporting transparency, accountability and collective learning.

The session will include a practical demonstration of the logic models in action, showing how they are used to guide decision making, monitor progress and support evaluation.

Overall, the session aims to share a practical approach to outcomes measurement, provide an example of how to bridge the evaluation gap for staff who may be less familiar with data collection, and demonstrate how strong evaluation and research integrity can be maintained while building organisational evaluation capacity.

Interactivity:
Time will be set aside for interactive discussion and debate, enabling participants to reflect on how similar approaches could be adapted to their own organisational contexts and how the approach could be enhanced.