aes26 International Evaluation Conference

Authors: Kristine Tuban, Save the Children, Martin Holmstrand, Save the Children
This presentation showcases MEAL Uplift, a regional initiative by Save the Children Australia that strengthens digital monitoring, evaluation, accountability, and learning (MEAL) across Pacific Country Offices through a close partnership between programme teams, MEAL, and the Technology team. The presentation focuses on how this collaboration produced a practical, co‑created digital solution while building long‑term organisational capability.

The objective of the presentation is to demonstrate why cross‑disciplinary partnership is critical to improving data quality, efficiency, and use in evaluation practice, and how evaluation can actively shape digital transformation, rather than simply adopting technological tools after they are introduced. This is important in contexts where teams face increasing reporting demands but limited capacity to manage fragmented or manual data systems.

The core argument is that sustainable digital MEAL is achieved when three elements are intentionally integrated:
(1) Digital systems and infrastructure, illustrated through a custom‑built data collection app and automated data pipeline co‑designed with the Technology team to address efficiency and quality challenges;
(2) Skills and behaviours, supported through targeted training, coaching, and practical use of real data; and
(3) Scaling and institutional support, through shared standards, governance, and regional scaffolding that embeds digital MEAL into everyday practice.

The presentation will follow this three‑part structure, using concrete examples and lessons from MEAL Uplift to show what worked, what changed, and why.

Participant engagement will be promoted through short reflection prompts on participants’ own digital MEAL challenges, followed by shared discussion and an open Q&A focused on transferable ideas and co‑created solutions across different organisational contexts.

Author: Alice Muller, World Vision
Evaluators and development practitioners need to understand how environmental changes impact and result from development programming.  Often finding practical ways to do is a challenge, particularly where complex environmental, biodiversity and climate systems are involved, and where evaluators and practitioners come from social or economic backgrounds with limited biophysical or ecological training. This paper shares how we can better value place and understand environmental consequences in international development projects by letting evaluations be locally-led, with local knowledge and values at the center.

Drawing on real‑world examples from World Vision’s Regreening Communities work, the paper presents three simple tools used by communities t Together, these approaches value place by centring local and Indigenous knowledge and values, assessing change from a local perspective, in a standardised way and using spatial tools that together still contribute to organisational‑level evidence.

First, a community mapping and planning exercise makes space for diverse groups within a community to define what regreening and wellbeing mean in their own environments, surfacing cultural, livelihood and social values that are often invisible in standard indicator frameworks. Second, a community‑assessed Regreening Index provides a structured and repeatable way to assess biophysical conditions and trends over time in relation to community priorities, while still enabling consistent comparison across sites and projects. Third, spatial mapping of regreening sites using a powerful, but freely available tool adapted from the WASH sector, enables evaluators to examine patterns and cumulative effects at project and organisational levels.

While none of these tools is unique in isolation, when combined they lower barriers to including environmental considerations in evaluation practice and generate insights that strengthen learning, adaptive management and sustainability discussions, ethically. The presentation will share engaging practical examples and invite participants to reflect on opportunities to better value place within their own work.

Author: Charlie Tulloch (Policy Performance)

This session at the start of the conference orients new, emerging and non-evaluators to the types of topics and content they will encounter over the coming days. It provides a foundational overview of evidence-related fields (e.g. evidence, monitoring, evaluation, learning), debunks key concepts/language, offers insights into what evaluators do day-to-day, discusses values/valuing, ethical considerations, competencies and key steps usually considered when completing high quality evaluation projects.The session strengthens defines the boundaries and intersections of evaluation and other disciplines to understand this field of endeavour, the pathways in and the opportunities it offers.

The session’s main aims are to share and discuss:

• How evidence is used in different policy and organisational landscapes

• The roles and types of projects led by evaluators

• Techniques frequently used by evaluators•Ways that evaluators think (logic, data, outputs, outcomes, impact)

• Why and when evaluations happen (or do not)

• Practical tips for evaluation commissioners and working with evaluators

• Capability building (skills and knowledge) to promote use of evaluation in-house

• A seven-stage approach to planning and conducting an evaluation project

• Essential skills of evaluators, including logic modelling, defining key/interview questions, forming value judgments, and selecting effective evaluation approaches/methods.The session will provide attendees with an opportunity to ask questions anonymously, so they feel very comfortable as the conference commences.

Questions will be addressed during the session, creating an open, interactive discussion.The session will also provide an overview of the conference structure, learning opportunities and flow, to provide a guide to attendees about how to gain the most value from their time.

Author: Jess Dart, Clear Horizon
This paper introduces the Collective Noticing Method (CNM) and explores its relevance for measurement, evaluation and learning (MEL). Developed by the author, CNM offers evaluators and changemakers a collaborative way to notice, learn from, and respond to subtle and emerging changes. It is a rigorous yet flexible method with a strong focus on learning and collective sensemaking, well suited to the “messy middle” of systems change and place-based work. It also has potential for community and grassroots work, program portfolios, and research impact tracking.

Collective noticing as a practice has a long history, evident in approaches such as Outcome Harvesting, Most Significant Change (MSC), citizen science, ripple effect monitoring, and context monitoring. More recently, this practice has been enabled and accelerated by emerging digital platforms and AI. Building on these foundations, CNM intentionally harnesses the eyes and ears of those implementing and experiencing an initiative to bring shared attention to what is changing.CNM is inherently participatory, inviting many people to observe, document, and interpret change collectively. It values lived experience as a central source of insight, embraces uncertainty, and supports adaptive action grounded in what is actually emerging. As a decolonising and grounded approach to MEL, CNM shares power by broadening who defines what matters and what counts as evidence. It centres learning-oriented measurement and collective sensemaking, bringing forward subtle signals, relational shifts, and everyday insights that conventional approaches often overlook.

CMN involves impact logging and tracking against agreed ways of working and learnings, whereas Outcome Harvesting focuses on outcomes and tends to be done as a one-off study. It differs from MSC in that CNM uses shorter, multi-perspective logs across diverse categories and evidence types.This session introduces CNM and invites participants to critically explore its benefits, limitations, and the contexts in which it is most—and least—useful.

Author: Salli Cohen (The Policy Room)
This presentation explores the uncomfortable space where policy and evaluation structurally reinforce harm. Its objective is to challenge assumptions of neutrality and examine how evaluative frameworks can either illuminate or obscure inequity.
The core argument is threefold: (1) evaluation and policy are never neutral and must interrogate power; (2) systems alignment and cultural authority determine whether outcomes are real or performative; and (3) accountability must ask “for whom” evaluation and policy work; and who bears the cost.
The session blends applied case insight with structured reflection and peer dialogue to provoke critical engagement and practical recalibration.

Authors: Kristian Paolo Torres, BRAC Ultra-Poor Graduation Initiative, Tania Dora Warokka, BRAC Ultra-Poor Graduation Initiative, Arnaldo Pellini, BRAC Ultra-Poor Graduation Initiative
Programmes aiming to influence governance systems and policy reforms—particularly those using experimental approaches—require MEL functions that are able to move across design, strategy, context analysis, and sensemaking.

Traditional M&E often prioritizes linear accountability, struggling to measure influence on policies and behaviors (as noted by the ODI RAPID programme, 2006–2020).

This presentation shares the experience of the BRAC Ultra-Poor Graduation Initiative, which partners with governments across six countries to scale the ‘Graduation’ approach through national social development systems (e.g. social protection, livelihood, labour and employment, etc.). Because UPGI’s success is measured by policy influence and system change rather than direct delivery, the MEL function must act as a strategic bridge rather than a neutral observer.

Drawing on our experience in Indonesia and the Philippines, we share how MEL supports country teams as adaptive units navigating two primary challenges: calibrating the interplay between political commitment, administrative feasibility, and technical quality; and designing for scale from the outset to ensure government ownership rather than isolated pilot results.

We argue that a team learning culture—rather than a "neutral" M&E function—is the essential infrastructure for this work. By establishing specific habits to capture field experience and inform strategic decisions, MEL acts as the "many-hatted" bridge required to navigate these dynamic systemic boundaries and prevent programmes from remaining stagnant.

We will share three core insights from our MEL experience:

1. How the evaluator’s role shifts to an active learning partner, framing field evidence to navigate political and administrative boundaries.
2. How practical tools—such as Capacity and Commitment Rubrics and Diaries—support teams in documenting for learning rather than reporting for compliance.
3. Real-world examples where structured reflection informed specific shifts in government engagement strategies.

The session will also engage the audience in a "Reflection Micro-Workshop" where participants will share insights from their own work and experiences.

Who really benefits from a program and how? Using personae to enrich logic models.
Author: Fiona Scott-Melton, Allen + Clarke
Not everyone experiences a program the same way. Yet logic models tend to obscure differences between population groups experiences, needs and aspirations. Incorporating personae helps address this limitation. By representing distinct population groups within a logic model, personae make visible the different routes through which people experience a program and what they gain from it. Drawing on a real life logic model developed for housing disabled people, the session will show how incorporating personae strengthens evaluation design, enriches social return on investment calculations and produces more compelling findings for stakeholders.

Help! The post-it notes have fallen off our logic model and been mixed up!
Author: Robert Grimshaw, ATO
Can you help us piece the logic model back together? This online interactive group activity is one tool used by a large public sector agency to familiarise people with key logic modelling concepts and thinking. This session will provide a practical demonstration of the activity and how it’s being used by workshop participants to apply their learning in a simple and fun way. Participant feedback has confirmed the value of this tool for engaging with and building the capability of a diverse audience who have limited evaluation capability and experience.

Method vs methodology – what’s the difference, and why it matters 
Author: Martina Donkers, Martina Donkers
We’ve got to stop using the words ‘method’ and ‘methodology’ interchangeably. ‘Methodology’ isn’t a slightly-fancier way of saying ‘method’ – it’s actually what conceptually underpins your method choices, and it’s an important thing to reckon with if you want to do evaluation that speaks to your stakeholders, is culturally responsive, is ethical and proportionate, and is ultimately useful. This quick-fire Ignite session will be a rapid deep-dive into practical ways to think about methodology in evaluation, apply it to traditional practice and emerging approaches, and leverage that thinking to do better evaluative work.

Exploring arts-based methods for project evaluation
Authors: Katie Ronson, Creative Australia, Kirstin Clements, Arts Centre Melbourne
In this presentation evaluation practitioners from Creative Australia and Arts Centre Melbourne share how arts-based evaluation methods can unlock insights about participant experience in contexts where traditional methods struggle.

Evaluating ConnectUp: preferences for a social connectivity app among people with disability and carers
Authors: Feby Savira, Deakin University, Dom Kwasnicka, Deakin University, Lucio Naccarella, Deakin University
Thou shall not: The 10 Commandments for Surveys 
Author: Louise Parker, Alinea International
The 10 Commandments for Surveys:
1) Thou shall not ask too many questions - survey fatigue is real and so is wasting people's time. Figure out what you absolutely must know from respondents and craft your questions around that.
2) Thou shall not ask double-barreled questions e.g. 'Did you find the workshop fun and informative?'
3) Thou shall not change the appearance order of repeating yes/no or Likert scale responses - if 'strongly agree' appears on the far right the first time, keep it that way throughout the whole survey.
...plus 7 more fabulous tips!

Cultural identity as a shield: Measuring the social value of Culture and Kinship
Authors: Louise Green, Victorian Aboriginal Community Controlled Health Organisation, Lily Edwards Wathaurong Aboriginal Co-operative
This presentation shares learnings from research by VACCHO and Victorian Aboriginal Community-Controlled Organisations (ACCOs), exploring how strengthening Identity and Cultural connectedness and practice, drives positive long-term health and wellbeing outcomes.
Building on a 2022 evaluation of the Culture and Kinship Model, the research is driven and sustained by Aboriginal leadership and Cultural governance and uses a SROI methodology to understand drivers of change and inform future recommendations. Supported by Kowa Collaboration, Aboriginal-led evaluation consultancy, the approach is grounded in culturally responsive stakeholder engagement — including Impact and Value Yarning — to enable participatory interpretation and translation of knowledge and evidence.

Authors: Alexander Gyles (La Trobe University), Allan Mua Illingworth (Muaakia Consulting), Glenda Napaljarri Wayne (La Trobe University), Mildred Napaljarri Spencer (La Trobe University), Raelene Jigili (Central Land Council), Marilyn Vilsoni (Solve Pacific)

“How might we most appropriately track and describe change about impact and wellbeing in Indigenous Australian and Fijian communities?”

Led by Indigenous community researchers and MEL practitioners, the session draws on two grounded case examples: the YWPP (Warlpiri Education and Training Trust, Tanami NT) approach to tracking learning and wellbeing, and community-centred MEL work supported by the Itaukei Trust Affairs Board (TTFB).

The session will be highly interactive: participants will break into small, facilitated groups with community and TTFB representatives to interrogate practice, and the table will close with a plenary synthesis that surfaces cross-case lessons, tensions, and practical next steps for culturally respectful MEL.

Authors: Andrew Boyle
Development programs often focus on whether interventions achieve intended outcomes, yet comparatively little attention is given to how the problems those interventions seek to address are defined. The way a problem is represented shapes which interventions are considered possible, influencing theories of change, program design, and the evaluation questions that follow.

Author: Doyen Radcliffe, Bajja Yungayimanha Collaboration
Indigenous-led systems change often begins with shifts in cultural authority, relationships, trust and institutional behaviour—long before formal outcomes or policy reform become visible. Yet many evaluation approaches continue to prioritise measurable outputs and short-term results, overlooking the relational, cultural and ethical foundations that drive meaningful change. This creates a gap between what First Nations communities experience as impact and what institutions recognise as evidence.

This presentation introduces the Danjoo Koorliny Five-Dimensional Indigenous Influence Model, developed through the Aboriginal-led impact evaluation of the Danjoo Koorliny movement in Western Australia. The model offers a practical and culturally grounded way to measure systems change by identifying and assessing influence across cultural, relational, behavioural, structural and transformational domains. Central to this work is a decolonising evaluation process that repositions cultural authority, Country and relational accountability as the foundation of ethical and rigorous evaluation. For First Nations evaluation, this means privileging Elder governance, community decision-making, Indigenous Data Sovereignty and culturally grounded methods such as yarning, storywork and outcome harvesting. For evaluation more broadly, it expands how evidence, value and impact are understood in complex systems and supports earlier identification of meaningful change.

The session will highlight three key messages. First, influence is the primary mechanism and measurable pathway of Indigenous-led systems change. Second, decolonising evaluation strengthens methodological rigour by integrating Indigenous and Western approaches rather than positioning them in opposition. Third, valuing place, relationships and cultural integrity improves the relevance, integrity and usefulness of evaluation for communities, policymakers and funders.

The presentation will combine short input, reflective discussion and interactive activities. Participants will explore practical tools, apply the framework to their own contexts, and reflect on how measuring influence can strengthen systems thinking and evaluation practice across diverse sectors.

Why you should leave evaluation (and come back!)
Author: Delyth Lloyd, Australian Centre for Evaluation
After 15 years as an AES member, I stepped away from evaluation for a one-year hiatus. What began as a brief stint in an implementation role quickly transitioned into a fast and furious year using every evaluation skill I had to offer. I negotiated in hard spaces, navigated power dynamics, held space for diversity and ethical complexities. Busy people asked for more rubrics and used them in practice! This presentation distils insights from my transition out of, and back into, the sector. It reflects on the evaluation competencies we hold, and highlights how deeply they are valued by others.

The Evaluation Pub Test: Simple Questions for Public Value 
Author: Amanda Taylor-Short, ARTD
Publicly funded evaluations carry an implicit responsibility to contribute to public good. In practice, evaluations are often viewed as mechanisms for accountability in government decision‑making. This can reinforce a focus on compliance, program‑specific detail, and predefined questions, limiting broader reflection about the contribution of a program and leaving simple but important questions unexplored. This creates a missed opportunity to connect evaluation work with the people and decisions it exists to inform. This presentation explores the Evaluation Pub Test: examining evaluator responsibility to support public value, the benefits of answering simple questions, and considerations for deciding which questions are worth answering.

From scientist to evaluator: Lessons beyond numbers 
Author: Heather Bryan, First Person Consulting
This Ignite traces one evaluator’s unlikely journey from a postgraduate ecologist in Latin America to an honours student modelling climate impacts, and later a data analyst at the Australian Bureau of Statistics - before arriving at the unexpected doorstep of evaluation. Along the way came a new appreciation for the value of qualitative evidence. This talk shares three lessons motivated by moving beyond numbers: outliers are insights rather than errors, stories explain what numbers can’t, and subjectivity, when acknowledged rather than ignored, can strengthen evaluation. Together, these lessons invite a broader view of evidence, where individual stories bring meaning.

Seeing through their eyes: Photo-Elicitation in Evaluation
Author: Estelle Gaillard, CSIRO
Images can ignite conversation, evoke emotions, elicit reflection and spark new ways of thinking. Photo-elicitation uses photographs in interviews to prompt participants to share insights into their lived experiences that might not surface through standard questions. This Ignite presentation will introduce the method, show practical examples of its use in evaluation and outline the photo-elicitation process. Attendees will leave reflecting on whether — and how — photo-elicitation could fit into their own evaluation toolkit.

Making Space for Boba and Beats: Valuing Grassroots Reality with an Evaluation MVP
Author: Anabelle (Pin-Ju) Chen, Taiwan Pride Sydney
For a volunteer-run grassroots NFP, traditional evaluation often occupies the very space intended for community action. When resources are scarce, every hour spent on data collection undermines the operational realities of the place where we operate. As the President and Evaluator of Taiwan Pride Sydney, I faced this tension at the 2026 Sydney Mardi Gras. How do you measure the cultural impact of a 'Lanterns of Ecstatica, Taiwan Lighting up Asia' float when the generator dies, the music cuts out, and you are forced to march giant illuminated bubble tea cups in total silence?
This Ignite presentation proposes a Minimum Viable Product (MVP) for evaluation. By shifting the focus to macro-validation and systemic consequences, I will demonstrate how grassroots organisations can engineer a structurally sound evidentiary argument that survives real-world disaster. Using aggressive dimensionality reduction and cognitive anchoring, we can design evaluation MVPs that extract maximum validated learning with minimum administrative friction. Let's spill the tea on how to make space for rigorous evaluation when absolutely nothing goes to plan.

Authors: Elisabeth Jackson (Centre For Human Security and Social Change, La Trobe University), Thushara Dibley Centre For Human Security and Social Change, La Trobe University, Shane D'Angelo Centre For Human Security and Social Change, La Trobe University)

This roundtable explores MEL as a practice that can challenge existing power structures and strengthen community voices. It aims to promote sharing and learning between evaluators in domestic and international contexts and build new connections between evaluators who share similar approaches and principles.

Across diverse approaches such as culturally responsive and Indigenous evaluation, realist approaches, and place-based methods, practitioners are asking: whose worldview shapes what counts as evidence? While the language used in different sectors varies, there are strong common threads: centring marginalised voices, working collaboratively, reflecting on our own assumptions, and valuing local knowledge.

This session is designed for intermediate to advanced evaluators who are thinking critically about power, partnership, and the politics of evidence. Participants will explore what is common across approaches in domestic and international contexts and how these are creating space for different worldviews and supporting forms of evidence that are meaningful to communities.

After a short framing and brief examples, participants will move into structured small-group discussions to surface shared tensions and practical insights. Prompts will include: What is one way your current evaluation practice may reinforce existing power dynamics? Where have you seen MEL genuinely shift decision-making power? What institutional constraints limit these approaches? Groups will then report back in a facilitated plenary. Participants will be invited to identify one insight they will take back into their own practice and one structural barrier the evaluation field needs to address collectively. Key insights will be shared more broadly through a blog developed after the conference.

The roundtable aligns with the aes26 sub-themes Traditional and New Ways and Ethics and Integrity, inviting participants to reflect on their real-world experience of applying participatory and culturally grounded approaches in different contexts and exploring how these can help disrupt existing power relations.

Authors: Alan Woodward, Alan Woodward Consulting, Leanne Kelly, Alfred Deakin Institute, Deakin University
Evaluations of community-based and place-based programs regularly require methods capable of examining context, relationships, and emergent outcomes. While case studies are a long-standing qualitative research approach, they are often under-utilized or misunderstood within evaluation practice. Drawing on the evaluation of Australian Red Cross’ Community Resilience Teams as an applied example, the presentation demonstrates how case study design enabled exploration of contextual dynamics, stakeholder perspectives, and underlying mechanisms that would not have been visible through survey or indicator-driven approaches alone. Participants will be offered practical guidance on when case studies are suitable and considerations for conducting case study activities.

Authors: Maud Mukova-Moses, Fred Hollows Foundation, A K M Badrul, Huq Fred Hollows Foundation, Jagath Happuhannadige, Fred Hollows Foundation
Reflecting on a predominantly qualitative evaluation, this paper explores how internally led evaluation can strengthen ethical practice, integrity, and inclusion in complex program settings. Drawing on The Fred Hollows Foundation’s mid-term evaluation of a Gender Equity, Disability and Social Inclusion (GEDSI) eye health project in Bangladesh, the paper examines how the evaluation navigated ethical practice in a context where power imbalances and language differences influenced whose voices were heard.

The evaluation engaged diverse groups, including ethnic minority communities and persons with disabilities. This raised practical ethical questions about language access, interpretation, voice and power. In navigating these complexities, the evaluation sought to incorporate equity, cultural sensitivity, and power-awareness to create space for diverse voices, reveal hidden barriers, and enable more ethical decision-making. Through use of document review, interviews, focus groups and a partner validation workshop, the evaluation intentionally foregrounded lived experience while maintaining analytical independence.

The paper demonstrates that ethical evaluation is not only about safeguarding participants, but also about how evaluators navigate competing priorities, institutional constraints, and contextual power dynamics. By conducting the evaluation internally, the team was able to deepen its understanding of gender, disability, and ethnic inclusion dynamics; build trust with community stakeholders; and generate insights that may have remained invisible.

The paper also explores tensions encountered during validation and interpretation: What does ethical evaluation look like when stakeholder priorities differ? How can evaluators recognise and address power dynamics within interviews, focus groups, and validation workshops? And how can evaluators transparently acknowledge limitations in ways that strengthen trust and learning? By sharing practical strategies and reflective insights, evaluators are invited to move beyond procedural compliance towards a deeper practice of relational accountability and integrity across diverse contexts. The paper offers practical examples of embedding ethical considerations into internal evaluation and using findings to inform practice improvement.

Author: Adrian Field, Dovetail Consulting
Value for Investment (VfI) is an emerging evaluation system that is gaining increasing traction internationally. The system is intentionally mixed methods, interdisciplinary and collaborative.
It is this collaborative positioning of VfI that this presentation is focused. In VfI framing, value draws from multiple perspectives of what makes a programme, intervention or policy valuable. Therefore, understanding and capturing value requires a fundamentally collaborative mindset, one that reaches beyond existing data sources and reporting, to explore from many different perspectives what it is that fundamentally matters about the intervention that is the evaluation’s focus.

This presentation draws on learning and approaches in exploring value through participatory design techniques. It will draw on approaches developed and refined across a range of VfI evaluations, in the health, mental health, justice, urban development, transport and creative sectors.
The session will describe a range of facilitative techniques adopted to draw on different stakeholders perspectives on value, and the strengths that they offer for building robust, relevant and actionable evaluations.

Using a VfI lens allied with participatory design techniques, the presentation will explore how value can be brought to life across key evaluation questions, theories of change and value propositions, evaluative rubrics, and collaborative sense-making.

Session participants will gain a clear sense of the possibilities and potential for applying participatory approaches in VfI, and the strengths these offer for evaluation practice.

Author: Kizzy Gandy, Jacaranda Partners

Learning objective: Participants will be able to construct core and alternative hypotheses from a theory of change, design evidence tests to distinguish between them and make defensible causal claims.

Contribution analysis is now widely used in Australian evaluation practice, but many evaluators stop short of the next step: systematically testing whether something other than the program explains the observed outcomes. Process tracing does exactly this. It raises the rigour of causal claims not by collecting more data, but by being more deliberate about what the data needs to show — and what it would need to show if the alternative explanation were true instead.

This workshop uses a real evaluation of the Embrace Multicultural Mental Health Project as a case study. The Embrace evaluation combined contribution analysis and process tracing with a mixed-method design including a quasi-experimental quantitative survey and qualitative interviews. It produced a result that many evaluations encounter but few handle well: the quantitative analysis found the program had no detectable effect on key outcomes, yet qualitative evidence suggested it was working. Process tracing provided the analytical framework to distinguish between three plausible explanations — the program failed, the effects take time to materialise, or competing programs are producing the same outcomes.

Participants work through three structured exercises: translating a theory of change into core and alternative hypotheses; designing evidence tests that can distinguish between them; and applying evaluator judgement to weight mixed evidence and reach a defensible conclusion. Each exercise uses the Embrace case, with reflection prompts connecting the method to participants' own evaluations.

The workshop is aimed at intermediate to advanced evaluators working on complex programs.

Author: Victoria Pilbeam (SPC), India Lynn (SPC)

How many robust evaluations get left on shelves? What can we do when good evaluations are overlooked? The true value of evaluative practice lies not in the reports or their recommendations, but what comes after, how we use them to broker organisational change is the route to success.
To ground these questions, our session will use a case study on capability development – a fundamental part of our work at the Pacific Community (SPC) Fisheries Aquaculture and Marine Ecosystems division. At SPC, our scientists and technical staff travel across the Pacific delivering training on a wide range of topics with the goal of enhancing the sustainable management and economic, food security, and cultural benefits of Pacific fisheries. Our capability development aims to achieve change on the ground, in the ministries, communities and industries of our member countries and territories. But successive evaluations have told us that, despite all this training, we are not necessarily seeing the desired results and that we need to explore different modalities to bring about true capability strengthening.
So, how did we dust off these evaluations and use them to chart a new pathway to impact? In this session, we will discuss how we used a combination of behaviour change research, co-design, and organisational change management to move the dial using existing reviews. This approach is rooted in theoretical, cultural and contextual considerations, including behaviour change models and Pacific pedagogies. Our paper will illustrate how taking a grounded approach can make evaluation relevant and learning strategic. Whilst, inviting participants to reflect on their own experiences and their roles in shepherding evaluations from good recommendations towards genuine organisational change.    

Authors: Stuart Raetz, Climate Resilient Communities, Primatia Romana Wulandari, Alinea International, Keren Winterford, University of Technology Sydney
This panel will explore and contrast Monitoring, Evaluation and Learning (MEL) insights from two Australian international development investments undertaking evaluative longitudinal case studies that aim to monitor changes in community and institutional resilience over an extended period: The Australian Water Partnership (AWP) and Climate Resilient Communities (CRC). AWP are retrospectively studying 10 years of water governance investments (2015–2025), while CRC are undertaking evaluative case studies in early stages of investments, exploring how communities are adapting to climate change in five countries (Cambodia, Fiji, Kiribati, Timor Leste and Tonga) in the Indo Pacific (2026 – 2029).
Drawing on emerging insights of evaluative practice, evidence and learnings the session will explore the enabling conditions that support community and institutional resilience in a changing climate. The panel will discuss:
1.Different vantage points reveal different resilience dynamics. AWP’s retrospective longitudinal analysis uncovers patterns of institutional strengthening, governance adaptation, and enabling conditions that only emerge over time, while CRC’s evaluative case studies will illuminate how climate resilience is context specific and driven by locally led adaptation practices.
2.Complementary methodologies strengthen evaluative insight. Both programs use participatory, outcome oriented, and complexity sensitive approaches—providing methodological alignment while generating distinct, mutually reinforcing evidence streams.
3.Integrated evidence supports better climate informed programming. When institutional governance evidence is paired with forward looking community insights, development programs gain stronger foundations for policy engagement, climate integration, and long-term investment planning.
The panel will provide examples from programming to contrast retrospective and forward-looking approaches to longitudinal case studies.
Audience interaction will be promoted through short provocations and facilitated reflection to elicit insights from the audience. The panel will support a practical understanding of the challenges and opportunities in defining resilience and stimulate discussion on how MEL can help programs to contribute to resilience outcomes, navigating complexity in a changing climate.

Author: Carina Calzoni, AES Rob Sale, nous

The AES has been exploring its vision of “quality evaluation that makes a difference” through a strategic project - engaging members and examining evaluation theory and practice. This work has surfaced a central challenge: while the phrase is compelling, its meaning is complex, contested, and shaped by context.

This roundtable begins from that complexity—but does not seek to resolve it. Instead, it focuses on what comes next.
We will briefly share insights from the journey so far, including the multiple dimensions of “evaluation”, “quality”, and “making a difference”. These span tensions between evaluation as process, product, and profession; competing perspectives on quality (e.g. standards, utility, impact, values); and diverse understandings of use and influence across contexts and stakeholders.

The primary purpose of the session is to co-design how this conversation continues across the AES community. Participants will engage in facilitated small-group discussions to explore key questions: What does quality evaluation mean in your context? Who defines it? What does “making a difference” look like—and for whom? How should these conversations evolve as contexts, practice, and membership change?

Participants will then work together to identify practical ways to sustain and deepen engagement, such as ongoing communities of practice, publications, podcasts, or future conference formats. The session will capture and share these ideas to inform AES’s ongoing work.

Designed for intermediate to advanced evaluators, this roundtable creates space for collective reflection and future-oriented dialogue. By centring plurality and participation, it supports the AES vision by keeping the conversation alive—recognising that what constitutes “quality evaluation that makes a difference” must continue to evolve with the field.

Authors: Patricia Rogers, Footprint Evaluation Initiative, Emil Laurberg Morgensen, University of Southern Denmark
Theories of change are now commonly used to design initiatives, including projects, programs and policies, and to shape monitoring and evaluation.  But 40 years after first being showcased at an AES conference, these are often developed and used in ways that don’t fit what is needed, especially to support adaptation across different settings and in changing contexts.  This session will present common mistakes, what they are, why they matter, and examples of better strategies.  Participants will engage with exercises to identify issues, try out strategies and explore possible applications in their own work and in shaping organisational requirements and procedures.

Culturally Grounded Evaluation: Innovative Methods from the Champions4Change Workshop
Author: Sunet Jordaan, Tennele Shields 
The Champions4Change program is a First Nations–led initiative supporting people with lived experience of acute rheumatic fever (ARF) and rheumatic heart disease (RHD) to deliver culturally safe education and advocacy in their communities. In 2025, the Heart Foundation delivered a workshop to strengthen Champions’ knowledge, confidence and leadership skills using culturally grounded and accessible evaluation methods.
This presentation shares the evaluation approach used to assess knowledge gained, confidence built and participants’ experiences. Instead of surveys, evaluators used Yarning circles, storytelling and visual self-assessment tools to create a culturally safe environment aligned with community ways of knowing, being and doing. First Nations facilitators with established relationships with Champions led the activities.
Champions used a visual “road journey” to represent changes in confidence and reflected on their understanding of ARF and RHD through yarning conversations and storytelling discussions. The session highlights how culturally grounded, visual and narrative methods can improve accessibility, participation and lived experience leadership while generating richer insights than traditional surveys.

Agile Evaluation Approaches to Combatting Antisemitism in Australia
Authors: Linda Gyorki, Milo McKay
This presentation explores agile evaluation through a real-world evaluation of place-based initiatives aimed at combating antisemitism and strengthening social cohesion. As global political developments, social polarisation and misinformation reshape communities, evaluators working in sensitive environments can no longer rely on fixed designs. This presentation demonstrates how evaluation can adapt without sacrificing rigour or credibility. Three key lessons highlight agile evaluation in practice. First, polarisation affects participation and engagement, making trust-building and relationship management essential evaluation skills. Second, iterative data collection enables evaluators to identify emerging issues, adjust methods and remain responsive to changing community dynamics. Third, responsiveness itself builds credibility. When evaluation processes visibly adapt to local events and stakeholder realities, findings become more useful and trusted by communities.
Aligning with the conference theme, “Making space, valuing place,” the presentation examines how evaluation can remain grounded in community experience while responding to broader global forces. The session will share practical lessons for evaluators navigating sensitivity, external events and methodological adaptation.

Co-design and evaluation as bridges: An adaptive approach to delivering technology for coral reef conservation
Author: Emily Maher
Delivering multilateral conservation projects requires working across disciplinary, cultural and institutional boundaries. In Southeast Asia, coral reef monitoring and management are further challenged by limited resources, capacity and coordination. This presentation shares the adaptive project management and evaluation approaches used by the Australian Institute of Marine Science to support coral reef monitoring through knowledge, technology and expertise exchange. A deliberate “year zero” planning phase aligned expectations, identified local needs and strengthened collaboration. Guided by co-design and adaptive management principles, evaluative processes structured dialogue between scientists, policymakers, managers and practitioners while integrating diverse priorities and cultural considerations. Evaluation functioned as connective infrastructure rather than simply monitoring progress. Regular reflection on partner feedback and evolving implementation needs supported timely adaptation and negotiation of trade-offs. This collaborative approach fostered strong local ownership and sustained government support, contributing to outcomes including a national coral reef monitoring plan in Brunei Darussalam and agreed monitoring standards in the Philippines and Vietnam. The presentation reflects on roadblocks, unexpected outcomes and lessons for complex projects operating across policy, technology, science and implementation.

Effects of community water fluoridation on child dental caries in remote Northern Territory, Australia
Author: Ramakrishna Chondur
Community water fluoridation (CWF) is a cost-effective intervention for reducing dental caries at a population level. This Northern Territory (NT) study used a difference-in-difference (DiD) analysis to examine dental caries outcomes among children exposed to CWF across 50 remote NT communities. Methods: Oral health data from the NT Department of Health (2008–2020) included 24,546 children aged 1–17 years. Drinking water fluoride data from the Power and Water Corporation were linked to the oral health dataset. The DiD analysis compared a treatment group with two control groups to assess the impact of CWF on dental caries outcomes using the decayed, missing and filled teeth (dmft/DMFT) index. Results: Dental caries significantly decreased among children in the treatment group following implementation of CWF, with greater reductions than both control groups over the same period.
Conclusion: CWF produced population-level reductions in dental caries among children in remote NT communities, supporting longstanding NT Department of Health policy and demonstrating improved oral health outcomes.

Enshittification: A new category for your next MEL framework?
Author: Duncan Rintoul
The term enshittification (or enpoopification for delicate ears) describes how digital services often begin as useful tools but degrade in quality and usability over time. This decline is commonly linked to increasing prioritisation of profit, resulting in cluttered interfaces, manipulated content, more advertising, rising costs and poorer user experience.
Coined by Canadian writer Cory Doctorow, the term was named “word of the year” by the American Dialect Society in 2023 and Australia’s Macquarie Dictionary in 2024. Drawing on Doctorow’s 2025 book Why Everything Suddenly Got Worse and What To Do About It, this Ignite presentation argues that enshittification deserves a place in risk matrices and MEL frameworks across multiple sectors. The presentation also argues for the value of longitudinal process evaluation, not only longitudinal outcome evaluation, to better understand how systems and user experiences deteriorate over time.

From Admin to Evidence: Transforming Data Quality Through Culture and Capability
Author: Cath Cooper
This Ignite presentation will offer practical insights into how redefining purpose, building capability, and embedding supportive systems can transform data from a burden into a meaningful asset for evaluation.
Key points: 1.Cultural Shift: we reframed “admin tasks” as “Evidence of Impact”, shifting the narrative to honour the value of data as a foundation for learning and accountability 2.System Redesign: A collaboratively built CMS, supported by guides, workshops and induction, strengthened capability and consistency. 3.Sustained Improvement: Monthly monitoring, mentoring and rapid feedback loops led to significantly improved data accuracy and reduced correction time, enabling reliable reporting and decision making.

Authors: Anne Stephens, Ethos of Engagement, Jill Thomas
This presentation explores the design of a systemic Monitoring and Evaluation (M&E) system for counter-trafficking in persons (CTIP) in the ASEAN region. Developed in 2023, the system supports the adoption and implementation of three victim-centred and gender-sensitive guides for counter trafficking in persons, in use across ASEAN member states. The presentation focuses on how systemic, participatory approaches to evaluation design can enhance uptake of evaluation and the role of a well-designed framework to support the capacity of individuals to monitor and evaluate their work.  

The objective of this presentation is to present the process used to develop a simple to use M&E system within a complex setting; show the public facing guidance documents and tools used to support novice and highly skilled evaluators to use the system; and describe our challenges and learnings.

This presentation offers a timely and practice-grounded contribution to the evaluation field by demonstrating how evaluators can design for relevance, capacity development and impact in complex, real-world settings. It provides actionable insights for practitioners seeking to strengthen the value and use of evaluation in increasingly uncertain and contested environments.

Authors: Eleanor Williams, ACE, Jade Maloney, ARTD, Jess Buchwald, Verian
Did you grow up wanting to be an evaluator? For most of us the answer is no, and there is value in the way we have all fallen into evaluation profession from diverse backgrounds.

But what does this mean for our career pathways and the way others think of evaluation and evaluators? What does it mean for evaluation as a profession?

As the leading voice for evaluation in Australian, the Australian Evaluation Society has long considered options for and pathways to professionalisation that strengthen our roots as a society, and develop the routes to a future in which evaluation profession is increasingly recognised and valued.

The 2024-2028 AES Strategic Plan firmly put considering pathways to professionalisation back on the agenda. A working group has since been exploring options for professionalisation, drawing on learnings from the review undertaking by Peersman and Rogers in 2017, consistent with the AES values, and informed by the experiences of other evaluation associations and other professionalisation associations in Australia.
Now it’s time to seek your views because professionalisation is for you, the Australian evaluation community.

Harnessing the success of the fishbowl approach to exploring quality evaluation that makes a difference at the 2025 conference, we are inviting everyone to jump in and share their thoughts about the value professionalisation would provide, and the risks to be managed to ensure the pathway are accessible, inclusive and respectful of diverse ways of knowing. The working group understands there are diverse views on professionalisation and invites these to be surfaced in this conversation.

The session will be facilitated by professionalisation working group co-chairs Eleanor Williams, Jade Maloney and Jess Buchwald with opportunities to contribute live or through written formats. The working group will use what you share in shaping the route forward.

And yes, there will again be chocolate fish rewards for contributors.

Just think: What would it look like if the next generation could grow up wanting to be an evaluator? What if when you became an evaluator you could see a clear pathway forward?

Author: Matthew Armstrong, CGIAR
This presentation provides practical insights on how and when to use economics in evaluation. Economic ideas shape public policy, funding, and program design. Economics promises efficiency, cost effectiveness, value for money and optimisation. Many evaluation theories, for example Stufflebeam’s CIPP model (2007), Scriven’s (2007) consumer orientated model, and Patton’s (2003) utilisation-focused model are biased towards the social concerns of evaluation.

Yet, evaluators are often tasked with incorporating economic concepts, methods and language into evaluations. But how do we make them fit-for-purpose, credible and culturally appropriate? This case study examines when and how economic concepts should be integrated into evaluation.

CGIAR, a global agricultural research network, is introducing tools to apply comparative advantage in portfolio design and evaluation. Comparative advantage suggests organisations should specialise in areas where they are relatively strongest and partner elsewhere to maximise impact. The theory is compelling: comparative advantage offers reduced duplication and deliberate partnerships at lower costs.

However, how do we credibly measure and make evaluative judgements about comparative advantage? This study found considerable variation in how comparative advantage was applied across the organisation and consternation in response to the roll-out that hindered its usefulness. This presentation describes an interdisciplinary negotiation undertaken to develop guidance for rigorous process and performance evaluations of comparative advantage within CGIAR. It involves a structured process to identify synergies and tensions between existing evaluation and economic practice relating to comparative advantage, a step-by-step approach to undertaking evaluations with comparative advantage criteria, and recommendations for developing an organisation-wide approach to comparative advantage to support consistent, high-quality. Furthermore, this case study illustrates a crucial requirement to reflect upon how economics is integrated into evaluation, including when it should be adapted or challenged.

In the interactive component, participants will reflect on how economics shapes evaluation in their organisations.