aes26 International Evaluation Conference

Authors: Kristine Tuban, Save the Children, Martin Holmstrand, Save the Children
This presentation showcases MEAL Uplift, a regional initiative by Save the Children Australia that strengthens digital monitoring, evaluation, accountability, and learning (MEAL) across Pacific Country Offices through a close partnership between programme teams, MEAL, and the Technology team. The presentation focuses on how this collaboration produced a practical, co‑created digital solution while building long‑term organisational capability.

The objective of the presentation is to demonstrate why cross‑disciplinary partnership is critical to improving data quality, efficiency, and use in evaluation practice, and how evaluation can actively shape digital transformation, rather than simply adopting technological tools after they are introduced. This is important in contexts where teams face increasing reporting demands but limited capacity to manage fragmented or manual data systems.

The core argument is that sustainable digital MEAL is achieved when three elements are intentionally integrated:
(1) Digital systems and infrastructure, illustrated through a custom‑built data collection app and automated data pipeline co‑designed with the Technology team to address efficiency and quality challenges;
(2) Skills and behaviours, supported through targeted training, coaching, and practical use of real data; and
(3) Scaling and institutional support, through shared standards, governance, and regional scaffolding that embeds digital MEAL into everyday practice.

The presentation will follow this three‑part structure, using concrete examples and lessons from MEAL Uplift to show what worked, what changed, and why.

Participant engagement will be promoted through short reflection prompts on participants’ own digital MEAL challenges, followed by shared discussion and an open Q&A focused on transferable ideas and co‑created solutions across different organisational contexts.

Author: Alice Muller, World Vision
Evaluators and development practitioners need to understand how environmental changes impact and result from development programming.  Often finding practical ways to do is a challenge, particularly where complex environmental, biodiversity and climate systems are involved, and where evaluators and practitioners come from social or economic backgrounds with limited biophysical or ecological training. This paper shares how we can better value place and understand environmental consequences in international development projects by letting evaluations be locally-led, with local knowledge and values at the center.

Drawing on real‑world examples from World Vision’s Regreening Communities work, the paper presents three simple tools used by communities t Together, these approaches value place by centring local and Indigenous knowledge and values, assessing change from a local perspective, in a standardised way and using spatial tools that together still contribute to organisational‑level evidence.

First, a community mapping and planning exercise makes space for diverse groups within a community to define what regreening and wellbeing mean in their own environments, surfacing cultural, livelihood and social values that are often invisible in standard indicator frameworks. Second, a community‑assessed Regreening Index provides a structured and repeatable way to assess biophysical conditions and trends over time in relation to community priorities, while still enabling consistent comparison across sites and projects. Third, spatial mapping of regreening sites using a powerful, but freely available tool adapted from the WASH sector, enables evaluators to examine patterns and cumulative effects at project and organisational levels.

While none of these tools is unique in isolation, when combined they lower barriers to including environmental considerations in evaluation practice and generate insights that strengthen learning, adaptive management and sustainability discussions, ethically. The presentation will share engaging practical examples and invite participants to reflect on opportunities to better value place within their own work.

Author: Charlie Tulloch (Policy Performance)

This session at the start of the conference orients new, emerging and non-evaluators to the types of topics and content they will encounter over the coming days. It provides a foundational overview of evidence-related fields (e.g. evidence, monitoring, evaluation, learning), debunks key concepts/language, offers insights into what evaluators do day-to-day, discusses values/valuing, ethical considerations, competencies and key steps usually considered when completing high quality evaluation projects.The session strengthens defines the boundaries and intersections of evaluation and other disciplines to understand this field of endeavour, the pathways in and the opportunities it offers.

The session’s main aims are to share and discuss:

• How evidence is used in different policy and organisational landscapes

• The roles and types of projects led by evaluators

• Techniques frequently used by evaluators•Ways that evaluators think (logic, data, outputs, outcomes, impact)

• Why and when evaluations happen (or do not)

• Practical tips for evaluation commissioners and working with evaluators

• Capability building (skills and knowledge) to promote use of evaluation in-house

• A seven-stage approach to planning and conducting an evaluation project

• Essential skills of evaluators, including logic modelling, defining key/interview questions, forming value judgments, and selecting effective evaluation approaches/methods.The session will provide attendees with an opportunity to ask questions anonymously, so they feel very comfortable as the conference commences.

Questions will be addressed during the session, creating an open, interactive discussion.The session will also provide an overview of the conference structure, learning opportunities and flow, to provide a guide to attendees about how to gain the most value from their time.

Author: Jess Dart, Clear Horizon
This paper introduces the Collective Noticing Method (CNM) and explores its relevance for measurement, evaluation and learning (MEL). Developed by the author, CNM offers evaluators and changemakers a collaborative way to notice, learn from, and respond to subtle and emerging changes. It is a rigorous yet flexible method with a strong focus on learning and collective sensemaking, well suited to the “messy middle” of systems change and place-based work. It also has potential for community and grassroots work, program portfolios, and research impact tracking.

Collective noticing as a practice has a long history, evident in approaches such as Outcome Harvesting, Most Significant Change (MSC), citizen science, ripple effect monitoring, and context monitoring. More recently, this practice has been enabled and accelerated by emerging digital platforms and AI. Building on these foundations, CNM intentionally harnesses the eyes and ears of those implementing and experiencing an initiative to bring shared attention to what is changing.CNM is inherently participatory, inviting many people to observe, document, and interpret change collectively. It values lived experience as a central source of insight, embraces uncertainty, and supports adaptive action grounded in what is actually emerging. As a decolonising and grounded approach to MEL, CNM shares power by broadening who defines what matters and what counts as evidence. It centres learning-oriented measurement and collective sensemaking, bringing forward subtle signals, relational shifts, and everyday insights that conventional approaches often overlook.

CMN involves impact logging and tracking against agreed ways of working and learnings, whereas Outcome Harvesting focuses on outcomes and tends to be done as a one-off study. It differs from MSC in that CNM uses shorter, multi-perspective logs across diverse categories and evidence types.This session introduces CNM and invites participants to critically explore its benefits, limitations, and the contexts in which it is most—and least—useful.

Author: Salli Cohen (The Policy Room)
This presentation explores the uncomfortable space where policy and evaluation structurally reinforce harm. Its objective is to challenge assumptions of neutrality and examine how evaluative frameworks can either illuminate or obscure inequity.
The core argument is threefold: (1) evaluation and policy are never neutral and must interrogate power; (2) systems alignment and cultural authority determine whether outcomes are real or performative; and (3) accountability must ask “for whom” evaluation and policy work; and who bears the cost.
The session blends applied case insight with structured reflection and peer dialogue to provoke critical engagement and practical recalibration.

Authors: Kristian Paolo Torres, BRAC Ultra-Poor Graduation Initiative, Tania Dora Warokka, BRAC Ultra-Poor Graduation Initiative, Arnaldo Pellini, BRAC Ultra-Poor Graduation Initiative
Programmes aiming to influence governance systems and policy reforms—particularly those using experimental approaches—require MEL functions that are able to move across design, strategy, context analysis, and sensemaking.

Traditional M&E often prioritizes linear accountability, struggling to measure influence on policies and behaviors (as noted by the ODI RAPID programme, 2006–2020).

This presentation shares the experience of the BRAC Ultra-Poor Graduation Initiative, which partners with governments across six countries to scale the ‘Graduation’ approach through national social development systems (e.g. social protection, livelihood, labour and employment, etc.). Because UPGI’s success is measured by policy influence and system change rather than direct delivery, the MEL function must act as a strategic bridge rather than a neutral observer.

Drawing on our experience in Indonesia and the Philippines, we share how MEL supports country teams as adaptive units navigating two primary challenges: calibrating the interplay between political commitment, administrative feasibility, and technical quality; and designing for scale from the outset to ensure government ownership rather than isolated pilot results.

We argue that a team learning culture—rather than a "neutral" M&E function—is the essential infrastructure for this work. By establishing specific habits to capture field experience and inform strategic decisions, MEL acts as the "many-hatted" bridge required to navigate these dynamic systemic boundaries and prevent programmes from remaining stagnant.

We will share three core insights from our MEL experience:

1. How the evaluator’s role shifts to an active learning partner, framing field evidence to navigate political and administrative boundaries.
2. How practical tools—such as Capacity and Commitment Rubrics and Diaries—support teams in documenting for learning rather than reporting for compliance.
3. Real-world examples where structured reflection informed specific shifts in government engagement strategies.

The session will also engage the audience in a "Reflection Micro-Workshop" where participants will share insights from their own work and experiences.

Who really benefits from a program and how? Using personae to enrich logic models.
Author: Fiona Scott-Melton, Allen + Clarke
Not everyone experiences a program the same way. Yet logic models tend to obscure differences between population groups experiences, needs and aspirations. Incorporating personae helps address this limitation. By representing distinct population groups within a logic model, personae make visible the different routes through which people experience a program and what they gain from it. Drawing on a real life logic model developed for housing disabled people, the session will show how incorporating personae strengthens evaluation design, enriches social return on investment calculations and produces more compelling findings for stakeholders.

Help! The post-it notes have fallen off our logic model and been mixed up!
Author: Robert Grimshaw, ATO
Can you help us piece the logic model back together? This online interactive group activity is one tool used by a large public sector agency to familiarise people with key logic modelling concepts and thinking. This session will provide a practical demonstration of the activity and how it’s being used by workshop participants to apply their learning in a simple and fun way. Participant feedback has confirmed the value of this tool for engaging with and building the capability of a diverse audience who have limited evaluation capability and experience.

Method vs methodology – what’s the difference, and why it matters 
Author: Martina Donkers, Martina Donkers
We’ve got to stop using the words ‘method’ and ‘methodology’ interchangeably. ‘Methodology’ isn’t a slightly-fancier way of saying ‘method’ – it’s actually what conceptually underpins your method choices, and it’s an important thing to reckon with if you want to do evaluation that speaks to your stakeholders, is culturally responsive, is ethical and proportionate, and is ultimately useful. This quick-fire Ignite session will be a rapid deep-dive into practical ways to think about methodology in evaluation, apply it to traditional practice and emerging approaches, and leverage that thinking to do better evaluative work.

Exploring arts-based methods for project evaluation
Authors: Katie Ronson, Creative Australia, Kirstin Clements, Arts Centre Melbourne
In this presentation evaluation practitioners from Creative Australia and Arts Centre Melbourne share how arts-based evaluation methods can unlock insights about participant experience in contexts where traditional methods struggle.

Evaluating ConnectUp: preferences for a social connectivity app among people with disability and carers
Authors: Feby Savira, Deakin University, Dom Kwasnicka, Deakin University, Lucio Naccarella, Deakin University

A national DCE (n=1,031) was conducted with people with disability, carers, and individuals identifying as both. Participants completed choice tasks comparing hypothetical versions of ConnectUp that varied across cost, communication modes, group structures, anonymity, support features, and information sources. Preferences were analysed using a conditional logit model, with latent class analysis used to identify user segments. Attitudinal questions explored additional features not captured within the DCE.

Three key findings emerge. First, participants were cost sensitive, preferring low- or no-cost options. Second, privacy and control were critical: users preferred small, private group settings and avoided features that compromised anonymity. Third, preferences were heterogeneous. Three segments were identified, including a highly price-sensitive group, a majority group prioritising anonymity and community-generated content, and a smaller segment (18%) more tolerant of cost but still favouring private group structures.

Attitudinal findings highlight additional design considerations. Flexible access features such as membership suspension (73%) and free trial options (77%) were widely valued. Around half of participants prioritised staff training in inclusivity (50%) and access in a preferred language (51%), while over half (54%) preferred human support over automated options.

This presentation demonstrates how combining DCE and attitudinal data can strengthen evaluation of digital services by capturing trade-offs and broader user expectations, and how segmentation can inform more inclusive design decisions. Participants will engage in scenario-based discussion to translate findings into practice.

Thou shall not: The 10 Commandments for Surveys 
Author: Louise Parker, Alinea International
The 10 Commandments for Surveys:
1) Thou shall not ask too many questions - survey fatigue is real and so is wasting people's time. Figure out what you absolutely must know from respondents and craft your questions around that.
2) Thou shall not ask double-barreled questions e.g. 'Did you find the workshop fun and informative?'
3) Thou shall not change the appearance order of repeating yes/no or Likert scale responses - if 'strongly agree' appears on the far right the first time, keep it that way throughout the whole survey.
...plus 7 more fabulous tips!

Authors: Andrew Boyle
Development programs often focus on whether interventions achieve intended outcomes, yet comparatively little attention is given to how the problems those interventions seek to address are defined. The way a problem is represented shapes which interventions are considered possible, influencing theories of change, program design, and the evaluation questions that follow.

Author: Doyen Radcliffe, Bajja Yungayimanha Collaboration
Indigenous-led systems change often begins with shifts in cultural authority, relationships, trust and institutional behaviour—long before formal outcomes or policy reform become visible. Yet many evaluation approaches continue to prioritise measurable outputs and short-term results, overlooking the relational, cultural and ethical foundations that drive meaningful change. This creates a gap between what First Nations communities experience as impact and what institutions recognise as evidence.

This presentation introduces the Danjoo Koorliny Five-Dimensional Indigenous Influence Model, developed through the Aboriginal-led impact evaluation of the Danjoo Koorliny movement in Western Australia. The model offers a practical and culturally grounded way to measure systems change by identifying and assessing influence across cultural, relational, behavioural, structural and transformational domains. Central to this work is a decolonising evaluation process that repositions cultural authority, Country and relational accountability as the foundation of ethical and rigorous evaluation. For First Nations evaluation, this means privileging Elder governance, community decision-making, Indigenous Data Sovereignty and culturally grounded methods such as yarning, storywork and outcome harvesting. For evaluation more broadly, it expands how evidence, value and impact are understood in complex systems and supports earlier identification of meaningful change.

The session will highlight three key messages. First, influence is the primary mechanism and measurable pathway of Indigenous-led systems change. Second, decolonising evaluation strengthens methodological rigour by integrating Indigenous and Western approaches rather than positioning them in opposition. Third, valuing place, relationships and cultural integrity improves the relevance, integrity and usefulness of evaluation for communities, policymakers and funders.

The presentation will combine short input, reflective discussion and interactive activities. Participants will explore practical tools, apply the framework to their own contexts, and reflect on how measuring influence can strengthen systems thinking and evaluation practice across diverse sectors.

Why you should leave evaluation (and come back!)
Author: Delyth Lloyd, Australian Centre for Evaluation
After 15 years as an AES member, I stepped away from evaluation for a one-year hiatus. What began as a brief stint in an implementation role quickly transitioned into a fast and furious year using every evaluation skill I had to offer. I negotiated in hard spaces, navigated power dynamics, held space for diversity and ethical complexities. Busy people asked for more rubrics and used them in practice! This presentation distils insights from my transition out of, and back into, the sector. It reflects on the evaluation competencies we hold, and highlights how deeply they are valued by others.

The Evaluation Pub Test: Simple Questions for Public Value 
Author: Amanda Taylor-Short, ARTD
Publicly funded evaluations carry an implicit responsibility to contribute to public good. In practice, evaluations are often viewed as mechanisms for accountability in government decision‑making. This can reinforce a focus on compliance, program‑specific detail, and predefined questions, limiting broader reflection about the contribution of a program and leaving simple but important questions unexplored. This creates a missed opportunity to connect evaluation work with the people and decisions it exists to inform. This presentation explores the Evaluation Pub Test: examining evaluator responsibility to support public value, the benefits of answering simple questions, and considerations for deciding which questions are worth answering.

From scientist to evaluator: Lessons beyond numbers 
Author: Heather Bryan, First Person Consulting
This Ignite traces one evaluator’s unlikely journey from a postgraduate ecologist in Latin America to an honours student modelling climate impacts, and later a data analyst at the Australian Bureau of Statistics - before arriving at the unexpected doorstep of evaluation. Along the way came a new appreciation for the value of qualitative evidence. This talk shares three lessons motivated by moving beyond numbers: outliers are insights rather than errors, stories explain what numbers can’t, and subjectivity, when acknowledged rather than ignored, can strengthen evaluation. Together, these lessons invite a broader view of evidence, where individual stories bring meaning.

Bridging Sectors Through Evaluation: Lessons from a University–Community, Community of Practice
Author: Hannah Morgan, UTS
In 2025, the University of Technology Sydney partnered with Inner Sydney Voice (ISV) to co-design an Evaluation Community of Practice, connecting university evaluators and community practitioners. We reflect on lessons learned from the collaboration, including the value of shared ownership and relational trust in cross-sector work. Community practitioners strengthened their confidence and capability, while UTS deepened its understanding of how to adapt evaluation capacity building to meet the needs of the community sector context. The experience illustrated the mutual benefits of collaboration and the potential of Communities of Practice to advance evaluation through reciprocal learning.

Seeing through their eyes: Photo-Elicitation in Evaluation
Author: Estelle Gaillard, CSIRO
Images can ignite conversation, evoke emotions, elicit reflection and spark new ways of thinking. Photo-elicitation uses photographs in interviews to prompt participants to share insights into their lived experiences that might not surface through standard questions. This Ignite presentation will introduce the method, show practical examples of its use in evaluation and outline the photo-elicitation process. Attendees will leave reflecting on whether — and how — photo-elicitation could fit into their own evaluation toolkit.

Making Space for Boba and Beats: Valuing Grassroots Reality with an Evaluation MVP
Author: Anabelle (pin-chu) Chen, Taiwan Pride
For a volunteer-run grassroots NFP, traditional evaluation often occupies the very space intended for community action. When resources are scarce, every hour spent on data collection undermines the operational realities of the place where we operate. As the President and Evaluator of Taiwan Pride Sydney, I faced this tension at the 2026 Sydney Mardi Gras. How do you measure the cultural impact of a 'Lanterns of Ecstatica, Taiwan Lighting up Asia' float when the generator dies, the music cuts out, and you are forced to march giant illuminated bubble tea cups in total silence?
This Ignite presentation proposes a Minimum Viable Product (MVP) for evaluation. By shifting the focus to macro-validation and systemic consequences, I will demonstrate how grassroots organisations can engineer a structurally sound evidentiary argument that survives real-world disaster. Using aggressive dimensionality reduction and cognitive anchoring, we can design evaluation MVPs that extract maximum validated learning with minimum administrative friction. Let's spill the tea on how to make space for rigorous evaluation when absolutely nothing goes to plan.

Authors: Alexander Gyles (La Trobe University), Allan Mua Illingworth (Muaakia Consulting), Glenda Napaljarri Wayne (La Trobe University), Mildred Napaljarri Spencer (La Trobe University), Raelene Jigili (Central Land Council), Marilyn Vilsoni (Solve Pacific)

“How might we most appropriately track and describe change about impact and wellbeing in Indigenous Australian and Fijian communities?”

Led by Indigenous community researchers and MEL practitioners, the session draws on two grounded case examples: the YWPP (Warlpiri Education and Training Trust, Tanami NT) approach to tracking learning and wellbeing, and community-centred MEL work supported by the Itaukei Trust Affairs Board (TTFB).

The session will be highly interactive: participants will break into small, facilitated groups with community and TTFB representatives to interrogate practice, and the table will close with a plenary synthesis that surfaces cross-case lessons, tensions, and practical next steps for culturally respectful MEL.