aes26 International Evaluation Conference

Authors: Kristine Tuban, Save the Children, Martin Holmstrand, Save the Children
This presentation showcases MEAL Uplift, a regional initiative by Save the Children Australia that strengthens digital monitoring, evaluation, accountability, and learning (MEAL) across Pacific Country Offices through a close partnership between programme teams, MEAL, and the Technology team. The presentation focuses on how this collaboration produced a practical, co‑created digital solution while building long‑term organisational capability.

The objective of the presentation is to demonstrate why cross‑disciplinary partnership is critical to improving data quality, efficiency, and use in evaluation practice, and how evaluation can actively shape digital transformation, rather than simply adopting technological tools after they are introduced. This is important in contexts where teams face increasing reporting demands but limited capacity to manage fragmented or manual data systems.

The core argument is that sustainable digital MEAL is achieved when three elements are intentionally integrated:
(1) Digital systems and infrastructure, illustrated through a custom‑built data collection app and automated data pipeline co‑designed with the Technology team to address efficiency and quality challenges;
(2) Skills and behaviours, supported through targeted training, coaching, and practical use of real data; and
(3) Scaling and institutional support, through shared standards, governance, and regional scaffolding that embeds digital MEAL into everyday practice.

The presentation will follow this three‑part structure, using concrete examples and lessons from MEAL Uplift to show what worked, what changed, and why.

Participant engagement will be promoted through short reflection prompts on participants’ own digital MEAL challenges, followed by shared discussion and an open Q&A focused on transferable ideas and co‑created solutions across different organisational contexts.

Authors: Jen Lorains (Children's Ground), Veronica Doolan (Children's Ground), Pauline Grant (Children's Ground), Jackie Treeves (Children's Ground)

For too long our people have been the subjects not the leaders of evaluation and research: “Our people have been researched to death. It’s time we researched ourselves back to life” (William Tilmouth, Senior Arrernte man).
Children’s Ground (CG) is disrupting the status quo in research and evaluation. From daily data collection and designing evaluation tools, to analysing evaluation data through community and culturally centred evaluation frameworks relevant to each place.

Using practice evidence and collaborative reflection about how First Nations communities are leading service/program evaluation for their families and place, the workshop learning objective is that participants will increase their understanding of culturally grounded evaluation and gain practical strategies and skills that can be applied to their evaluation context.

The workshop will consist of two parts, including CG sharing practice evidence, followed by collaborative group/table strategy development.

Firstly, CG’s evaluation principles will be outlined, with First Nations leaders sharing experiences in action. Participants will reflect on 2-3 principles, documenting their effective and challenging experiences of working in line with the principes, then sharing with the larger group. CG’s First Nations leaders will respond, building on the knowledge being generated by the participants.

Secondly, CG’s First Nations leaders will share experiences of developing community and culturally centred evaluation frameworks, including a visual walk through of 2-3 frameworks developed by First Nations communities across three culturally and geographically diverse regions. Comparative examples of evaluation data analysis between CG’s cultural and western evaluation frameworks will also be shared, including methodological implications.

Participants will collaboratively document ideas for supporting First Nations people/communities to develop community and culturally centred evaluation frameworks in their context, then sharing with the larger group.

We believe learning how to embed culturally grounded evaluation from First Nations community’s real-world experience is an important contribution to holistic learning, complements theoretical learning.

Author: Jo van Twest Farmer (Rooftop Social)

Independence is often posited as a primary means of ensuring evaluation is rigorous, trustworthy and fair. Independence is Norm 4 of the United Nation’s Evaluation Group Norms and Standards for Evaluation (2016), which states it is “necessary for credibility”. Built into this standard are two concepts – behavioural independence (evaluating without undue influence) and organisational independence (structures that support independent practice). For many in the evaluation landscape, the best means of ensuring independence has been for ‘independent evaluators’ to conduct ‘independent evaluations’.
Discussions around what constitutes independence have long been a feature of evaluation literature, but arguably there have never been so many pressures on maintaining independence in evaluation. There is growing interest in evaluation approaches where the relationships between program delivery and evaluation is less clear-cut, such as developmental, participatory and lived experience-led evaluations. As evaluation capability grows, evaluation functions are more widespread across teams and embedded in business-as-usual learning practice, rather than contracted out. Less positively, changing fiscal conditions place more pressure on organisations to undertake evaluation with less capacity for external expertise and support. The need to demonstrate impact to access funding in tightening funding environments means evaluations are higher stakes than ever, and there are often vested interests in demonstrating programs are effective.
This short paper will discuss what independence means in 2026, arguing that evaluation has moved beyond ‘organisational independence’ and must instead focus on how to support ‘behavioural independence’. Contemporary evaluation practice requires understandings of independence grounded in ethics and integrity. This approach directly challenges traditional role boundaries in evaluation, such as external versus internal or participant versus evaluator. In responding to the challenges of maintaining independence in 2026 and beyond, the presentation will suggest criteria to support evaluators to guide the ethics of their own practice, and a framework for upholding independent practice.

Authors: Nic Vogelpoel (Day Four Projects), Malika Reese (Lived Experience Advisor), Sandra Collins (Lived Experience Advisor)
What happens when evaluation is not just informed by lived experience, but led by it?

This presentation offers a rare, practice-based account of a lived experience-led evaluation undertaken with the National Centre for Action on Child Sexual Abuse, where victim survivors worked as co-evaluators across all stages of the process. Established to centre victim survivors in awareness-raising, help-seeking, advocacy and best-practice responses, the Centre provided a powerful context to rethink how evaluation knowledge is produced, and by whom.

The session argues that lived experience-led evaluation is not an ethical “add-on”, but a fundamental shift in values, power and epistemology. Drawing on the evaluation findings, presenters will explore three core propositions.

First, nothing about us without us: moving from evaluation on people to evaluation led by people with lived experience reshapes evaluation questions, evidence, outcomes and definitions of rigour. Second, epistemic justice: lived experience leadership challenges entrenched assumptions about who gets to ask questions, whose knowledge counts, and how institutions respond to new forms of evidence. Third, beyond advice to synthesis: lived experience cannot simply “advise” evaluation, it must be integrated as a distinct way of knowing that transforms the whole evaluation.

Co-presented by lived experience evaluators and Day Four Projects evaluators, the session will combine reflective storytelling, concrete practice examples and facilitated dialogue. Participants will be invited to critically examine their own evaluative assumptions, engage in small-group reflection, and explore practical strategies for making space for multiple knowledge systems while maintaining evaluative integrity and meeting institutional requirements.This session will be particularly valuable for evaluators, commissioners, researchers and practitioners seeking more just, credible and impactful approaches to lived-experience evaluation in complex and sensitive contexts.

Who really benefits from a program and how? Using personae to enrich logic models.
Author: Fiona Scott-Melton, Allen + Clarke
Not everyone experiences a program the same way. Yet logic models tend to obscure differences between population groups experiences, needs and aspirations. Incorporating personae helps address this limitation. By representing distinct population groups within a logic model, personae make visible the different routes through which people experience a program and what they gain from it. Drawing on a real life logic model developed for housing disabled people, the session will show how incorporating personae strengthens evaluation design, enriches social return on investment calculations and produces more compelling findings for stakeholders.

Help! The post-it notes have fallen off our logic model and been mixed up!
Author: Robert Grimshaw, ATO
Can you help us piece the logic model back together? This online interactive group activity is one tool used by a large public sector agency to familiarise people with key logic modelling concepts and thinking. This session will provide a practical demonstration of the activity and how it’s being used by workshop participants to apply their learning in a simple and fun way. Participant feedback has confirmed the value of this tool for engaging with and building the capability of a diverse audience who have limited evaluation capability and experience.

Method vs methodology – what’s the difference, and why it matters 
Author: Martina Donkers, Martina Donkers
We’ve got to stop using the words ‘method’ and ‘methodology’ interchangeably. ‘Methodology’ isn’t a slightly-fancier way of saying ‘method’ – it’s actually what conceptually underpins your method choices, and it’s an important thing to reckon with if you want to do evaluation that speaks to your stakeholders, is culturally responsive, is ethical and proportionate, and is ultimately useful. This quick-fire Ignite session will be a rapid deep-dive into practical ways to think about methodology in evaluation, apply it to traditional practice and emerging approaches, and leverage that thinking to do better evaluative work.

Exploring arts-based methods for project evaluation
Authors: Katie Ronson, Creative Australia, Kirstin Clements, Arts Centre Melbourne
In this presentation evaluation practitioners from Creative Australia and Arts Centre Melbourne share how arts-based evaluation methods can unlock insights about participant experience in contexts where traditional methods struggle.

Evaluating ConnectUp: preferences for a social connectivity app among people with disability and carers
Authors: Feby Savira, Deakin University, Dom Kwasnicka, Deakin University, Lucio Naccarella, Deakin University

A national DCE (n=1,031) was conducted with people with disability, carers, and individuals identifying as both. Participants completed choice tasks comparing hypothetical versions of ConnectUp that varied across cost, communication modes, group structures, anonymity, support features, and information sources. Preferences were analysed using a conditional logit model, with latent class analysis used to identify user segments. Attitudinal questions explored additional features not captured within the DCE.

Three key findings emerge. First, participants were cost sensitive, preferring low- or no-cost options. Second, privacy and control were critical: users preferred small, private group settings and avoided features that compromised anonymity. Third, preferences were heterogeneous. Three segments were identified, including a highly price-sensitive group, a majority group prioritising anonymity and community-generated content, and a smaller segment (18%) more tolerant of cost but still favouring private group structures.

Attitudinal findings highlight additional design considerations. Flexible access features such as membership suspension (73%) and free trial options (77%) were widely valued. Around half of participants prioritised staff training in inclusivity (50%) and access in a preferred language (51%), while over half (54%) preferred human support over automated options.

This presentation demonstrates how combining DCE and attitudinal data can strengthen evaluation of digital services by capturing trade-offs and broader user expectations, and how segmentation can inform more inclusive design decisions. Participants will engage in scenario-based discussion to translate findings into practice.

Thou shall not: The 10 Commandments for Surveys 
Author: Louise Parker, Alinea International
The 10 Commandments for Surveys:
1) Thou shall not ask too many questions - survey fatigue is real and so is wasting people's time. Figure out what you absolutely must know from respondents and craft your questions around that.
2) Thou shall not ask double-barreled questions e.g. 'Did you find the workshop fun and informative?'
3) Thou shall not change the appearance order of repeating yes/no or Likert scale responses - if 'strongly agree' appears on the far right the first time, keep it that way throughout the whole survey.
...plus 7 more fabulous tips!

Authors: Andrew Boyle
Development programs often focus on whether interventions achieve intended outcomes, yet comparatively little attention is given to how the problems those interventions seek to address are defined. The way a problem is represented shapes which interventions are considered possible, influencing theories of change, program design, and the evaluation questions that follow.

Author: Jack Rutherford, ARTD
Have you ever felt that the Monitoring and Evaluation (M&E) Framework you developed or had to implement was potentially too complicated?

This presentation introduces Key Monitoring Questions (KMQs); a novel approach that builds on the well‑established logic of Results‑Based Accountability (RBA) to simplify M&E frameworks while strengthening their usefulness for decision‑making and reflection.

M&E Frameworks often become unintentionally complex. In the pursuit of rigour and comprehensiveness, evaluators and program designers can generate long lists of indicators, data sources and measurement requirements. While well‑intentioned, these expansive frameworks can overwhelm those responsible for implementation, leading to delays in data collection and reporting or abandoned learning processes.

Instead of attempting to measure every activity, output or incremental outcome, KMQs concentrate attention on purposeful questions drawn from RBA that the monitoring system must answer. This shift helps teams focus on what matters most by clarifying the purpose of monitoring and identifying the minimum information required to support meaningful learning.

The presentation will introduce four KMQs adapted from RBA and demonstrate how they streamline data collection and support evaluation and reflection:

1.How much was done? 
2.How well was it done? 
3.Who was affected? 
4.How has the context changed?

The presentation will share three key messages:

1.Complex frameworks can undermine learning by overwhelming users with excessive data expectations.
2.KMQs provide a structured, disciplined way to sharpen focus and streamline indicator selection.
3.Using KMQs supports more sustainable monitoring practice, enabling teams to engage more deeply with the data that matters.

Before the session, you will be asked to reflect on your experiences developing or implementing M&E Frameworks, recognising the shared issues that KMQs address.

This session is targeted at foundational and intermediate evaluators, and commissioners and program staff who suspect their frameworks could be simpler and more effective.

Authors: Jessi Gidgup-Lovett (Rooftop Social), Duncan Rintoul (Rooftop Social)

First Nations peoples of Australia have evaluated outcomes, impacts and responsibilities since time immemorial through Indigenous systems of lore, knowledge and accountability to Country, kin and future generations. The Cultural Wisdom and Story Gathering Artefact (CWISGA) responds to this context by reframing monitoring and evaluation as practices of accountability, care, truth telling and improvement rather than extraction and surveillance. The objective of this presentation is to introduce CWISGA and show why culturally responsive evaluation that begins with relationships and engagement is essential for better outcomes across the sector.

CWISGA provides an accessible framework that operationalises culturally responsive evaluation through clear principles aligned with the four Rs of reconciliation, respect, reciprocity and responsibility, and with the interconnected wisdoms of Knowing, Doing and Being.

Three key messages guide the work: embed cultural governance from the outset rather than as an afterthought, uphold Indigenous Data Sovereignty and governance, and interpret outcomes through holistic wellbeing and relational accountability.

The session will open with a concise framing of the developmental context in a national organisation that supports equity focused curriculum in schools, followed by a guided walk through the CWISGA principles and a brief case example. Interactivity will be promoted through a short yarning prompt and small group reflection on local application of CWISGA, followed by commitments to action to support translation into practice.

Author: Doyen Radcliffe, Bajja Yungayimanha Collaboration
Indigenous-led systems change often begins with shifts in cultural authority, relationships, trust and institutional behaviour—long before formal outcomes or policy reform become visible. Yet many evaluation approaches continue to prioritise measurable outputs and short-term results, overlooking the relational, cultural and ethical foundations that drive meaningful change. This creates a gap between what First Nations communities experience as impact and what institutions recognise as evidence.

This presentation introduces the Danjoo Koorliny Five-Dimensional Indigenous Influence Model, developed through the Aboriginal-led impact evaluation of the Danjoo Koorliny movement in Western Australia. The model offers a practical and culturally grounded way to measure systems change by identifying and assessing influence across cultural, relational, behavioural, structural and transformational domains. Central to this work is a decolonising evaluation process that repositions cultural authority, Country and relational accountability as the foundation of ethical and rigorous evaluation. For First Nations evaluation, this means privileging Elder governance, community decision-making, Indigenous Data Sovereignty and culturally grounded methods such as yarning, storywork and outcome harvesting. For evaluation more broadly, it expands how evidence, value and impact are understood in complex systems and supports earlier identification of meaningful change.

The session will highlight three key messages. First, influence is the primary mechanism and measurable pathway of Indigenous-led systems change. Second, decolonising evaluation strengthens methodological rigour by integrating Indigenous and Western approaches rather than positioning them in opposition. Third, valuing place, relationships and cultural integrity improves the relevance, integrity and usefulness of evaluation for communities, policymakers and funders.

The presentation will combine short input, reflective discussion and interactive activities. Participants will explore practical tools, apply the framework to their own contexts, and reflect on how measuring influence can strengthen systems thinking and evaluation practice across diverse sectors.

Authors: Martina Donkers, Julie Elliott

Professional isolation is an experience that, ironically, many evaluators share. As evaluators, we often find ourselves in a team of one – the only evaluator in the room, the only evaluator at the organisation, one bringing an evaluative lens to the problem. This can be hard. It can feel like people are turning to us for more answers than we have, or expecting something unrealistic. Sometimes we find ourselves feeling isolated due to our culture, our methodological approach or our disciplinary background.
There are structural ways isolation shows up – evaluators who work remotely in regional areas, evaluators who are self-employed, evaluators who balance their role with kids and family responsibilities. These experiences can leave us feeling like an outsider, with no one to test ideas, no one to help when the going gets rough, and no one to help grow our capabilities.
This panel session will explore AES members’ experiences of professional isolation, and what has worked (and not worked!) to help them combat it. We’ll present anonymised experiences from AES members about professional isolation to show attendees they aren’t alone.
We’ll then hear from 3 panellists at different stages of their professional journey, and what they’ve done to address professional isolation they’ve felt as evaluators. Finally, we’ll open the floor to questions – how can we feel more connected in our work? – and explore a range of ways that we can combat professional isolation in our field. This panel is presented by the AES Peer Group Mentoring Program Working Group, and considers how the program helps strengthen connection to peers. It also extends thinking beyond AES initiatives, and considers other ways that evaluators can combat professional isolation in a disconnected world. If you’ve ever felt lonely as an evaluator, this session is for you.

Why you should leave evaluation (and come back!)
Author: Delyth Lloyd, Australian Centre for Evaluation
After 15 years as an AES member, I stepped away from evaluation for a one-year hiatus. What began as a brief stint in an implementation role quickly transitioned into a fast and furious year using every evaluation skill I had to offer. I negotiated in hard spaces, navigated power dynamics, held space for diversity and ethical complexities. Busy people asked for more rubrics and used them in practice! This presentation distils insights from my transition out of, and back into, the sector. It reflects on the evaluation competencies we hold, and highlights how deeply they are valued by others.

The Evaluation Pub Test: Simple Questions for Public Value 
Author: Amanda Taylor-Short, ARTD
Publicly funded evaluations carry an implicit responsibility to contribute to public good. In practice, evaluations are often viewed as mechanisms for accountability in government decision‑making. This can reinforce a focus on compliance, program‑specific detail, and predefined questions, limiting broader reflection about the contribution of a program and leaving simple but important questions unexplored. This creates a missed opportunity to connect evaluation work with the people and decisions it exists to inform. This presentation explores the Evaluation Pub Test: examining evaluator responsibility to support public value, the benefits of answering simple questions, and considerations for deciding which questions are worth answering.

From scientist to evaluator: Lessons beyond numbers 
Author: Heather Bryan, First Person Consulting
This Ignite traces one evaluator’s unlikely journey from a postgraduate ecologist in Latin America to an honours student modelling climate impacts, and later a data analyst at the Australian Bureau of Statistics - before arriving at the unexpected doorstep of evaluation. Along the way came a new appreciation for the value of qualitative evidence. This talk shares three lessons motivated by moving beyond numbers: outliers are insights rather than errors, stories explain what numbers can’t, and subjectivity, when acknowledged rather than ignored, can strengthen evaluation. Together, these lessons invite a broader view of evidence, where individual stories bring meaning.

Bridging Sectors Through Evaluation: Lessons from a University–Community, Community of Practice
Author: Hannah Morgan, UTS
In 2025, the University of Technology Sydney partnered with Inner Sydney Voice (ISV) to co-design an Evaluation Community of Practice, connecting university evaluators and community practitioners. We reflect on lessons learned from the collaboration, including the value of shared ownership and relational trust in cross-sector work. Community practitioners strengthened their confidence and capability, while UTS deepened its understanding of how to adapt evaluation capacity building to meet the needs of the community sector context. The experience illustrated the mutual benefits of collaboration and the potential of Communities of Practice to advance evaluation through reciprocal learning.

Seeing through their eyes: Photo-Elicitation in Evaluation
Author: Estelle Gaillard, CSIRO
Images can ignite conversation, evoke emotions, elicit reflection and spark new ways of thinking. Photo-elicitation uses photographs in interviews to prompt participants to share insights into their lived experiences that might not surface through standard questions. This Ignite presentation will introduce the method, show practical examples of its use in evaluation and outline the photo-elicitation process. Attendees will leave reflecting on whether — and how — photo-elicitation could fit into their own evaluation toolkit.

Making Space for Boba and Beats: Valuing Grassroots Reality with an Evaluation MVP
Author: Anabelle (pin-chu) Chen, Taiwan Pride
For a volunteer-run grassroots NFP, traditional evaluation often occupies the very space intended for community action. When resources are scarce, every hour spent on data collection undermines the operational realities of the place where we operate. As the President and Evaluator of Taiwan Pride Sydney, I faced this tension at the 2026 Sydney Mardi Gras. How do you measure the cultural impact of a 'Lanterns of Ecstatica, Taiwan Lighting up Asia' float when the generator dies, the music cuts out, and you are forced to march giant illuminated bubble tea cups in total silence?
This Ignite presentation proposes a Minimum Viable Product (MVP) for evaluation. By shifting the focus to macro-validation and systemic consequences, I will demonstrate how grassroots organisations can engineer a structurally sound evidentiary argument that survives real-world disaster. Using aggressive dimensionality reduction and cognitive anchoring, we can design evaluation MVPs that extract maximum validated learning with minimum administrative friction. Let's spill the tea on how to make space for rigorous evaluation when absolutely nothing goes to plan.

Authors: Alexander Gyles (La Trobe University), Allan Mua Illingworth (Muaakia Consulting), Glenda Napaljarri Wayne (La Trobe University), Mildred Napaljarri Spencer (La Trobe University), Raelene Jigili (Central Land Council), Marilyn Vilsoni (Solve Pacific)

“How might we most appropriately track and describe change about impact and wellbeing in Indigenous Australian and Fijian communities?”

Led by Indigenous community researchers and MEL practitioners, the session draws on two grounded case examples: the YWPP (Warlpiri Education and Training Trust, Tanami NT) approach to tracking learning and wellbeing, and community-centred MEL work supported by the Itaukei Trust Affairs Board (TTFB).

The session will be highly interactive: participants will break into small, facilitated groups with community and TTFB representatives to interrogate practice, and the table will close with a plenary synthesis that surfaces cross-case lessons, tensions, and practical next steps for culturally respectful MEL.

Authors: Samantha Abbato
Learning objective: Participants will be able to identify and practise managing the eight major enemies of evaluative thinking in evaluation contexts.

Why this skill matters: Even experienced evaluators fall prey to thinking traps. Individual biases (including emotional reasoning, fast thinking, confirmation bias and overconfidence) undermine rigour. Group biases such as in-group favouritism and cascading effects distort collective judgement. Noise, both between evaluators and within a single evaluator at different times, creates inconsistency in decisions and recommendations. Together, these eight enemies threaten the quality of evaluations at every stage: from scoping and data synthesis to communicating findings.

This session equips participants with practical strategies to recognise and manage these threats for clearer, more defensible evaluative thinking.

How the skill will be taught: Using the Thinking-Bee Obstacles board game, participants work in small groups through evaluation-based scenarios that activate each of the eight thinking enemies. The game provides ego-safe, attention-directing play. Participants think through bee personas rather than as themselves, making it easier to surface and examine real thinking traps. A brief facilitator-led debrief anchors the game experience to participants’ own evaluation practice.

How participants will engage: Participants will play the Thinking-Bee Obstacles game in small groups of four to six, applying the eight enemies of thinking to realistic evaluation decisions.

The session closes with a structured reflection linking game insights to participants’ own work contexts.