aes26 International Evaluation Conference

Welcome to Country
Opening address: President, Australian Evaluation Society

Being comfortable with discomfort
Robyn Ober, Batchelor Institute of Indigenous Tertiary Education, Lead Researcher and Educator

This keynote challenges evaluators and commissioners to get comfortable with discomfort, and to rethink what ethical, rigorous evaluation looks like when it happens on Country.

Responding to the aes26 theme Making Space, Valuing Place, Dr Robyn Ober draws on three decades of practice in Aboriginal communities to show how traditional knowledge systems and contemporary evaluation can work together.

Through vivid stories from remote and very remote contexts, the talk brings to life the tensions at the heart of evaluation practice: timelines versus relationships, control versus trust, and methodological neatness versus lived reality. At the centre is the Community Researcher Approach, where local people are co-researchers who shape the questions in collaboration with evaluators, lead conversations in language, and make meaning on their own terms.

The talk argues that ethics and integrity are enacted in how we show up, who holds authority, and whether participants’ voices are recognised, respected and valued so they feel safe to share their own truths. Integrity is also enacted when Aboriginal people can see that the purpose of evaluation is for the benefit of Aboriginal people. You’ll leave with practical ways to commission and conduct evaluation on Country to strengthen voice, evidence quality and impact.

Authors: Caitlin Morton, Maggie Hawkins (Attorney General's Department)
This session poses the question – how should we evaluate legal policy? Evaluators across all sectors encounter legal frameworks, yet few forums explicitly address how law itself can and should be evaluated. This session seeks to help carve out that space within evaluation practice.

Strong approaches to evaluating legal policy are critical to achieving just, fair, and secure society - the remit of the Attorney-General's Department. In this session, past and current evaluators from the AGD reflect on observations, challenges, and conversations, and invite discussion on what it looks like when legal policy is working well. We present an overview of existing dialogue on this question, and argue that it is critical to examine the social and ethical foundations of legal policy, the principles that inform legal policy, and how the law can support the operation of impactful legal policy.

We will investigate how law does and does not align with the beliefs and assumptions of the communities it touches, and how law is interpreted and put into practice. Together we will explore localisation as a major challenge and opportunity in evaluating legal policy – noting the persistent regional/metro divide in accessing legal services, and diversity between communities and across states, which always requires collaboration.

This session is critically important to evaluators as law touches all public policy, which in turn impacts the operations of not just federal governments, but local and state governments, private businesses, community organisations, not-for-profits, and more.

Authors: Jen Lorains (Children's Ground), Veronica Doolan (Children's Ground), Pauline Grant (Children's Ground), Jackie Treeves (Children's Ground)

For too long our people have been the subjects not the leaders of evaluation and research: “Our people have been researched to death. It’s time we researched ourselves back to life” (William Tilmouth, Senior Arrernte man).
Children’s Ground (CG) is disrupting the status quo in research and evaluation. From daily data collection and designing evaluation tools, to analysing evaluation data through community and culturally centred evaluation frameworks relevant to each place.

Using practice evidence and collaborative reflection about how First Nations communities are leading service/program evaluation for their families and place, the workshop learning objective is that participants will increase their understanding of culturally grounded evaluation and gain practical strategies and skills that can be applied to their evaluation context.

The workshop will consist of two parts, including CG sharing practice evidence, followed by collaborative group/table strategy development.

Firstly, CG’s evaluation principles will be outlined, with First Nations leaders sharing experiences in action. Participants will reflect on 2-3 principles, documenting their effective and challenging experiences of working in line with the principes, then sharing with the larger group. CG’s First Nations leaders will respond, building on the knowledge being generated by the participants.

Secondly, CG’s First Nations leaders will share experiences of developing community and culturally centred evaluation frameworks, including a visual walk through of 2-3 frameworks developed by First Nations communities across three culturally and geographically diverse regions. Comparative examples of evaluation data analysis between CG’s cultural and western evaluation frameworks will also be shared, including methodological implications.

Participants will collaboratively document ideas for supporting First Nations people/communities to develop community and culturally centred evaluation frameworks in their context, then sharing with the larger group.

We believe learning how to embed culturally grounded evaluation from First Nations community’s real-world experience is an important contribution to holistic learning, complements theoretical learning.

Author: Salli Cohen (The Policy Room)
This presentation explores the uncomfortable space where policy and evaluation structurally reinforce harm. Its objective is to challenge assumptions of neutrality and examine how evaluative frameworks can either illuminate or obscure inequity.
The core argument is threefold: (1) evaluation and policy are never neutral and must interrogate power; (2) systems alignment and cultural authority determine whether outcomes are real or performative; and (3) accountability must ask “for whom” evaluation and policy work; and who bears the cost.
The session blends applied case insight with structured reflection and peer dialogue to provoke critical engagement and practical recalibration.

From evidence to impact: reclaiming evaluation as a means to an end
Lígia Teixeira, Founder and Chief Executive of the Centre for Homelessness Impact

Evaluation has never been more sophisticated, yet many challenges it seeks to address remain stubbornly persistent. Across sectors, we generate high-quality evidence, but too often struggle to translate it into meaningful, sustained impact. The risk is not just a lack of rigour, but a loss of connection to purpose: the way evaluation is used within systems can allow it to become an end in itself, rather than a means to improve lives.This keynote argues for renewed focus on evaluation as a discipline of impact. Drawing on international efforts to tackle homelessness, it explores how outcomes are shaped not by individual interventions alone, but by the systems in which they operate. Without a systems lens, even strong and diverse forms of evidence – quantitative, qualitative and lived - can lead to fragmented action, missing the broader dynamics that ultimately determine success.

The keynote reflects on how evaluation can better connect global insight with local context, enabling faster learning while respecting place-based realities. It also argues that effective systems prevent harm before it occurs, rather than responding once it is entrenched.

This is a call to re-centre evaluation on its core purpose: not just to understand the world, but to change it.

Authors: Nic Vogelpoel (Day Four Projects), Malika Reese (Lived Experience Advisor), Sandra Collins (Lived Experience Advisor)
What happens when evaluation is not just informed by lived experience, but led by it?

This presentation offers a rare, practice-based account of a lived experience-led evaluation undertaken with the National Centre for Action on Child Sexual Abuse, where victim survivors worked as co-evaluators across all stages of the process. Established to centre victim survivors in awareness-raising, help-seeking, advocacy and best-practice responses, the Centre provided a powerful context to rethink how evaluation knowledge is produced, and by whom.

The session argues that lived experience-led evaluation is not an ethical “add-on”, but a fundamental shift in values, power and epistemology. Drawing on the evaluation findings, presenters will explore three core propositions.

First, nothing about us without us: moving from evaluation on people to evaluation led by people with lived experience reshapes evaluation questions, evidence, outcomes and definitions of rigour. Second, epistemic justice: lived experience leadership challenges entrenched assumptions about who gets to ask questions, whose knowledge counts, and how institutions respond to new forms of evidence. Third, beyond advice to synthesis: lived experience cannot simply “advise” evaluation, it must be integrated as a distinct way of knowing that transforms the whole evaluation.

Co-presented by lived experience evaluators and Day Four Projects evaluators, the session will combine reflective storytelling, concrete practice examples and facilitated dialogue. Participants will be invited to critically examine their own evaluative assumptions, engage in small-group reflection, and explore practical strategies for making space for multiple knowledge systems while maintaining evaluative integrity and meeting institutional requirements.This session will be particularly valuable for evaluators, commissioners, researchers and practitioners seeking more just, credible and impactful approaches to lived-experience evaluation in complex and sensitive contexts.

Authors: Andrew Boyle
Development programs often focus on whether interventions achieve intended outcomes, yet comparatively little attention is given to how the problems those interventions seek to address are defined. The way a problem is represented shapes which interventions are considered possible, influencing theories of change, program design, and the evaluation questions that follow.

Authors: Ian Falk (Mission Australia), Paul Royce (Mission Australia)

Evaluation can often be treated as a technical process, shaped by existing evaluative ‘types’, such as formative, summative, realist evaluation among others, and often focused on program performance and measurable outcomes. In complex, place-based contexts such as youth homelessness in Australia’s Northern Territory (NT), however, evaluation needs to be much more than this – it becomes a contested and adaptive practice shaped by existing evidence, culture, geography and structural constraint.
This paper presents emerging results from a layered multisite, multimethod evaluative scoping study examining the accommodation needs of young people aged 8–24 who are at risk of, or experiencing, homelessness, where First Nations young people comprise around 90% of those impacted. The place-based, First Nations-led consultation process functioned as a responsive, continually adjusted evaluative design. Rather than applying a single evaluation lens, the project adopted a layered evaluation model integrating the evidence base, relational consultation, iterative and adaptive methodological design, program case study analysis, and system-level mapping. Methods included multi-media narrative engagement in remote communities, semi-structured interviews with urban stakeholders, and for both urban and regional/remote sites, iterative reflection on emerging methodologies and insights.
This approach revealed that accommodation pathways are shaped less by individual behaviour than by structural misalignment between Western housing systems and young people’s lived realities, including kinship, mobility and connection to Country. In addition, the project established evidence-based criteria for ‘successful’ programs and services, which informed the program and service mapping process. In terms of the evaluation methodology, the paper found that layered and iterative place-based evaluation provides a more practical, useful and credible framework for understanding youth homelessness in complex, diverse and remote settings, enabling evaluation to move beyond judgement toward explanation, learning and culturally grounded, co-designed place-based system design.

Author: Doyen Radcliffe, Bajja Yungayimanha Collaboration
Indigenous-led systems change often begins with shifts in cultural authority, relationships, trust and institutional behaviour—long before formal outcomes or policy reform become visible. Yet many evaluation approaches continue to prioritise measurable outputs and short-term results, overlooking the relational, cultural and ethical foundations that drive meaningful change. This creates a gap between what First Nations communities experience as impact and what institutions recognise as evidence.

This presentation introduces the Danjoo Koorliny Five-Dimensional Indigenous Influence Model, developed through the Aboriginal-led impact evaluation of the Danjoo Koorliny movement in Western Australia. The model offers a practical and culturally grounded way to measure systems change by identifying and assessing influence across cultural, relational, behavioural, structural and transformational domains. Central to this work is a decolonising evaluation process that repositions cultural authority, Country and relational accountability as the foundation of ethical and rigorous evaluation. For First Nations evaluation, this means privileging Elder governance, community decision-making, Indigenous Data Sovereignty and culturally grounded methods such as yarning, storywork and outcome harvesting. For evaluation more broadly, it expands how evidence, value and impact are understood in complex systems and supports earlier identification of meaningful change.

The session will highlight three key messages. First, influence is the primary mechanism and measurable pathway of Indigenous-led systems change. Second, decolonising evaluation strengthens methodological rigour by integrating Indigenous and Western approaches rather than positioning them in opposition. Third, valuing place, relationships and cultural integrity improves the relevance, integrity and usefulness of evaluation for communities, policymakers and funders.

The presentation will combine short input, reflective discussion and interactive activities. Participants will explore practical tools, apply the framework to their own contexts, and reflect on how measuring influence can strengthen systems thinking and evaluation practice across diverse sectors.

Authors: David Stuart (Creative Australia)
Strategies are everywhere involving all sorts of topics and all kinds of goals, actions and stakeholders.  Strategy defines important goals and outlines how an organisation's activities will meet those goals. You would think that strategy and evaluation were great friends. But why is strategy evaluation so low profile in evaluation literature, and so hard to pull off? This paper reflects on several attempts at strategy-based evaluation and the lessons for fruitful strategy evaluation including the role for evaluators and evaluation in supporting strong strategy design and success.​​​​

Why you should leave evaluation (and come back!)
Author: Delyth Lloyd, Australian Centre for Evaluation
After 15 years as an AES member, I stepped away from evaluation for a one-year hiatus. What began as a brief stint in an implementation role quickly transitioned into a fast and furious year using every evaluation skill I had to offer. I negotiated in hard spaces, navigated power dynamics, held space for diversity and ethical complexities. Busy people asked for more rubrics and used them in practice! This presentation distils insights from my transition out of, and back into, the sector. It reflects on the evaluation competencies we hold, and highlights how deeply they are valued by others.

The Evaluation Pub Test: Simple Questions for Public Value 
Author: Amanda Taylor-Short, ARTD
Publicly funded evaluations carry an implicit responsibility to contribute to public good. In practice, evaluations are often viewed as mechanisms for accountability in government decision‑making. This can reinforce a focus on compliance, program‑specific detail, and predefined questions, limiting broader reflection about the contribution of a program and leaving simple but important questions unexplored. This creates a missed opportunity to connect evaluation work with the people and decisions it exists to inform. This presentation explores the Evaluation Pub Test: examining evaluator responsibility to support public value, the benefits of answering simple questions, and considerations for deciding which questions are worth answering.

From scientist to evaluator: Lessons beyond numbers 
Author: Heather Bryan, First Person Consulting
This Ignite traces one evaluator’s unlikely journey from a postgraduate ecologist in Latin America to an honours student modelling climate impacts, and later a data analyst at the Australian Bureau of Statistics - before arriving at the unexpected doorstep of evaluation. Along the way came a new appreciation for the value of qualitative evidence. This talk shares three lessons motivated by moving beyond numbers: outliers are insights rather than errors, stories explain what numbers can’t, and subjectivity, when acknowledged rather than ignored, can strengthen evaluation. Together, these lessons invite a broader view of evidence, where individual stories bring meaning.

Bridging Sectors Through Evaluation: Lessons from a University–Community, Community of Practice
Author: Hannah Morgan, UTS
In 2025, the University of Technology Sydney partnered with Inner Sydney Voice (ISV) to co-design an Evaluation Community of Practice, connecting university evaluators and community practitioners. We reflect on lessons learned from the collaboration, including the value of shared ownership and relational trust in cross-sector work. Community practitioners strengthened their confidence and capability, while UTS deepened its understanding of how to adapt evaluation capacity building to meet the needs of the community sector context. The experience illustrated the mutual benefits of collaboration and the potential of Communities of Practice to advance evaluation through reciprocal learning.

Seeing through their eyes: Photo-Elicitation in Evaluation
Author: Estelle Gaillard, CSIRO
Images can ignite conversation, evoke emotions, elicit reflection and spark new ways of thinking. Photo-elicitation uses photographs in interviews to prompt participants to share insights into their lived experiences that might not surface through standard questions. This Ignite presentation will introduce the method, show practical examples of its use in evaluation and outline the photo-elicitation process. Attendees will leave reflecting on whether — and how — photo-elicitation could fit into their own evaluation toolkit.

Making Space for Boba and Beats: Valuing Grassroots Reality with an Evaluation MVP
Author: Anabelle (pin-chu) Chen, Taiwan Pride
For a volunteer-run grassroots NFP, traditional evaluation often occupies the very space intended for community action. When resources are scarce, every hour spent on data collection undermines the operational realities of the place where we operate. As the President and Evaluator of Taiwan Pride Sydney, I faced this tension at the 2026 Sydney Mardi Gras. How do you measure the cultural impact of a 'Lanterns of Ecstatica, Taiwan Lighting up Asia' float when the generator dies, the music cuts out, and you are forced to march giant illuminated bubble tea cups in total silence?
This Ignite presentation proposes a Minimum Viable Product (MVP) for evaluation. By shifting the focus to macro-validation and systemic consequences, I will demonstrate how grassroots organisations can engineer a structurally sound evidentiary argument that survives real-world disaster. Using aggressive dimensionality reduction and cognitive anchoring, we can design evaluation MVPs that extract maximum validated learning with minimum administrative friction. Let's spill the tea on how to make space for rigorous evaluation when absolutely nothing goes to plan.