Authors: Jen Lorains (Children's Ground), Veronica Doolan (Children's Ground), Pauline Grant (Children's Ground), Jackie Treeves (Children's Ground)
For too long our people have been the subjects not the leaders of evaluation and research: “Our people have been researched to death. It’s time we researched ourselves back to life” (William Tilmouth, Senior Arrernte man). Children’s Ground (CG) is disrupting the status quo in research and evaluation. From daily data collection and designing evaluation tools, to analysing evaluation data through community and culturally centred evaluation frameworks relevant to each place.
Using practice evidence and collaborative reflection about how First Nations communities are leading service/program evaluation for their families and place, the workshop learning objective is that participants will increase their understanding of culturally grounded evaluation and gain practical strategies and skills that can be applied to their evaluation context.
The workshop will consist of two parts, including CG sharing practice evidence, followed by collaborative group/table strategy development.
Firstly, CG’s evaluation principles will be outlined, with First Nations leaders sharing experiences in action. Participants will reflect on 2-3 principles, documenting their effective and challenging experiences of working in line with the principes, then sharing with the larger group. CG’s First Nations leaders will respond, building on the knowledge being generated by the participants.
Secondly, CG’s First Nations leaders will share experiences of developing community and culturally centred evaluation frameworks, including a visual walk through of 2-3 frameworks developed by First Nations communities across three culturally and geographically diverse regions. Comparative examples of evaluation data analysis between CG’s cultural and western evaluation frameworks will also be shared, including methodological implications.
Participants will collaboratively document ideas for supporting First Nations people/communities to develop community and culturally centred evaluation frameworks in their context, then sharing with the larger group.
We believe learning how to embed culturally grounded evaluation from First Nations community’s real-world experience is an important contribution to holistic learning, complements theoretical learning.
Jen Lorains is the Director of Research & Evaluation at Children’s Ground. She works with each community to evaluate and evidence the impact of Children’s Ground’s empowerment, systems reform and integrated service platform.
Jen has undergraduate and postgraduate qualifications in applied social research and over 15 years experience designing and undertaking research and evaluation with communities and services. Her interest lies in working with communities to implement and evaluate approaches (within... Read More →
Authors: Ethel Karskens (Clear Horizon), Maree Dibella (Clear Horizon) What participants will learn: This hands-on session teaches evaluators how to configure and use AI assistants for real evaluation tasks. Participants will leave with a working AI tool customised for their own work, whether that's coding interview transcripts, analysing open-ended survey responses, or synthesising progress reports.
Why this skill matters: Most evaluators have experimented with ChatGPT or similar tools, but few have moved beyond ad-hoc prompting to building reusable, reliable AI workflows. The gap between "asking ChatGPT a question" and "using AI as a systematic evaluation tool" is significant. This session bridges that gap by teaching the practical setup skills that turn general-purpose AI into specialized evaluation assistants that produce consistent, auditable outputs.
As AI becomes standard in evaluation practice, knowing how to configure these tools properly - with the right instructions, quality controls, and workflow integration - is becoming a core professional capability. This session responds directly to evaluators' need for practical AI implementation skills, not just conceptual understanding.
How we'll teach the skill: Minutes 0-5: Quick orientation - what makes an AI "assistant" different from a chatbot Minutes 5-15: Live demonstration - the facilitator builds an assistant for interview coding from scratch, narrating decisions Minutes 15-40: Guided hands-on practice -participants configure their own assistant for a task they choose (interview coding, survey analysis, or report summarisation), using either sample data provided or their own files Minutes 40-50: Group debrief -participants share one success and one challenge; facilitator troubleshoots common issues
How participants will engage: Participants will work on their own laptops throughout the session, following a structured build process with real-time facilitator support. They'll leave with a configured tool, a workflow template, and practical troubleshooting strategies they can apply immediately in their work.
I lead the data and insights strategy of Clear Horizon. This includes dashboard development and other data solutions to create insights for our clients. I am interested in innovation, data for good, and creating a data-driven culture in organisations.
Effective evaluation increasingly depends on strong partnerships across communities, commissioners, service providers and evaluators. Yet partnering is often guided by goodwill rather than shared principles, leaving teams vulnerable to power imbalances, ethical drift and unspoken assumptions. This skill building session focuses on principles-based partnering, defined as the deliberate use of a small, shared set of agreed principles to guide roles, behaviours and decision making within evaluation partnerships.
The objective of the session is to build participants’ capability to use partnering principles intentionally and appropriately in real world evaluation contexts, particularly where values, authority and accountabilities differ. Drawing on applied evaluation practice, the session introduces principles based partnering not as a universal solution, but as a supporting mechanism that must be applied judiciously and adapted to context and place.
Participants will develop three core skills: 1.Identifying when partnering principles are likely to be helpful and when they are unlikely to add value or may even create risk. 2.Understanding and applying practical processes for establishing partnering principles, including who should be involved, how principles can be co-created, and how they can be revisited over time. 3. Using principles to navigate tension, power dynamics and ethical dilemmas as they arise during the evaluation lifecycle.
The session is designed as an interactive workshop. Participants will work in small groups to explore short evaluation scenarios, test whether principles-based partnering is appropriate, and practice establishing and applying principles in context. This will be followed by whole group discussion to surface lessons and challenges. Participants will leave with a clear, adaptable approach for deciding when and how to establish and use partnering principles. The session is suited to foundational and intermediate evaluators seeking hands on skills grounded in real world practice.
Associate Director, Grosvenor Public Sector Advisory
Dana is a public sector expert, possessing over 17 years of deep experience advising government organisations on program evaluation, organisational review, service optimisation and performance management. She is a member of Grosvenor’s Executive Leadership Team as Head of Strategy... Read More →
Wednesday September 16, 2026 2:30pm - 3:30pm ACST Rooms 1+2Stokes Hill Rd, Darwin City NT, Australia
Authors: Samantha Abbato Learning objective: Participants will be able to identify and practise managing the eight major enemies of evaluative thinking in evaluation contexts.
Why this skill matters: Even experienced evaluators fall prey to thinking traps. Individual biases (including emotional reasoning, fast thinking, confirmation bias and overconfidence) undermine rigour. Group biases such as in-group favouritism and cascading effects distort collective judgement. Noise, both between evaluators and within a single evaluator at different times, creates inconsistency in decisions and recommendations. Together, these eight enemies threaten the quality of evaluations at every stage: from scoping and data synthesis to communicating findings.
This session equips participants with practical strategies to recognise and manage these threats for clearer, more defensible evaluative thinking.
How the skill will be taught: Using the Thinking-Bee Obstacles board game, participants work in small groups through evaluation-based scenarios that activate each of the eight thinking enemies. The game provides ego-safe, attention-directing play. Participants think through bee personas rather than as themselves, making it easier to surface and examine real thinking traps. A brief facilitator-led debrief anchors the game experience to participants’ own evaluation practice.
How participants will engage: Participants will play the Thinking-Bee Obstacles game in small groups of four to six, applying the eight enemies of thinking to realistic evaluation decisions.
The session closes with a structured reflection linking game insights to participants’ own work contexts.
My twenty-plus years of evaluation experience are built on academic training in qualitative and quantitative disciplines, including mathematics, health science, epidemiology, biostatistics, and medical anthropology. I am passionate about effective communication and evaluation capacity-building... Read More →
Wednesday September 16, 2026 4:30pm - 5:30pm ACST Rooms 3+4Stokes Hill Rd, Darwin City NT, Australia
Scaling is often treated as a straightforward ambition: test an intervention in one place, then replicate it in others. In complex systems, and particularly in place-based settings where challenges are systemically entrenched in local history, relationships and power, this logic breaks down. What works is frequently inseparable from where and with whom it works. Scaling across places is not replication. It is a fundamentally different process, and evaluators need tools adequate to that complexity. This workshop equips evaluators, program designers and commissioners to challenge lock-step and linear models of scaling and apply a complexity-informed, place-sensitive approach in their own practice. Despite the proliferation of place-based initiatives across Australia and the Asia Pacific, most evaluators lack a coherent framework for assessing scalability or monitoring fidelity in contextually sensitive ways. Three connected arguments run through the session. First, scalability is a question before it is a plan: interventions need to be tested for readiness and direction before any scaling begins. Second, scaling in complex systems requires holding the tension between fidelity and adaptation, not resolving it prematurely. Third, scaling across places demands attention to what is systemically entrenched in each context: the relationships, trust and power dynamics that cannot be lifted and shifted. These arguments are anchored in three practical tools: a reworked scalability assessment; the Scaling in Place Framework, a new conceptual tool mapping fidelity, adaptation, mechanism and context; and the Scaling Canvas and Fidelity Checker for planning and monitoring. The session is structured around a short conceptual input, a worked case example, and a facilitated small group activity in which participants apply the tools to a program or context from their own practice. Structured reflection closes the session. It is targeted at intermediate-level evaluators, program designers and commissioners working in place-based, health, community or government settings.
Principal Consultant | Co-Founder, First Person Consulting
I'm Matt, Co-Founder of First Person Consulting. I work at the intersection of systems thinking, evaluation and design — helping people make sense of messy, complex problems with a healthy dose of humour and humility. I also co-host the It Depends Podcast, where the honest answer... Read More →
Thursday September 17, 2026 10:30am - 11:30am ACST Rooms 3+4Stokes Hill Rd, Darwin City NT, Australia
Author: Elisabeta Torava Evaluation practice in Indigenous communities across the Pacific and Australia often rely on Western tools that overlook relational obligations, kinship structures, and place based ethics. This session introduces a practical, culturally grounded evaluation approach based on vanua ontology, a relational worldview that positions land, people, and relationships as inseparable. Drawing from my doctoral research with iTaukei communities in Fiji, the session demonstrates how evaluators can design and implement evaluations that honour Indigenous values, strengthen relational accountability, and generate findings that communities recognise as meaningful. The objective is to demonstrate how evaluators can redesign Western tools and methods to honour Indigenous relational ethics, strengthen cultural integrity, and generate findings that communities recognise and relate with. This work is important because many evaluation tools used across the Pacific and Australia continue to erase relational systems, producing invisibility, misinterpretation, and unintended harm. The core argument is that evaluation practice must shift from individualistic, decontextualised measures to relational, place‑based approaches grounded in Indigenous worldviews. Three key messages will be shared: - Western evaluation tools often embed assumptions that conflict with Indigenous relational logics. - Vanua‑aligned principles offer a culturally coherent foundation for ethical, rigorous evaluation. - Practical redesign is possible when evaluators centre relationships, place, and collective wellbeing. Designed as a skill‑building session, the presentation uses hands‑on activities rather than lecture. Participants will analyse a standard Western evaluation tool, identify where invisibility occurs, and collaboratively redesign selected questions using vanua‑based principles. A case vignette and mapping template will guide this process. Interactivity is promoted through small‑group work, collective mapping, movement‑based clustering, and facilitated dialogue. Participants will leave with a practical mini‑framework and concrete tools they can apply immediately in their own evaluation practice.
Learning objective: Participants will be able to construct core and alternative hypotheses from a theory of change, design evidence tests to distinguish between them and make defensible causal claims.
Contribution analysis is now widely used in Australian evaluation practice, but many evaluators stop short of the next step: systematically testing whether something other than the program explains the observed outcomes. Process tracing does exactly this. It raises the rigour of causal claims not by collecting more data, but by being more deliberate about what the data needs to show — and what it would need to show if the alternative explanation were true instead.
This workshop uses a real evaluation of the Embrace Multicultural Mental Health Project as a case study. The Embrace evaluation combined contribution analysis and process tracing with a mixed-method design including a quasi-experimental quantitative survey and qualitative interviews. It produced a result that many evaluations encounter but few handle well: the quantitative analysis found the program had no detectable effect on key outcomes, yet qualitative evidence suggested it was working. Process tracing provided the analytical framework to distinguish between three plausible explanations — the program failed, the effects take time to materialise, or competing programs are producing the same outcomes.
Participants work through three structured exercises: translating a theory of change into core and alternative hypotheses; designing evidence tests that can distinguish between them; and applying evaluator judgement to weight mixed evidence and reach a defensible conclusion. Each exercise uses the Embrace case, with reflection prompts connecting the method to participants' own evaluations.
The workshop is aimed at intermediate to advanced evaluators working on complex programs.
This Skill Building session introduces three practical, culturally grounded evaluation tools developed through Churches of Christ Vanuatu’s (CCCV) mental wellbeing work: the Faith + Data Model, the Youth Risk‑Mapping Tool and Trauma‑Informed Storian Circles. These tools emerged from a multi‑year program involving a 1,110‑household Urban Study, youth behavioural data, earthquake trauma responses and community‑driven interventions. The session addresses a core challenge in evaluation: how to design methods that are rigorous, culturally resonant and effective in low‑resource, cross‑cultural settings. The objective of the presentation is to equip evaluators with adaptable tools that integrate community evidence, kastom practices and faith‑based strengths to strengthen mental wellbeing systems. This topic is important because evaluators increasingly work in culturally diverse contexts where Western evaluation methods alone are insufficient for capturing lived experience, trauma, and relational dynamics. The presentation advances three key messages: 1.Evaluation must integrate cultural and spiritual knowledge with data to produce meaningful insights. 2.Youth wellbeing requires rapid, context‑specific assessment tools that identify patterns of risk and guide targeted interventions. 3.Trauma‑aware, culturally grounded qualitative methods can generate rich data while supporting community healing and resilience. Each tool is introduced through a short demonstration using real CCCV examples, followed by a structured group activity where participants apply the tool to a scenario. This design ensures that participants not only understand the concepts but also practice using them in a supportive environment. The session will be interactive through small‑group exercises, reflective discussions and scenario‑based problem‑solving. Participants will map youth risks, design a Storian Circle prompt and apply the Faith + Data Model to a community case study. These activities encourage peer learning, cultural reflection and practical skill development. By the end of the session, participants will leave with three adaptable tools they can apply immediately in their own evaluation contexts.
Authors: Patricia Rogers, Footprint Evaluation Initiative, Emil Laurberg Morgensen, University of Southern Denmark Theories of change are now commonly used to design initiatives, including projects, programs and policies, and to shape monitoring and evaluation. But 40 years after first being showcased at an AES conference, these are often developed and used in ways that don’t fit what is needed, especially to support adaptation across different settings and in changing contexts. This session will present common mistakes, what they are, why they matter, and examples of better strategies. Participants will engage with exercises to identify issues, try out strategies and explore possible applications in their own work and in shaping organisational requirements and procedures.
Founder of BetterEvaluation and former Professor of Public Sector Evaluation at RMIT University. Now working as consultant and advisor. My work has focused on supporting appropriate choice and use of evaluation methods and approaches to suit purposes and context. I am currently working... Read More →
Authors: Arun Jyothi Callapilli (Policy Performance) Charlie Tulloch (Policy Performance)
The session focuses on building skills in designing infographics and making effective data visualisation choices to enhance evaluation reporting. The session objective is to equip new and experienced evaluators with practical skills to transform traditional evaluation reports into clear, engaging and actionable visual based outputs.
This is important because evaluation reports are often text heavy and dense with evidence and associate findings. However, without effective visualisation, critical insights may be overlooked or misunderstood. Infographics can help bridge this gap by transforming evaluation evidence and findings into clear, engaging visual reports that enhance reader understanding and engagement and are more likely to be used.
The session is grounded in utilisation-focused evaluation principles (developed by Michael Quinn Patton), emphasising the importance of designing reports that facilitate use and effectively communicate significant findings to intended audiences.
The session’s main aims are to share and build knowledge and skills in: • visual data story telling techniques • applying effective design principles • understanding the strengths and limitations of different visual choices • accessibility considerations • avoiding common data visualisation pitfalls • tips and techniques on creating infographics • promoting consistency and accessibility in data presentations
This is a skill-building session using real-life case studies. Attendees will explore a range of tools, tips and visualisation choices, making live design decisions. They will work in small groups round tables to discuss options and apply principles of good design to create their own visuals, creating a collaborative and participatory learning environment.
Policy Performance is a proud conference sponsor! Charlie delivers evaluation projects, capability building support and drives public sector improvement. Charlie loves to help those who are new to evaluation or transitioning from related disciplines. He is a past AES Board member... Read More →
Friday September 18, 2026 10:30am - 11:30am ACST Rooms 3+4Stokes Hill Rd, Darwin City NT, Australia
Author: Liesl Harrold, Australian Taxation Office Problematisation is a deliberate process of dismantling a problem to understand the different ways of thinking that lead to the classification of phenomena as a problem. It goes beyond the construction of problem statements to focus on the effort required to understand historical and theoretical assumptions underpinning its framing. Problematisation is a way to test assumptions, generate new ideas, and make new connections to theoretical understandings. In evaluation, it has the potential to provide rigour to practices associated with judging through structuring evaluative thinking.
Using a skill building format, this paper will help participants understand the role of problems in evaluation. The format will follow an explain-model-apply in a group teaching format including practical application of selected trans-disciplinary theories and approaches. It will include a brief overview of:
•Problem logics and how they can be constructed •Problem representation and the genealogy of problems •Key theories that can support evaluators to think differently e.g. social identity and psychological safety theoretical frameworks.
Problematisation provides a systematic approach that offers evaluators support to think differently, rather than using existing knowledge to validate existing thoughts. Evaluators’ worldviews and skills influence their competence which may manifest in generalisations of the problem.
Problem-solving is a role in evaluation, as it supports the purpose of interventions in directing social change. They are primarily considered in the needs analysis phase of an evaluation to anchor program logics. However, this foundation has implications for intervention design, defining outcomes and establishing criteria of merit. Monitoring frameworks, particularly when using sentinel indicators, are also influenced by problem framing and assumptions. Indigenous and transformative approaches, where the rectification of historical power imbalances is essential, would find this particularly relevant. Problematisation can prepare participants for truth-telling, a step in reconciling intergenerational trauma and stopping systemic violence (Payne & Norman, 2025).
Assistant Director, Small Business Evaluation Hub, Australian Taxation Office
Liesl works in the Australian Taxation Office (ATO), helping business areas deliver quality evaluations and to build their evaluation culture, capacity and practice. With over 25 years of evaluation experience, Liesl has also worked for Queensland Treasury and Trade where she assisted... Read More →
Friday September 18, 2026 11:30am - 12:30pm ACST Rooms 3+4Stokes Hill Rd, Darwin City NT, Australia
Authors: Su-Ann Drew, Grosvenor, Jo van Twest Farmer, Rooftop Social, Eleanor Williams, ACE, Emma Williams, Martina Donkers Ethical pressures arise due to a range of conflicting incentives that for those who commission and deliver evaluations. Evaluators may try to maintain methodological rigour while meeting tight timeframes or limited budgets. Commissioners may need defensible evidence while navigating organisational expectations, political sensitivities or shifting priorities. These pressures are real and often lead to ethical tensions for all involved, without an agreed or shared language for discussion. This session creates space for attendees to discuss challenges openly, safely and constructively, helps participants recognise and make sense of pressures shaping commissioning decisions, and builds participants’ confidence in responding in ways that support both quality and working relationships.
Building on previous AES presentations on 'everyday ethics', we give participants tools to apply in real-world commissioning contexts by introducing a simple organising framework, the Evaluation Pressure System, which helps participants identify the mix of pressures influencing a situation and why tensions arise. The framework is a guide to support reflection and conversation rather than a technical model.
Using the framework, we will explore fictional but realistic scenarios that illustrate common pressure points in commissioning and evaluation delivery. Participants will be invited, through anonymous polling, to indicate the extent to which each scenario reflects situations they have encountered. Through in-room conversations, attendees will use the framework to examine what helps maintain integrity and constructive working relationships when pressures collide. The intention is not to analyse cases in depth, but to build a clearer shared understanding of tensions that arise and how they can be handled well.
To support ongoing application, attendees will receive a Trade‑off Log to clarify constraints and integrity risks, and practical communication strategies for raising concerns early and negotiating expectations. These will help participants recognise tensions earlier, discuss them more openly and navigate them in ways that support quality and collaboration.
Su-Ann is a Manager specialising in program evaluation within Grosvenor’s public sector advisory practice. Su-Ann has more than a decade of rich and diverse professional experience, which enables her to offer a unique perspective and critical lens to solving complex problems for... Read More →
