aes26 International Evaluation Conference: Full Schedule

Presenter and chair information page, resources for conference presenters, authors and session chairs.

arrow_back View All Dates

11:00am ACST

Measuring what matters: Culturally responsive evaluation of a statewide ECEC strategy

Wednesday September 16, 2026 11:00am - 11:30am ACST

Authors: Phillip Belling, NSW Department Of Education Jessica de Jesus, NSW Department Of Education, Rowan Savage, Aboriginal Education & Communities, NSW Department of Education

Presenter note: Rowan has been an important contributor to this work and was originally hoped to join the presentation in person. As the opportunity to present at aes26 was only confirmed in the past two days, the short timeframe unfortunately made it difficult to arrange for him to travel to Garrmalang | Darwin. We’re sorry Rowan can’t be here in person and acknowledge the perspective and contribution he has brought to the work being shared.

Government strategies to improve Aboriginal children’s access to culturally safe, high-quality early childhood education and care (ECEC) are increasingly ambitious, and also increasingly measured. Yet many mainstream outcome frameworks privilege what is easy to count (enrolments, attendance, completions) over what Aboriginal families and communities say really matters (cultural safety, belonging, relationships, self-determination, and thriving). And while there is a growing number of evaluations of pilots, projects and small scale programs, there are few examples of culturally responsive evaluations of large-scale, multi-program and strategy-level initiatives.

This session shares an approach being used in the NSW Department of Education to build culturally meaningful process indicators and outcome measures for a statewide ECEC strategy. Drawing on lessons from recent evaluations of state-level ECEC programs and evaluation designs, we present a practical method for moving from policy intent to measurable outcomes without collapsing cultural concepts into deficit metrics.

The approach combines: (1) outcome-domain mapping that starts with Aboriginal-led commitments and community-informed constructs; (2) a set of core indicators that deliberately pairs quantitative indicators with qualitative evidence (yarning, narrative case examples, workforce and service reflections) so that relationships and cultural safety are visible; and (3) governance and data sensemaking processes that keep Aboriginal authority central in interpreting evidence and deciding what “good” looks like across diverse local contexts, while remaining accountable to statewide commitments.

We also discuss why “measuring what matters” is a safeguarding task. In recent evaluation work we observed that indicators can unintentionally shift attention, create compliance behaviours, redistribute decision rights, and generate cultural load. In this session we outline design choices that mitigate these risks, including co-design checkpoints, minimum evidence standards, and using indicators as prompts for shared interpretation rather than as league tables.

Participants will leave with an adaptable template for developing culturally meaningful indicators and outcome measures, and examples they can translate to their own settings and evaluation partnerships.

Chair

Deb Moyle

Speakers

Phillip Belling

Senior Evaluator, Centre for Education Statistics and Evaluation, NSW Department of Education

Talk to me about evaluation that transforms lives and enables social change. I’m interested in realist, developmental, embedded and responsive evaluation; evaluation in complex education systems; evaluation capacity building; and how we make credible evaluative judgements about... Read More →

Wednesday September 16, 2026 11:00am - 11:30am ACST
Rooms 1+2 Stokes Hill Rd, Darwin City NT, Australia

Ethics and integrity

Format Short paper
Audience Level Foundational, Intermediate
Industry Community and social services, Not for profit Philanthropy
ID 230

11:00am ACST

Introduction to Evidence for New, Emerging and Non-Evaluators

Wednesday September 16, 2026 11:00am - 12:00pm ACST

Hall 2

Author: Charlie Tulloch (Policy Performance)

This session at the start of the conference orients new, emerging and non-evaluators to the types of topics and content they will encounter over the coming days. It provides a foundational overview of evidence-related fields (e.g. evidence, monitoring, evaluation, learning), debunks key concepts/language, offers insights into what evaluators do day-to-day, discusses values/valuing, ethical considerations, competencies and key steps usually considered when completing high quality evaluation projects.The session strengthens defines the boundaries and intersections of evaluation and other disciplines to understand this field of endeavour, the pathways in and the opportunities it offers.

The session’s main aims are to share and discuss:

How evidence is used in different policy and organisational landscapes

The roles and types of projects led by evaluators

Techniques frequently used by evaluators•Ways that evaluators think (logic, data, outputs, outcomes, impact)

Why and when evaluations happen (or do not)

Practical tips for evaluation commissioners and working with evaluators

Capability building (skills and knowledge) to promote use of evaluation in-house

A seven-stage approach to planning and conducting an evaluation project

Essential skills of evaluators, including logic modelling, defining key/interview questions, forming value judgments, and selecting effective evaluation approaches/methods.The session will provide attendees with an opportunity to ask questions anonymously, so they feel very comfortable as the conference commences.

Questions will be addressed during the session, creating an open, interactive discussion.The session will also provide an overview of the conference structure, learning opportunities and flow, to provide a guide to attendees about how to gain the most value from their time.

Chair

Sophie Henness

Consultant, ARTD Consultants

Speakers

Charlie Tulloch

Director, Policy Performance

Policy Performance delivers evaluation projects, capability building support and supports public sector and organisational improvement. Charlie also helps those who are new to evaluation or transitioning from related disciplines. He is a past AES Board member and current member of... Read More →

Wednesday September 16, 2026 11:00am - 12:00pm ACST
Hall 2

Special

Format Big room
Audience Level Foundational
Industry Public administration and safety, International development, Community and social services, Not for profit Philanthropy

11:30am ACST

When no one has time for evaluation - building learning cultures that survive pressure

Wednesday September 16, 2026 11:30am - 12:00pm ACST

Waterfront 1

Author: Su-Ann Drew (Grosvenor)
Many organisations value evaluation in principle but struggle to sustain it in practice, particularly when time is limited, priorities shift, or politically sensitive issues arise. Under pressure, evaluation is often treated as an optional task rather than a way of thinking embedded in everyday work. This short paper examines what allows evaluative thinking to persist under these conditions, especially where leadership support is intermittent, symbolic or short lived.

Chair

Lindsay Reilly

Speakers

Su-Ann Drew

Senior Manager, Grosvenor

Su-Ann is a Senior Manager specialising in program evaluation within Grosvenor’s public sector advisory practice. She has more than a decade of experience across management consulting, state government (including health, mental health, and alcohol and other drug policy and commissioning... Read More →

Wednesday September 16, 2026 11:30am - 12:00pm ACST
Waterfront 1 Stokes Hill Rd, Darwin City NT, Australia

Roots and routes

Format Short paper
Audience Level Foundational
Industry Public administration and safety, Not for profit Philanthropy, Community and social services
ID 188

11:30am ACST

What! No survey? Introducing the Collective Noticing Method (CNM)

Wednesday September 16, 2026 11:30am - 12:30pm ACST

Waterfront 2

Author: Jess Dart, Clear Horizon
This paper introduces the Collective Noticing Method (CNM) and explores its relevance for measurement, evaluation and learning (MEL). Developed by the author, CNM offers evaluators and changemakers a collaborative way to notice, learn from, and respond to subtle and emerging changes. It is a rigorous yet flexible method with a strong focus on learning and collective sensemaking, well suited to the “messy middle” of systems change and place-based work. It also has potential for community and grassroots work, program portfolios, and research impact tracking.

Collective noticing as a practice has a long history, evident in approaches such as Outcome Harvesting, Most Significant Change (MSC), citizen science, ripple effect monitoring, and context monitoring. More recently, this practice has been enabled and accelerated by emerging digital platforms and AI. Building on these foundations, CNM intentionally harnesses the eyes and ears of those implementing and experiencing an initiative to bring shared attention to what is changing.CNM is inherently participatory, inviting many people to observe, document, and interpret change collectively. It values lived experience as a central source of insight, embraces uncertainty, and supports adaptive action grounded in what is actually emerging. As a decolonising and grounded approach to MEL, CNM shares power by broadening who defines what matters and what counts as evidence. It centres learning-oriented measurement and collective sensemaking, bringing forward subtle signals, relational shifts, and everyday insights that conventional approaches often overlook.

CMN involves impact logging and tracking against agreed ways of working and learnings, whereas Outcome Harvesting focuses on outcomes and tends to be done as a one-off study. It differs from MSC in that CNM uses shorter, multi-perspective logs across diverse categories and evidence types.This session introduces CNM and invites participants to critically explore its benefits, limitations, and the contexts in which it is most—and least—useful.

Chair

Jake Clark

Principal Advisor, Impact and Evaluation, CSIRO

Dr Jake Clark is a Principal Advisor within CSIRO’s Education and Outreach Impact and Evaluation Team, where he leads the design and implementation of monitoring, evaluation and learning (MEL) tools for the STEM Community Partnerships Program. He brings extensive expertise in quasi-experimental... Read More →

Speakers

Jess Dart

CEO, Clear Horizon

Dr Jess Dart is the founder and CEO of Clear Horizon, an Australian-based specialist evaluation company. Having received the 2018 Outstanding Contribution to Evaluation Award from the Australian Evaluation Society (AES), Jess is a recognised leader with over 25 years of experience... Read More →

Wednesday September 16, 2026 11:30am - 12:30pm ACST
Waterfront 2 Stokes Hill Rd, Darwin City NT, Australia

Traditional and new ways

Format Long paper
Audience Level Foundational, Intermediate
Industry Community and social services, Not for profit Philanthropy, Environment and climate change, International development
ID 251

12:00pm ACST

The ethics of independence – what does ‘independent evaluation’ mean in 2026?

Wednesday September 16, 2026 12:00pm - 12:30pm ACST

Hall 2

Author: Jo van Twest Farmer (Rooftop Social)

Independence is often posited as a primary means of ensuring evaluation is rigorous, trustworthy and fair. Independence is Norm 4 of the United Nation’s Evaluation Group Norms and Standards for Evaluation (2016), which states it is “necessary for credibility”. Built into this standard are two concepts – behavioural independence (evaluating without undue influence) and organisational independence (structures that support independent practice). For many in the evaluation landscape, the best means of ensuring independence has been for ‘independent evaluators’ to conduct ‘independent evaluations’.
Discussions around what constitutes independence have long been a feature of evaluation literature, but arguably there have never been so many pressures on maintaining independence in evaluation. There is growing interest in evaluation approaches where the relationships between program delivery and evaluation is less clear-cut, such as developmental, participatory and lived experience-led evaluations. As evaluation capability grows, evaluation functions are more widespread across teams and embedded in business-as-usual learning practice, rather than contracted out. Less positively, changing fiscal conditions place more pressure on organisations to undertake evaluation with less capacity for external expertise and support. The need to demonstrate impact to access funding in tightening funding environments means evaluations are higher stakes than ever, and there are often vested interests in demonstrating programs are effective.
This short paper will discuss what independence means in 2026, arguing that evaluation has moved beyond ‘organisational independence’ and must instead focus on how to support ‘behavioural independence’. Contemporary evaluation practice requires understandings of independence grounded in ethics and integrity. This approach directly challenges traditional role boundaries in evaluation, such as external versus internal or participant versus evaluator. In responding to the challenges of maintaining independence in 2026 and beyond, the presentation will suggest criteria to support evaluators to guide the ethics of their own practice, and a framework for upholding independent practice.

Chair

Sophie Henness

Consultant, ARTD Consultants

Speakers

Jo van Twest Farmer

Rooftop Social

Wednesday September 16, 2026 12:00pm - 12:30pm ACST
Hall 2

Boundaries and bridges

Format Short paper
Audience Level Foundational, Intermediate
Industry Community and social services, Education and training, Health and disability, Not for profit Philanthropy, Public administration and safety
ID 214

12:00pm ACST

Change that Matters People in Place

Wednesday September 16, 2026 12:00pm - 12:30pm ACST

Waterfront 1

Author: Salli Cohen (The Policy Room)
This presentation explores the uncomfortable space where policy and evaluation structurally reinforce harm. Its objective is to challenge assumptions of neutrality and examine how evaluative frameworks can either illuminate or obscure inequity.
The core argument is threefold: (1) evaluation and policy are never neutral and must interrogate power; (2) systems alignment and cultural authority determine whether outcomes are real or performative; and (3) accountability must ask “for whom” evaluation and policy work; and who bears the cost.
The session blends applied case insight with structured reflection and peer dialogue to provoke critical engagement and practical recalibration.

Chair

Lindsay Reilly

Speakers

Salli Cohen

Founder, The Policy Room

Salli is a strategist, storyteller and a quiet force re-imagining what policy should be.

Guided by her values of equity, self-determination and respectful collaboration, and grounded in her principle of change that matters, she works with organisations and communities to ensure they have the tools, relationships and confidence to drive the changes desired by those they... Read More →

Wednesday September 16, 2026 12:00pm - 12:30pm ACST
Waterfront 1 Stokes Hill Rd, Darwin City NT, Australia

Ethics and integrity

Format Short paper
Audience Level Intermediate, Advanced
Industry Not for profit Philanthropy, International development, Public administration and safety, Community and social services, Health and disability
ID 82

2:30pm ACST

Evaluation as a Bridge: The Multiple Hats of MEL in Scaling Government-led Graduation Programmes

Wednesday September 16, 2026 2:30pm - 3:00pm ACST

Waterfront 3

Authors: Kristian Paolo Torres, BRAC Ultra-Poor Graduation Initiative, Tania Dora Warokka, BRAC Ultra-Poor Graduation Initiative, Arnaldo Pellini, BRAC Ultra-Poor Graduation Initiative
Programmes aiming to influence governance systems and policy reforms—particularly those using experimental approaches—require MEL functions that are able to move across design, strategy, context analysis, and sensemaking.

Traditional M&E often prioritizes linear accountability, struggling to measure influence on policies and behaviors (as noted by the ODI RAPID programme, 2006–2020).

This presentation shares the experience of the BRAC Ultra-Poor Graduation Initiative, which partners with governments across six countries to scale the ‘Graduation’ approach through national social development systems (e.g. social protection, livelihood, labour and employment, etc.). Because UPGI’s success is measured by policy influence and system change rather than direct delivery, the MEL function must act as a strategic bridge rather than a neutral observer.

Drawing on our experience in Indonesia and the Philippines, we share how MEL supports country teams as adaptive units navigating two primary challenges: calibrating the interplay between political commitment, administrative feasibility, and technical quality; and designing for scale from the outset to ensure government ownership rather than isolated pilot results.

We argue that a team learning culture—rather than a "neutral" M&E function—is the essential infrastructure for this work. By establishing specific habits to capture field experience and inform strategic decisions, MEL acts as the "many-hatted" bridge required to navigate these dynamic systemic boundaries and prevent programmes from remaining stagnant.

We will share three core insights from our MEL experience:

1. How the evaluator’s role shifts to an active learning partner, framing field evidence to navigate political and administrative boundaries.
2. How practical tools—such as Capacity and Commitment Rubrics and Diaries—support teams in documenting for learning rather than reporting for compliance.
3. Real-world examples where structured reflection informed specific shifts in government engagement strategies.

The session will also engage the audience in a "Reflection Micro-Workshop" where participants will share insights from their own work and experiences.

Chair

Dru Armstrong

Evaluation and Impact Manager, Health and Wellbeing Queensland

Much of my work sits in the space of finding practical ways to bring systems thinking, equity and learning into public-sector evaluation while meeting the demand for quantification, attribution and economic value. I love the challenge of evaluating in complexity and initiatives that... Read More →

Speakers

Kristian Paolo Torres

Research, Evaluation, and Knowledge Management Specialist, BRAC Ultra-Poor Graduation Initiative

Tania Dora Warokka

Lead MEL and Research, BRAC Ultra-Poor Graduation Initiative

Wednesday September 16, 2026 2:30pm - 3:00pm ACST
Waterfront 3 Stokes Hill Rd, Darwin City NT, Australia

Boundaries and bridges

Format Short paper
Audience Level Foundational, Intermediate
Industry Community and social services, Not for profit Philanthropy, International development
ID 234

2:30pm ACST

Epistemic Justice: victim survivors of child sexual abuse as co-evaluators

Wednesday September 16, 2026 2:30pm - 3:30pm ACST

Waterfront 1

Authors: Nic Vogelpoel (Day Four Projects), Malika Reese (Lived Experience Advisor), Angela Obradovic (National Centre for Action on Child Sexual Abuse)

What happens when evaluation is not just informed by lived experience, but led by it?

This presentation offers a rare, practice-based account of a lived experience-led evaluation undertaken with the National Centre for Action on Child Sexual Abuse, where victim survivors worked as co-evaluators across all stages of the process. Established to centre victim survivors in awareness-raising, help-seeking, advocacy and best-practice responses, the Centre provided a powerful context to rethink how evaluation knowledge is produced, and by whom.

The session argues that lived experience-led evaluation is not an ethical “add-on”, but a fundamental shift in values, power and epistemology. Drawing on the evaluation findings, presenters will explore three core propositions.

First, nothing about us without us: moving from evaluation on people to evaluation led by people with lived experience reshapes evaluation questions, evidence, outcomes and definitions of rigour. Second, epistemic justice: lived experience leadership challenges entrenched assumptions about who gets to ask questions, whose knowledge counts, and how institutions respond to new forms of evidence. Third, beyond advice to synthesis: lived experience cannot simply “advise” evaluation, it must be integrated as a distinct way of knowing that transforms the whole evaluation.

Co-presented by lived experience evaluators and Day Four Projects evaluators, the session will combine reflective storytelling, concrete practice examples and facilitated dialogue. Participants will be invited to critically examine their own evaluative assumptions, engage in small-group reflection, and explore practical strategies for making space for multiple knowledge systems while maintaining evaluative integrity and meeting institutional requirements.This session will be particularly valuable for evaluators, commissioners, researchers and practitioners seeking more just, credible and impactful approaches to lived-experience evaluation in complex and sensitive contexts.

Chair

Anona Armstrong

Director Research, Victoria University

Professor Anona Armstrong AM is Director of Research and Research Training, Professor of Governance and Head of the Governance Research Program in the College of Law and Justice, Victoria University. She was Foundation President, and is a Life Member and Fellow of the Australasian... Read More →

Speakers

Nic Vogelpoel

Director, Day Four Projects

We specialise in the theory and practice of good collaboration. We have a particular interest in learning and evaluation for partnerships, platforms and collaborative initiatives. We work with international and domestic partners from multilateral organisations, governments, NGOs... Read More →

Malika Reese

Lived Experience Advisor

Talk to me about Lived Experience awareness, consultation and healing. Talk to me about helping your organisation be more trauma informed.

Angela Obradovic

National Centre for Action on Child Sexual Abuse

Angela has been actively involved with the National Centre for Action on Child Sexual Abuse for the past 3 years, committed to improving how lived experience can inform prevention of CSA and responses to Adult Survivors of CSA. With 30 years of experience across social work and mental... Read More →

Wednesday September 16, 2026 2:30pm - 3:30pm ACST
Waterfront 1 Stokes Hill Rd, Darwin City NT, Australia

Boundaries and bridges

Format Long paper
Audience Level Advanced, Intermediate
Industry Not for profit Philanthropy, Health and disability, Community and social services, Public administration and safety, Education and training
ID 45

2:30pm ACST

Beyond Symbolic Inclusion - Building a Collective Mandate for Indigenous-Led Understanding Measurement Evaluation & Learning (UMEL)

Wednesday September 16, 2026 2:30pm - 3:30pm ACST

Hall 2

Author: Liz Wren (Gilibanga), Gulwangyang Moran, Slade Sibosado (Kimberley Aboriginal Medical Services)
Corrine Hodson (Barang Regional Alliance), Jessica Yamaguchi (National Indigenous Australians Agency)

Overview: For too long, Aboriginal and Torres Strait Islander practitioners have navigated a ‘middle space’—a complex intersection where Cultural obligations to Community and Country often collide with the rigid, competitive structures of Western bureaucracy. This session, presented by Gilibanga, synthesises critical insights from the Gilibanga Blak Think Tank based on the original work of the First Nations UMEL Peer Learning Circles, a national initiative conducted between 2025 and 2026 as part of a collaboration between Kowa and the Social Enterprise Development Initiative (SEDI).
The Collective Mandate: Moving beyond deficit-based ‘capacity building,’ this session articulates a Collective Mandate for the evaluation sector. We challenge the industry to shift from a model of individual scarcity to one of communal abundance. Drawing on the "U" in UMEL—Understanding—we prioritise early investment in relationships, local context, and Community aspirations as the foundational bedrock of all evaluative work.
Key Themes for Discussion: Participants will engage with four transformative thematic areas identified by the First Nations evaluation community:
• Structural Reformation: Moving from competitive procurement models that force "Mob to compete" toward collaborative contracting and ecosystem thinking.
• Dual Accountability: Acknowledging the emotional and professional labour required to hold ‘Two-Worlds’ practice, balancing contractual obligations with Cultural integrity.
• Broadening the Definition of "Evaluator": Validating place-based, relational, and lived expertise that exists outside traditional Western academic credentials.
• Material Decolonisation: Shifting from symbolic language to the practical transformation of contracts, reporting templates, and the active upholding of Indigenous Data Sovereignty.

Chair

Rick Cummings

Emeritus Professor, Murdoch University

Rick Cummings is an Emeritus Professor in Public Policy at Murdoch University. He has 40 years of experience conducting evaluation studies in education, training, health, and crime prevention primarily for the state and commonwealth government agencies and the World Bank. He currently... Read More →

Speakers

Corinne Hodson

Barang Regional Alliance

Jess Yamaguchi

Gilibanga / National Indigenous Australians Agency

Liz Wren

Gilibanga

Slade Sibosado

Kimberley Aboriginal Medical Services

Gulwanyung Moran

Gilibanga

Wednesday September 16, 2026 2:30pm - 3:30pm ACST
Hall 2

Traditional and new ways

Format Big room
Audience Level Foundational
Industry Public administration and safety, Community and social services, Not for profit Philanthropy
ID 328

4:00pm ACST

Developmental Evaluation or a Learning Organisation?

Wednesday September 16, 2026 4:00pm - 4:30pm ACST

Waterfront 2

Author: Caroline Henwood, The Ian Potter Foundation
Increasingly organisations are leaning into the “L” in MEL. Learning is a critical component of evaluation it is the opportunity to turn findings and insights into something practical to inform adaptation. However, evaluators often reflect on the report on the desk, or of evaluations occurring after decisions are made. Shifting the focus to learning creates a different space and dynamic for conversations to occur – a key practice in Developmental Evaluation.

Chair

Vanessa Hood

Director, Rooftop Social

I've been working as a facilitator and evaluator for over 20 years, in a wide range of contexts, including environment and financial literacy. I am a Director at Rooftop Social, a medium sized consulting firm, which provides services in evaluation, social research, facilitation and... Read More →

Speakers

Caroline Henwood

Research, Evaluation and Learning Manager, The Ian Potter Foundation

I am always intrigued by how we come to definitions, how words resonate differently for individuals and what it all means. On a more practical note! I am interested in people's perceptions on proportionality, how we think about it, and how we support it when it is often not what appears... Read More →

Laura Bird

Paul Ramsay Foundation

Wednesday September 16, 2026 4:00pm - 4:30pm ACST
Waterfront 2 Stokes Hill Rd, Darwin City NT, Australia

Boundaries and bridges

Format Short paper
Audience Level Foundational, Intermediate
Industry Not for profit Philanthropy
ID 171

4:00pm ACST

Cultural Wisdom and Story Gathering: a participatory reflection session

Wednesday September 16, 2026 4:00pm - 4:30pm ACST

Rooms 1+2

Authors: Jessi Gidgup-Lovett (Rooftop Social), Duncan Rintoul (Rooftop Social)

This session invites participants into a short, participatory exploration of Cultural Wisdom and Story Gathering, grounded in relational practice in how to do business with First Nations peoples and communities.
Before choosing methods, indicators or measures of success, we need to ask more fundamental questions: Who are we doing business with? Who holds power? How will power be shared? Whose knowledge will guide us? What are we responsible for.
Using simple sensory and reflective activities, this session will invite you to internalise respect, reciprocity, responsibility, reconciliation, being, knowing and doing as practical foundations for ethical evaluation practice. It will also ask participants to think critically about their own positionality and how they enter relationships with First Nations peoples.
You will leave with a one-page grounding resource to support ongoing reflective and reflexive practice, with a simple commitment to listen, learn, reflect, act and carry forward what you are learning through this conference.

Chair

John Gilbert

Manager Monitoring & Evaluation, Country Fire Authority

I am a program evaluator with over 20 years of experience in bushfire and natural hazards community safety and emergency management. This has included roles in the emergency services, universities and peak bodies. In my current role at the Country Fire Authority, I lead the Monitoring... Read More →

Speakers

Jessi Gidgup-Lovett

Rooftop Social

Duncan Rintoul

Managing Director, Rooftop Social

Wednesday September 16, 2026 4:00pm - 4:30pm ACST
Rooms 1+2 Stokes Hill Rd, Darwin City NT, Australia

Ethics and integrity

Format Short paper
Audience Level Foundational, Intermediate
Industry Education and training, Community and social services, Not for profit Philanthropy
ID 330

4:00pm ACST

Who Shapes What Counts? Collaboration as Ethical Design in Large-Scale Evaluation

Wednesday September 16, 2026 4:00pm - 4:30pm ACST

Waterfront 1

Author: Stefano Verrelli (The Salvation Army)
How do you build a national outcomes framework without flattening local realities, sidelining frontline practice wisdom, or reducing lived experience to an input rather than a shaping influence?

This presentation shares insights from an outcomes and impact evaluation of one of Australia’s largest homelessness service providers, spanning more than 100 programs, 700 practitioners, and around 40,000 clients annually. The evaluation aimed to develop and pilot a nationally relevant outcomes measurement framework before broader rollout, one that could work across diverse service models, jurisdictions, funding contexts, and client groups.

The challenge was not only technical, but ethical. A standardised framework risked privileging some perspectives over others, adding burden to already stretched services, and embedding measures that did not reflect frontline service realities or add value to people accessing support.

This presentation argues that, in large-scale evaluation, a staged and deliberate collaborative process across design, piloting, and refinement is a core ethical strategy. Using this case example, it shows how this approach made space for perspectives not always given meaningful influence in shaping outcomes evaluation at this scale, including frontline practitioners, practice leads, and people with lived and living experience. In doing so, it helped ensure that decisions about what outcomes mattered, how they were measured, and how the framework would work in practice were shaped by frontline realities and lived experience alongside competing system priorities.

The presentation offers a practical lesson for evaluators working across multiple sites and systems: ethical evaluation in practice depends on how frameworks are collaboratively developed, tested, and refined before implementation. The session will conclude with brief guided reflection questions to help attendees consider implications for their own evaluation practice.

Chair

Kizzy Gandy

Head, Evidence and Impact, Sax Institute

Speakers

Stefano Verrelli

Research and Evaluation Practitioner, The Salvation Army

Wednesday September 16, 2026 4:00pm - 4:30pm ACST
Waterfront 1 Stokes Hill Rd, Darwin City NT, Australia

Ethics and integrity

Format Short paper
Audience Level Foundational
Industry Not for profit Philanthropy, Community and social services, Health and disability
ID 157

4:30pm ACST

Professional isolation in evaluation: AES members’ experiences, and ways to strengthen peer connection and community

Wednesday September 16, 2026 4:30pm - 5:30pm ACST

Hall 2

Authors: Martina Donkers, Julie Elliott

Professional isolation is an experience that, ironically, many evaluators share. As evaluators, we often find ourselves in a team of one – the only evaluator in the room, the only evaluator at the organisation, one bringing an evaluative lens to the problem. This can be hard. It can feel like people are turning to us for more answers than we have, or expecting something unrealistic. Sometimes we find ourselves feeling isolated due to our culture, our methodological approach or our disciplinary background.
There are structural ways isolation shows up – evaluators who work remotely in regional areas, evaluators who are self-employed, evaluators who balance their role with kids and family responsibilities. These experiences can leave us feeling like an outsider, with no one to test ideas, no one to help when the going gets rough, and no one to help grow our capabilities.
This panel session will explore AES members’ experiences of professional isolation, and what has worked (and not worked!) to help them combat it. We’ll present anonymised experiences from AES members about professional isolation to show attendees they aren’t alone.
We’ll then hear from 3 panellists at different stages of their professional journey, and what they’ve done to address professional isolation they’ve felt as evaluators. Finally, we’ll open the floor to questions – how can we feel more connected in our work? – and explore a range of ways that we can combat professional isolation in our field. This panel is presented by the AES Peer Group Mentoring Program Working Group, and considers how the program helps strengthen connection to peers. It also extends thinking beyond AES initiatives, and considers other ways that evaluators can combat professional isolation in a disconnected world. If you’ve ever felt lonely as an evaluator, this session is for you.

Chair

Andrew Boyle

Andrew Boyle Consulting

Andrew Boyle is an Australian monitoring, evaluation and learning (MEL) specialist and Principal Consultant at Andrew Boyle Consulting. His work focuses on organisational learning, Theory of Change, evaluation capacity strengthening and culturally responsive practice, particularly... Read More →

Speakers

Martina Donkers

Independent evaluator

I'm a self-employed evaluator who gets to works on lots of different interesting projects with lots of different excellent teams - how good is that! I like qual methods, complexity, making sure evaluation is useful, and helping people grapple with what data really mean. I run qual... Read More →

David Roberts

Principal Consultant, RobertsBrown

David Roberts is a Fellow of the AES has a M.Eval.Ass and BA(Hons). He is semi-retired as a self-employed consultant. He has wide experience over 40 years in evaluations using both qualitative and quantitative methods. David has training and experience in Anthropology, Evaluation... Read More →

Jill Thomas

Senior Consultant, Jill Thomas Evaluation

Jill Thomas (MEval, MBusSys) is the Director of Jill Thomas Evaluation, a collaborative consultancy dedicated to advancing social good through the development of effective evaluation systems, projects, and products. With extensive experience as a consultant evaluator and analyst... Read More →

Paula White

Equity / Impact Measurement, Independent Evaluator

Bethany Hanson

Manager Review and Evaluation, TAFE NSW

Beth combines her extensive NSW Government and consultancy experience on evaluation projects spanning environment, sustainability, education and First Nations programs. Beth has led the design of several large portfolio evaluation frameworks at state and federal level. She's now applying... Read More →

Wednesday September 16, 2026 4:30pm - 5:30pm ACST
Hall 2

Special

Format Panel
Audience Level Intermediate
Industry Public administration and safety, Community and social services, Education and training, Not for profit Philanthropy, Health and disability
ID 311

4:30pm ACST

Evaluating projects using social capital framework in refugee communities in Australia

Wednesday September 16, 2026 4:30pm - 5:30pm ACST

Waterfront 1

Authors: Ali Rasoli (STARTTS), Samantha Cherian (STARTTS)
Community development programs in refugee settlement contexts are often evaluated using frameworks designed for clinical or mainstream service environments. These approaches can struggle to capture the trauma-related, relational, collective and culturally embedded forms of change that occur in refugee communities.

In response to this gap, the NSW Service for the Treatment and Rehabilitation of Torture and Trauma Survivors (STARTTS), together with more than 130 refugee community leaders, has co-designed a Social Capital Evaluation Framework for community development programs working with refugee and migrant communities in Australia. The framework adapts established social capital theory to the realities of settlement work, enabling evaluation of changes in bonding, bridging and linking social capital, while also introducing the concept of social capital enablers.

This presentation introduces the STARTTS Social Capital Framework and demonstrates how it functions as a culturally responsive and trauma-informed evaluation approach for refugee community development initiatives. Drawing on multiple program evaluations conducted across New South Wales, the session will show how the framework captures outcomes often overlooked by conventional evaluation models, including strengthened community networks, emerging leadership, increased access to institutions, and collective wellbeing.

The presentation will outline the conceptual foundations of the framework, describe the evaluation tools and indicators developed by STARTTS, and present practical examples from community programs. Participants will be invited to reflect on how social capital concepts can be applied within their own evaluation contexts through guided discussion during the session.

The session will offer practical insights for evaluators and practitioners working in settlement and community development contexts, demonstrating how social capital can be operationalised as both a conceptual framework and a practical evaluation tool for evaluating refugee community programs.

Chair

Kizzy Gandy

Head, Evidence and Impact, Sax Institute

Speakers

Ali Rasoli

Team Leader - Community Development Evaluation, STARTTS

I am an M&E (monitoring & evaluation) specialist with 12 years of experience in different sectors. I use evidence, insights, and data to support effective decision‑making, learning, and continuous improvement across programs and organisations. My key strength lies in fostering collaboration... Read More →

Samantha Cherian

Community Development Evaluation Officer, STARTTS

I am Community Development Evaluation Officer at STARTTS with a background in research and mixed-methods evaluation. I hold a Bachelor of Economics (Honours) and a Master’s degree in International Security. My work focuses on evaluating community development and mental health initiatives... Read More →

Wednesday September 16, 2026 4:30pm - 5:30pm ACST
Waterfront 1 Stokes Hill Rd, Darwin City NT, Australia

Traditional and new ways

Format Long paper
Audience Level Foundational, Intermediate
Industry Not for profit Philanthropy, Community and social services
ID 101

11:00am ACST

11:00am ACST

11:30am ACST

11:30am ACST

12:00pm ACST

12:00pm ACST

2:30pm ACST

2:30pm ACST

2:30pm ACST

4:00pm ACST

4:00pm ACST

4:00pm ACST

4:30pm ACST

4:30pm ACST

Get help with the event