Authors: Kristine Tuban, Save the Children, Martin Holmstrand, Save the Children This presentation showcases MEAL Uplift, a regional initiative by Save the Children Australia that strengthens digital monitoring, evaluation, accountability, and learning (MEAL) across Pacific Country Offices through a close partnership between programme teams, MEAL, and the Technology team. The presentation focuses on how this collaboration produced a practical, co‑created digital solution while building long‑term organisational capability.
The objective of the presentation is to demonstrate why cross‑disciplinary partnership is critical to improving data quality, efficiency, and use in evaluation practice, and how evaluation can actively shape digital transformation, rather than simply adopting technological tools after they are introduced. This is important in contexts where teams face increasing reporting demands but limited capacity to manage fragmented or manual data systems.
The core argument is that sustainable digital MEAL is achieved when three elements are intentionally integrated: (1) Digital systems and infrastructure, illustrated through a custom‑built data collection app and automated data pipeline co‑designed with the Technology team to address efficiency and quality challenges; (2) Skills and behaviours, supported through targeted training, coaching, and practical use of real data; and (3) Scaling and institutional support, through shared standards, governance, and regional scaffolding that embeds digital MEAL into everyday practice.
The presentation will follow this three‑part structure, using concrete examples and lessons from MEAL Uplift to show what worked, what changed, and why.
Participant engagement will be promoted through short reflection prompts on participants’ own digital MEAL challenges, followed by shared discussion and an open Q&A focused on transferable ideas and co‑created solutions across different organisational contexts.
Authors: Caitlin Morton, Maggie Hawkins (Attorney General's Department) This session poses the question – how should we evaluate legal policy? Evaluators across all sectors encounter legal frameworks, yet few forums explicitly address how law itself can and should be evaluated. This session seeks to help carve out that space within evaluation practice.
Strong approaches to evaluating legal policy are critical to achieving just, fair, and secure society - the remit of the Attorney-General's Department. In this session, past and current evaluators from the AGD reflect on observations, challenges, and conversations, and invite discussion on what it looks like when legal policy is working well. We present an overview of existing dialogue on this question, and argue that it is critical to examine the social and ethical foundations of legal policy, the principles that inform legal policy, and how the law can support the operation of impactful legal policy.
We will investigate how law does and does not align with the beliefs and assumptions of the communities it touches, and how law is interpreted and put into practice. Together we will explore localisation as a major challenge and opportunity in evaluating legal policy – noting the persistent regional/metro divide in accessing legal services, and diversity between communities and across states, which always requires collaboration.
This session is critically important to evaluators as law touches all public policy, which in turn impacts the operations of not just federal governments, but local and state governments, private businesses, community organisations, not-for-profits, and more.
Authors: Jess Moniodis (North Australian Aboriginal Family Legal Service), Mona Roberts (North Australian Aboriginal Family Legal Service) This session will examine how NAAFLS is embedding Indigenous Data Sovereignty and Governance principles into program evaluation. Our evaluation applies Indigenous-led qualitative methodologies that are explicitly aligned with nationally recognised victim-survivor principles. These methods prioritise safety, choice, voice, control, dignity, healing, and accountability across stages of the evaluation.
Guided by a two-worlds approach, and place-based victim-survivor led solutions, we recognise that well-intentioned initiatives can sometimes unintentionally create negative impacts rather than support community-defined outcomes.
NAAFLS aims to address this by co-creating an evaluation approach that places First Nations perspectives on safety and wellbeing at the forefront, while aligning with nationally recognised victim-survivor and organisational principles. We will explore how evaluation has helped restore ethics and integrity in a complex setting to support collective learning across stakeholders and shift power back to our communities. Our approach prioritises women’s voices, lived experience, cultural knowledge, and the inclusion of diverse perspectives to ensure evaluation is grounded in the lived realities of those most affected.
Informed by a two-worlds approach and Our Ways – Strong Ways – Our Voices: National Aboriginal and Torres Strait Islander Plan to End Family, Domestic and Sexual Violence 2026–2036, this work centres victim-survivors, Elders, community members, service providers, and experienced professionals in defining success, shaping accountability, and guiding learning and improvement. This presentation will demonstrate NAAFLS practical application of a two-way lens - sharing our approach and reflections in translating Indigenous Data Sovereignty from principle into practice. It will also discuss lessons learned in embedding best-practice principles, supporting place-based understanding, and strengthening sustainable, community-led pathways for support.
Participants will be encouraged to reflect on their own practice, share their experiences and challenges, and discuss practical ways to embed Indigenous Data Sovereignty into evaluation practices.
Authors: Catherine Wade (Parenting Research Centre), Matt Healy (First Person Consulting), Fiona May (Parenting Research Centre) Social Network Analysis (SNA) is a systems methodology that maps and measures the relationships between actors in a system - revealing how information flows, where influence sits, and where collaboration can be strengthened. While widely used in research and policy contexts internationally, SNA remains underutilised in Australian evaluation practice, particularly in the social services sector where understanding partnership networks is critical to achieving outcomes.
This paper presents SNA as both a practical evaluation tool and a method for surfacing the relational dynamics that traditional approaches often miss. Drawing on applied examples from recent social services projects, we demonstrate how SNA can be used to assess whether programs are building the collaborative networks they intend to, identify structural gaps in service systems, and track network change over time. One example used to showcase the method will be a project exploring the social networks that mothers from culturally and linguistically diverse backgrounds rely on when making decisions about infant sleep practices.
The project aims to map these networks, identify the types of information shared, and examine how they influence mothers’ engagement with safe-sleep education.
Three key messages will be explored in the paper: how SNA complements existing evaluation methods rather than replacing them; what it takes to design and implement SNA in complex service environments; and how findings can be communicated to diverse stakeholders in ways that are actionable and meaningful.
The presentation combines a conceptual introduction with worked examples, and will include structured discussion inviting participants to consider how SNA might apply to their own evaluation contexts.
Principal Consultant | Co-Founder, First Person Consulting
I'm Matt, Co-Founder of First Person Consulting. I work at the intersection of systems thinking, evaluation and design — helping people make sense of messy, complex problems with a healthy dose of humour and humility. I also co-host the It Depends Podcast, where the honest answer... Read More →
Principal Research Specialist, Parenting Research Centre
Dr Catherine Wade is Principal Research Specialist at the Parenting Research Centre and a Research Affiliate with the Faculty of Health Sciences at the University of Sydney. Catherine is leading a programme of research at the Parenting Research Centre investigating aspects of the... Read More →
Wednesday September 16, 2026 11:00am - 11:30am ACST Rooms 3+4Stokes Hill Rd, Darwin City NT, Australia
Author: Alice Muller, World Vision Evaluators and development practitioners need to understand how environmental changes impact and result from development programming. Often finding practical ways to do is a challenge, particularly where complex environmental, biodiversity and climate systems are involved, and where evaluators and practitioners come from social or economic backgrounds with limited biophysical or ecological training. This paper shares how we can better value place and understand environmental consequences in international development projects by letting evaluations be locally-led, with local knowledge and values at the center.
Drawing on real‑world examples from World Vision’s Regreening Communities work, the paper presents three simple tools used by communities t Together, these approaches value place by centring local and Indigenous knowledge and values, assessing change from a local perspective, in a standardised way and using spatial tools that together still contribute to organisational‑level evidence.
First, a community mapping and planning exercise makes space for diverse groups within a community to define what regreening and wellbeing mean in their own environments, surfacing cultural, livelihood and social values that are often invisible in standard indicator frameworks. Second, a community‑assessed Regreening Index provides a structured and repeatable way to assess biophysical conditions and trends over time in relation to community priorities, while still enabling consistent comparison across sites and projects. Third, spatial mapping of regreening sites using a powerful, but freely available tool adapted from the WASH sector, enables evaluators to examine patterns and cumulative effects at project and organisational levels.
While none of these tools is unique in isolation, when combined they lower barriers to including environmental considerations in evaluation practice and generate insights that strengthen learning, adaptive management and sustainability discussions, ethically. The presentation will share engaging practical examples and invite participants to reflect on opportunities to better value place within their own work.
Senior Monitoring and Evidence Advisor, World Vision
An environmental scientist, working in international development, interested in evaluation and learning about all things community, trees, ecosystem restoration, climate action, scaling and systems transformation. I also really like coffee and chatting about gardening, travel and animal anecdotes if you need a break anytime... Read More →
Wednesday September 16, 2026 11:00am - 11:30am ACST Waterfront 3Stokes Hill Rd, Darwin City NT, Australia
Author: Su-Ann Drew (Grosvenor) Many organisations value evaluation in principle but struggle to sustain it in practice, particularly when time is limited, priorities shift, or politically sensitive issues arise. Under pressure, evaluation is often treated as an optional task rather than a way of thinking embedded in everyday work. This short paper examines what allows evaluative thinking to persist under these conditions, especially where leadership support is intermittent, symbolic or short lived.
Su-Ann is a Manager specialising in program evaluation within Grosvenor’s public sector advisory practice. Su-Ann has more than a decade of rich and diverse professional experience, which enables her to offer a unique perspective and critical lens to solving complex problems for... Read More →
Wednesday September 16, 2026 11:30am - 12:00pm ACST Waterfront 1Stokes Hill Rd, Darwin City NT, Australia
Independence is often posited as a primary means of ensuring evaluation is rigorous, trustworthy and fair. Independence is Norm 4 of the United Nation’s Evaluation Group Norms and Standards for Evaluation (2016), which states it is “necessary for credibility”. Built into this standard are two concepts – behavioural independence (evaluating without undue influence) and organisational independence (structures that support independent practice). For many in the evaluation landscape, the best means of ensuring independence has been for ‘independent evaluators’ to conduct ‘independent evaluations’. Discussions around what constitutes independence have long been a feature of evaluation literature, but arguably there have never been so many pressures on maintaining independence in evaluation. There is growing interest in evaluation approaches where the relationships between program delivery and evaluation is less clear-cut, such as developmental, participatory and lived experience-led evaluations. As evaluation capability grows, evaluation functions are more widespread across teams and embedded in business-as-usual learning practice, rather than contracted out. Less positively, changing fiscal conditions place more pressure on organisations to undertake evaluation with less capacity for external expertise and support. The need to demonstrate impact to access funding in tightening funding environments means evaluations are higher stakes than ever, and there are often vested interests in demonstrating programs are effective. This short paper will discuss what independence means in 2026, arguing that evaluation has moved beyond ‘organisational independence’ and must instead focus on how to support ‘behavioural independence’. Contemporary evaluation practice requires understandings of independence grounded in ethics and integrity. This approach directly challenges traditional role boundaries in evaluation, such as external versus internal or participant versus evaluator. In responding to the challenges of maintaining independence in 2026 and beyond, the presentation will suggest criteria to support evaluators to guide the ethics of their own practice, and a framework for upholding independent practice.
Author: Salli Cohen (The Policy Room) This presentation explores the uncomfortable space where policy and evaluation structurally reinforce harm. Its objective is to challenge assumptions of neutrality and examine how evaluative frameworks can either illuminate or obscure inequity. The core argument is threefold: (1) evaluation and policy are never neutral and must interrogate power; (2) systems alignment and cultural authority determine whether outcomes are real or performative; and (3) accountability must ask “for whom” evaluation and policy work; and who bears the cost. The session blends applied case insight with structured reflection and peer dialogue to provoke critical engagement and practical recalibration.
Authors: Kristian Paolo Torres, BRAC Ultra-Poor Graduation Initiative, Tania Dora Warokka, BRAC Ultra-Poor Graduation Initiative, Arnaldo Pellini, BRAC Ultra-Poor Graduation Initiative Programmes aiming to influence governance systems and policy reforms—particularly those using experimental approaches—require MEL functions that are able to move across design, strategy, context analysis, and sensemaking.
Traditional M&E often prioritizes linear accountability, struggling to measure influence on policies and behaviors (as noted by the ODI RAPID programme, 2006–2020).
This presentation shares the experience of the BRAC Ultra-Poor Graduation Initiative, which partners with governments across six countries to scale the ‘Graduation’ approach through national social development systems (e.g. social protection, livelihood, labour and employment, etc.). Because UPGI’s success is measured by policy influence and system change rather than direct delivery, the MEL function must act as a strategic bridge rather than a neutral observer.
Drawing on our experience in Indonesia and the Philippines, we share how MEL supports country teams as adaptive units navigating two primary challenges: calibrating the interplay between political commitment, administrative feasibility, and technical quality; and designing for scale from the outset to ensure government ownership rather than isolated pilot results.
We argue that a team learning culture—rather than a "neutral" M&E function—is the essential infrastructure for this work. By establishing specific habits to capture field experience and inform strategic decisions, MEL acts as the "many-hatted" bridge required to navigate these dynamic systemic boundaries and prevent programmes from remaining stagnant.
We will share three core insights from our MEL experience:
1. How the evaluator’s role shifts to an active learning partner, framing field evidence to navigate political and administrative boundaries. 2. How practical tools—such as Capacity and Commitment Rubrics and Diaries—support teams in documenting for learning rather than reporting for compliance. 3. Real-world examples where structured reflection informed specific shifts in government engagement strategies.
The session will also engage the audience in a "Reflection Micro-Workshop" where participants will share insights from their own work and experiences.
Author: Jack Rutherford, ARTD Have you ever felt that the Monitoring and Evaluation (M&E) Framework you developed or had to implement was potentially too complicated?
This presentation introduces Key Monitoring Questions (KMQs); a novel approach that builds on the well‑established logic of Results‑Based Accountability (RBA) to simplify M&E frameworks while strengthening their usefulness for decision‑making and reflection.
M&E Frameworks often become unintentionally complex. In the pursuit of rigour and comprehensiveness, evaluators and program designers can generate long lists of indicators, data sources and measurement requirements. While well‑intentioned, these expansive frameworks can overwhelm those responsible for implementation, leading to delays in data collection and reporting or abandoned learning processes.
Instead of attempting to measure every activity, output or incremental outcome, KMQs concentrate attention on purposeful questions drawn from RBA that the monitoring system must answer. This shift helps teams focus on what matters most by clarifying the purpose of monitoring and identifying the minimum information required to support meaningful learning.
The presentation will introduce four KMQs adapted from RBA and demonstrate how they streamline data collection and support evaluation and reflection:
1.How much was done? 2.How well was it done? 3.Who was affected? 4.How has the context changed?
The presentation will share three key messages:
1.Complex frameworks can undermine learning by overwhelming users with excessive data expectations. 2.KMQs provide a structured, disciplined way to sharpen focus and streamline indicator selection. 3.Using KMQs supports more sustainable monitoring practice, enabling teams to engage more deeply with the data that matters.
Before the session, you will be asked to reflect on your experiences developing or implementing M&E Frameworks, recognising the shared issues that KMQs address.
This session is targeted at foundational and intermediate evaluators, and commissioners and program staff who suspect their frameworks could be simpler and more effective.
Jack joinedARTD in 2018 after completing his Honours thesis in behavioural ecology the previous year, during which he tried to teach colours to jumping spiders. He augments his skills and insights from ecology to the public policy ecosystem, thinking critically about evaluation and analysis... Read More →
Wednesday September 16, 2026 3:00pm - 3:30pm ACST Waterfront 3Stokes Hill Rd, Darwin City NT, Australia
Author: Caroline Henwood, The Ian Potter Foundation Increasingly organisations are leaning into the “L” in MEL. Learning is a critical component of evaluation it is the opportunity to turn findings and insights into something practical to inform adaptation. However, evaluators often reflect on the report on the desk, or of evaluations occurring after decisions are made. Shifting the focus to learning creates a different space and dynamic for conversations to occur – a key practice in Developmental Evaluation.
First Nations peoples of Australia have evaluated outcomes, impacts and responsibilities since time immemorial through Indigenous systems of lore, knowledge and accountability to Country, kin and future generations. The Cultural Wisdom and Story Gathering Artefact (CWISGA) responds to this context by reframing monitoring and evaluation as practices of accountability, care, truth telling and improvement rather than extraction and surveillance. The objective of this presentation is to introduce CWISGA and show why culturally responsive evaluation that begins with relationships and engagement is essential for better outcomes across the sector.
CWISGA provides an accessible framework that operationalises culturally responsive evaluation through clear principles aligned with the four Rs of reconciliation, respect, reciprocity and responsibility, and with the interconnected wisdoms of Knowing, Doing and Being.
Three key messages guide the work: embed cultural governance from the outset rather than as an afterthought, uphold Indigenous Data Sovereignty and governance, and interpret outcomes through holistic wellbeing and relational accountability.
The session will open with a concise framing of the developmental context in a national organisation that supports equity focused curriculum in schools, followed by a guided walk through the CWISGA principles and a brief case example. Interactivity will be promoted through a short yarning prompt and small group reflection on local application of CWISGA, followed by commitments to action to support translation into practice.
Author: Stefano Verrelli (The Salvation Army) How do you build a national outcomes framework without flattening local realities, sidelining frontline practice wisdom, or reducing lived experience to an input rather than a shaping influence?
This presentation shares insights from an outcomes and impact evaluation of one of Australia’s largest homelessness service providers, spanning more than 100 programs, 700 practitioners, and around 40,000 clients annually. The evaluation aimed to develop and pilot a nationally relevant outcomes measurement framework before broader rollout, one that could work across diverse service models, jurisdictions, funding contexts, and client groups.
The challenge was not only technical, but ethical. A standardised framework risked privileging some perspectives over others, adding burden to already stretched services, and embedding measures that did not reflect frontline service realities or add value to people accessing support.
This presentation argues that, in large-scale evaluation, a staged and deliberate collaborative process across design, piloting, and refinement is a core ethical strategy. Using this case example, it shows how this approach made space for perspectives not always given meaningful influence in shaping outcomes evaluation at this scale, including frontline practitioners, practice leads, and people with lived and living experience. In doing so, it helped ensure that decisions about what outcomes mattered, how they were measured, and how the framework would work in practice were shaped by frontline realities and lived experience alongside competing system priorities.
The presentation offers a practical lesson for evaluators working across multiple sites and systems: ethical evaluation in practice depends on how frameworks are collaboratively developed, tested, and refined before implementation. The session will conclude with brief guided reflection questions to help attendees consider implications for their own evaluation practice.
I am a researcher and evaluator in The Salvation Army's research and outcomes measurement team. I care deeply about using rigorous, inclusive, and accessible research methods to address social justice issues. I earned my PhD in experimental social psychology from The University of Sydney in 2019 and have over a decade of research experience in the field of applied behavioural science. In previous roles, my work primarily focused on understanding the causes and consequences of prejudice... Read More →
Wednesday September 16, 2026 4:00pm - 4:30pm ACST Waterfront 1Stokes Hill Rd, Darwin City NT, Australia
Authors: Ian Falk (Mission Australia), Paul Royce (Mission Australia)
Evaluation can often be treated as a technical process, shaped by existing evaluative ‘types’, such as formative, summative, realist evaluation among others, and often focused on program performance and measurable outcomes. In complex, place-based contexts such as youth homelessness in Australia’s Northern Territory (NT), however, evaluation needs to be much more than this – it becomes a contested and adaptive practice shaped by existing evidence, culture, geography and structural constraint. This paper presents emerging results from a layered multisite, multimethod evaluative scoping study examining the accommodation needs of young people aged 8–24 who are at risk of, or experiencing, homelessness, where First Nations young people comprise around 90% of those impacted. The place-based, First Nations-led consultation process functioned as a responsive, continually adjusted evaluative design. Rather than applying a single evaluation lens, the project adopted a layered evaluation model integrating the evidence base, relational consultation, iterative and adaptive methodological design, program case study analysis, and system-level mapping. Methods included multi-media narrative engagement in remote communities, semi-structured interviews with urban stakeholders, and for both urban and regional/remote sites, iterative reflection on emerging methodologies and insights. This approach revealed that accommodation pathways are shaped less by individual behaviour than by structural misalignment between Western housing systems and young people’s lived realities, including kinship, mobility and connection to Country. In addition, the project established evidence-based criteria for ‘successful’ programs and services, which informed the program and service mapping process. In terms of the evaluation methodology, the paper found that layered and iterative place-based evaluation provides a more practical, useful and credible framework for understanding youth homelessness in complex, diverse and remote settings, enabling evaluation to move beyond judgement toward explanation, learning and culturally grounded, co-designed place-based system design.
Author: Doyen Radcliffe, Bajja Yungayimanha Collaboration Indigenous-led systems change often begins with shifts in cultural authority, relationships, trust and institutional behaviour—long before formal outcomes or policy reform become visible. Yet many evaluation approaches continue to prioritise measurable outputs and short-term results, overlooking the relational, cultural and ethical foundations that drive meaningful change. This creates a gap between what First Nations communities experience as impact and what institutions recognise as evidence.
This presentation introduces the Danjoo Koorliny Five-Dimensional Indigenous Influence Model, developed through the Aboriginal-led impact evaluation of the Danjoo Koorliny movement in Western Australia. The model offers a practical and culturally grounded way to measure systems change by identifying and assessing influence across cultural, relational, behavioural, structural and transformational domains. Central to this work is a decolonising evaluation process that repositions cultural authority, Country and relational accountability as the foundation of ethical and rigorous evaluation. For First Nations evaluation, this means privileging Elder governance, community decision-making, Indigenous Data Sovereignty and culturally grounded methods such as yarning, storywork and outcome harvesting. For evaluation more broadly, it expands how evidence, value and impact are understood in complex systems and supports earlier identification of meaningful change.
The session will highlight three key messages. First, influence is the primary mechanism and measurable pathway of Indigenous-led systems change. Second, decolonising evaluation strengthens methodological rigour by integrating Indigenous and Western approaches rather than positioning them in opposition. Third, valuing place, relationships and cultural integrity improves the relevance, integrity and usefulness of evaluation for communities, policymakers and funders.
The presentation will combine short input, reflective discussion and interactive activities. Participants will explore practical tools, apply the framework to their own contexts, and reflect on how measuring influence can strengthen systems thinking and evaluation practice across diverse sectors.
CEO & AES Fellow, Bajja Yungayimanha Collaboration
Doyen Radcliffe is a Yamatji Naaguja man from the Midwest Region of Western Australia. Doyen is a community minded individual with a passion for empowering Indigenous communities to reach their real potential to improve quality of life, health, social and economic wellbeing, and inclusion... Read More →
Wednesday September 16, 2026 4:00pm - 4:30pm ACST Waterfront 3Stokes Hill Rd, Darwin City NT, Australia
Authors: David Stuart (Creative Australia) Strategies are everywhere involving all sorts of topics and all kinds of goals, actions and stakeholders. Strategy defines important goals and outlines how an organisation's activities will meet those goals. You would think that strategy and evaluation were great friends. But why is strategy evaluation so low profile in evaluation literature, and so hard to pull off? This paper reflects on several attempts at strategy-based evaluation and the lessons for fruitful strategy evaluation including the role for evaluators and evaluation in supporting strong strategy design and success.
