Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/aes40246/public_html/2014/libraries/rokcommon/RokCommon/Service/ContainerImpl.php on line 464
Monday pre-conference workshops

Monday pre-conference workshops

Monday 8 September 2014

A printable version (PDF) of the pre-conference workshop program can be downloaded here.
View Tuesday program here.

Click on the workshop titles to view the workshop description.

9am –12:30pm

Real-Time Evaluation, Theory and Practice (half day)

John Owen

> register


'Evaluation by us, for us': What is required of AES to strengthen, advance and support Indigenous Evaluation? (full day)

Amohia Boulton

This email address is being protected from spambots. You need JavaScript enabled to view it.

Quantifying the impact of a non-randomised intervention: Statistical methods for non-statisticians
(full day)

Jim Gaetjens

> register


12:30pm LUNCH
1:30 – 5pm

Quality in Qualitative Research and Evaluation - what do we need to know?
(half day)

> register

Karen Kellard

[Amohia Boulton


[Jim Gaetjens


Real-Time Evaluation, Theory and Practice HALF-DAY (am)

presented by John Owen; Centre for Program Evaluation; The University of Melbourne

The purpose of this workshop is to equip participants to undertake evaluation work that requires the use of real time or rapid response techniques. The objectives are enable participants to:

  • understand the principles underlying the real-time evaluation paradigm
  • practise the application of real-time methods in a simulated situation
  • consider ways in which they could transfer their understandings to their own practice.

The workshop is based on a translation of recent research underlying real-time evaluation practice, and addresses skills not typically taught in evaluation training programs.
These include; the ability to work in uncertain, ambiguous and complex situations, use of high level negotiation skills, understanding the need a for the pragmatic; flexibility in order to respond quickly to requests, and use of innovative communication and listening skills.

The workshop is targeted at participants who have some evaluation experience and wish to extend their evaluation repertoire. The workshop will be based on a simulated case example, and participants will work in groups on a set of scenarios related to the case. In addition to information acquired during the workshop, participant will be provided with take-away workshop notes and readings to consolidate designed to consolidate their knowledge.

> back to overview  > register

“Evaluation by us, for us”: What is required of AES to strengthen, advance and support Indigenous Evaluation? – A workshop for Indigenous participants only (no charge)

presented by Amohia Boulton; Whakauae Research for Māori Health and Development; New Zealand

The AES Constitution currently makes no mention of the unique place Indigenous peoples have in the make-up of societies in and around the Pacific, including Australia and New Zealand. Indigenous evaluation, as the ontological and epistemological expression of the lived reality of Indigenous peoples, as theory and as practice, is however, increasingly being recognised as a legitimate discipline in its own right. Furthermore, Indigenous evaluation – evaluation undertaken by Indigenous peoples for Indigenous peoples – is being demanded by Indigenous communities who are often in receipt of services and programmes developed without their input or consultation. Indigenous evaluation is regarded by these communities therefore, as an emancipatory and transformative force.

Despite the constitutional “silence” on the issue of Indigenous peoples, the AES Board is keen to advance and support the field of Indigenous evaluation as appropriate, and seeks guidance on how to do this from Indigenous participants at the 2014 conference. In this facilitated workshop for Indigenous participants only, workshop attendees will be asked to identify the key issues in Indigenous Evaluation in our wider Pacific region; how the AES can best support the growth and advancement of Indigenous Evaluation in our region; and how the AES can best support the growth and development of Indigenous members of the Society.

NOTE: This workshop is for Indigenous participants only. There is no charge. Please register your interest in securing a place by emailing This email address is being protected from spambots. You need JavaScript enabled to view it. with the subject line 'Expression of interest in attending the “Evaluation by us, for us” workshop'.

> back to overview  This email address is being protected from spambots. You need JavaScript enabled to view it.

Quantifying the impact of a non-randomised intervention: Statistical methods for non-statisticians

presented by Jim Gaetjens; WorkCover SA; Adelaide, South Australia

This workshop is designed for evaluators with reasonably good quantitative skills (perhaps a “quantitative bent”) but who certainly are not statisticians. It introduces a methodology which is a powerful tool for impact evaluation in situations where

  1. large amounts of data are available for both participants and non-participants in a program;
  2. the intervention is clearly defined as received or not received (i.e. ignores dosage effects); and
  3. the intervention was not randomly assigned.

Although good impact evaluation designs will yield useful information, there may exist data which require solid quantitative methods to exploit their full potential. Due to budget constraints or other reasons, employing a statistician may not be an option – or even if it is, the evaluator may prefer to explore the data themselves at least initially.

The workshop provides hands-on training to better equip the non-statistician in this task. It explains how to produce “propensity scores” to reduce the differences between intervention and non-intervention groups before comparing their outcomes. Conceptually the method seeks to match the two groups on characteristics (such as age, gender, occupation etc.); however propensity scores have definite advantages over other matching techniques, especially when there are many non-intervention records and many matching characteristics available. A quick literature review may lead to the conclusion that the methodology is exclusively the domain of statisticians. However, the truth is that many of the usual statistical difficulties do not apply, and the non-statistician is able to produce meaningful and credible results.

Participation in the workshop requires a laptop (or the willingness look over someone’s shoulder) with Excel installed. Prior to the workshop a data file will be provided for download, and instructions given to set up Excel with the required “Add-in”. Participants should have a basic knowledge of Excel, and a basic knowledge of quantitative methods, but no other pre-requisites are required.

> back to overview  > register

Quality in Qualitative Research and Evaluation – what do we need to know? HALF DAY (pm)

presented by Karen Kellard; The Social Research Centre, Melbourne, Australia

The main aim is to bring issues of ‘quality’ in qualitative evaluation to the forefront of debate and discussion.

The notion of ‘quality’ in qualitative research methods used in evaluation is at best vague.  Researchers and evaluators have long debated the notion of quality – how it is defined and measured – and have demonstrated a shared commitment to rigour, robustness and relevance but without a solid agreement on what this means, and indeed to what extent such standards or measures can be formalised given the nature of qualitative enquiry.

This workshop will focus on:

  • Four central principles of qualitative research in evaluation (that it adds new knowledge or evidence, that the chosen research design is defensible, that the conduct is rigorous and that the findings are credible and clearly linked to the data)
  • Quality at different stages of the qualitative research evaluation process (design, sampling, recruitment, fieldwork, analysis and reporting) – what kind of ‘indicators’ of quality could we be looking at, how and why?  How does this differ in face-to-face, telephone and online qualitative settings?
  • Quality elements that run throughout the research and evaluation process (relating to, for example, research ethics, neutrality, reflexivity, skills and so forth).

The intended outcome is to prompt further thinking, dialogue and debate about how we define and judge quality in qualitative approaches and to begin to think about the acceptability of frameworks for assessment in this area. At an individual level, it will also encourage self-reflective practice to continue to raise the standard of qualitative practice, and to build knowledge, capacity and expertise.

The intended target group is those who use qualitative techniques in evaluation, those who commission qualitative evaluations and those who use evidence from qualitative evaluations.  Some knowledge of qualitative methodologies is therefore assumed as well as (ideally) some qualitative practice experience.

> back to overview  > register