Evidence use in education policymaking: Key findings

There is a disconnect between education research and policy

Findings from the desktop review include:

• Education research is often seen to be disconnected from policymaker priorities and needs. For example, research may not be seen to address pressing issues for policymakers or clearly outline how findings could translate into policy change.  
• Collaboration between researchers and policymakers can assist with evidence use in policymaking.

All senior policymakers interviewed believed a strong evidence base was foundational for effective policy. However, most policymakers reported that: 

• Research available to them generally did not reflect system priorities or illuminate the decisions they needed to make, the practical options available, or the sequence of actions required.
• Relationships with researchers can be useful, but only if they shed light on how research could best be translated into policy, are productive and positive, and offer a viable way forward on key issues facing systems and sectors.

Evidence is not the only factor affecting education policy decision-making

Findings from the desktop review include:

• Policymaking is often not linear, and there is no specific stage of policymaking where evidence is more or less likely to be used.
• Factors such as political pressures and time imperatives affect decision-making.

Similarly, most policymakers reported that:

• It is important to understand the competing imperatives for decision-makers developing policy, including the political realities, obstacles, opportunities and costs.
• Priorities or plans of education stakeholders may not reflect the evidence-base.
• A strong, authoritative, even contrarian voice is needed to assert the strength of the evidence base in some instances.

Deep expertise in education and research is needed

Findings from the desktop review include:

• Specific expertise is needed to delineate issues and draw on rigorous evidence to develop effective policy responses. For example:
  • policymakers who are subject experts in education will likely use evidence differently from generalists working in education policy for the first time.
  • it is important that policymakers understand what makes quality research and what evidence to ask for depending on the situation.
  • it is important for researchers to consider how the certainty and context of research findings might be interpreted or misinterpreted.

Similarly, most policymakers reported that:

• Within education departments, expertise in education practice and learning domains is important for identifying problems in detail and understanding the research literature.
• Policymakers want to feel more confident that the curricular, teaching and learning, and assessment issues in Australian classrooms are known.
• they want greater consensus in the evidence base to reflect the quality of research evidence on particular questions.

Practical actions

From these themes (and from more detailed project findings), we can draw practical implications for improving evidence-based education policymaking. This table sets out actions for researchers, policymakers and AERO.

To support more effective use of evidence in policymaking:

Build stronger relationships between policymakers and those who generate evidence

Actions for researchers

• engage in more productive conversations (publicly and with the education community) that offer a viable way forward on key issues facing systems and sectors
• actively involve policymakers in research, where relevant and possible

Actions for policymakers

• engage in more productive conversations (publicly and with the research community) that offer a viable way forward on key issues facing systems and sectors
• be actively involved in research, where relevant and possible

Actions for AERO

• continue to consult with education departments and sectors about their research interests and policy priorities, to ensure AERO’s work is relevant and useful
• continue to engage with the research community about existing projects and future possibilities, to ensure AERO’s work draws on relevant expertise and supports or extends relevant existing research
• continue to build trust, credibility and provide a national authoritative voice on education research and evidence

Generate more relevant evidence

Actions for researchers

• consider how research can best address system priorities and address key issues facing ECEC services and schools
• ensure research addresses specific issues and goes beyond high-level, already well-known findings such as ‘teaching quality matters’

Actions for policymakers

• ensure system priorities are clearly communicated and identify the key, specific issues facing ECEC services and schools

Actions for AERO

• assist with identifying the key issues facing ECEC services and schools, and make this information publicly available
• generate research that reflects the key issues in ECEC services and schools and addresses system priorities

Carefully examine and clearly communicate the quality of evidence on specific questions

Actions for researchers

• use rigorous evidence synthesis methodologies to present clear recommendations reflecting the quality of evidence on specific questions or topics
• clearly translate certainty in research findings – for example, rankings of effect size alone are potentially misleading

Actions for policymakers

• make evidence-based decisions in the best way possible by ensuring there is deep expertise in education practice, learning domains and research methodology available at a range of levels within the organisation
• go beyond high-level research findings to seek rigorous evidence that answers specific questions

Actions for AERO

• use rigorous evidence synthesis methodologies to present clear recommendations reflecting the quality of evidence on specific questions or topics
• develop additional services and resources for policymakers, as part of AERO’s implementation strategy
• acknowledge context and any known constraints in AERO advice

Explicitly consider how to translate evidence findings into meaningful policy

Actions for researchers

• explicitly address how research findings could translate to meaningful policy, including by considering questions of sequencing and prioritisation and what options could exist for policymakers at specific decision-points.

Actions for policymakers

• clarify issues faced by policymakers so that researchers can better guide policymakers to make decisions — for example, discuss questions of sequencing and prioritisation or outline the sequence of steps required to make good decisions.

Actions for AERO

• develop additional services and resources for policymakers, as part of AERO’s implementation strategy.

Background

Relatively little is understood about how evidence does and should inform education policy development in Australia, and what are the best ways to maximise its impact.

To explore these issues, AERO commissioned Learning First to:

• complete a desktop review of existing international literature on the use of evidence in education policymaking, and barriers and enablers to the use of evidence
• interview senior policymakers from across Australia about how evidence is used in education policy development in Australia, barriers and enablers to the use of evidence, and how the use of evidence in policymaking can and should be improved. Most interviewees were former system leaders who had responsibility for school education and/or early childhood education and care (ECEC) over the last decade.

‘Evidence’ was defined in this project to include both research evidence (that is, academic research usually published as books, reports, articles, summaries or podcasts) and sector-generated evidence (which includes data and information drawn from national or state assessments or collected from ECEC services, schools and related sectors). However, most of the literature and interviews focused on research evidence.

Keywords: educational leadership, evidence-based practice