Evidence use in schools: National snapshot

Introduction

In examining the use of evidence, we distinguish between:

• Research evidence: academic research, such as causal research or synthesis research, which uses rigorous methods to provide insights into educational practice.
• Teacher-generated evidence: Evidence generated by teachers through their daily practice (for example, teacher observations, information gained from formative or summative assessments or insights from student feedback on teacher practices).

Evidence makes a difference when it is rigorous, reliable and implemented well. As such, we also distinguish between:

• quality of evidence use: when evidence is engaged with thoughtfully, appropriately and implemented well
• using high-quality evidence: when evidence is rigorous and reliable, and teachers and leaders have the skills and confidence to assess rigour and reliability.

This snapshot is based on findings from our Evidence use survey, a rapid review of existing literature on evidence use and early findings from interviews with school teachers and leaders. Survey data includes responses from 932 teachers, middle leaders and principals. We discuss differences between teachers and leaders (middle leaders and principals) where these are relevant and statistically significant. It is important to note that survey responses often present a more optimistic view of the state of evidence use in schools than interviews or other techniques such as direct observation or analysis of school artefacts, like lesson plans.

Of the 22 studies included in the rapid review, most measured rates and quality of evidence use through self-report questionnaires. Specifically, 16 of the studies were larger-scale, quantitative studies that surveyed over 150 participants. All these studies, except for one, sourced participants from multiple (if not all) Australian States and Territories. The remaining 7 studies were small-scale qualitative or mixed studies that conducted interviews, surveys and/or focus groups with less than 100 participants and sourced participants from only one or two jurisdictions. As such, the findings from these studies may be context specific and limited by study methodology and sample size. The rapid review most often provided context and additional information to findings from our Evidence use survey. Where findings from the rapid review and the Evidence use survey were similar, we reported findings from our Evidence use survey as measures in this survey are comprehensive and specifically designed for the purpose of this snapshot. See Methodology for further information on the qualitative and quantitative data sources and analyses for this research.

Purpose

Better use of evidence lifts student outcomes, so it’s important to understand how evidence is being used. The purpose of this national snapshot is to:

1. Assess the current level of evidence use in schools across Australia. While there is scattered information about the use of evidence in schools, an overall assessment of evidence use at national level has not been completed to date. Knowing how evidence is used across Australia will:
   • help us improve student outcomes through effective use of evidence
   • provide baseline information (how and how much evidence is currently used) against which changes in evidence use over time can be described, monitored, tracked and compared.
2. Identify gaps in the way evidence use is described, measured, tracked and reported. While some measures of evidence use exist, we don’t know whether they are comprehensive enough to provide a detailed picture of the ways evidence is used. Identifying what we are currently measuring and what is missing will help us to develop ways to better describe, measure, track and report use of evidence and how evidence use contributes to improved student outcomes.

Why is this important?

Evidence use improves student outcomes

Embedding evidence in teaching practice improves student outcomes (ACER 2018). Data tells us that Australian students’ learning is stagnant or in decline (NAPLAN 2021; PISA 2018). While there are multiple causes for the deteriorating quality of student outcomes, teaching practice is an important avenue through which student outcomes can be improved.

Establish a baseline for evidence use in Australia

To understand progress in evidence use it is important to understand where Australia is now in terms of how and how much evidence is used in schools. This snapshot is the baseline against which future progress can be monitored.

Identify ways to measure evidence use and gaps in measurement and knowledge

We need to know what we don’t know. On the one hand, this snapshot presents the current state of evidence use, as assessed from a range of sources (surveys, interviews, rapid review of literature). On the other hand, it identifies gaps in the ways we currently describe, measure and report on use of evidence and evidence-based practices. The gaps identify areas where research efforts should be focused, for example to assess the quality of existing measures of evidence use and develop better measures.

Identify how schools and individual teachers and leaders can be supported to use evidence

This snapshot identifies enablers of evidence use at system-, school-, and individual-level. Ensuring these enablers are in place, and addressing challenges, will facilitate better use of evidence in schools.

Key findings

Use of evidence

School teachers and leaders more frequently report using evidence generated through their daily practice (such as findings from analysis of student learning data) than research evidence (such as the findings of academic research).

Our survey found that:

• 67% of respondents report using forms of teacher-generated evidence “often” or “very often”.
• 41% of respondents report using forms of research evidence “often” or “very often”.

Research evidence is more often used by leaders than teachers.

In the past year:

• 52% of leaders and 38% of teachers “often” or “very often” refined a practice based on recommendations from academic research.
• 56% of leaders and 34% of teachers “often” or “very often” referred to academic research in staff meetings when discussing instructional practices.
• 60% of leaders and 40% of teachers “often” or “very often” consulted academic research to improve their knowledge about the effectiveness of an instructional practice.

Analysing student learning data allows teachers to see the impact of any changes to their practice, making them more confident about recommendations from research on practices that work.

• 78% of survey respondents “agree” or “strongly agree” they can only be confident a practice works after they’ve trialled it with their students.

While a culture of sharing and discussing evidence within schools is emerging, many teachers do not report cultures of evidence use that are embedded and strong enough to the point where staff encourage each other to change their practice based on the evidence.

• 45% of survey respondents “agree” or “strongly agree” they will encourage colleagues to stop using an instructional practice if evidence from academic research shows it doesn’t work.
• 36% of survey respondents “agree” or “strongly agree” they will encourage colleagues to stop using an instructional practice if evidence they collected from their classroom shows it doesn’t work.

Enablers of evidence use 

Teachers and leaders identify professional learning activities as an effective source of research evidence. Professional learning takes many shapes, including formal training courses as well as opportunities for collaboration, and knowledge-exchange to build skills to assess quality, rigour and relevance of evidence.

• 64% of survey respondents report that at their school, coaching is available to help staff use evidence to change their practice.
• 73% of respondents report that at their school, they have set aside regular times or meetings to discuss evidence that could improve their practice.
• 66% indicate that their school system (Government, Catholic or Independent) provides easily accessible information, resources, training or other support to help them use evidence to inform their practice.
• 86% of respondents agree that they discuss evidence that could improve their practice with colleagues.

Professional learning activities represent enablers of evidence use and are provided by many schools and systems, but the extent to which activities are based on quality evidence, or the ways to ensure professional learning is effective in embedding evidence is not clear. Furthermore, professional learning needs to be targeted where support is needed, and to the staff who need it most.

• Confidence, particularly confidence to assess rigour and relevance of academic research, helps teachers and leaders to use findings from this research to focus on practices that are most likely to improve student outcomes.

Findings from the AERO evidence use survey suggest that teachers and leaders who are confident in their skills in assessing the quality and rigour of academic research, are around twice as likely to report that they “often” or “very often” consulted academic research to improve their knowledge about the effectiveness of an instructional practice.

• 54% of teachers and 81% of leaders who were confident in their skills reported consulting academic research “often” or “very often”.
• Only 29% of teachers and 37% of leaders who were not confident in their skills reported consulting academic research “often” or “very often”.

Leaders champion use of evidence to inform, discuss or change practice and lead school-wide change. This is often seen by teachers as an enabler of evidence use.

• 80% of teachers “agree” or “strongly agree” that leaders share and discuss evidence that could improve teacher practice.

It is to be noted, however, that compared to this relatively high proportion of teachers indicating their leaders encourage them to use evidence, a lower proportion report the availability of coaching or times to meet and discuss evidence (64% and 73% respectively).

Access to information and resources provided by schools or systems is less commonly considered to be an effective way to gather or use evidence.

• Only 38% of survey respondents identify instructional practices recommended by schools or school systems as effective ways of gathering or using evidence.

Time to engage with research, connect with peers and networks and engage in professional learning is an asset for teachers and leaders and an enabler of use of evidence in schools. However, teachers often identify lack of time as a barrier to engaging with evidence.

Terminology and gaps in tracking evidence use

• Teachers use the term “research evidence” to describe a wide variety of things, not all of which might be rigorous or relevant. When discussing research evidence, teachers sometimes refer to academic publications, other times “research evidence” is understood more broadly to include opinion pieces or pre-service training. In addition, teachers generally feel research is overly academic, not contextually relevant, and overall, not “fit-for-purpose”.
• Compared to findings from interviews, findings from surveys paint a more positive picture of evidence use. Surveys often do not inquire about the quality of use of evidence and may carry reporting biases, overstating use of evidence.
• Tracking of evidence use should also focus on quality of evidence use. There are 2 types of quality to consider in relation to evidence use:
  • quality use of evidence: when evidence is engaged with thoughtfully, appropriately and implemented well
  • using high-quality evidence: when evidence is rigorous and reliable, and teachers and leaders have the skills and confidence to assess rigour and reliability.

Before using findings of evidence appropriately, teachers and leaders need to be confident that the evidence they are using is rigorous and reliable. Findings from AERO’s evidence use survey indicate some concerns with use of quality evidence:

• 35% of teachers and leaders who trialled and refined their practice based on academic research are not confident to assess the quality or rigour of that evidence.
• 38% of teachers and leaders who consulted academic research to improve their knowledge about the effectiveness of an instructional practice are not confident to assess the quality or rigour of that evidence.

Student learning data and professional learning communities are commonly identified as effective ways of gathering and using evidence

Teachers and leaders sometimes have different understandings of what constitutes “evidence” or “evidence-based practice” (Gleeson et al. 2020b). Despite these differences, teachers and leaders commonly report that using student data to generate evidence (that is, teacher-generated evidence) and engaging in professional learning communities are good ways to learn about, gather and use evidence (Prendergast and Rickinson 2019; Mills et al. 2021; White 2021).

Similarly, in AERO’s evidence use survey, respondents identify the following activities as effective ways of gathering or using evidence (Figure 1):

• using student learning data to determine the effectiveness of a particular practice (78% of teachers and leaders)
• using student learning data to compare different practices to decide which is the most effective (72% of teachers and leaders)
• engaging in professional learning communities that discuss effective instructional practices (73% of teachers and leaders).

Figure 1: Effective sources of evidence, as identified by AERO evidence use survey

Notably, using instructional practices recommended by academic research, recommended by school systems, or learned during initial teacher education, are least often identified as effective ways to gather or use evidence (55%, 38% and 25% of survey respondents, respectively).

Some differences emerge between teachers and leaders. A higher proportion of leaders compared with teachers identify both the analysis of student data to determine the effectiveness of a practice (85% and 77% respectively) and using instructional practices recommended by academic research (68% and 52% respectively) as effective ways to gather and use evidence.

A range of difficulties in accessing and using academic research may contribute to academic research being considered a less effective source of evidence than teacher-generated evidence, and more so by teachers than by leaders. Examples of some of these difficulties include information not being in an easy to access format, language being predominantly academic and hard to understand, or lack of examples of how academic ideas apply to practice (Marsh 2015; White 2021; Prendergast and Rickinson 2019). Some of these challenges with using research evidence are overcome by accessing academic research via (most often external) professional learning activities (Parker et al. 2020; Prendergast and Rickinson 2019; White et al. 2021). It is therefore encouraging that teachers and leaders who answered AERO’s evidence use survey indicated that professional learning communities that discuss effective instructional practices are effective ways to learn more about practices that work (72% and 79% respectively). However, professional learning activities can take different forms and in addition to understanding frequency of engagement in such activities, the quality of evidence use during these learning activities should also be investigated.

Teacher-generated evidence is more commonly used than research evidence

Given that teachers and leaders more frequently identify teacher-generated evidence than research evidence as effective sources of evidence, it is not surprising that a large proportion of teachers and leaders use this type of evidence to also inform their practice (Q Project 2020b). Teachers are exposed to and use student learning data more frequently than research evidence. They often use formative assessment and collect student learning data to report on student performance throughout a school term. Hence, collecting, accessing and using the findings from this learning data is a frequent occurrence. One AERO interviewee explained how they use data they generate alongside research evidence:

‘Evidence that you collect yourself and [you notice that] learning is taking place is powerful because it’s exactly the situation that you’re talking about and exactly what’s going on. Sometimes it’s tough to go against that ... But I’ll take the research-based stuff and I’ll implement that in a broad way and I’ll try and bring those practices in, and then if they work cool, if it’s not working.. I’ll go with the thing that’s working in reality over the thing that’s not working in theory.’

On the other hand, using research evidence requires teachers to seek the evidence; this can be from published academic work or from summaries created by organisations like think tanks and “what works” agencies (Mills et al. 2021).

Teachers also seek evidence via external professional learning providers, or via other filtered, indirect, social ways, for example via social media and blogs, and conversations with colleagues (Rickinson et al. 2021a; Mills et al. 2021; Prendergast and Rickinson 2019). An interview respondent explained the need to navigate the range of research available:

‘There’s lots of research and theories out there, but you’ve just got to pick and choose which ones are going to be best for you as a teacher but as well for the students’.

The higher use of teacher-generated evidence than research evidence is reflected in survey findings:

• 67% of respondents report using forms of teacher-generated evidence “often” or “very often”:
  • 59% explicitly referenced student learning data when discussing instructional practices with colleagues
  • 60% consulted student learning data before planning a lesson
  • 73% changed the next lesson plan based on student learning data they collected
  • 77% used student learning data to evaluate the effectiveness of their practice.
• 41% of respondents report using forms of research evidence “often” or “very often”:
  • 41% consulted a document that summarises effective instructional practices when planning a lesson or unit
  • 42% trialled or refined an instructional practice based on recommendations from academic research
  • 39% referred to academic research in staff meetings when discussing instructional practices
  • 44% consulted academic research to improve their knowledge about the effectiveness of an instructional practice.

However, there are some differences between teachers’ and leaders’ use of evidence (Figure 2), particularly in terms of use of research evidence. A higher proportion of leaders than teachers report they “often” or “very often”:

• consulted a document that summarises effective instructional practices when planning a lesson (50% of leaders and 38% of teachers)
• trialled or refined an instructional practice based on recommendations from academic research (52% of leaders and 39% of teachers)
• referred to academic research in staff meetings when discussing instructional practices (56% of leaders and 34% of teachers)
• consulted academic research to improve their knowledge about the effectiveness of an instructional practice (60% of leaders and 40% of teachers).

Figure 2: Use of evidence, teachers and leaders, AERO evidence use survey

Use of research evidence by leaders is essential. Leaders can embed research evidence findings in school plans and goals, to achieve a whole-school approach to evidence use. Leaders also act as an enabler of evidence use for teachers. They may disperse relevant research to teachers and provide the environment to engage with research, further supporting teachers to use evidence to improve their practice. While leaders championing use of research evidence is generally seen as a positive activity, some studies have found this top-down approach can sometimes be problematic. It may be perceived that other staff are excluded from genuine participation in sourcing, analysing and interpreting evidence. In these cases, teachers feel their preferences and wisdom are disregarded (Jackson 2022; Rickinson et al. 2021a). It is then important that champions of evidence use play a support role, guiding others and improving their confidence to use evidence in their work.

Confidence increases the likelihood of using evidence

Confidence to assess the quality and rigour of academic research appears to increase the likelihood of academic research being used by both teachers and leaders (Figure 3). Being confident allows teachers and leaders to use evidence for a range of activities related to their role, such as planning lessons or to refine instructional practice. In our survey, the proportion of teachers and leaders who “often” or “very often” use academic research is twice as high if they are confident to assess the rigour and quality of academic research. For example:

• The percentage of leaders who “often” or “very often” consult academic research to improve their knowledge about the effectiveness of an instructional practice is:
  • 81% for leaders who are confident to assess the quality and rigour of academic research
  • 37% for leaders who are not confident to assess the quality and rigour of academic research.
• The percentage of teachers who “often” or “very often” consult a document that summarises effective instructional practices when planning a lesson or unit is:
  • 53% for teachers who are confident to assess the rigour and quality of academic
  • 26% for teachers who are not confident to assess the rigour and quality of academic research.

An AERO interviewee explained the step from identifying research evidence to implementing findings of research evidence:

‘If it’s relevant and easy to implement, I might implement it straight away. Other than that, I might file it for next unit next term. I might trial it on one particular class.’

Interestingly, teachers who are confident to assess the quality and rigour of research are more likely to use academic research than leaders who are not confident to assess quality and rigour. For example:

• 55% of teachers who are confident to assess the quality and rigour of academic research “often” or “very often” trial or refine instructional practices based on recommendations from academic research, compared to only 33% of leaders who are not confident to assess the quality and rigour of academic research.

On the other hand, a large gap emerges between leaders who are confident to assess the quality and rigour of academic research and teachers who are not confident to do so. For example:

• 74% of leaders who are confident to assess the quality and rigour of academic research will also refer to academic research in staff meetings when discussing instructional practice, compared to only 23% of teachers who are not confident to assess the quality and rigour of academic research.

Figure 3: Use of academic research by confidence to assess its rigour and quality, teachers and leaders, AERO evidence use survey

Notes: Confident: Mostly or very confident to assess the quality and rigour of academic research. Not confident: not at all confident, slightly or somewhat confident to assess the quality and rigour of academic research. Sample size: confident teachers=279; not confident teachers=355; confident leaders=83; not confident leaders=73. Results should be interpreted in the context of a relatively small sample size, especially for leaders.

Confidence to use teacher-generated evidence to assess the effectiveness of a practice also increases the use of this type of evidence for both teachers and leaders. However, there are very little differences between teachers and leaders (Figure 4).

Figure 4: Use of student learning data by confidence to use it to change practice, AERO Evidence use survey

Notes: Confident: very or mostly confident to assess the effectiveness of a practice based on student learning data; not at all, slightly confident or somewhat confident to assess the effectiveness of a practice based on student learning data. Sample size: confident teachers=434; not confident teachers=202; confident leaders=121; not confident leaders=36. Results should be interpreted in the context of a relatively small sample size, especially for leaders.

Quality is key: evidence makes a difference when it is rigorous, reliable and implemented well

The Quality Use of Research Evidence (QURE) framework (Rickinson et al. 2022) developed by the Monash Q Project emphasises the importance of quality use of research evidence, referring to evidence needing to be appropriate, and engagement and implementation to be thoughtful (Rickinson et al. 2021b). Observational studies have identified examples of differing levels of evidence use among Australian schools and educators (for example, Parker et al. 2020). Further, some teachers report using a practice, but they may be only using aspects of that practice (Parker et al. 2020). They report using a type of evidence but might be unsure about the quality of the evidence they are using. This highlights the importance of investigating the frequency of evidence use along with the quality of that evidence and how it is used. Two aspects of quality need to be understood when tracking evidence use:

• The quality of evidence use, referring to how evidence is used, for example is it engaged with thoughtfully, appropriately and implemented well?
• The quality of evidence, referring to whether the evidence is reliable and rigorous, and teachers and leaders have the skills and confidence to assess rigour and reliability.

Using poor-quality evidence or picking and choosing parts of the evidence to use can have unexpected outcomes, including making practice less effective.

Findings from AERO’s evidence use survey suggest use of quality evidence may be a concern:

• 35% of teachers and leaders who trialled and refined their practice based on academic research are not confident to assess the quality or rigour of that evidence.
• 38% of teachers and leaders who consulted academic research to improve their knowledge about the effectiveness of an instructional practice are not confident to assess the quality or rigour of that evidence.

Early findings from interviews with teachers and leaders revealed various levels of interpretation of what constitutes good quality of evidence use. For one teacher, high quality teacher-generated evidence refers to capacity to replicate student data, or that measures can be used with different cohorts:

‘If I can record it and if I can repeat it. So, if I can collect it across different classes or collect it in the same class but in different lessons.’

Another teacher explained they are more likely to perceive research evidence as high quality if it has been trialled by other teachers and demonstrated to work:

‘I think if another teacher has taken on that research and the evidence from it and implemented it into their classroom, and it’s worked for their group of students, that gives me the initiative and I guess the motivation to try it for my students. Because obviously, each and every student and group of students is completely different. But if I can see it’s working in their class, then it kind of motivates me to then try it into my class. And if it doesn’t work, it doesn’t work. Or, I have to change it. Or research into it a little bit more to find a full understanding as how I should do it better and for it to work.’

A teacher explained high quality evidence as data that is regularly and consistently collected:

‘I think that’s where you’re having a variety of assessment pieces. We use the PAT assessment at our school. And that I know is from infants right up to Year 6. So, that evidence is quite reliable, because you can actually do Box and Whisker charts and graphs. And you can compare and … you can see what’s our lower bands, what’s our higher bands. And it’s all Australia-based. So that is high-quality evidence, because you can compare and it helps us unpack and analyse that data.’

Several education research institutes have provided guidance on whether it is “safe” or appropriate to label a practice as “evidence-based” depending on the amount and quality of existing research on the practice. For example, the What Works Clearinghouse, a branch of the USA’s Institute of Education Sciences, reviews research on interventions in education and rates them on a three-point scale according to the strength and credibility of the intervention effects reported. Similarly, AERO has developed Standards of evidence and Evidence decision-making tools that aim to guide teachers and educator’s assessment of the confidence they should have that a particular practice will ‘work’ based on the amount, rigour and contextual relevance of research evidence available.

Student learning data is analysed to test if practices proposed by academic research work

Findings from our survey suggest that the analysis of student learning data is often used to test the applicability of academic research, or if “it works” in practice. Two-thirds (65%) of respondents report that research on effective practices confirms what they already know about what works in their classroom. This suggests they feel they are already implementing practices that are endorsed by the research they have come across. Despite this, their confidence in a practice is often dependent on the research they generate themselves, with over three quarters (78%) of survey respondents agreeing or strongly agreeing they can only be confident a practice works after they’ve trialled it with their students (Figure 5).

While a culture of sharing and discussing evidence within schools is emerging, many teachers do not report cultures of evidence use that are embedded and strong enough to the point where staff encourage each other to change their practice based on the evidence. Additionally, more respondents are likely to advise a colleague against an instructional practice based on academic research rather than data they collected in their classroom (45% and 36%, respectively). It is possible that academic research is the source of ideas that are further cemented by the analysis of student learning data.

A teacher explained that while they valued and trusted research evidence, data collected in their classrooms was a necessary validation that a practice works:

‘I feel like evidence from academic research, because of obviously the protocols around collecting that evidence, around the way that it’s gone about, and it’s measured and things like that that we can’t always control in schools. But in saying that, we need to really value the evidence that we collect too because the contextual element that it provides for engaging in that work. I feel like I do naturally trust more what’s out there, but you also know you can’t just pick up something and implement it in a school and expect the same outcome. So, I guess I trust slightly more that, but they really need to be happening together.’

Similarly, a teacher explained they would advise colleagues to use a practice recommended by research evidence only after having tested it in their classroom:

‘If it did work, and then the research says it works and I found that it worked, that’s usually when I go and tell the other teachers, ‘I’m doing this and it worked really well. It’s time consuming, but it’s getting really good results.’ Or something like that. I think when you have both of them aligned; the research and my own evidence.'

Interestingly, some teachers and leaders seem to rely on teacher-generated evidence without needing to understand if research evidence supports a practice:

‘If it’s a teacher who’s experienced and who I trust and it’s working for them I may not always go towards the evidence, I may sometimes take their word for it. It depends on the teacher and the skill, and whether it was a complex thing or whether it was an easy thing. Those things would change my decision.’

‘Evidence that you collect yourself and learning is taking place is powerful because it’s exactly the situation that you’re talking about and exactly what’s going on. Sometimes it’s tough to go against that.’

Figure 5: Beliefs about evidence use, AERO evidence use survey

Not surprisingly, a higher proportion of leaders compared to teachers are likely to encourage a colleague to change their practice, be it based on academic research (65% of leaders and 39% of teachers) or on teacher-generated evidence (44% of leaders and 33% teachers). Confidence in assessing the rigour and quality of academic research plays an important role in whether both teachers and leaders would recommend against a practice if research evidence showed it did not work (Figure 6):

• 54% of respondents who are confident to assess the rigour and quality of academic research are also likely to encourage their colleagues against a practice if academic research demonstrates it is not effective. A large gap emerges between leaders and teachers in this category, with 75% of leaders and 47% of teachers who are confident to assess the rigour and quality of academic research likely to encourage against a practice based on academic research.
• 37% of respondents who are not confident to assess the rigour and quality of research are also likely to encourage their colleagues against a practice if research evidence demonstrates it is not effective. Even if they are not confident to assess the rigour and quality of research, 33% of teachers and 54% of leaders would still advise against a practice if research showed it didn’t work.

To avoid the use of recommendations from non-rigorous research it is hence important that both teachers and leaders can assess the rigour and quality of research evidence.

Figure 6: Encouraging colleagues to stop using a practice by confidence to use evidence, AERO Evidence use survey

Support at system- and school-level enables evidence use

Enablers of evidence use

Enablers of evidence use are elements that support or facilitate engagement with evidence. A range of enablers of evidence use in Australian education were identified from published studies (for example, White 2021; Parker et al. 2020; Rickinson et al. 2021a; Jackson 2022; Mills et al. 2021; Q Project 2021b), the AERO evidence use survey and from early findings from interviews with teachers.

School-level enablers

In schools, support refers to both organisational structures and workplace “culture”. Our survey found a positive association between teachers’ use of evidence, particularly teacher-generated evidence, and evidence being regularly used at school level, for example in deciding on school policies. A teacher explained the importance of a learning culture:

‘I feel quite supported here. I know that my principal is very much about research informed practices and being research led. And the fact that we’ve got a learning culture leader and not a learning and teaching leader says a lot, so I feel supported here if I want to lead with anything about research.’

However, while the school-level culture of sharing seems to be emerging, many teachers do not report a culture of evidence use that is embedded and strong enough for staff to encourage each other to change their practice based on evidence.

School-level enablers include:

• embedding evidence in a whole-of-school approach, for example by establishing dedicated roles to assist with research implementation, and explicitly including research use in the school improvement plan (Rickinson et al. 2021b)

A leader explained the importance of dedicated roles to facilitate use of research evidence across the school:

‘We have really very clear practices at our school about what we call facilitated planning. So, my learning culture leader, who is my research lead, she is part of facilitated planning with the literacy leader, and then that team that they’re working with, which is the team of teachers. So, at those facilitated planning meetings, we bring the research to the table, so that you are enabling our teachers to engage in research.’

A teacher emphasised the importance of a school culture open to learning:

‘I haven’t got a heap of experience but in some schools you go there and as you do professional development stuff people are like, ‘I already know this’ or, ‘This doesn’t work’ or those negative comments, they can cause the non-adoption of those practices. But I think having a good environment and a good culture that wants to learn those things and wants to try and implement at least some of them is probably a good thing.’

• leaders’ championing evidence use, for example by using evidence to guide school policies and programs and encouraging staff to implement evidence in their practice (Q Project 2021a; Rickinson et al. 2021a, 2021b)

Teachers appreciate leaders engaging with research evidence and sharing evidence with the team:

‘So my principal sits on a research committee that has a range of representatives […] principals, directors, researchers themselves. So, I’m really lucky in that space that she is able to share what’s occurring at a system level for me as a support. I know that the system now put out – they call them research bursts, so they put out information and share that to schools around what’s happening in the research space, what to look out for, articles to look at, or projects, or however it is.’

• facilitating opportunities for collaboration and knowledge exchange within and across schools by establishing professional learning networks (White 2021; Parker et al. 2020)

‘Professional learning team meetings. We always engage in research in the professional learning teams, and we have those meetings twice a week.

• ensuring professional development opportunities are available to foster staff evidence use capabilities and give step-by-step assistance with evidence implementation.

Many teachers and leaders who answered our survey report school-level enablers are in place at their school:

• 80% of teachers feel their leaders share and discuss evidence that could improve their practice.
• 75% of respondents indicate that “often” or “very often” at their school they use evidence when deciding on school policies and program.

See also Evidence use in schools – how do teachers say they are supported?

For information on the relationship between school-level support and evidence use, see also Supporting teachers to use research evidence well in practice.

However, further investigation is necessary to understand what happens past this support and encouragement, whether evidence is used where it is needed most.

Individual-level enablers

At the individual level, factors that appear to make teachers more likely to use evidence include:

• receiving peer support by sharing and discussing evidence with colleagues
• holding a postgraduate degree, particularly a research-based one, for example, a Master of Research or a PhD (Q Project 2020a)
• feeling confident with collecting and analysing research evidence and teacher-generated evidence and knowing how to translate findings to their classroom (Q Project 2021a; Parker et al. 2020; Rickinson et al. 2021a)
• believing that evidence use facilitates professional growth (Rickinson et al. 2021b)
• holding a formal leadership position in their school (Rickinson et al. 2021a).

Encouragingly, findings from our survey suggest that teachers often experience a good level of support from colleagues and leaders:

• 85% of teachers “agree” or “strongly agree” that colleagues discuss evidence that could improve their practice.
• 83% of teachers “agree” or “strongly agree” that leaders encourage teachers to use student learning data (a form of teacher-generated evidence) to determine whether a practice works.

System-level enablers

Enablers at a systems level include:

• clear definitions of what constitutes “evidence” from trusted organisations (Parker et al. 2020)
• funding and incentives from jurisdictions to develop teachers’ evidence use skills (White et al. 2021)
• including ‘engagement with evidence’ within the professional standards, for example the expectations of teachers as research consumers, or emphasis on building research into professional learning development (White 2021)
• providing resources from trusted organisations, such as:
  • comprehensive and accessible syntheses of research evidence (for example, see AERO’s practice guides with annotated reference lists)
  • practical, step-by-step guidance on how to implement evidence-based practices (for example, see AERO’s Implementation checklists)
  • concrete examples of what practices look like in the classroom (see AERO’s case studies and rubrics for teachers.)
  • ‘one-stop’ research banks or repositories that are curated and catalogued (so teachers and leaders have less searching and sifting to do). See, for example, AERO’s collection or the Education Hub in New Zealand. 

About two thirds of respondents to our survey indicate that their school system (Government, Catholic or Independent) provides easily accessible information, resources, training or other support to help them use evidence to inform their practice (66%).

School systems and sectors commonly use system-level strategic priorities to support and increase evidence use. This system-level support is frequently extended to the school level across the system through school improvement initiatives and school leadership.

Confidence to assess the quality of evidence is a critical enabler of evidence use

As presented in previous sections of this report, confidence to use research is a key factor for whether research is used to change instructional practice. School-based support plays an important role in increasing confidence to use evidence, and indirectly, the use of evidence and evidence-based practices. Respondents to the AERO Evidence use survey feel more confident to use student learning data than academic research to inform their practice. Further, half or less are confident to assess the quality of academic research or whether the academic research is relevant to their context. Differences emerge between teachers and leaders (Figure 7):

• 44% of teachers and 53% of leaders are confident to assess the quality of academic research.
• 56% of teachers and 70% of leaders are confident to assess the relevance of academic research to own context.
• 53% of teachers and 64% of leaders are confident to use academic research to inform and change their practice.
• 68% of teachers and 77% of leaders are confident to use student learning data to inform and change their practice.

Confidence to refine practice based on recommendations from academic research corresponds with confidence to determine whether academic research is rigorous or of high quality and whether the research is relevant or applicable to one’s own situation or context. Of respondents who are confident to change or refine their practice based on academic research:

• 68% are also confident to determine the quality and rigour of academic research.
• 83% are also confident to determine whether academic research is relevant or applicable to their situation or context.

This is an indication that to improve confidence to change practice based on academic research, it is important that teachers and leaders can assess the quality and rigour of academic research and whether that research is relevant or applicable to their situation or context. To support teachers and leaders, AERO has developed a range of resources:

• Standards of evidence – for making consistent and transparent judgements when assessing evidence
• Evidence decision-making tools – to help make decisions about using a program or practice
• Research reflection guide – for reflecting on a piece of research
• Rubrics – for reflecting on and improving specific evidence-based practices.

Professional learning is a doorway to evidence

Teachers and leaders report a common barrier to using evidence effectively is that staff lack the necessary skills, such as the ability to sift through academic search engines to find quality, relevant research evidence and the ability to collect and analyse student learning data (Marsh 2015; White 2021; Parker et al. 2020; Louden 2019; Rickinson et al. 2021a; Mills et al. 2021; Prendergast and Rickinson 2019). While lack of access to research is an issue, the skills to sift through research once it is accessed and the skills to identify quality research are a further barrier. Professional learning, administered by external providers or by internal school staff like middle and school leaders, can help address these skill gaps (Gleeson et al. 2021; Q Project 2021b; White 2021; Parker et al. 2020; Maher and Prescott 2017; White et al. 2021; Jackson 2022; Rickinson et al. 2021a). Aside from instructing teachers how to source research evidence or collect their own data, professional learning is also commonly described as a source of current research – a way that time-poor teachers can get a summary of the most important takeaways from a trusted “expert”. Professional learning is also regarded as useful for adapting ideas from research into classroom practice and ensuring the changes to practice are feasible and adjusted to local needs. A primary school teacher explained:

‘The reason why I am changing my practice quite a bit lately is because I have the support, and has been that (…) support from the regional office (…), support for sites and workers and teachers one-on-one in their practice; having those people that are third party that know the direction, that have read a lot of stuff, in a job that can support you has been really beneficial’

While professional learning is important, the quality of professional learning activities matters. Mills et al. (2021) found that 46% of Australian school practitioners surveyed didn’t receive good quality professional learning on engaging with education research and, similarly, 40% said they didn’t get good quality professional learning on how to collect and interpret their own data.

Cost of training activities is also a barrier to engaging in professional learning activities, for example, a teacher explained:

‘Having that professional development, we also can find our own professional development and we’re encouraged to. But not too much because then that costs the school money'

Professional learning communities are seen by teachers as an opportunity to identify, access and assess research evidence, and, importantly, to plan how it can be used to improve their practice and student outcomes.

Findings from our survey support the use of professional learning and skill building (by allowing time to discuss evidence) as an enabler to evidence use and indicate that use of teacher-generated evidence more often than research evidence is directly linked with:

• having coaching available to help staff use evidence to change their practice
• setting aside regular times or meetings to discuss evidence that could improve practice.

Encouragingly, many survey respondents report the availability of coaching and meeting times to discuss evidence in their schools:

• 64% of respondents report that at their school, coaching is available to help staff use evidence to change their practice.
• 73% of respondents report that at their school, they have set aside regular times or meetings to discuss evidence that could improve their practice.
• 86% of respondents report that they discuss with their colleagues evidence that could improve their practice.

However, the proportion of respondents reporting availability of coaching and time for meetings to discuss evidence (64 to 73%) is well below that of respondents reporting encouragement by their leaders to use evidence (80 to 83%). This may indicate that while leaders are aware of the importance of evidence use, and of their role to encourage evidence use, further supports need to be in place for teachers to be able to access and use evidence to improve their practice.

Barriers to using evidence

Barriers to evidence use can be thought of, in simple terms, as the absence of any of the enablers described above. Additional barriers, somewhat distinct to what has already been discussed, are described below.

Time

A lack of time is one of the most commonly cited barriers to the use of both research evidence and teacher-generated evidence (Marsh 2015; White 2021; Parker et al. 2020; Q Project 2021b; Maher and Prescott 2017; White et al. 2021; Mills et al. 2021; Jackson 2022; Prendergast and Rickinson 2019; Rickinson et al. 2021a).A recent survey found that half of teachers and leaders don’t have enough time to source and properly evaluate research (Rickinson et al. 2021a). This figure is unsurprising given Australian teachers have one of the highest teaching loads among OECD (Organisation for Economic Co-operation and Development) countries (Teaching hours (indicator), OECD 2018), and, thus, there is likely relatively little time leftover to search for and consult research. A burdensome teaching load may partially explain why classroom teachers are more likely to say they “don’t have enough time” to use evidence compared with school leaders (84% vs 61%) (Q Project 2021b). In a rural and remote context, a lack of time may prohibit access to face-to-face professional learning opportunities that are mostly metropolitan-based and, thus, would require hours or days-long commutes. A related barrier is that there is a smaller pool of casual relief teachers in rural and remote settings that could cover teaching staff during their off-site professional learning activities (Maher and Prescott 2017). Encouragingly, there is preliminary evidence that online professional learning programs, particularly those involving video, are a viable alternative to face-to-face professional learning programs (Education Endowment Foundation 2020). Under three quarters (73%) of respondents to our survey “agree” or “strongly agree” that at their school they have set aside regular times or meetings to discuss evidence that could improve their practice.

Nevertheless, being time poor is often a barrier to using evidence, acknowledged by teachers and leaders, as someone said:

‘I love it when I get a chance to look at - which isn’t very often, the latest trends in education and what is best practice. I haven’t even had a chance to update that recently, except for our professional development. We don’t get time to learn. I understand why some teachers don’t use practices, because they probably haven’t even had a chance to look at them or know what they are.’ ‘But when you’re teaching - we have a five-period day. When most of your days are four periods a day. And you spend the one period trying to plan or mark or catch up with students who have missed work. And there’s not really much time for learning and growing as educators, which is sad. It’s only that I have smaller classes, I have a little bit more time to try things and a bit more space for it. But time is going to be the big one.’

Someone else said:

‘We have teachers that are away left, right and centre. We can’t staff particular classes. Our profession at the moment is like 45% down on what it was five years ago. Even our really good teachers that are still here are struggling, because if we can’t get relief for a class - experienced teachers are taking two classes. So, it’s not sustainable but it’s a very big barrier for bringing in any sort of change. I feel like the profession is just coping at the moment.’

Change fatigue

Some studies identified “change fatigue” resulting from frequent policy changes at a system level as a challenge for teachers and leaders (Parker et al. 2020; White et al. 2021). If schools must adapt to frequent policy changes this may mean they have less time and energy leftover to dedicate to implementing research findings that could benefit education in the long term (White et al. 2021). In addition, a barrier cited, particularly in special education contexts, was the recommendation of practices that are not evidence-based and may not work in the respective contexts.

Approach to the dissemination and use of evidence

Some studies suggest that teachers feel there is a top-down approach in their schools; while leaders champion and facilitate evidence use at school level (which is an enabler of evidence use), sometimes other staff feel excluded from genuine participation in sourcing, analysis and interpretation of evidence. This top-down dissemination of knowledge is particularly clear when it comes to NAPLAN use: school leaders often tend to be solely responsible for analysing NAPLAN data and then sharing key takeaways with teaching staff. The extent to which teaching staff are precluded from directly learning from NAPLAN data depends on who you ask; while 48% of school leaders indicated all teaching staff could access NAPLAN data, only 9% of teachers said this was the case (Jackson 2022). Interviews with Australian teachers demonstrate that a consequence of top-down approaches to evidence use is that teachers may feel their preferences and wisdom are disregarded (Jackson 2022; Wescott 2021). It appears that in schools, more informal types of teacher-generated evidence, like teachers’ day-to-day observation of students, are often seen as less valuable than other types of evidence, like standardised achievement scores. Reflecting this, more often than leaders, teachers feel that their observations and experience should be prioritised (56% of teachers and 17% of leaders)(Q Project 2020a) and not brushed off as “anecdotal evidence” (Wescott 2021). Teachers feel that their expertise should be valued alongside other types of evidence to inform decision-making.

Concerns with evidence

Common concerns with research evidence are that:

• the content and language are inaccessible for teachers and leaders, a case of “academics writing for other academics”
• the findings aren’t contextually relevant, which is particularly a concern for regional/remote, special needs and/or English as Additional Language teachers given research is typically euro-centric, English-centric, metropolitan-centric and involves neurotypical children
• the practice recommendations given (if any) lack specificity, leaving the teacher feeling the research isn’t “fit-for-purpose”
• concrete examples aren’t provided to assist with implementation (Jackson 2022; Prendergast and Rickinson 2019; Parker et al. 2020).

Overall, these concerns with evidence point towards a perceived disconnect between academic research and what teachers and leaders actually need in terms of evidence. Additionally, research evidence is often behind a prohibitively expensive paywall (Mills et al. 2021; Parker et al. 2020).

Common concerns with teacher-generated evidence are the limitations, particularly of standardised achievement tests, to give a valid and reliable indication of where a student, class or school is at. NAPLAN, specifically, described as just a “moment in time” that “doesn’t tell the whole story” of a class or student (White 2021; Louden 2019; Spina 2019; Ehrich et al. 2015; Wescott 2021)

Getting everyone understanding that evidence improves student outcomes

Some teachers and leaders identified the lack of approval from parents or reluctances on behalf of some teachers as barriers to engage with new practices as barriers to evidence use:

‘parents and knowledge and approval from parents’

‘teachers being closed to new ideas… it’s about a fixed mindset’

Recommendations and next steps

The national snapshot of evidence use provides several next steps for organisations who engage with teachers and leaders, as well as AERO.

1. Promote the ultimate purpose of using evidence: to improve outcomes for students

While some teachers and leaders understand how using evidence may improve student outcomes, there needs to be a broader dissemination of how evidence-based teaching practice improves student outcomes.

Next steps for AERO

• Continue providing accessible and practical information to support evidence use.
• Conduct or support large-scale research to demonstrate effectiveness of use of evidence.
• Socialise findings to normalise the positive relationship between evidence and student outcomes.

2. Clarify terminology around evidence use

A lack of a common understanding of what constitutes evidence and types of evidence can become a barrier to understanding how evidence is used, risking comparing apples and oranges. For example, when discussing research evidence, some teachers and leaders may be referring to publications in journal articles, while others may think more broadly, including summaries and resources from think tanks, opinion pieces or pre-service training.

Next steps for AERO and others

Researchers, academics, think tanks, any other organisations engaging with teachers and/or evidence (for example, responsible for collection of evidence, or training teachers in evidence use) must be clear and consistent in their use of terminology. Consistency across published resources for teachers and leaders and in data collection tools will help avoid confusion around what evidence is and misreporting use.

3. Investigate how professional learning activities can more effectively support the use of evidence in practice

Teachers and leaders often engage in external professional learning activities (Prendergast & Rickinson, 2019; Mills et al, 2021; White, 2021) but engagement in any type of professional learning does not immediately mean accessing research evidence or using it in practice. External professional learning activities commonly use research evidence; they translate academic research for teachers and leaders and further explain or showcase how it works in practice (Parker et al., 2021; Prendergast & Rickinson, 2019; White et al., 2018; Gleeson et al., 2021). However, the ways to ensure professional learning can be effective in embedding evidence are not clear. Furthermore, professional learning activities, particularly those run internally at a school (for example coaching within a school or learning communities) must be informed by evidence.

Next steps for AERO

• Identify effective professional learning mechanisms, that is, what contributes to effective professional learning and how these activities lead to better use of evidence.
• Provide guidance for effective internal professional learning activities.
• Provide guidance for ways to implement what is learnt through professional learning.

4. Focus on quality

In addition to tracking use of evidence, 2 aspects of quality need to be monitored: quality use of evidence and use of quality evidence. It is important to assess the quality of use of evidence, referring to evidence needing to be appropriate, and engagement and implementation to be thoughtful (Rickinson et al, 2021b). In addition, the evidence teachers and leaders use must be of quality (evidence needs to be reliable and rigorous) and teaches and leaders must know how to assess the quality of research they engage with. For example, some teachers and leaders report they use a practice, but they may be only using aspects of that practice. They report using a type of evidence but may be unsure about the quality of the evidence they are using. This may create a distorted image of evidence being frequently used and subsequent high expectations of improvement in student outcomes if there is no investigation of the quality of that evidence or how the evidence is used.

Next steps for AERO

Expand on current tools and resources by further examining the quality or depth of use of evidence.

5. Better understand how schools, teachers and leaders can successfully use evidence to inform practice

The characteristics of schools, teachers and leaders who use evidence well are not fully understood. Understanding what individual- and school-level factors facilitate evidence use will help schools, researchers and policymakers understand how they can support schools to use evidence to inform practice.

Next steps for AERO

• Continue to learn about the ways that characteristics already linked to evidence use (such as confidence, leadership and collaboration) can further enhance use of evidence in practice.
• Continue to identify characteristics and enablers that help schools and individuals to implement and sustain use of evidence to inform practice, including how these interact with each other.

6. Teachers and leaders should regularly access and use evidence-based system resources

Next steps for AERO/others

Support systems by creating and promoting evidence-based resources.

Keywords: educational dataset, data analysis