Survey findings and practical guidance from an AERO and Monash Q Project collaboration

This report describes ways that schools can build teachers’ and leaders’ confidence in using research evidence, and why this is important. It presents findings from independent surveys conducted by the Australian Education Research Organisation (AERO) and the Monash Q Project.

Overall findings and guidance

Nearly 1,400 school teachers and leaders were surveyed across all Australian states and territories in 2020 and 2021.

AERO and the Q Project used the survey data to examine the relationship between support provided by schools, confidence in using research evidence and use of research evidence in practice. This report therefore uses empirical data to examine the importance of school support, and the role that confidence plays in the use of research evidence.

We found that when teachers and leaders are supported by their school to use research evidence, they are:

  • more confident in using research evidence
  • more likely to use research evidence in their practice.

The support can include being given dedicated time, professional learning, and access to a coach or school leader in a specialised research dissemination role or with specialised research expertise.

Additional findings include:

  • Many teachers and leaders indicated that their school provides support to use research evidence.
  • While teachers and leaders believed in the value of using research evidence, research evidence was not often used to help improve knowledge or practice or to make decisions.
  • Teachers and leaders had some concerns about their abilities to assess or analyse research evidence. These confidence gaps represent important opportunities to target improvement initiatives in schools.

Context

Increasingly, teachers, leaders, schools and school systems are becoming aware that using evidence generated from academic research (also referred to as research evidence) can improve their practice (Nelson and Campbell 2019). When school leaders and teachers engage with research evidence, their own teaching skills can improve (for example, Bell et al. 2010; Godfrey 2016), and learning and school performance outcomes also improve (for example, Mincu 2014; Supovitz 2015; Rose et al. 2017). These findings provide powerful reasons to use research evidence to inform practice.

But using research evidence well and incorporating it into practice is complex and demanding work.

‘Evidence is important, but what is more important is […] teacher expertise and professionalism [to] make better judgments about when, and how, to use research.’
(Wiliam 2019)

This ‘expertise and professionalism’ requires teachers and school leaders to value research evidence and to want to use it to improve their practice. It also requires teachers and leaders to have confidence in their own skills and knowledge to engage with research evidence (Rickinson et al. 2021a). Recent Australian studies have made clear that teachers and leaders require support and resources to do this (for example, Parker et al. 2020; Prendergast and Rickinson, 2019; Rickinson et al. 2021b; Walsh et al. 2022).

Purpose of this report

This report offers guidance to build teachers’ and leaders’ confidence in using research evidence, and to increase the use of research evidence in schools.
The report does this by:

  • explaining the importance of different individual and school characteristics that can enable the increased and improved use of research evidence
  • demonstrating, with empirical findings, how support provided by a school to use research evidence increases teachers’ and leaders’ confidence in using, as well as actual use of, research evidence (Figure 1)
  • guiding schools to focus on key supports that can help teachers and leaders to build their confidence and skills to use research evidence, and use it well.

The report draws on 2 independent surveys; one conducted by the Monash Q Project in 2020 and one conducted by AERO in 2021, with a total of 1,389 teachers and school leaders across all Australian states and territories. The methods used to collect and analyse survey responses are summarised in Appendices 1 and 2.

‘If educators are more research engaged and connected…teaching and learning improvements are likely to follow’
(Malin and Brown 2020)

The Australian Education Research Organisation (AERO) and the Monash Q Project have similar aims to promote the use of quality research and evidence in Australian educational practice.

Both AERO and the Q Project are currently:

  • working with teachers and school leaders across Australia to understand what is involved in using research and evidence well in practice
  • developing resources and interventions to support increased and improved use of research evidence in practice.

Factors that enable the use of research evidence

The use of research evidence in schools is influenced by a range of factors (Dagenais et al. 2012; van Schaik et al. 2018). Some of these factors include the characteristics of individual teachers and school leaders themselves. Several previous studies have found that individuals’ beliefs, motivations and confidence regarding the use of research evidence are important influences on their use of research evidence in practice (for example, Judkins et al. 2014; Lysenko et al. 2014; Penuel et al. 2017; Williams and Coles 2007). Other influencing factors are contextual and include school-based supports, such as scheduled time, professional learning opportunities, effective leadership, and a school culture that encourages and facilitates research evidence use (Cain 2019; Cooper and Levin 2013; Godfrey and Brown 2019).

AERO identifies 2 main types of evidence:

  • teacher-generated evidence (such as evidence gained from using student learning data)
  • research evidence is interpreted in this report as evidence generated through academic research, which uses rigorous methods to provide insights into educational practice.

This report focuses on research evidence. Using research evidence means the process of engaging with and drawing on research evidence to inform, change and improve decision-making and practice.

Quality Use of Research Evidence (QURE) Framework

One way to imagine how these different enablers of research evidence use can come together in a school context is through the Monash Q Project’s Quality Use of Research Evidence (QURE) Framework (Rickinson et al. 2020). As depicted in Figure 2, this framework sees the quality use of research evidence as ‘thoughtful engagement with and implementation of appropriate research evidence’. Quality use of research evidence is supported by a blend of 3 individual enabling components – skillsets, mindsets and relationships – and 3 organisational enabling components – culture, leadership and infrastructure. Together, these individual and organisational factors have the potential to increase and improve the use of research evidence in practice.

The complex interactions and inter-dependencies across the education sector to support thoughtful engagement with and implementation of appropriate research evidence.

  • Appropriate research evidence – The need for research evidence to be not only methodologically rigorous, but also appropriate for the educational issue, the context and intended use.
  • Thoughtful engagement and implementation – Critical engagement with the research evidence, shared deliberation about its meaning and effective integration of aspects of the evidence within practice.

Individual level

  • Skillsets - The knowledge and capabilities that are required to thoughtfully engage with and implement appropriate research evidence.
  • Mindsets - The dispositions, attitudes and values that are required to thoughtfully engage with and implement appropriate research evidence.
  • Relationships - The interpersonal processes and connections that are required to thoughtfully engage with and implement appropriate research evidence.

Organisational level

  • Leadership - The organisational vision, commitments and role models that support thoughtful engagement with and implementation of appropriate research evidence.
  • Culture - The organisational ethos, values and norms that support thoughtful engagement with and implementation of appropriate research evidence.
  • Infrastructure - The organisational structures, resources and processes that support thoughtful engagement with and implementation of appropriate research evidence.

Teachers’ and leaders’ perspectives

Teachers’ and leaders’ perspectives play an important role in moving discussions about research evidence use beyond the theoretical (for example, Boaz and Nutley 2019). They also shed light on what is involved in using research evidence well in practice and how it can be supported.

Supports to use research evidence are available

Overall, teachers and school leaders generally agree that supports to use research evidence are available at their school (see Box 1 for types of supports). For instance, 80% of respondents in AERO’s survey agreed/strongly agreed that their leaders share and discuss evidence to improve practice (Figure 3). In the Q Project survey, 87% of respondents agreed/strongly agreed that their school facilitates a professional learning community that supported their evidence use (Figure 4).

Some types of support, however, were not as common as others, such as the availability of coaching within schools (in AERO’s sample) or the provision of time by schools (in the Q Project’s sample). Overall, though, supports reported by teachers and school leaders provide a good opportunity for enabling greater and improved research evidence use.

AERO survey

✓ School leaders share and discuss evidence to improve our practice
✓ School sets aside regular times or meetings to discuss evidence that could improve practice
✓ School uses evidence when deciding on school policies and programs
✓ School has coaching available to help staff use evidence to change their practice

Q Project survey

✓ School leaders foster professional learning communities or support collaborative learning opportunities
✓ Schools refer to evidence of ‘what works’ when deciding on initiatives to implement
✓ Schools consult a variety of information sources when making decisions
✓ School leaders encourage informed risk-taking in teaching practice
✓ Schools implement formal processes to help staff engage with evidence
✓ Schools make time available for staff to engage with research
✓ Schools implement informal processes to help staff engage with evidence

Schools value research evidence more than they use it

Previous research from the Q Project has highlighted that teachers and school leaders believe in the value of using research to inform their practice (Rickinson et al. 2021b; Walsh et al. 2022). The Q Project’s survey in this study showed that 83% of respondents agreed or strongly agreed that using research will help improve student outcomes.

Despite these strong beliefs in the value of research evidence, just under half of teachers and school leaders use research evidence often in practice (Figure 5). For instance, 43% of respondents in the Q Project survey said they consulted university-based research articles always/often to help inform their decisions. The same percentage (43%) of respondents in the AERO survey said they consulted academic research often/very often for their own knowledge.

‘We make sure that all the staff have an understanding [of the research] … and [we] support them to get them along the way. … We develop an essential agreement; how it will be taught, what’s expected. … From there, we support them in the classroom … and it’s a continual process of supporting the teachers.’
Senior Leader, Government Primary School,
Q Project

Opportunities to build confidence in using research evidence

One reason for research evidence not being used often may be a lack of confidence. Teachers and school leaders in both AERO’s survey (Figure 6) and the Q Project’s survey (Figure 7) expressed some concerns about their own abilities to use research evidence in practice. In particular, just under half (46%) of the respondents in the AERO survey were mostly/very confident about determining the rigour and quality of research, while 45% of respondents in the Q Project survey agreed/strongly agreed that they felt confident to initiate discussions about research with their colleagues.

These confidence gaps represent important opportunities to target improvement initiatives within schools. Work conducted by the Q Project (Walsh et al. 2022) found that these initiatives should focus on (in rank order):

  1. identifying issues where consulting research evidence would be appropriate
  2. assessing the usability of research evidence
  3. assessing research evidence for contextual relevance and fit.

‘There’s almost barriers to entry with [research use], because when people don’t feel that they’re particularly confident with it…[then they don’t use it]’
Middle Leader, Government Secondary School, Q Project

Building confidence through support improves research evidence use

Across both studies there was a clear relationship between schools’ support of research evidence use, teachers’ and leaders’ confidence, and the frequency of their research evidence use1.

When teachers and leaders are supported by their school to use research evidence, they are:

  • more confident in using research evidence
  • more likely to use research evidence in their practice.

Confidence in using research evidence is important because it explains some of the relationship between support provided by a school and use of research evidence in practice. Specifically, provision of support can improve confidence, which in turn can foster greater use of research evidence in practice (Figure 8).

These findings emphasise the importance of providing support to improve use of research evidence in practice. Therefore, focusing on improving a school’s culture and infrastructure around using research evidence can help to support teachers’ confidence to use research evidence, as well as their use of this evidence in practice.

Conclusion

This collaborative report between AERO and the Monash Q Project outlines ways to increase and improve use of research evidence in a school’s practices and decision-making.

The report shows that teachers and leaders are more confident in using research evidence and are more likely to use research evidence in their practice when supported to do so by their school.

In summary, focusing on providing supports to use research evidence though a school’s culture and infrastructure can improve research evidence use. The findings of this study can also provide a frame for school leaders and teachers to reflect on the use of research evidence in their context and the supports provided by their schools. By drawing on the guidance provided in this report, school leaders and teachers can work together to harness the potential of research evidence use for their own professional improvement, as well as to drive school-wide improvements in teaching and learning.

Guidance for providing support at your school

Insights from both AERO’s and the Q Project’s work suggest several supports can help improve a school’s culture and infrastructure around using research evidence, which can then lead to increased and improved use of research evidence in practice below.

The Monash Q Project conceptualises infrastructure as the school-based structures, resources and processes that support quality research use (Rickinson et al., 2020). These may include:

  • the allocation of time, space, facilities (for example, a research library or centre) and budget
  • access to research
  • the creation of school-based coaches or research leaders
  • professional learning opportunities
  • research use being linked to key planning, inquiry and performance cycles
  • research use being incorporated into meetings and formal discussions.

Effective leadership is required for supporting infrastructure to be developed and embedded within the school culture (Cutler et al., 2022).

Benefits/supports

This supports:

  • more frequent engagement with research evidence
  • more collaboration to make sense of evidence.

Resources

Benefits/supports

Having someone to drive and guide research evidence use within schools is highly valued by teachers and leaders, helping them to develop their skills and confidence.

Resources

Benefits/supports

These can help teachers and leaders build their confidence and skills to locate appropriate research and assess its relevance and rigour.

Resources

  • AERO’s Standards of evidence are helpful for making consistent and transparent judgements when assessing evidence.
  • AERO’s Research reflection guide can be used for reflecting on a piece of research evidence.
  • The Q Project has a resource suite focused on finding and using research as well as sharing evidence.
  • The Q Project self-assessment tool provides teachers, leaders and schools with an immediate report of their research use capacities and access to tailored support resources.

References

Bell M, Cordingley P, Isham C and Davis R (2010) Report of professional practitioner use of research review: Practitioner engagement in and/or with research, CUREE, GTCE, LSIS and NTRP, http://www.curee.co.uk/files/publication/[site-timestamp]/Practitioner%….

 

Boaz A and Nutley S (2019) ‘Using evidence’, in Boaz A, Davies H, Fraser A and Nutley S (eds), What works now? Evidence-informed policy and practice, pp. 251–277, Policy Press.

 

Cain T (2019) Becoming a research-informed school: Why? What? How? Routledge.

 

Cooper A and Levin B (2013) ‘Research use by leaders in Canadian school districts’, International Journal of Education Policy & Leadership 8(7), https://files.eric.ed.gov/fulltext/EJ1016293.pdf.

 

Cutler B, Gleeson J, Rickinson M, Walsh L, Salisbury M and Cirkony C (2022) ‘From “finding time” to “investing time”: The importance of time and leadership for cultivating cultures of quality research’, Australian Educational Leader, 44(2):30-33. https://search.informit.org/doi/10.3316/informit.518363592839442.

 

Dagenais C, Lysenko L, Abrami PC, Bernard RM, Ramde J and Janosz M (2012) ‘Use of research-based information by school practitioners and determinants of use: A review of empirical research’, Evidence & Policy, 8(3):285–309, doi.org/10.1332/174426412X654031.

 

Godfrey D (2016) Leadership of schools as research-led organizations in the English educational environment: Cultivating a research engaged school culture. Educational Management Administration & Leadership, 44(2):301–321, doi.org/10.1177/1741143213508294.

 

Godfrey D and Brown C (Eds.) (2019) An ecosystem for research-engaged schools: Reforming education through research. Routledge.

 

Hayes AF (2009) ‘Beyond Baron and Kenny: Statistical mediation analysis in the new millennium’, Communication Monographs, 76(4):408–420, doi.org/10.1080/03637750903310360.

 

Judkins M, Stacey O, McCrone T and Inniss M (2014) Teachers’ use of research evidence: A case study of United Learning schools. National Foundation for Educational Research.

 

Lysenko L, Abrami P, Bernard R, Dagenais C and Janosz, M (2014) ‘Educational research in educational practice: Predictors of use’, Canadian Journal of Education/Revue canadienne de l’éducation, 37(2):1–26, www.jstor.org/stable/canajeducrevucan.37.2.06.

 

Malin JR and Brown C (2020) ‘Joining worlds: Knowledge mobilization and evidence-informed practice’, in JR Malin and C Brown (eds), The role of knowledge brokers in education: Connecting the dots between research and practice, pp. 1–9, Routledge.

 

Malin JR, Brown C, Ion G, van Ackeren I, Bremm N, Luzmore R, Flood J and Rind GM (2020) ‘World-wide barriers and enablers to achieving evidence-informed practice in education: What can be learnt from Spain, England, the United States, and Germany?’, Humanities and Social Sciences Communications, 7(1):1–14, doi.org/10.1057/s41599-020-00587-8.

 

Maxwell SE and Cole DA (2007) ‘Bias in cross-sectional analyses of longitudinal mediation’, Psychological Methods, 12(1):23–44, doi.org/10.1037/1082-989X.12.1.23.

 

Maxwell SE, Cole DA and Mitchell MA (2011) ‘Bias in cross-sectional analyses of longitudinal mediation: Partial and complete mediation under an autoregressive model’, Multivariate Behavioral Research, 46(5): 816–841, doi.org/10.1080/00273171.2011.606716.

 

Mincu M (2014) ‘Inquiry paper 6: Teacher quality and school improvement – What is the role of research?’, in BERA & RSA (eds), The role of research in teacher education: Reviewing the evidence, pp. 28–32, BERA, www.bera.ac.uk/wp-content/uploads/2014/02/BERA-RSA-Interim-Report.pdf.

 

Nelson J and Campbell C (2019) ‘Using evidence in education’, in Boaz A, Davies H, Fraser A and Nutley S (eds), What works now? Evidence-informed policy and practice revisited, pp. 131–149, Policy Press.

Parker B, Steele T, Rose V and Taylor D (2020) Getting evidence moving in schools (GEMS): Investigation paper. Evidence for Learning, evidenceforlearning.org.au/research-and-evaluation/investigations-and-insights/getting-evidence-moving-in-schools-gems/.

 

Penuel WR, Briggs DC, Davidson KL, Herlihy C, Sherer D, Hill HC, Farrell CC and Allen AR (2017) ‘How school and district leaders access, perceive, and use research’, AERA Open, 3(2):1–17, doi.org/10.1177/2332858417705370.

 

Prendergast S and Rickinson M (2019) Understanding school engagement in and with research. Australian Educational Researcher, 46:17–39, doi.org/10.1007/s13384-018-0292-9.

 

Rickinson M, Walsh L, Cikony C, Salisbury M and Gleeson J (2020) Quality Use Of Research Evidence (QURE) Framework Report, Monash University, https://doi.org/10.26180/14071508.v2.

 

Rickinson M, Gleeson J, Walsh L, Cutler B, Cirkony C and Salisbury M (2021a) Research and evidence use in Australian schools: Survey, analysis and key findings. Monash University, doi.org/10.26180/14445663.

 

Rickinson M, Gleeson J, Walsh L, Salisbury M, Cutler B and Cirkony C (2021b) Using research well in Australian schools. Monash University, doi.org/10.26180/14783637.

 

Rohrer JM, Hünermund P, Arslan RC and Elson M (2021) ‘That’s a lot to process! Pitfalls of popular path models’, PsyArXiv, doi.org/10.31234/osf.io/paeb7.

 

Rose J, Thomas S, Zhang L, Edwards A, Augero A and Rooney P (2017) Research learning communities: Evaluation report and executive summary. Education Endowment Foundation, educationendowmentfoundation.org.uk/projects-and-evaluation/projects/research-learning-communities.

 

Rosseel Y (2012) ‘lavaan: An R package for structural equation modeling’, Journal of Statistical Software, 48(2):1–36, www.jstatsoft.org/v48/i02/.

 

Supovitz J (2015) ‘Teacher data use for improving teaching and learning’, in Brown C (eds), Leading the use of research and evidence in schools, pp. 117–125, IOE Press.

 

Van Schaik P, Volman M, Admiraal W and Schenke W (2018) ‘Barriers and conditions for teachers’ utilisation of academic knowledge’, International Journal of Educational Research, 90:50–63, doi.org/10.1016/j.ijer.2018.05.003.

 

Walsh L, Gleeson J, Cutler B, Rickinson M, Cirkony C and Salisbury M (2022) What, why, when and how: Australian educators’ use of research in schools, Monash University, doi.org/10.26180/17192990.v1.

 

Wiliam D (30 May 2019) Dylan Wiliam: Teaching not a research-based profession, Tes Magazine, accessed 6 May 2022.

 

Williams D and Coles L (2007) ‘Teachers’ approaches to finding and using research evidence: An information literacy perspective’, Educational Research, 49(2):185-206, doi.org/10.1080/00131880701369719.

Appendices

Download the full report (PDF, 3.6MB) to view the appendices:

  • Appendix 1: Sample and data collection methods 
  • Appendix 2: Statistical model 

Keywords: educational data, data analysis, evidence-based education, evidence-based teaching