QuickSmart in Schools involves students being withdrawn from class for 30 minutes on three occasions a week for 30 weeks (at least 90 lessons are recommended for full benefit). It is an exit program with clear criteria to identify when students have acquired certain skills and are to exit the program.
QuickSmart is an example of an evolutionary student intervention program with a practical and student-focused interpretation of learning, and theoretical underpinning, drawn from advances in educational neuroscience and cognition.
QuickSmart stands as one of a few interventions either nationally or internationally in which evaluations examine implementation at all sites of its development. Analyses of QuickSmart since 2001 have identified impressive statistically significant end-of-program and longitudinal gains using probability measures and effect sizes that confirm verbal and written reports by principals, teachers, teacher aides and parents.
Independent (federal, state-wide or standardised tests) assessments gathered from QuickSmart and comparison students since 2001 consistently show that Indigenous and non-Indigenous QuickSmart students have made substantial academic improvement. Research data collected from across Australia report:
- average effect-size results for thousands of QuickSmart Indigenous and non-Indigenous students (of 0.60 to 0.94) that translates into growth of two- to three-years in one year compared to the gains made by average-achieving students. (An effect size of 0.3 represents an expected yearly average growth for non-QuickSmart students.)
- substantial improvement on standardised test results in the first year of implementation is increased, and sometimes doubled, with new students in the second year as schools and Instructors become more experienced.
- learning continues to deepen years after students exit the program, with academic gains maintained or enhanced in subsequent years.
- Indigenous students receive great benefit from the program with their results mirroring those of non-Indigenous students and increased student engagement in class and improvements in attendance evident in many schools.
The QuickSmart team (Professor John Pegg, Associate Professor Lorraine Graham, Ms Jenny Thomas, Ms Eve Croeser and Ms Noelene Raymond) were awarded the inaugural University of New England Vice-Chancellor’s Award for Excellence in Research in 2009.
QuickSmart schools are encouraged to submit pre- and post-test program data for participating and comparison students to SiMERR for analysis. Each school that submits data receives an annual report detailing the performance of students as a group, on OZCAAS tests and standardised tests measuring improvement e.g. speed and accuracy from pre-test to post-test intervention and a comparison of this improvement to those of average students over the same time period.
All annual national and independent reports and can be viewed here: https://quicksmart.une.edu.au/program-evidence/
Other QuickSmart publications can be viewed here: https://simerr.une.edu.au/quicksmart/publications/
Ongoing Research Aims
QuickSmart is an applied research activity that is “undertaken to acquire new knowledge but directed towards a specific, practical aim or objective”**. This program of research does not represent a single research activity. Underpinning and informing QuickSmart is a longitudinal programmatic coordinated set of research projects that are:
- aimed at understanding and addressing numeracy and literacy under-performance evidenced by school students and adults who can be classified as at-risk learners across Australia; as well as
- the consequences (and learnings) of taking such an intervention to scale involving a wide variety of educational contexts and geographic locations across Australia.
It is the position of members of the SiMERR National Research Centre that making meaningful progress in a contested and difficult area of social-science enquiry requires extensive and ongoing longitudinal and programmatic research. In our view, this is the only justifiable and efficient way to move the field and practice forward.
The significance of the QuickSmart research program is that the research questions of most interest evolve as the activity continues, with deeper and more complicated issues addressed as the quantum of data allows for increased sophistication in the techniques used. Likewise, the issues addressed offer deeper insights into the activity as the research program continues (e.g., What are the mechanisms of deliberate practice and feedback within the QuickSmart lessons? Why does performance uniformly improve for QuickSmart students in schools that implement the program in the second and subsequent years?)
Funding for the research comes from federal and state education authorities and schools, and competitive research grants as well as through philanthropic and business education sponsors.
Researchers within the SiMERR National Research Centre use QuickSmart data in a systematic way for theory development concerning the real social problems that accompany the need to support:
- students in today’s classrooms in different locations, under different funding arrangements, with varying levels of support mechanisms; and
- adults seeking employment or increased practical skill levels.
There are logical connections and progressions within the developments and findings of QuickSmart that evolve over each year. As new learning and practice emerges the resulting ideas are continually trialled and evaluated.
These research drivers result in the “creation of new knowledge and the use of existing knowledge in a new and creative way so as to generate new concepts, methodologies and understandings. This … includes synthesis and analysis of previous research to the extent that it leads to new and creative outcomes.”**
The research that informs QuickSmart is focused particularly on cognitive processing, the conditions necessary to be in place for school students and adults to gain facility with fundamental lower-order tasks or basic academic skills, and the potential complementary effects of the improved mastery of these skills on higher-order learning processes.
Accordingly, the initial research funded in 2001 had three overall goals that still underpin our programmatic research endeavours:
- to investigate the nature of improved fluency on the acquisition of basic academic skills;
- to observe whether improved fluency with the basics has any effect on the performance of more demanding academic tasks, such as comprehension and mathematical problem solving, as reflected in students’ performance on state-wide tests or standardized achievement tests; and
- to monitor longitudinally the retention or further development/refinement in QuickSmart participants of both basic and higher-order skills.
Given the increase in the number of schools and employer groups interested in implementing and refining QuickSmart programs, the initial research goals have evolved and been greatly expanded. Currently, QuickSmart programmatic research aims to:
- Develop a deeper understanding of the role of working-memory load in information processing, and how this is implicated in literacy and numeracy problems students encounter.
- Develop detailed descriptions of cognitive obstacles that preclude low-achieving students attaining acceptable standards of literacy and numeracy.
- Understand ways that learning obstacles might be approached, once identified, in a variety of learning contexts with the expectation that over time these would be addressed.
- Prepare detailed profiles of individual students, documenting their development in literacy/numeracy over the period of an academic year and seeking patterns of similarity across large cohorts of students.
- Gain insights into how the procedures developed for individual application in small-class or group settings through QuickSmart may be generalised to suit whole or part classroom situations.
- Explore ways of adapting the technology used in QuickSmart to identify and target particular problems that learners face in areas of literacy and numeracy.
- Develop a set of design features that can be used to support learning disabilities in the areas of literacy and numeracy.
- Collect, analysis and interpret pre- and post-test data collected for each intervention group as well as from comparison ‘learners’, using two forms of assessment, namely:
- the Computer-based Academic Assessment System (OZCAAS) tests of accuracy and speed of recall and recognition of basic literacy tasks or numeracy facts; and
- independently prepared tests in the form of standardised achievement tests to provide information about the transfer of basic facts to more complex academic and cognitive tasks.
Since QuickSmart has been implemented in an increasing number of schools across Australia and adult learning contexts, the focus of research has also developed to consider:
- Ways to improve the provision of QuickSmart training by a deeper analysis of the results highlighting difference in approaches of groups achieving greater learning growth from their students.
- Documenting the occurrences of ‘learnable’ features that are evident after students have graduated out of the program.
- Further assessment of ‘self-factors’ such as student self-efficacy, self-confidence, and scaffolded risk-taking that are an important part of the QuickSmart research framework.
- Linking the qualitative and quantitative data associated with particular learning obstacles.
- Developing models for taking evidence-based interventions to scale across numbers of schools in different sectors, jurisdictions, states and territories as well as for different industry groups.
- Identifying models of success that underpin the QuickSmart implementation.
- Creating closer links between learning theory and educational neuroscience.
There is a systematic element to the QuickSmart research that precisely fits the definition of: “creative work undertaken on a systematic basis in order to increase the stock of knowledge … and the use of this stock of knowledge to devise new applications”**.
This creative use of new knowledge informs the investigation of the implementation of QuickSmart as an evolving program guided by research evidence. Our approach is systematic, particularly in the processing of information that is informing the development of new programs and the work of SiMERR’s Higher Degree Research (HDR) students.
One HDR student, Anne Bellert a PhD candidate at the SiMERR National Research Centre, was the recipient of the prestigious Learning Difficulties Australia Tertiary Student Research Award. Anne submitted her thesis (The Effects Of Improved Automaticity In Basic Academic Skills On Test Performance: A Study Of Learning Difficulties In The Middle-School Years), which uses the QuickSmart intervention. In making this Award the Judging Committee noted the particular strengths of this research in terms of its broader application for intervention for students with learning difficulties in a classroom situation, with a specific focus on mathematics intervention.
The information acquired from research with the QuickSmart program has also acted as a catalyst for the development and operationalisation of new intervention programs based on QuickSmart learnings focusing on different areas of learning difficulty including those associated with: primary and secondary schooling; at-risk adult learners; technical and further education; and, industry and workplace contexts.
**Acknowledgement: Quotes are taken from Higher Education Research Data Collection, Specifications for the collection of 2010 data. Research Funding and Policy Branch, Department of Innovation, Industry, Science and Research, January, 2011
Outcomes and Results
Independent (federal, state-wide or standardised tests) assessments gathered from QuickSmart and comparison students over fourteen years consistently show that QuickSmart students make substantial academic improvement. Research data collected across Australia from over 67,000 QuickSmart students report:
- effect-size results of 0.60 to 0.94 that translate into growth of two- to three-years in one year when compared to the gains made by average-achieving students. (An effect size of 0.3 represents the expected yearly average growth for non-QuickSmart students.) In terms of an individual student’s growth this improvement can be as high as a factor of 7;
- substantial improvement on standardised test results in the first year of implementation that increases, and sometimes doubles, during the second year of implementation as schools and instructors become more experienced;
- academic gains are maintained or enhanced in subsequent years;
- Indigenous students receive great benefit from the program, with their results mirroring those of non-Indigenous students, and reports indicating increased student engagement in class and improvements in school attendance; and
- verbal and written comments from principals, teachers, teacher aides and parents confirm the positive impact of the QuickSmart programs.
Levels of students’ higher-order thinking are measured before and after the QuickSmart intervention. It is important that the assessment measures chosen for this purpose are rigorous, independent of the instructional program, and relevant to the Australian student population.
For the purposes of the QuickSmart program, higher-order thinking in reading is conceptualised as word and text comprehension. Similarly, higher-order thinking in numeracy involves knowing how to effectively problem-solve using quickly and accurately recalled basic number facts and strategies.
Therefore, for the purposes of this research, students’ improvement in higher-order thinking processes, such as comprehension and problem-solving, can be shown by their improved performance on standardised tests.
Comparison Student Data
In order to gain a clearer indication of the effectiveness of the QuickSmart intervention for improving accuracy and automaticity of basic academic skills, Computer-based Academic Assessment System (OZCAAS) and standardised test data are collected from other students in the same grade as the participants in the study.
In general, the group of comparison students included in the assessments consists of average-achieving students as nominated by their teachers in each of the areas of reading and numeracy. These comparison students complete the selected OZCAAS sub-tests in literacy or numeracy at the beginning and the end of the intervention and also participate in the standardised testing sessions.
Cognitive Aptitude Assessment System (OZCAAS)
The Australian version of the Cognitive Aptitude Assessment System (OZCAAS), a computer-assisted assessment application, is a unique component of the QuickSmart program. Developed by researchers from the Laboratory for the Assessment and Training of Academic Skills (LATAS) at the University of Massachusetts to obtain reliable assessments of student performance, the assessment tasks used are designed and sequenced in order to target and identify the exact nature of the literacy/numeracy problems a student is experiencing. The techniques developed have been successful with students who have specific reading and/or mathematics disabilities.
The OZCAAS system provides measures of how rapidly students complete tasks by speaking into a microphone once a word, a cloze sentence, or a number fact is displayed on the computer screen. An instructor scores each response for accuracy.
Students’ assessment results are automatically averaged per item and made available in either a graph or report form that is easily interpretable by both students and teachers. Students record their results onto graphs that, over time, become powerful visual representations of their progress. Most QuickSmart lessons conclude with an assessment on the OZCAAS system. During these assessments, students aim to increase their accuracy and decrease response time as a means of demonstrating increased automaticity.
The OZCAAS system thus provides ongoing monitoring of students’ basic academic skills during lessons and supports the instructional focus of the QuickSmart intervention both throughout the duration of the program and when used for data collection at pre-test and post-test.
Developed at the SiMERR National Research Centre, QuickSmart drew upon extensive analyses of the research literature (e.g., Swanson & Hoskyn, 1998), while its initial implementation was supported by research grants from the Australian Research Council, the Federal Government, project funds from SiMERR, and extensive cash and in-kind support from the Northern Territory and New South Wales.
The research program associated with QuickSmart is unique because it explored a programmatic intervention conducted in a wide variety of Australian schools. Since 2001, systematic data collection and analysis has accrued substantial evidence regarding the value and applicability of the QuickSmart Numeracy (basic mathematics) and QuickSmart Literacy (reading, vocabulary and comprehension) programs as they have been implemented to an increasingly expansive scale. As of 2022, the programs have been implemented in over 1,500 schools in Australia, with data collected for over 67,000 students.
The QuickSmart research program is one of only a few programmatic interventions that has accrued substantial evidence regarding value and applicability from research conducted in Australian schools targeting low-achieving middle-school students. In May 2009, the NSW Department of Education and Training announced that QuickSmart Numeracy was an approved numeracy intervention program to be offered under the new state/federal school partnership funding arrangement for schools with literacy and numeracy needs and schools in areas of low socioeconomic status.
The following principles have guided both the development and scaling up of QuickSmart:
- Research evidence should inform policy positions and systemic approaches to addressing the needs of low-achieving middle-school students.
- Programs designed to address the learning needs of low-achieving middle-school students should be intense, of significant duration, and conducted in small-class instructional settings.
- An extensive professional learning program for teachers, teacher aides and executive members of schools and education jurisdictions should be an important component of any sustainable instructional intervention.
- Improving the skill base of teacher aides should be a focus of attention for all support programs, especially those in rural and remote areas or difficult-to-staff schools where teaching staff mobility is a significant factor.
- To ensure sustainability, national, state, regional and school level stakeholders need to coordinate their efforts and collaborate to ensure the fidelity of the program, and the viability of its implementation and scaling up processes.
- Costs of the program should be shared across national, state, regional and school-level stakeholders.
Some key features of QuickSmart that underpin its effectiveness since 2001 include:
- A whole school approach to intervention for low-achieving students which includes professional learning sessions geared to principals and administrators;
- The establishment of a QuickSmart team within each school consisting of a QuickSmart coordinator and QuickSmart instructors as well as a member of the senior executive;
- Active participation in a six-day (three sets of two-days) professional learning program throughout the year that introduces, then consolidates, the QuickSmart approach;
- An intervention that is based on research evidence and supported by extensive resource materials, manuals, software and levels of practical and troubleshooting advice.
Since 2009, the Ascent Business Services group in Armidale have been subcontracted by SiMERR to administer the packing and sending out of QuickSmart flashcard kits and other resources to schools. More information here: https://www.une.edu.au/connect/news/2009/01/une-maths-program-adds-confidence