Outcomes and Results
Independent (federal, state-wide or standardised tests) assessments gathered from QuickSmart and comparison students over fourteen years consistently show that QuickSmart students make substantial academic improvement. Research data collected across Australia from over 63,000 QuickSmart students report:
- effect-size results of 0.60 to 0.94 that translate into growth of two- to three-years in one year when compared to the gains made by average-achieving students. (An effect size of 0.3 represents the expected yearly average growth for non-QuickSmart students.) In terms of an individual student’s growth this improvement can be as high as a factor of 7;
- substantial improvement on standardised test results in the first year of implementation that increases, and sometimes doubles, during the second year of implementation as schools and instructors become more experienced;
- academic gains are maintained or enhanced in subsequent years;
- Indigenous students receive great benefit from the program, with their results mirroring those of non-Indigenous students, and reports indicating increased student engagement in class and improvements in school attendance; and
- verbal and written comments from principals, teachers, teacher aides and parents confirm the positive impact of the QuickSmart programs.
Levels of students’ higher-order thinking are measured before and after the QuickSmart intervention. It is important that the assessment measures chosen for this purpose are rigorous, independent of the instructional program, and relevant to the Australian student population.
For the purposes of the QuickSmart program, higher-order thinking in reading is conceptualised as word and text comprehension. Similarly, higher-order thinking in numeracy involves knowing how to effectively problem-solve using quickly and accurately recalled basic number facts and strategies.
Therefore, for the purposes of this research, students’ improvement in higher-order thinking processes, such as comprehension and problem-solving, can be shown by their improved performance on standardised tests.
Comparison Student Data
In order to gain a clearer indication of the effectiveness of the QuickSmart intervention for improving accuracy and automaticity of basic academic skills, Computer-based Academic Assessment System (OZCAAS) and standardised test data are collected from other students in the same grade as the participants in the study.
In general, the group of comparison students included in the assessments consists of average-achieving students as nominated by their teachers in each of the areas of reading and numeracy. These comparison students complete the selected OZCAAS sub-tests in literacy or numeracy at the beginning and the end of the intervention and also participate in the standardised testing sessions.
QuickSmart schools are encouraged to submit pre- and post-test program data for participating and comparison students to SiMERR for analysis. Each school that submits data receives an annual report detailing the performance of students as a group, on OZCAAS tests and standardised tests measuring improvement e.g. speed and accuracy from pre-test to post-test intervention and a comparison of this improvement to those of average students over the same time period.
All annual national reports can be viewed here: https://quicksmart.une.edu.au/program-evidence/
Cognitive Aptitude Assessment System (OZCAAS)
The Australian version of the Cognitive Aptitude Assessment System (OZCAAS), a computer-assisted assessment application, is a unique component of the QuickSmart program. Developed by researchers from the Laboratory for the Assessment and Training of Academic Skills (LATAS) at the University of Massachusetts to obtain reliable assessments of student performance, the assessment tasks used are designed and sequenced in order to target and identify the exact nature of the literacy/numeracy problems a student is experiencing. The techniques developed have been successful with students who have specific reading and/or mathematics disabilities.
The OZCAAS system provides measures of how rapidly students complete tasks by speaking into a microphone once a word, a cloze sentence, or a number fact is displayed on the computer screen. An instructor scores each response for accuracy.
Students’ assessment results are automatically averaged per item and made available in either a graph or report form that is easily interpretable by both students and teachers. Students record their results onto graphs that, over time, become powerful visual representations of their progress. Most QuickSmart lessons conclude with an assessment on the OZCAAS system. During these assessments, students aim to increase their accuracy and decrease response time as a means of demonstrating increased automaticity.
The OZCAAS system thus provides ongoing monitoring of students’ basic academic skills during lessons and supports the instructional focus of the QuickSmart intervention both throughout the duration of the program and when used for data collection at pre-test and post-test.