Cognitive Aptitude Assessment System (CAAS)
The Cognitive Aptitude Assessment System (CAAS), a computer-assisted assessment system, is a unique component of the QuickSmart program. Developed by researchers from the Laboratory for the Assessment and Training of Academic Skills (LATAS) at the University of Massachusetts to obtain reliable assessments of student performance, the assessment tasks used are designed and sequenced in order to target and identify the exact nature of the literacy/numeracy problems a student is experiencing. The techniques developed have been successful with students who have specific reading and/or mathematics disabilities.
The CAAS system provides measures of how rapidly students complete tasks by speaking into a microphone once a word, a cloze sentence, or a number fact is displayed on the computer screen. An instructor scores each response for accuracy.
Students' assessment results are automatically averaged per item and made available in either a graph or report form that is easily interpretable by both students and teachers. Students record their results onto graphs that, over time, become powerful visual representations of their progress. Most QuickSmart lessons conclude with an assessment on the CAAS system. During these assessments, students aim to increase their accuracy and decrease response time as a means of demonstrating increased automaticity.
The CAAS system thus provides ongoing monitoring of students' basic academic skills during lessons and supports the instructional focus of the QuickSmart intervention both throughout the duration of the program and when used for data collection at pre-test and post-test.
Overview of CAAS Numeracy Component
Tasks generally used to assess and instruct the QuickSmart mathematics group include:
- addition number facts;
- subtraction number facts;
- multiplication number facts;
- division number facts; and
- triple addition tasks (for example, 7 + 4 + 3).
Overview of CAAS Literacy Component
Tasks used to assess and instruct the QuickSmart reading group include:
- simple word recognition;
- middle word recognition;
- simple close sentence comprehension;
- middle close sentence comprehension; and
- non-word reading tasks (non-word reading is recognized as a difficult task for students with learning disabilities).
Levels of students’ higher-order thinking are measured before and after the QuickSmart intervention. It is important that the assessment measures chosen for this purpose are rigorous, independent of the instructional program, and relevant to the Australian student population.
For the purposes of the QuickSmart program, higher-order thinking in reading is conceptualised as word and text comprehension. Similarly, higher-order thinking in numeracy involves knowing how to effectively problem-solve using quickly and accurately recalled basic number facts and strategies.
Therefore, for the purposes of this research, students’ improvement in higher-order thinking processes, such as comprehension and problem-solving, can be shown by their improved performance on standardised tests.
In order to gain a clearer indication of the effectiveness of the QuickSmart intervention for improving accuracy and automaticity of basic academic skills, Computer-based Academic Assessment System (CAAS) and standardised test data are collected from other students in the same grade as the participants in the study.
In general, the group of comparison students included in the assessments consists of average-achieving students as nominated by their teachers in each of the areas of reading and numeracy. These comparison students complete the selected CAAS sub-tests in literacy or numeracy at the beginning and the end of the intervention and also participate in the standardised testing sessions.
In order to monitor the more affective aspects of participation in the QuickSmart program qualitative data are also collected from students and other stakeholders.
Students' perceptions of the QuickSmart Program
The students' perceptions of the usefulness and importance of their QuickSmart learning experience are gathered through interview and questionnaire data collected at the conclusion of the program. Students are asked to comment on how QuickSmart helped them with their work in the classroom and in their lives outside of school, as well as on what they enjoyed most about the QuickSmart program and what they would change to make it better.
Instructors', Parents', Teachers' and Principals' perceptions of the QuickSmart Program
Other stakeholders, such as QuickSmart instructors, classroom teachers, parents, and principals, are also provided with an opportunity to share their evaluations of the QuickSmart program. Questionnaires are designed to elicit valuable information and insights from each of these groups. An example of a question asked in these questionnaires is what effect the QuickSmart program has had on the students' performance in terms of their academic achievement, attitude to learning, self-confidence, self-esteem, and behaviour.