Interpreting Test Results and Error Analysis to Get Beneath the Scores

dFOCUS works with schools to help staff analyze test data, prioritize strengths and weaknesses, identify patterns or trends in results, and integrate the results into the curriculum. By training principals and teachers to accurately interpret test results, EdFOCUS consultants help districts bring their data to life and help it to actually drive classroom instruction.

One component is a Data Wall, compiled separately for each subject. It uses green ink to highlight strengths and red ink to designate weaknesses. Some schools add a third layer—yellow. Their “greens” are those tested concepts for which 75% of the students scored well; their “reds” are concepts where fewer than 50% of the students showed understanding. “Yellow” indicates those concepts for which 50-74% of the students showed success. Most districts post the Data Wall in the faculty work room where staff can apply sticky-notes with student names to indicate progress in specific skill areas.

Another technique is to help school staffs look for Error Patterns. School staffs determine which concepts have proven the most problematic—and for which particular students. By drilling down beneath the test items that measure each concept, school staffs can identify the types of errors made by students.

For multiple choice items, the distractors chosen by each student provide insight into his and her level of understanding.

• When a majority of students select the correct answer, test items reveal a level of understanding and mastery that can and should be replicated. This “correct” answer also informs the re-teaching or intervention that must be provided to students who did not make the correct choice.
• When a several students select the same wrong answer, teachers should determine what misunderstanding that represents and use it as the basis for remediation. In most cases, when several students select one or more wrong answers, the concept should be re-taught.
• Incorrect answers—if they are properly worded—are just as instructive to teachers as the correct answers. To really take advantage of the testing situation, smart practitioners want to know as much about what students were thinking as possible.
  

EdFOCUS helps each grade level team Prioritize their needs and strengths. That is, they identify the 5-10 concepts that need to be addressed (“red”) as well as 5-10 concepts that have been mastered (“green”). As the teams discuss these two extremes, it is important that they distinguish the kinds of instruction used for the “green” concepts from that used with the “red.” More importantly, they can see the connections between the teaching strategies and student responses to determine the impact of teacher-effects on their students’ test performance.

Once the types of errors and any patterns are identified by grade-level or AYP group, the school staff should next turn to the curriculum and ask themselves several driving questions:

1. Where do the concepts appear in the Curriculum Map or Pacing Guide? Are the concepts repeated across the school year, or are they taught one time only?
2. What does the test item ask students to do? Many of the high stakes test questions are contextualized with real-world scenarios and events. It is important that students are taught at the same level of difficulty or conceptual thinking as is used in the test. Since the content standards expect application, then rote-level memory will be inadequate for students to be successful.
3. Look at the predominant mistakes made by students to determine which students made the mistakes. Did the best-performing students miss the same questions as the poor-performing students? Who is making the errors and the type of errors made inform the team what interventions are needed going forward. In addition, how did Students with Special Needs perform on these items?
4. In terms of the priority needs, teachers must examine how the concept is taught in comparison to what is being asked of students on high stakes tests. Are students being given the opportunity to master the concept at the level and depth required in the standards? Does the test match the standard in terms of this cognitive demand?
5. b What will be done for students in the various RTI levels? Should different materials and resources be obtained to offer the appropriate instruction? What Professional Development is provided for concepts where teachers feel they need some additional support?