QCEA- 2006: Facts and Figures – What do they mean?

For the second consecutive year, Qatari students in grades 4-11 were tested against standards of international quality, set by the Supreme Education Council. The results of Qatar's Comprehensive Education Assessment (QCEA) 2006 showed a relative loss from 2004-2005 standards levels – including Indepedent Schools. This has spurred a nationwide debate about student performance, which is one of the purposes.

Before we take a comprehensive look at the QCEA-2006 results, let us stand back and remember that educators and thought-leaders agree that education assessment is a complex and multidimensional process that should not be assigned black and white terms, such as 'success' or 'failure.' Rather, assessments should guide the process of educating children – after all, annual standards-based exams are not the single gauge of genuine subject performance.

In this article, we measure test results against 2005 levels. We break down the results, discuss the meaning of individualized reports, and analyze student performance in the four subjects in which students are tested: Arabic language, English language, mathematics, and science.

A priority for the Evaluation Institute of the Supreme Education Council is student assessment and school evaluation through standards and outcomes-based tests. Transparent, objective dissemination and analysis of results is critical. Only with accurate and complete information can decision-makers make wise choices about K-12 education.

The Evaluation Institute uses an array of tools to assess students' achievement and compare their performance against international peers. Some of these tools include a battery of standardized tests under the name of Qatar Comprehensive Educational Assessment (QCEA), School Report Cards (SRCs), participation in three international studies, triennial reviews and continuous monitoring.

The QCEA is one component of the assessment system. This year the exam was administered to 70,000 students in all Qatari schools, including Independent, Ministry of Education (MoE), and private Arabic schools. Its results measure students' proficiency according to Qatar's curriculum standards set by the SEC as part of the reform effort.

About the QCEA 2006 Reporting:
QCEA results are contained in three reports: Student report: An individualized report containing information on students' performance. Identifies strengths, weaknesses and suggestions for improvement in the subjects of Arabic, English, mathematics and science.

Classroom report: Report for teachers of the four tested subject areas showing how students perform in their subject matters in the previous year. This report identifies achievement levels against new curriculum standards and includes examples of questions the students were required to answer.

National report: Report presents data by grade, gender, and school type for all tested students in independent, MoE and private Arabic schools, in the subjects of Arabic, English, mathematics, and science. The report includes other valuable educational indicators and allows comparison of student performance from 2005-2006.

Performance Levels' Results
How it works: Each student receives a scale score in each content area (Arabic, English, mathematics, science).In order to be more educationally useful scale scores are converted into performance levels that indicate whether a student has met standards or not. There are five performance levels: Meet standards – level 5
Approaches standards – level 4
Below standards levels – 1 through 3

Arabic: Four percent of students scored at level 5 (meet standards). We found that females performed better than males; students in Independent Schools performed better than all other schools. Four percent of students met standards; the percentage of students in Independent Schools at level 5 reached 6%.

English: Two percent of students scored at level 5 (meet standards); 89 percent scored at levels 1-3 (below standards). We found females performed better than males in all grades; students in Independent Schools outperformed in all other schools.

Mathematics: One percent of students scored at level 5 (meet standards); 20 percent scored at level 4 (approach standard). We found that females performed better than males; students in Independent Schools performed better than those in all other schools.

Science: Less than one percent of all students scored at level 5 (meet standards); 81 percent scored level 4 (approach standards). We found that the females outperformed males; students in Independent Schools outperformed those in all other schools.

Overall student performance level by grade in the four subjects:
Arabic: By grade - Level 5 ranges from 2% (grade 4) to 5% (grades 6 and 11). Level 4 ranges from 20% (grade 9) to 26% (grade 11). Levels 1-3 range from 69% (grade 11) to 77% (grade 9).

English: By grade - Level 5 ranges from 2% (grades 4, 5, 7, 8 and 9) to 3% (grade 6, 10, and 11). Level 4 ranges from 7% (grade 9) to 11% (grade 5, 6 and 11). Levels 1-3 range from 84% (grade 6 and 11) to 91% (grade 9).

Mathematics: By grade - Level 5 ranges from less than 1% (grades 4 to 9 and 11) to just over 1% (grade10). Level 4 ranges from 9% (grade 4) to 30% (grade 6 and 11). Levels 1-3 range from 70% (grades 6, 10 and 11) to 91% (grade 4).

Science: By grade - Level 5 for all grades is 0%. Level 4 ranges from 15% (grade 4) to 26% (grade 11). Levels 1-3 range from 74% (grade 11) to 85% (grade 4).

Overall students' performance by school level in the four subjects:
Arabic: By school -– Level 5 in MoE ranges from 2% (grade 4 and 10) to 4% (grades 6, 7 and 11). Level 5 in private Arabic school ranges from 1% (grade 4) to 4% (grade 6). Level 5 in Independent Schools ranges from 3% (grade 4) to 9% (grade 11), with most grades at 6 -7%.

English: By school - Level 5 in MoE ranges from 0% (grades 4, 5 and 7) to 1% (grades 6, 8, 9, 10 and 11). Level 5 in private Arabic schools ranges from 0% (grades 4, 5, 7 and 9) to 2% (grades 6, 8 and 10). Level 5 in Independent Schools ranges from 4% (grades 7 and 9) to 13% (grade 11).

Mathematics: By school - Level 5 in MoE is 0% in grades 4, 8 and 11. In private Arabic schools the only student who achieved "Meets Standards" is in grade 10. In Independent Schools, Level 5 ranges from 0% (grades 4, 7 and 8) to 5% (grade 10).

Science: By school - the only student who achieved Level 5 in the MoE schools is in grade 8. In private Arabic schools, the one student who achieved Meets Standards is in grade 5. In Independent Schools, the percentage of students who Meet Standards is 0% in grades 6, and 9 to 11.

QCEA comparisons 2005 to 2006 (grade, gender, school type)
Arabic: Comparison by grade reveals that the percentage of students attaining Level 5 increased slightly from 2005 to 2006, in grades 6 and 11 and declined in grades 4, 7, 8, 9, and 10. The percentage remained the same in grade 5. Comparisons by gender shows that boys stayed at the same level; girls declined by one percent. Comparisons by school type reveals that MoE schools remained the same, private Arabic schools decreased, and Independent Schools declined by four percent.

English: Comparison by grade reveals that the percentage of students at Level 5 increased in grade 6, remained the same in grades 4, 5 and 7 to 11. Comparison by gender reveals that the percentage of both boys and girls who "Meet Standards" remained the same. Comparison by school type reveals that MoE schools remained the same, private Arabic schools declined by 2 percent, and Independent Schools declined by 5 percent.

Mathematics: Comparison by grade reveals that the percentage of students achieving Level 5 remained the same in grades 4 to 9 and 11. Comparison by gender shows that boys and girls both stayed at the same levels. Comparison by schools demonstrates that MoE and private Arabic schools remained the same. Independent Schools increased by 1 percent.

Science: Comparison by grade reveals that the percentage of students who "Meet Standards" remained the same at less that 1 percent in all grades. Comparisons by gender show that both boys and girls remained at the same levels. Comparison by school type show that MoE and private Arabic and independent schools remained the same.

Comparison by Cohort:
The comparison of independent school cohorts reveals that the performance of cohort 1 in both "Meets Standards" and "Approaches Standards" was better than the performance of Cohort 2 in Independent Schools which denotes a positive indicator because of accumulated experience over time in schools and a longer time for students been exposed to improved methods.

Student motivation:
Some people wonder if Qatari students taking QCEA 2006 tests did not regard them as serious. But Evaluation Institute Director Mr. Adel Al Sayed says that students' lack of motivation exists throughout the world. Certainly an absence of motivation affects exam outcomes, but the purpose of filtering and screening processes (psychometric and statistical controls) is to lessen that impact.

Though lack of motivation affects students' individual performance and perhaps school performance, it is not a factor in general performance.

Mr. Al Sayed added, "Although educators struggled with student motivation before we instituted reform, this is a problem that we need to address. In general, we are seeing students more interested in the classroom experience, but we need to address students' individual issues outside of testing situations."

Independent School performance:
There has been substantial discussion about the difference in performance levels by school type. Attention has been focused on the reduction in independent school performance despite deep SEC support. Mr. Al Sayed points out that reform is still in its early stage. "There was slight progress in independent school performance in many cases, but not to the degree that we'd hoped, given abundant support for these schools, as well as freedom and autonomy in the areas of curriculum standards and employment," he said.

But if one examines the School Report Card findings, there is clear upward movement in Independent Schools' performance. Mr. Al Sayed said, "We must continue to undertake serious reform efforts to boost school performance. We will be researching avenues for further improvement, and we must not forget that parental involvement will be an important component of any lasting change."

Although last year's independent school results were better than 2006 results, the reality is that results in both years were low. MoE school performance did not substantially change, but MoE performance is also described as low.

Dr. Enrique Froemel, the Student Assessment Office Director said, "The reform needs more time to stand on its own. Just as we cannot force a child to grow faster, we must allow reform to develop on its own, naturally. In some countries, reform has taken 30 years to truly take root. We have no choice but to be patient."

Next Steps:
After the dissemination of the QCEA results, Mr. Adel Al Sayed says each school needs to develop a plan to promote performance. "The key message of the reform is that principals must be accountable for his or her school's performance. We will provide support and services, but the role of the Evaluation Institute is to disseminate information and objectively analyze the results."