Nationwide test reinforces need for reform initiative

Performance Speaks to Need for Continued Reform

The results are in from Qatar’s new nationwide assessment test for primary, preparatory and secondary students, supplying a persuasive and pressing argument in favor of continued educational reform.

Qatar has gone to great lengths to develop assessment tools that comprehensively judge, every year and at every grade level, what Qatari students know and can do with their knowledge. The offspring is the Qatar Comprehensive Educational Assessment (QCEA), a set of tests that evaluates students’ knowledge and skills in Arabic, English, math and science. The tests are administered to K-12 students at all public and private Arabic schools. Over 80,000 students were tested in 2005, and the results of that screening have just been released.

Those results, as had been expected, are often lacking. The exams are evaluated in two ways: by so-called scale scores, in which results are assigned a simple numeric value, and by performance category, in which results are judged according to how close a student is to meeting national curriculum standards enforced by the Supreme Education Council (SEC).

"In general, Qatari students are way below international standards in the four subject matters,”

 said Dr. Enrique Froemel, the director of the Office of Student Assessment at the SEC’s Evaluation Institute. But he noted that around one-half of countries that carry out similar tests fall short. Believe me, they (Qatari students) are in good and wide company.”

Outside experts also note that the standards set by Qatar are particularly rigorous when compared to other countries, and that the well-established frailties of the country’s education system will require many years to mend. The mediocre performance argues in favor, rather than against, change: if results had been exemplary, the logic behind reform would be tenuous. Definitely the reform is necessary,” Dr. Froemel said.

While the scale-score numbers are useful for research purposes and for giving the public a straightforward reading of the test, SEC officials say that performance category reporting is the more meaningful evaluation tool since it shows how close students are to mastering subjects. Performance levels let you see what is behind the numbers,” said Dr. Markus Broer, a psychometrics and educational statistics officer at the Evaluation Institute. They lend meaning to the numbers.”

For 2005, virtually no students met the curriculum standards in math or science; in English and Arabic at most grade levels no more than 10%, and often significantly less than that, met the standards. The vast majority of students fell into approaches standards” or below standards' categories.

A look at the results of eighth graders -- a randomly chosen grade level in the middle of the educational cycle -- is indicative of the farther-reaching trends.

”Those students who should do better are starting to do better”

The Evaluation Institute did not release composite figures on performance level by grade level. But a look at the data broken down by gender shows that no students met standards in science or math, and only 1% of boys and 3% of girls met the English-language curriculum standards. Students fared slightly better in Arabic, where 4% of boys and 7% of girls met standards. While a percentage of students were judged to be approaching standards in the four subjects, the vast majority were below standards, to one degree or another.

The eighth-grade figures point to another consistent tendency, which has been seen in previous tests: girls outperform boys, often by a significant margin. In Arabic, 28% of Grade 8 girls approached standards to 17% of boys. In English 12% of girls did to 6% of boys. In science, 20% of girls approached standards to 17% of boys; in math 24% of girls approached to 18% of boys.

Evaluation specialists note there is a growing global trend for women to outperform men academically, even in math and sciences where in the past male students were stronger. That gap may be larger in countries such as Qatar where, for cultural reasons, women tend to spend more time at home and dedicate more time to study.

By the same token, non-Qataris tended to perform better on the screenings than Qatari nationals, although the gap in performance normally was not as striking as in the case of gender. In Arabic, 17% of Qataris approached standards compared to 30% of non-Qataris; in English, 8% of Qataris and 10% of non-Qataris approached standards. In science, 13% of Qataris vs. 28% of non-Qataris approached standards. In math, 15% of Qataris approached compared to 30% of non-Qataris.

Broken down by school type, Independent School students are beginning to edge out their counterparts in other public and private Arabic schools. The statistics showed no clear advantage to MoE-run or private Arabic schools in terms of test performance.

For those same eighth graders, 34% of students in IS schools approached

Arabic standards, compared to 20% in MoE and 22% in private schools. In English, 27% of IS pupils approached standards, to 6% of MoE and 12% of private. For science, 49% of IS, 15% of MoE and 16% of private students approached standards. In math, 51% of IS, 17% of MoE and 19% of private approached standards.

Critics say that it is to be expected that Independent School students do better on these exams, since their study programs are built around the SEC curriculum standards that all IS schools adopted in September 2004; other schools do not follow the same curricula. That apparently gives them a built-in advantage over their counterparts at other schools. The critics note that Independent Schools represent a small percentage of total schools, and that some were recently converted to IS after being the highest-achieving MoE schools.

But the Evaluation Institute believes these criticisms are not entirely valid, since the curriculum standards are in turn based around what experts determined Qataris need to be successful and productive members of society, rather than any values specifically linked to Independent Schools. Tinkering with the tests to reflect other curricula would be illogical, the Institute believes.

Dr. Froemel said that it is the performance of the IS students -- which will need to borne out in future examinations -- together with the Qatari national leadership’s well-demonstrated political will to reform education, and commitment of economic resources to that end, that are most important to highlight.

”Those students who should do better are starting to do better,” said Dr. Froemel. These are the first hints that things appear to be going in the right direction.”