By Hyacinth Evans, Contributor
Hyacinth Evans
THE PUBLICATION of the ranking of Jamaican high schools on the basis of students' performance in the 2003 CXC examination has generated much discussion about the quality of our schools and the state of education.
In many ways the publication of these examination results and the public debate that has ensued are positive developments in that they point to the achievements of some schools and highlight schools that need additional resources. The debate has shown once again as well, that education and the performance of our schools are of paramount importance to Jamaicans.
The report has ranked 148 technical and high schools according to the performance of their students in the 2003 CXC examination. To do so, the authors have relied on data compiled by the National Council on Education (NCE). Contrary to what has been reported in sections of the media, the authors of the report were not commissioned by the NCE to do the study. They merely used the data already compiled and available to the public. The NCE data reported CXC results in Mathematics and English only - the compulsory subjects sat by all CXC candidates.
Using this base data, the authors of the report analysed the performance of schools on 16 of the more than 30 subjects that CXC now examines. These 16 subjects are for the most part the academic subjects and, apart from English and Mathematics, do not fully represent the subjects typically taken by the technical and upgraded high schools a fact that places many of the schools at a disadvantage. Many students in some traditional high schools also take the subjects excluded from the analysis.
The authors have calculated a rating for each school by combining the percentage of students who gained a pass in a subject with the proportion of that year's cohort of students sitting the examination in that subject. The ratings assigned to each subject are added and divided by the number of subjects sat by students at that school.
SMALL PROPORTION
Each subject is assigned a score of between -8 and +8 with a score of +8 assigned when 96 per cent or more of the students pass the subject. The rating also takes into account the percentage of the cohort of students entered for the examination. If a school does very well in the subject passes, the rating can be lowered depending on the percentage of the cohort of students entered for the exam. Schools that enter a small proportion of the cohort of students get a low score. Those schools that enter a high proportion of students in a subject will get a higher score. It must be noted that the rating is that of the school and of its performance in the subjects. In other words the unit of analysis is the school and the subject not the student. The rating does not indicate how well students did on the examination (gaining a grade 1, 2, or 3), or how many subjects the student took (6, 8, 10?) or the percentage of subjects passed by each student.
In the NCE report of the CXC results of 2003, Dr. Minott was quoted as saying that many schools adopt ways of 'making results look good' by screening students and preventing some from taking the examination. The results are then reported as a percentage of those entered rather than as a percentage of the entire cohort. Using the percentage of the cohort who were entered for the examination to calculate the score addresses this problem. A quick glance at the rating of the schools shows that those schools that entered close to 100 per cent of their students are rated highest.
In fact many schools ended up with a negative score in part because they entered a small percentage of students. It must be noted that I was unable to speak with the authors of the report to confirm the method. This description is based on my reading of the Key to the report and a description provided by a representative of the NCE based on her understanding of it.
So the authors of the report have used the raw CXC scores in a subject, together with the percentage of students entered for the CXC examination to develop a rating of the school. They end up with a rating which does not take into account many of the variables which educators and policy makers in other countries have now decided to use in determining a useful and just assessment of schools. One important variable that has been omitted is the prior attainment of students. But there are other variables that are considered necessary depending on the school and the community served by that school.
The practice of publicising the performance of schools has become more prevalent in countries such as the United Kingdom and the United States where there is increasing marketisation of education, and parental choice of schools. The rationale is that information on schools' performance will increase accountability and aid parental choice.
It has been recognised, however, that such information on the schools' performance (called league tables in the UK) has to be accompanied by other information which contextualises the results and which facilitates parental decision making. There is a substantial body of literature which suggests that in an open enrolment system, contextual information is vital to assess the performance of schools in relation to the socio-economic characteristics of the community served by the school.
Such information includes the socio-economic status of the area, the social composition of the school, the prior attainment of students, the quality of the teaching staff and the like.
Many schools in the UK also include other outcome measures apart from academic performance of students such as truancy rates, and school leavers' destination (college, work, unemployment?) as a tool to aid parental choice. These additional variables yield an adjusted score for the school.
Hills et al and other resear-chers and policy analysts argue that unadjusted scores when used to rate schools are flawed because they fail to take into account these important variables.
THE CHALLENGE OF EQUAL EDUCATIONAL OPPORTUNITY
Whether or not we accept our own inadequate Jamaican version of the league tables (inadequate because so many important variables are omitted as many principals have pointed out in the past two weeks), it is important for us to understand the challenges that the education system faces today. We have to place the performance of the 148 schools in some historical and social context. Included in the 148 schools are 59 traditional high schools, 75 upgraded high schools formerly called comprehensive high schools, and 14 technical schools (these numbers are taken from the 2002 report of the NCE).
Many of these comprehensive (later high schools) were first established as new secondary schools or junior secondary schools which offered a different curriculum and prepared students for a different examination. The society held a different set of expectations for the graduates of the new secondary schools later comprehensive high schools. The three types of school were established at different periods, and have different histories and legacies which in many ways continue to exercise an influence on what happens in these schools.
High schooling in Jamaica was historically very selective, reserved for certain social groups. It is true to say that Jamaica inherited a dual system of education that was established for two social groups. The high school what we now call the traditional high school to distinguish it from the upgraded high school was established for one group the whites and a few browns with connections. Those of us who attended high school in the 1950s and 1960s entered schools that had only recently embarked on a process of integration socially. The slow process of changing from an elitist and very selective high schooling had begun, but even then the numbers who received scholarships and free places were pitiably small.
We cannot underestimate the magnitude of the task that this society faces in equalising a school system that was separate and unequal since its inception. Every post-colonial society and societies such as the U.S. and the UK which have historically excluded racial groups from a good quality education face this mammoth task. Jamaica is one of those countries.
THE WAY FORWARD
In the end what is the usefulness of the assessment of schools as it is conducted at present? Many criticisms have been made of the study and its conclusions, some of which have been discussed in this paper. The authors have said they are aware of some the flaws and will take these into account in future studies. However, it is clear that public ratings of schools must take into account more than the raw scores on examinations. To do an equitable and just job of assessing schools, we must have adjusted ratings which take into account prior attainment and characteristics of communities at the very least, though school level factors should ideally be included.
If we decide that these ratings are important for public information and debate then other types of data now available such as those from the Survey of Living Conditions must be included in the analysis, and more sophisticated statistical analyses carried out. This may mean that it is the Ministry of Education which
ought to be involved in such studies. The present method of assessing schools has too many flaws, and thus lacks validity. Some of these have already been discussed. But it is not made clear that many of the methodological decisions made in carrying out the study and in rating the schools are quite arbitrary. One such is the assigning of a score of between -8 and +8 for performance in a subject, and assigning a letter grade to a school based on the calculations. One has to read carefully to realise that when a school has 56-59 per cent of its students passing a subject it is assigned a score of 0, and assigned a score of -1 if 50 - 55 per cent of its students pass that subject. These are arbitrary decisions.
When such scores are then multiplied by a fraction in cases where fewer than the total cohort are entered for that subject, we can understand why so many schools received a failing grade. It is not difficult for schools to obtain failing a grade in this assessment!
Unsubstantiated
Gary Orfield of the Harvard Graduate School of Education in his article in the edited book 'Unequal Schools: Unequal Chances', has this to say of the use of unadjusted test scores such as those used in the report: "They are used so much because they are cheap, readily measurable, and easy to reify into an apparently valid measure of school accomplishment because there is considerable public confidence in them". Unfortunately the public discussion over the past two weeks suggests that the Jamaican public has a lot of confidence in this assessment. And then the authors draw conclusions that are not substantiated by the data. For example, the authors claim that there is a problem of leadership at the school level, because all schools have the same resources, and therefore all schools should be able to perform well. Wow! The only similarity in these schools is the prescribed curriculum. Those of us who work in schools know that all schools do not have the same resources.
Students' academic achievement at Immaculate Conception High School can be explained by a host of factors including: students' prior attainment (and the social capital that they bring to school), teachers' qualifications, the quality of the material and physical facilities at the school, parents' ability to afford extra lessons, parents' ability to help with home work, the value students place on education and many others. The performance of students at Trench Town High School and other poorly-rated schools can be explained on the basis of these same variables.
In my view, what is most needed in many of our schools - especially the upgraded schools - is more pedagogical support at the school level to help teachers craft appropriate methods to cater to the needs and abilities of our students and to enact a curriculum that requires more ambitious student centred teaching. Teachers in all countries that have adopted similar curricular reforms that demand more ambitious teaching - as the ROSE curriculum does - need this kind of support. Above all we need teachers who have and can communicate high expectations for students from poor backgrounds. The new curriculum requires teachers who can help students to think critically, problem solve, and work harmoniously with peers. All this goes beyond academic achievement, though this is important. If we continue with this public assessment and ranking of schools, I would like to see an expanded set of criteria on which to judge schools. Criteria such as self confidence, self esteem, a social conscience, commitment to country and a well developed sense of self. It is instructive to note that recent reports out of the U.K. indicate that Scotland, Wales and Northern Ireland have now decided to abandon the league tables in favour of more school specific descriptions, because they omit much of what is important in understanding the work of schools.
Hyacinth Evans is professor of Teacher Education and director of the School of Education, University of the West Indies, Mon.
