One standardised assessment tool has become the key benchmark for national governments to judge their schools’ successes. But the academic rankings generated by the Program in International Student Assessment (PISA) are eclipsing important questions such as how particular groups of students are doing in school or graduation rates.
Educational excellence globally has become synonymous with outstanding achievement as gauged by PISA, which is steered by the Organisation for Economic Cooperation and Development (OECD).
But what does this mean? PISA captures both global and domestic or local snapshots of student achievement. PISA ranks, for example, how Finland measures up against Poland and other countries, but also provides in-depth reports that summarise how population groups — such as immigrants and non-immigrants — are doing within a country’s school systems.
PISA academic rankings increasingly dominate global education policy discussions. That means that opportunities to consider how particular groups of students are faring, and even the basic question of school completion rates, are being eclipsed by concerns over national scores.
A sole focus on achievement standards can obscure relatively poor high school graduation rates as is the case in parts of Canada.
Lost original PISA aims
The OECD developed a standardised measure of achievement after recognising that many children who attended school were not learning and accumulating skills.
The initial aim of PISA was to develop indicators that could be used to complement information on quantity of education — such as participation, access and attainment — with information on the quality of formal and informal learning opportunities the students received.
PISA evolved as a triennial measure to test 15-year-old students from all over the world in key domains such as reading, mathematics and science. The OECD has also pioneered PISA assessment in areas such as financial literacy, collaborative problem-solving and global competence, and is working on new tools to assess creativity and critical thinking.
When PISA captures differences in how population groups are doing (for instance, immigrants and non-immigrants, or students from low versus high socio-economic backgrounds), it also associates these outcomes with other important influences. PISA asks students to report how they feel, how satisfied they are and what motivates them in school and in life.
But among many policy-makers, discussions about PISA often focus on how scores influence the behaviour of teachers and other educational professionals.
Score-dominated educational policy-making has led many educators to criticise and deride PISA as an instrument that encourages short-term fixes to education, narrows the curriculum to tested subject areas and increases stress among teachers and students.
High standards, equity and graduation rates
But rather than discuss such a laundry list of points that have been repeatedly noted by academics and teachers, we propose that policy-makers consider that it’s possible to have high standards, equity and completion rates all at once.
This is evident in countries like Japan and Finland that have consistently boasted top PISA scores, small gaps across different student subgroups and exemplary high school graduation rates at 97 percent.
Some countries have been able to increase educational participation and high school graduation rates while improving or maintaining achievement standards, as was the case in Turkey.
Canada’s education system is decentralised to the provincial level. This means that each province is free to develop its own curriculum and regulate its education system without oversight by the federal government.
While the mainstream media has increasingly recognised Canada for having a strong education system — based almost exclusively on PISA performance — this recognition obscures important provincial differences.
The table below shows how Canadian provinces ranked nationally in their PISA 2015 reading, mathematics and science results. (The table does not consider global PISA rankings: for example, in 2015, Alberta was first in Canada for science but Singapore was first globally.) The table also shows provincial graduation rates for 2015 and 2016 from Statistics Canada.
Graduation figures reflect the extended time of up to two additional years provided to students to complete their diploma
Alberta and Québec — two of the highest achieving Canadian provinces comparatively with PISA scores — have the lowest secondary school completion rates. Conversely, Newfoundland and New Brunswick, which both fall in the bottom half of the provincial PISA rankings, had the highest graduation rates.
New litmus test
The 2018 PISA results will be released on December 3. They will undoubtedly, and for good reasons, attract a great deal of attention. Those countries that rank high in academic achievement will be applauded and those that fall to the bottom may consider implementing reforms to align their policies to those of high-achieving jurisdictions.
The Canadian example highlights the importance of viewing such results in conjunction with other important educational indicators such as graduation rates. Such data are available from the OECD through its annual Education at a Glance publication and in many decentralised systems through national statistical agencies.
While governments around the world may be tempted to model education reforms after high-achieving systems, we would like to remind them that they should never lose sight of the bigger picture.
Students who fail to complete secondary school face significant challenges that result in lasting personal, social and economic hardships. And correlating completion rates with data about student sub-populations could provide systems the opportunity to develop targeted policies to respond to at-risk student groups.
Implementing policies that promote long-term achievement and educational attainment is possible but requires going beyond news headlines.
By Louis Volante, Professor, Faculty of Education, Brock University; Francesca Borgonovi, British Academy Global Professor, UCL, and Jo Ritzen, Professor of International Economics of Education, Science and Technology, Maastricht University