Evidence to date on the quality of education in Vietnam – including previous Young Lives findings, PISA 2012 and PISA 2015 – indicates high levels of learning among Vietnamese students, particularly in subjects like Maths. However, there are concerns that this reflects rote learning and proficient test-taking skills, rather than ‘meaningful learning’ or the ability to use knowledge and skills in a variety of new situations. A recent World Bank report revealed concerns that young people in Vietnam do not have the required skills for a 21st century labour market, such as problem solving, critical thinking, communication and teamwork. Vietnam’s Ministry of Education and Training (MOET) has similar concerns, and reforms are underway to develop a new ‘competency-based’ curriculum which will support the development of these skills.
To date, there is little evidence on whether such concerns are fair, and if Vietnamese students actually lack skills like problem solving and critical thinking. In the 2016-17 Young Lives Vietnam school survey, which included 8,860 Grade 10 students across 5 provinces, we therefore designed a ‘transferable skills’ test to assess students’ problem solving and critical thinking skills [i]. This included problem-solving items adapted from PISA 2003, and critical-thinking items adapted from the College Work and Readiness Assessment (CWRA+). Since the original tests were not developed for use in low- or middle-income contexts, all items were adapted to ensure that they were relevant for the students in Vietnam – for example, a metro map in one problem-solving unit was changed to a bus map. The items were then translated into Vietnamese and piloted with students across diverse school contexts in Vietnam to establish the difficulty level of the tests, and to further improve their suitability for the Vietnamese context [ii].
So what have we learned about problem solving and critical thinking skills from these tests – are Vietnamese students developing so-called ‘21st century skills’ at school?
Vietnamese students are not just ‘rote learners’
Using Item Response Theory (IRT) analysis, all students were given a problem-solving score and a critical-thinking score, on a scale with a mean of 500 and standard deviation of 100 (we’ll be sharing a technical note with details of how we confirmed the validity of our tests soon). In terms of problem-solving skills, based on their scores and difficulty of the items that they were able to answer, we identified:
- 42% of students as weak or emergent problem solvers (below 480)
- 43% of students as basic problem solvers (480 – 579)
- 15% of students as competent problem solvers (above 580)
A fairly high proportion of students were identified as weak or emergent problem solvers – i.e., they were only able to answer basic comprehension questions, and did not answer any problem-solving questions correctly. However, over half the students were either basic problem solvers (able to answer easier problem-solving questions correctly) or competent problem solvers (able to solve more complex problems).
In terms of critical thinking skills, we identified:
- 25% of students as emergent critical thinkers (below 420)
- 39% of students as basic critical thinkers (420 – 539)
- 36% of students as competent critical thinkers (above 540)
While a quarter of students were only able to answer basic comprehension questions and simple critical thinking questions (‘emergent critical thinkers’), three-quarters of the students were able to answer more complex critical thinking questions, and so were identified as either basic or competent critical thinkers. Contrary to widely expressed concerns in Vietnam, these results indicate that Vietnamese students are more than just rote learners – the majority of students in our survey have at least basic problem-solving and critical-thinking skills.
Do high performers in school subjects also perform highly on problem-solving and critical-thinking tests?
Our problem-solving and critical-thinking tests aimed to assess ‘cross-curricular’ skills, rather than subject-specific skills. However, a key question is whether problem solving and critical thinking really are ‘higher order’ cognitive skills, or whether those who perform highly on these tests are also high performers in school subjects (which may in turn simply reflect proficient test-taking skills). As a first step to explore this question, we looked at district-level performance on the four tests that were administered in the Young Lives Vietnam school survey at the end of the academic year: Maths, English, problem solving and critical thinking (see Figure 1).
Figure 1: District-level performance on end-of-year Maths, English, problem solving and critical thinking tests in Vietnam
Figure 1 shows that students in Da Nang (a city in central Vietnam) and Hung Yen (a province close to Hanoi) are, on average, the higher performers in the end-of-year Maths and English tests, with students in Lao Cai (a disadvantaged Northern province) as the lowest performers on both these tests. Based on previous Young Lives studies and beginning-of-year tests in the current school survey, these are largely the results we would expect. However, the graphs for problem-solving and critical-thinking scores show a completely different pattern, with students in Phu Yen (a disadvantaged Southern province) as the highest performers in both tests. Notably, one district in Da Nang moves from a mid-level performer in Maths and English to the lowest performing district in problem solving and critical thinking. This suggests that high performance in school subjects does not necessarily mean high performance in problem solving and critical thinking.
Discussion
The 2016-17 Young Lives Vietnam school survey provides unique evidence on the development of 21st century skills. While it is clear that young people in Vietnam do have 21st century skills – contrary to the common claim that Vietnamese students are simply ‘rote learners’ – it also seems that there is a distinction between high performance in school subjects and skills such as problem solving and critical thinking. This clearly raises further questions – which students are better at problem solving and critical thinking, and what kinds of schools do they attend? Do their teachers adopt different classroom strategies compared to students who perform highly in school subjects such as Maths? These are some of the questions we aim to answer with forthcoming analysis; in the meantime, these initial results indicate that 21st century skills add an important dimension to our definitions and measurement of ‘school effectiveness’.
[i] The transferable skills test was also administered to students in the 2016-17 Young Lives India school survey – we will be sharing cross-country findings on transferable skills over the next few months.
[ii] More details on the adaptation and piloting of the transferable skills tests in India and Vietnam are included in this article.
Evidence to date on the quality of education in Vietnam – including previous Young Lives findings, PISA 2012 and PISA 2015 – indicates high levels of learning among Vietnamese students, particularly in subjects like Maths. However, there are concerns that this reflects rote learning and proficient test-taking skills, rather than ‘meaningful learning’ or the ability to use knowledge and skills in a variety of new situations. A recent World Bank report revealed concerns that young people in Vietnam do not have the required skills for a 21st century labour market, such as problem solving, critical thinking, communication and teamwork. Vietnam’s Ministry of Education and Training (MOET) has similar concerns, and reforms are underway to develop a new ‘competency-based’ curriculum which will support the development of these skills.
To date, there is little evidence on whether such concerns are fair, and if Vietnamese students actually lack skills like problem solving and critical thinking. In the 2016-17 Young Lives Vietnam school survey, which included 8,860 Grade 10 students across 5 provinces, we therefore designed a ‘transferable skills’ test to assess students’ problem solving and critical thinking skills [i]. This included problem-solving items adapted from PISA 2003, and critical-thinking items adapted from the College Work and Readiness Assessment (CWRA+). Since the original tests were not developed for use in low- or middle-income contexts, all items were adapted to ensure that they were relevant for the students in Vietnam – for example, a metro map in one problem-solving unit was changed to a bus map. The items were then translated into Vietnamese and piloted with students across diverse school contexts in Vietnam to establish the difficulty level of the tests, and to further improve their suitability for the Vietnamese context [ii].
So what have we learned about problem solving and critical thinking skills from these tests – are Vietnamese students developing so-called ‘21st century skills’ at school?
Vietnamese students are not just ‘rote learners’
Using Item Response Theory (IRT) analysis, all students were given a problem-solving score and a critical-thinking score, on a scale with a mean of 500 and standard deviation of 100 (we’ll be sharing a technical note with details of how we confirmed the validity of our tests soon). In terms of problem-solving skills, based on their scores and difficulty of the items that they were able to answer, we identified:
- 42% of students as weak or emergent problem solvers (below 480)
- 43% of students as basic problem solvers (480 – 579)
- 15% of students as competent problem solvers (above 580)
A fairly high proportion of students were identified as weak or emergent problem solvers – i.e., they were only able to answer basic comprehension questions, and did not answer any problem-solving questions correctly. However, over half the students were either basic problem solvers (able to answer easier problem-solving questions correctly) or competent problem solvers (able to solve more complex problems).
In terms of critical thinking skills, we identified:
- 25% of students as emergent critical thinkers (below 420)
- 39% of students as basic critical thinkers (420 – 539)
- 36% of students as competent critical thinkers (above 540)
While a quarter of students were only able to answer basic comprehension questions and simple critical thinking questions (‘emergent critical thinkers’), three-quarters of the students were able to answer more complex critical thinking questions, and so were identified as either basic or competent critical thinkers. Contrary to widely expressed concerns in Vietnam, these results indicate that Vietnamese students are more than just rote learners – the majority of students in our survey have at least basic problem-solving and critical-thinking skills.
Do high performers in school subjects also perform highly on problem-solving and critical-thinking tests?
Our problem-solving and critical-thinking tests aimed to assess ‘cross-curricular’ skills, rather than subject-specific skills. However, a key question is whether problem solving and critical thinking really are ‘higher order’ cognitive skills, or whether those who perform highly on these tests are also high performers in school subjects (which may in turn simply reflect proficient test-taking skills). As a first step to explore this question, we looked at district-level performance on the four tests that were administered in the Young Lives Vietnam school survey at the end of the academic year: Maths, English, problem solving and critical thinking (see Figure 1).
Figure 1: District-level performance on end-of-year Maths, English, problem solving and critical thinking tests in Vietnam
Figure 1 shows that students in Da Nang (a city in central Vietnam) and Hung Yen (a province close to Hanoi) are, on average, the higher performers in the end-of-year Maths and English tests, with students in Lao Cai (a disadvantaged Northern province) as the lowest performers on both these tests. Based on previous Young Lives studies and beginning-of-year tests in the current school survey, these are largely the results we would expect. However, the graphs for problem-solving and critical-thinking scores show a completely different pattern, with students in Phu Yen (a disadvantaged Southern province) as the highest performers in both tests. Notably, one district in Da Nang moves from a mid-level performer in Maths and English to the lowest performing district in problem solving and critical thinking. This suggests that high performance in school subjects does not necessarily mean high performance in problem solving and critical thinking.
Discussion
The 2016-17 Young Lives Vietnam school survey provides unique evidence on the development of 21st century skills. While it is clear that young people in Vietnam do have 21st century skills – contrary to the common claim that Vietnamese students are simply ‘rote learners’ – it also seems that there is a distinction between high performance in school subjects and skills such as problem solving and critical thinking. This clearly raises further questions – which students are better at problem solving and critical thinking, and what kinds of schools do they attend? Do their teachers adopt different classroom strategies compared to students who perform highly in school subjects such as Maths? These are some of the questions we aim to answer with forthcoming analysis; in the meantime, these initial results indicate that 21st century skills add an important dimension to our definitions and measurement of ‘school effectiveness’.
[i] The transferable skills test was also administered to students in the 2016-17 Young Lives India school survey – we will be sharing cross-country findings on transferable skills over the next few months.
[ii] More details on the adaptation and piloting of the transferable skills tests in India and Vietnam are included in this article.