Student assessment and how we can never just teach to learn

For students to be successful in an unstructured knowledge economy they must not only be able to implement their knowledge, but also be able to assess solutions to determine their appropriateness or chance of success. 

I believe that implementation of theoretical or taught knowledge alone cannot drive, or be expected to drive, a knowledge economy forward. What we learn and understand depends on the context within which we learn it. I think knowledge gained in one situation cannot be directly applied to another, but might offer very good hints as to the appropriate course of action. Rather I think it is knowledge combined with critical assessment of its implementation that ultimately adds value. Furthermore, this assessment or evaluation should be done iteratively and the gained insights applied to further advance the process.

I think “Teaching” and “Learning” in Higher Education is often grouped together simply as “Teaching and Learning” for expediency’s sake. There are probably many academics who regard them as equal, specifically with the understanding that teaching = learning (in that logical order). For me, this is far from the truth because teaching can at best facilitate one part of learning (and not the most important part).  In mathematically rigorous subjects I think the one aspect of learning which is most important, but almost impossible to teach, is self-assessment.

Self-assessment speaks to having confidence in your knowledge and being able to critically evaluate it in order to further or improve it by identifying areas where your knowledge is still lacking. It also implies ownership of one’s knowledge and one’s learning. With the realization of the importance of self-assessment in place, the (higher) education system becomes an accessible library of knowledge instead of a battery hen force-feeding factory. 

In my opinion the realization of true self-assessment is also the biggest hurdle when moving from high school into tertiary education. In mathematics this is the shift from knowing the “right” answer to motivating the rationale behind, and the steps used, to obtain an answer. 

However, the self-assessment cannot stop there. It has to be entrenched in the act of facilitating the mastering of a subject by equipping students with the tools to critically evaluate their own progression outside of formal assessments. I think the biggest obstacle to this entrenchment is once again the current methods of assessment by higher education institutions. 

For timetabling purposes subjects are compartmentalized into modules. Each of these teaching modules are individually assessed in order for the student to progress.  Of course the goal is that these modules should together form a coherent body of knowledge. However, I believe it would pay to remember that knowledge is not modular, but rather a continuous flow. Unfortunately in the modern era lecturers’ teaching and research work are often experienced to be competing interests and time spent on improving teaching and learning is time taken away from research. Thus I believe there is a real danger that teaching is becoming even more modular as most lecturers probably only consider the specific module content in preparing lectures and not necessarily exactly how the module fits into the bigger scheme: this is mostly left to the programme design committee. Thus as teaching becomes more modular the knowledge flow is interrupted since students experience disjointed facts and pockets of knowledge. A direct consequence of the interrupted flow is that deduction (and intuition) is hindered and as a consequence the modular knowledge is more memorized than truly understood. Furthermore, since each module is assessed in isolation, there are little incentives for students to integrate knowledge.  

To encourage self-assessment and integration of knowledge I believe there must be a place in every degree programme where the overall body of knowledge is assessed as well as feedback on the learning experience is given.  One way of achieving this is to have an oral examination, where the student is encouraged to share what they have learned as well as understood during the programme.  Such an oral examination could serve as preparation for any interviews that the potential employee will undergo in the future. 

Structured and unstructured knowledge economies

As I understand it, part of BIUST’s mandate is to help transform the economy of Botswana from a resource based one to a knowledge based economy. The sense I’m currently getting is that BIUST wants to achieve this by establishing a technology/innovation park (part of the university’s founding mandate). However, this venture is still in the planning phase. On the academic side the general consensus seem to be that this transformation would be driven by increasing the number of postgraduate degrees. I believe that will definitively contribute to making this transition, but to achieve this it has to inform all the activities at BIUST. In particular the undergraduate teaching has to be transformed to deliver students who will be equipped to facilitate this transformation through to postgraduate level and beyond.

In addition to what is being taught, I think the environment in which recent graduates will be seeking employment needs to be considered since this will inform how the what is being taught. I propose classifying the environment in which graduates might find themselves in into two (sub)types of knowledge economies:

(More) structured knowledge economy: a knowledge economy where job descriptions, entry requirements, and promotion trajectories are (mostly) defined and known.

This is typical in a more traditional, well-established, industrialized economies where universities deliver, amongst other things, industry-specific trained graduates.

(More) unstructured knowledge economy: typically smaller knowledge economy where graduate absorption is limited and employment outside of traditional occupations are highly dependent on innovation and entrepreneurial skills. In other words this type of economy is more prevalent amongst developing nations which lack well-established and diverse industries. Furthermore in these knowledge economies the specific demand for highly skilled graduates is difficult to quantify.

I don’t think the above are mutually exclusive, both could be prevalent in sectors of most countries’ economies. In most resource-based economies the mining industry could probably be considered structured:  problems that skilled labour are tasked to solve are within their knowledge parameters or confined to the tools available to solve it. Thus out of the box thinking is not necessarily required or encouraged on a large scale.  On the other hand sectors which are technology based or innovation driven (even within the most developed countries/economies) could be considered an unstructured skills economy: large scale, almost continuous, evaluation and improvement of solutions are needed to drive the economy forward.

From what I have read about Silicone Valley it is clear that at least some tech companies are more interested in hiring employees based on their interests and skills set than their formal education. An extreme example of this trend is probably the Thiel Fellowship which pays high schoolers to not go to college but to rather pursue their ideas. In the same way PhD graduates are attractive to potential employers due to their demonstrated independence, determination and higher order thinking skills: outside of the formal research environment the graduate’s highly specialized PhD research project is only attractive to a very narrow interest group. Therefore any investment or inceptive aimed to increase the number of PhD graduates have to consider the probable absorption of these graduates into the economy. South Africa is currently a prime example of how highly skilled individuals are trapped in limbo (see here for more detail): for a country the oversupply of highly skilled individuals is considered a “good” problem to have, but it sucks for the individual who has invested so much and is consequently being treated as an unwanted stepchild with limited prospects.  

To avoid a similar bottle neck in graduate employment, Botswana (or any developing nation that is expanding its tertiary education sector) has to consider the challenges faced by graduates: Either in terms of employment in sectors where the graduates’ skill set do not directly match that which is required by the position which will necessitates some reinvention; or as change agents/transformation drivers (predominantly manifested as, but not limited to, entrepreneurs). This should be address as part and parcel of the drive towards a knowledge economy and on the appropriate level: tertiary education. Due consideration need to be given to these challenges when planning undergraduate curriculum as well as postgraduate research projects.

I do believe that the incentives are lacking in the tertiary education environment to achieve the transfer of the skills set needed to deal adequately with the realities of an unstructured knowledge economy. In particular with regards to postgraduate students who are probably most expected to drive the transformation from a resource to a knowledge economy. I think one of the main reasons for this is an unhealthy fixation on research publications as measure of academic quality, especially at institutions who lack other formal infrastructure to evaluate all the aspects of academic life (research, teaching, and, service/community interaction activities). To my mind, if the number of publications and years spent in academia are the only measures of an academic’s quality it could lead to severe complacency, in particular amongst senior academics. Furthermore, the situation is compounded in environments where the number of researchers in a specific field is limited.  In these cases researchers may well become a law unto themselves and have low tolerance for critical engagement since they see themselves as the only recognizable expert.

Even when high impact and high quality papers are produced in developing economies this must still be done with due consideration for students’ education as a primary concern. Highly specialized research implies highly specialized skills. If there is no local absorption of highly skilled graduates, we are essentially producing graduates for a structured skills economy elsewhere (in this sense I think an argument around decolonizing higher education in Africa can be made).

However, I don’t think any philosophical arguing would do much to change the situation since, as with any human behaviour, the situation will only change if the incentives, rewards, and in effect what is truly valued, changes. Check back later for my suggestions.

Observations from HRDC fair and STEM festival

The HRDC (Human Resources Development Council) recently held a job/skills/tertiary education fair aimed at school leavers in Gaborone.  A similar one was also held in Francistown. I attended on behalf of the Department of Physics and Astronomy as a scientist on the ground to give insight into the field and to also try and answer the more scientific questions from learners.  The HRDC fair serves as the main enrolment vehicle for BIUST since it is the best way to access learners in the more populated areas of Gaborone and Francistown.

BIUST with the trophy for second best stall at HRDC fair.

I was on duty for a day and a half and during this time I interacted with a number of students. Apart from the expected procedural questions the main question I was confronted with was something along the lines of which course is “most marketable”.  I understood this to mean “which course would deliver me a BMW or a Golf5 the quickest” with little consideration for their own scientific/academic interest.  The question was most prevalent amongst males. I tried to counter this question by asking them what they were interested in, but had little success with answers.

Some came with their parents and in most of these instances the learners did little talking. However, the parents were very engaging, but only to the point where I could support their opinion (and thus choices they have already made on behalf of their offspring).

Only a handful of learners seemed to be informed and/or have given their study choices some thought. Thus it was apparent to me that learners had little knowledge of the job market as well as the options available to them. Consequently, these learners were probably only considering traditional graduate careers or careers to which they have seemingly been exposed to through the media (“chemical and forensic sciences” were the top course for which students enrolled).

I am convinced that these learners are in dire need of guidance and career counselling.  The number of learners successfully enrolling at universities in Botswana may be manageable enough to at least have every candidate complete a personality questionnaire. This can inform their own as well as the university’s decision on whether to enrol them in a specific course.  

At the STEM (Science, Technology, Engineering, and Mathematics) festival in Francistown students were also enrolled, but the main aim of the festival was to expose learners to the STEM subjects. I gave a talk on the first evening and it was well received. I gathered that mostly teachers were in attendance. From the audience questions, it was clear that there is a great need for extensive scientific enrichment for both learners and teachers alike.

I came away from these two events with a renewed sense of responsibility as a scientist, especially one in a developing country where the job market cannot (currently) absorb all the graduates which universities produce. I think the question I will be asking myself for some time will be whether our teaching and degrees actually deliver on the promises we make so liberally to entice young people to enrol for our courses.

The Ubuntu test

Teaching the Modern Physics (introductory Quantum Mechanics) course at BIUST I had difficulty in getting the students to not only to engage the course material, but their class mates as well.

The first test I gave the class did not go so well.   In the spirit of giving them a chance to better their mark and also to force more cooperation between them I gave them a second test.  I called it the Ubuntu test (after the IsiXhosa saying made famous by Nelson Mandela that goes something like "I am well if you are well").

For the Ubuntu test, l set aside a two hour tutorial period. At the start of the period I gave them one question that needed to be in the Ubuntu test. For the next 30 minutes they had to decide what the other questions in the test will be.  The test had to have a total of a certain amount. All homework, tutorial or other test questions were fair game. However, everyone in the class had to be in agreement with regards to the questions.

 After thirty minutes they had to give me the questions so that I could go and set the test which they will start to write individually with an hour left in the period.  The kicker was that since this is an Ubuntu test the class average will be awarded to all. The idea was that in the remaining they could discuss and bring each other up to speed.

It worked relatively well, but unfortunately I think their confidence let them down and thus they could not use the time constructively.  At least the average was a pass.