The Troublesome Nature of Learning Outcomes

In higher education learning outcomes are part of the bedrock that informs assessment, qualifications and course approval processes.  

They are important statements and as such we should give serious consideration as to the nature of learning outcomes and how we use them.

I have a growing sense of unease that we have collectively bought into a set of assumptions about learning, teaching and the nature of knowledge that limits our understanding of the processes involved. 

Most particularly, I am concerned about poorly formed and limited thinking that surrounds the conceptualisation and use of ‘learning outcomes’ in third level teaching contexts.

There is an instrumental view of learning is dangerously simplistic.  It regards education in industrial terms and hence learning outcomes constitute the produce. Unfortunately this view is pervasive as it seems to fit with the current obsession with economic rationality.  

In this bizarre world-view learning outcomes are given numerical value, assigned to levels, added together, divided up, stated as percentages and generally treated as though they were clearly defined, uniform and self-contained entities.

Much of the prevailing dogma and practice in higher education supports this commodification model of learning outcomes.  Some of the blame rests with the quality and regulatory processes.   There seems to be a relentless quest for the normalisation of practices across the sector.  In my view doing the same thing across different teaching, learning and assessment contexts is seldom an indication of quality.  In addition there is too much emphasis on procedural rather than conceptual documentation. 

Most detrimentally they may be guilty of the same mistake as many religions, albeit unwittingly, they encourage the ‘outsourcing’ of deep thinking. Many teachers regard learning outcomes as unquestionable 'givens' within a course or subject area.  As a consequence there is no incentive to think deeply about what they are trying to achieve. 
This can lead to passive acceptance of handed down templates and and safe formulae. 

I argue that learning outcomes are troublesome concepts and we should treat them with a great deal of caution and critical scrutiny. By arguing that learning outcomes are troublesome I am suggesting that they open up questions about the nature of knowledge, the essence of a discipline or subject area and the appropriateness of the assessments.  I like this kind of trouble.

The implicit assumptions of learning-outcome-based pedagogic design

Generating learning outcome statements is an important task in pedagogic design.  It is often considered as the first step.

At first glance the process may appear to be simple and straightforward "we just need to describe what the learner needs to know and what they can do" but as we shall see, this often brings up some critical questions as to the essence of subject knowledge and the nature learning.

When we underpin pedagogic design on the basis of a series of learning outcome statements we should be aware of the implicit assumptions we are making.  I highlight in particular three fundamental questions that we need to consider:

To what extent should we focus on the learning process rather than the learning outcome?

To what extent is a learning outcome capable of being described, verified or assigned a particular value or quality?

To what extent should we associate learning outcomes with individual versus group or community competence?

Process versus outcome orientations

One of the first muddles in which we tend to find ourselves is the degree of emphasis we place on either the process of learning versus the outcome of learning.

I suggest that the need for accountability is at the heart of an outcome-orientation of learning.  Accountability is necessary as there is a societal value for certain skills and competence (e.g. doctors, engineers, accountants etc.) and hence the need to assess what people know and what they can do.  There is also a need for educational accountability.  People need to be assured that the course they pursue will lead to the skills and capabilities that they desire.  A third area of accountability is individual - we set goals for ourselves and therefore we need to identify milestones of achievement and learning outcomes often play this role.

In contrast, a process-orientation emphasises the intrinsic value of learning.  The purpose is participation and engagement in directed inquiry.  Our natural resources such as curiosity, creativity and discussion help to drive and direct our learning.  Obviously the learning process is directed toward a particular task or goal but the purpose is the process not the goal.

Here is an analogy, each day I take the dog for a walk and we either go to the park or to the river.  It is easy to see that my purpose is not to get to these destinations but to walk the dog - the destination is secondary.

Consider another situation, suppose most days I do a cryptic crossword and I enjoy this process.  After many years I get better (slowly in my case) and people might say that I was competent at crosswords. My goal is to continue to enjoy working on each puzzle.  I develop my competence so that I can continue to engage in the process.

These days, there is much discussion about the need to develop scientific and analytical thinking in our young people.  There are many calls to improve the effectiveness of science teaching and to increase the numbers of graduates in the science, technology, engineering and mathematics (STEM) disciplines.

Now ask yourself which of the two orientations discussed above - an outcome-orientation or a process-orientation - would be most likely to engender a passion for science? People often talk about a 'love' for a particular subject; I think what they really mean is their love for the practices associated with the field.  They learned to love these practices through a continuous process of participation.  
 

The elusive description

The second critical question that we need to address in relation to learning outcomes is that of description. The following definition of a learning outcome is provided by the ECTS Users Guide (ECTS stands for the European Credit Transfer System):

Learning outcomes are statements of what a student is expected to know, understand and/or be able to demonstrate after completion of a process of learning.  

(p 47)

How useful is this definition?  Let's take a closer look.  The definition characterises learning outcomes (LOs) as statements of expected competence - for expediency I'm going to cluster knowledge, understanding and demonstrableness together under the broader term 'competence'.

Suppose for a particular course, we are presented with a list of learning outcomes as 'statements of expected competence' defined above. We would need to investigate further in order to fully appreciate what we are dealing with.

For example, we might ask does each statement capture the entirety of the competence?  Clearly this would be very difficult. It is challenging to find a single statement that encapsulates the essence of science or management. One of the first to point this out and to provide an example of good troublesome thinking in relation to learning outcomes was Plato in the Meno dialogue.

More likely we would interpret the series of learning outcome statements as follows: we would say that the learning outcomes are a series of statements such that when taken together, they adequately describe the expected competence.  This is an important implicit assumption often overlooked in course and assessment design.

With this in mind I propose a new (and hopefully more useful) definition:

Learning outcomes comprise a series of statements such that when taken together, they adequately describe an expected competence.

Notice the effect of my additional condition requiring that LO statements exist as part of a set or series such that when taken together they adequately describe the expected competence.  In other words each learning outcome is defined in relation to its membership of the set and overall the set provides an adequate description of the competence.  Two points of note; first, you cannot deal with each LO in isolation and second the term 'adequate description' needs to be referenced to some outside value or authority.

Furthermore, we need to bear in mind the ideas of Russell and Bateson in relation to logical types.  Essentially we need to be careful about the distinction between statements and 'statements about statements'.

Individual versus group competence

The third critical question that I wish to discuss is individual versus group competence. The ECTS definition above clearly refers to competence as an individual (student) attribute.  This seems to be the standard approach and given our cultural emphasis on individual performance it seems inevitable that we focus on the person rather than the group.

But are we missing something?  Surely there are numerous contexts in which collective effort is essential and abilities are valued in so far as they contribute symbiotically to a collective competence. Here I'm not talking about the weak troublesome construct of 'ability to work as a team' - ironically regarded as an individual attribute.  I am referring to competence that manifests itself through collaboration.

I came across a good example of this recently.  I observed two men boarding a tram: one was in a wheelchair and had a serious physical disability, the other man who pushed to wheelchair was blind.  Together they negotiated the city streets, used public transport and managed their affairs.  Considered together each was mobile and visually aware, considered as individuals each was deficient on one of these abilities.

My worry is that when we concentrate on individual competence we miss the potential that only becomes apparent in group effort.  This point and indeed all of the points I make above are not meaningless musings on the theory of knowledge.  They have practical implications for education and society.

As teachers and academics we are challenged to inquire deeply about the learning outcomes we use.  I hope I have convinced you that we should not take this task lightly and hopefully troubled thinking will arise in the process.

Plato's Meno

Plato's Meno

One of the first accounts of the troublesome nature of learning outcomes is given in Plato's Meno. 

Plato used a series dramatically constructed dialogues as vignettes to illustrate philosophical points he wished to make. In the Meno Plato describes a conversation between Socrates, Meno (hence the title), a slave boy and Anytus.

Meno puts the following problem to Socrates:

"Can you tell me, Socrates, can virtue be taught? Or is it not teachable but the result of practice, or is it neither of these, but men possess it by nature or in some other way?"

Socrates and Meno proceed by agreeing that whereas they would recognise instances of virtue, as actions or as a quality in a person, it is difficult to know the essence of what it means to be virtuous.

So herein lies Meno's paradox how can we recognise examples of virtuous behaviour while not knowing the entirety, or the common form, of the concept. In other words, how can look for something (a form of knowledge) when we don't know what it is?

The important point is that Meno's initial question on how we learn virtue inevitably draws us toward a conceptual  examination of the meaning of virtue itself.

In the end the pair fail to resolve the mater and later in the dialogue Plato (through Socrates) goes on to provide a theory of knowledge based on pre-existing memory and the use of questioning as a means of recollecting what was there in the first place.

Socrates uses a series of questions to elicit a mathematical proof from the slave boy as a means of illustrating his point.

Learning Outcomes

Where we find learning outcomes

All learning outcomes are descriptive, they are attempts to capture in a series of statements the results and consequences of instruction or experience.
For anyone taking on a course of study, particularly a third level course, they are likely to want access to a description of that course and the modules associated with it. 

A key part of any such course or module description will be a series of statements that define the purpose and intent of the learning involved - these are known as the "Learning Outcomes".

Learning outcomes can be defined at all levels of course participation:

• Programme Level Learning Outcomes are statements that describe the range, depth and kind of knowledge and competence expected of a student on completion of an entire programme such as a degree or a diploma.

• Module Level Learning Outcomes are statements that describe the knowledge and competence expected by the student on completion of a particular module or subject area within a programme.

• Class Level Learning Outcomes are indications of what is expected to be achieved by the students on completion of a specific class or tutorial session.