Widgepedia Widgepedia

Like most teachers today, we take it as a given that communicative ability in a second language must be considered holistically. That is, communicative ability includes not only vocabulary, pronunciation and grammar skills, but also the capacity to use these in real-world contexts. It is this last point which is often missed by traditional assessment tools, largely because it is considered too time-consuming and subjective to test.

Here, we hope to demonstrate that task-based assessment is neither time-consuming nor any more subjective than other methods, and in fact includes other advantages as well.

Task-based assessment can be easy, straightforward and meaningful for learners if one primarily looks at the appropriate outcome of a given task, and secondarily looks at the accuracy of the language used. In other words, if the learner can achieve the task appropriately, then the learner passes. Conversely, if they cannot achieve the task in a generally acceptable manner, they fail. This presupposes two important factors, which are discussed later: 1) that the task was well selected to begin with; and 2) what "appropriate" means.

Here is a hypothetical task, including some specific parameters to be met:

Task:

Tell a story

Parameters:

On a simple, familiar topic (e.g., family trip)

To a single sympathetic listener (e.g., a friend)

For a certain length of time (e.g., two minutes)

With visual support (e.g., a photo album)

Note that parameters are important in order to keep tasks at a specific level. For example, if we were to remove the visual support, or change the “single sympathetic listener” to “an audience of English teachers during a high-stakes university entrance interview” it would make the same basic task far more challenging.

Now, let us say we are grading the task outcome on a 10-point scale. If the learner appropriately completes the task (i.e. they tell a story according to these parameters, regardless of how “good” it actually was), then they pass. Their score is somewhere between 6 and 10.

If they could not accomplish the task (i.e. they could barely be understood, or if what they produced would not reasonably be called “a story”) then they fail. Also, if they spoke reasonably well but did not stay within the parameters—for example if they spoke for only one minute or spoke about an entirely different topic—then they did not complete the task and they fail. Their score is between 1 and 5.

The next step is to assess how well the task was achieved. Now—secondarily—we can look at things such as pronunciation, vocabulary and grammar. Bear in mind that, if an appropriate outcome has been achieved, then by definition we already know that the learner's pronunciation, vocabulary and grammar are at least at an acceptable level for the task. They could not have managed to complete the task otherwise.

In short, task-based assessment works like this:

Step 1

Was the task appropriately completed?

Would the outcome be reasonably recognizable by an “average” native speaker of English as an example of its “type”?

In this case, Was it a story? Was there a beginning, a middle and an end? Were the events in the story linked to each other coherently? Was it clear enough to understand the meaning, despite any possible language problems such as poor pronunciation or grammar mistakes?

If yes, PASS.

If no, FAIL.

Step 2

If PASS, then how good was the story? Read the descriptors below and assign a grade from 6 to 10.

If FAIL, what were the issues? Were there at least some redeeming qualities? Assign a grade from 1 to 5.

Example scoring criteria for a pass:

10 No grammar mistakes worth mentioning. Vocabulary use was very appropriate. Pronunciation was exceptionally clear. Speech was very smooth and fluent. Gestures, facial expressions and manner were effective and natural. Speaker was very confident and engaging.

9 A few small grammar, vocabulary or pronunciation mistakes that did not affect meaning. Speech was smooth, near-fluent, and very easy to understand. Gestures, facial expressions and manner were effective and natural, with perhaps a minor slip. Speaker was confident and engaging.

8 A considerable number of small grammar, vocabulary or pronunciation difficulties sometimes affected meaning. Despite these, the overall speech was easy to follow and understand. Gestures, facial expressions and manner were generally appropriate. Speaker was mostly confident and engaging.

7 Occasional serious difficulties with grammar, vocabulary or pronunciation. Speech was not always smooth and clear, but overall quite understandable. Speaker did not revert to L1. Gestures, facial expressions or manner may have been marked or distracting.

6 Serious difficulties with basic grammar, vocabulary or pronunciation. Required some support from the listener. Occasionally reverted to L1. Gestures, facial expressions or manner may have been distracting. Speech not always clear, but overall complete and understandable.

Example scoring criteria for a fail:

5 Serious difficulties with basic grammar, vocabulary or pronunciation. Required considerable support and patience from the listener. Often reverted to L1; nevertheless, short sections of the speech could sometimes be understandable, or contained other redeeming features.

4 Serious difficulties with basic grammar, vocabulary or pronunciation. Required considerable support and patience from the listener. Often reverted to L1. Understandable only to a very sympathetic listener who is familiar with the learner's L1.

3 Did not display an ability to use basic grammar structures. Spoke in two- or three-word utterances using limited but appropriate vocabulary. Used other means to support speech, including heavy use of L1. Difficult to understand even for a very sympathetic listener.

2 Did not display an ability to use basic grammar structures at all. Spoke in one- or two-word utterances using basic vocabulary. Used other means to support speech, including heavy use of L1. Extremely difficult to understand even for a very sympathetic listener.

1 Could not respond to or use basic set greetings such as "How are you?" Made only single-word utterances in basic vocabulary, or none at all. Used L1 almost exclusively, perhaps making an attempt to anglicize some words. Impossible for an average listener to understand.

Task Selection

The real trick in TBL assessment is to select tasks and task parameters in a thoughtful way. [to be continued...]

What Does "Appropriate" Mean?

One of the more difficult points in task-based assessment is arriving at a good definition of what "appropriate" means. For example, when we say that the point of a task such as Order a pizza by telephone is that a hypothetical pizza restaurant employee would either understand the order "well enough" to deliver the pizza, what do we mean?

At what point do we draw the line between clear and unclear, appropriate or inappropriate, etc? Do we assume that the typical interlocutor knows the learner's L1? Is familiar with other learners of the L1? Is not familiar with those learners, but is experienced with language learners in general? What about attitude—is the interlocutor patient and sympathetic to learners? Are they even in a position to be sympathetic; e.g. a waiter vs. a police officer? These questions can go on and on, and there is no easy answer.

One way of solving the issue is to simply arrive at a clear definition of one's own, and to accept that these variables will be interpreted differently by different teachers. Some teachers prefer to imagine a person whom they know to be a "reasonable" interlocutor. For example, a teacher in Japan who is too familiar with Japanese L1 learners could distance themselves somewhat by asking the question, Would my brother/mother/friend understand this? That is, instead of trying to establish what a hypothetical "pizza guy" might do, they might imagine a more specific person in that role, at least for the purpose of making that crucial initial pass/fail distinction.