lnu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Drift of Meaning: University Tutors' Understanding of Examination Papers
Linnaeus University, Faculty of Social Sciences, Department of pedagogy.ORCID iD: 0000-0002-0928-4308
2015 (English)In: ECER 2015, Education and Transition. Contributions from Educational Research. Network: 22. Research in Higher Education, 2015Conference paper, Abstract (Refereed)
Abstract [en]

Students' understanding of feedback is of great interest whenever good education is underway. Effective feedback presupposes that students understand the task on which feedback is given. If they do not understand the question it is, of course, difficult to understand the aspects that generated the wrong answers they gave to the question.

But what about the tutors formulating the task? Do they always understand it as intended and in the same way? Could it even be that a task is interpreted differently by the same tutor? And if so, feedback on what?

The purpose of this study is to examine how university tutors understand tasks issued to students. Does interpretation differ if the tutors themselves try to solve the task, discuss the solution and how it could be interpreted with other tutors, as well as trying to formulate better versions of the task?

Previous work has showed that tutors interpret tasks in a somewhat different manner and thereby give feedback on a somewhat different task, in spite of having the same text as point of departure (Crisp, 2007; Glover and Brown, 2006; Nicol, 2010). Handley and Williams (2011) discuss tacit knowledge regarding assessment criteria. It is therefore important that the tutors interpret the criteria in a consistent manner so students receive almost the same assessment of the quality of their work. However tutors are mostly unable to express these qualities and thereby only a sense of shared understanding is possible.

The meaning of words such as “analyse” and “discuss” are of special interest, as they signal key skills in higher education. Sadler (2010) argues that one precondition for students to convert feedback into actions of improvement is a working knowledge of higher cognitive skills involving such capacities as evaluation, critical thinking, creativity and analysis.

Lea and Street (1998) argue that when developing academic literacy, there must be an awareness that the meaning of the concept differs between institutions, staff and students. Tutors thus differ in what constitutes valid knowledge depending on context. Ivanič et al (2000) found that tutors' responses included micro-messages which are discipline-specific, for example as to what may be considered a sufficient justification or an acceptable explanation. Chanock (2000) states that disparity in the interpretation of key concepts has its roots in different traditions across disciplines

In recent research, the power of dialogue is often in focus. The problem of misunderstanding of feedback could thus be solved by tutors and students engaging  in dialogue. Higgins, Hartley and Skelton (2002) recommend, when commenting on how to prevent conflicting advice based on different meanings across disciplines, more discussion between tutors and students regarding tutors' expectations. Handley and Williams (2011) argue that tutors, representing different disciplines, by discussing examples, will realise that they hold contrasting, multiple interpretations of the meaning of “quality”.

In this study, focus is on how tutors themselves interpret a task over a span of time. More specifically, does interpretation differ if the tutors try to solve the task, discuss the solution and how it could be interpreted with other tutors, as well as trying to formulate better versions of the task?

Method

Five university teachers and one senior lecturer representing three departments, one humanistic and two social science, took part in a development work. The aim was to improve the scientific competence of teachers working in teacher education. Three tasks used in teacher education, and also formulated by at least one of the participators, were presented and elaborated within the group. The tasks were short, two-three sentences, to allow for an in-depth examination of their meaning. A typical arrangement was that a) each of the teachers tried to solve the task, b) solutions were assessed by other teachers, c) problems in solving the task and making the assessment were discussed within the entire group focussed on different interpretations, d) each teacher tried to formulate a less ambiguous version, and e) final discussion of advantages and disadvantages of the new versions.

The development work continued for eight sessions, totally twelve hours. All session were, after permission was gained from the participants, recorded. The material was listened through several times and sections where the participants expressed uncertainty considering the interpretation of the task were transcribed, as well as sections were uncertainty was discussed.

Expected outcomes

The empirical material shows that tutors interpret a task somewhat differently when examining it more carefully.This result is in line with previous research (Crisp, 2007; Glover and Brown, 2006; Nicol, 2010; Handley and Williams, 2011). The empirical material also shows that the same tutor may vacillate in his/her interpretation of a task when examined more thoroughly. Consequently feedback given to students also differs. When assessing say fifty students, consequently, the first student receives feedback on a somewhat different task than the last student does, even if the same tutor assesses the task! The drift of meaning is probably quite minor, but still noteworthy.

One tutor stated: “we had different pictures and we did discuss together to form a common one”. In her opinion different interpretations were due to lack of communication. So if tutors discuss enough they are able to understand the task in the same way. Recent research points in the same direction. (Handley and Williams, 2011; Nicol, 2010; Price et al, 2010; Carless, 2006). The empirical material indicates another direction. The more you engage in dialogue the more you realise that the words and sentences used could be understood differently, and if more words are used to try to explain what you mean, there are more words to be misunderstood.

The ambiguity of words, and hence of the dialogue, cannot be bridged as tutors interpret what others say in the light of their previous understanding. They will “hear” what they have, so to speak, already decided to hear. Dialogue often serves to confirm a certain degree of understanding, whereby preliminary interpretations are stabilized. We believe in accuracy of dialogue due to force of habit.. This is especially so when more abstract concepts such as reflect, reason and discuss are used.

References

Carless, David (2006). Differing perceptions in the feedback process. Studies in Higher Education. 31 (2) 219-233.

Chanock, K. (2000). Comments on essays: do students understand what tutors write? Teaching in Higher Education, 5(1), 95-105.

Crisp, Beth (2007). Is I worth the effort? How feedback influences students´ subsequent submission of assessable work. Assessment & Evaluation in Higher Education, 32(5) 571-581.

Glover, Chris & Brown, Evelyn (2006). Written feedback for Students: too much, too detailed or too incomprehensible to be effective? Bioscience Education, 7, 1 http://www.bioscience.heacademy.ac.uk/journal/vol7/beej-7-3.pdf

Handley, Karen & Williams, Lindsay (2011). From copying to learning: using exemplars to engage students with assessment criteria and feedback.  Assessment & Evaluation in Higher Education, 36(1) 95-108.

Higgins, Richard, Hartley, Peter and Skelton, Alan (2002). The Conscientious Consumer: reconsidering the role of assessment feedback in student learning. Studies in Higher Education. 27(1) 53-64.

Ivanič, Roz, Romy, Clark & Rimmeshaw, Rachel (2000). What Am I supposed to Make of This? The Messages Conveyed to Students by Tutors’ Written Comments. In Lea, Mary & Stierer, Barry (Ed.) Student writing in Higher Education. Suffolk: Open University Press. University of Chicago Press.

Lea, Mary & Street, Brian (1998). Student Writing in Higher Education: an academic literacies approach. Studies in Higher Education. 23 (2) 157-172.

Nicol, David (2010). From monologue to dialogue: improving written feedback processes in mass higher education. In Assessment & Evaluation in Higher Education, 35(5) 501-517.

Price, Margaret, Handley, Karen, Millar, Jill & O´Donovan, Berry (2010). Feedback: all effort, but what is the effect?. In Assessment & Evaluation in Higher Education, 35(3) 277-289.

Sadler, Royce (2010). Beyond feedback: developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35(5) 535-550.

Place, publisher, year, edition, pages
2015.
Keyword [en]
feedback, university teachers, tasks
National Category
Pedagogical Work
Research subject
Pedagogics and Educational Sciences, Pedagogics
Identifiers
URN: urn:nbn:se:lnu:diva-43373OAI: oai:DiVA.org:lnu-43373DiVA: diva2:814429
Conference
European Conference on Educational Research (ECER) 2015, Budapest, September 7-11, 2015
Available from: 2015-05-27 Created: 2015-05-27 Last updated: 2017-01-10Bibliographically approved

Open Access in DiVA

No full text

Other links

Abstract

Search in DiVA

By author/editor
Sellbjer, Stefan
By organisation
Department of pedagogy
Pedagogical Work

Search outside of DiVA

GoogleGoogle Scholar

Total: 36 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf