How far does media coverage of international large-scale assessment help hold governments to account for their education commitments? Posting on UNESCO’s World Education Blog by Mary Hamilton

UNESCO’s Global Education Monitoring Report blog: World Education has just published a blog by Mary Hamilton, Associate Director of the Lancaster Literacy Research Centre and Co-Director of the Lab for International Assessment Studies.

A key rationale for carrying out international comparative surveys of skills such as the Organisation for Economic Co-operation and Development (OECD) Programme for International Student Assessment (PISA) is that the findings can positively influence policy and therefore educational outcomes. Such claims implicate the media as part of a chain of influence. The argument runs that the media publicise the findings, which influence public opinion and in turn this puts pressure on politicians to respond. The media can also compare past successes, failures and improvements through a running commentary on trends in the test scores.

However, the impact of media on educational policy is assumed but not widely researched. My colleagues and I have followed media coverage of the OECD Programme for the International Assessment of Adult Competencies (PIAAC) in France, Japan and the United Kingdom as well as in Greece, New Zealand, Singapore and Slovenia, which took part in the second wave of (PIAAC-2).

Head over the World Education Blog to read the remainder of the story.

 

Advertisements

Academic publishing practices in universities in England

Members of the ESRC-funded ‘Dynamics of Knowledge Creation: Academics’ writing practices in the contemporary university workplace’, research project team have posted here before about different aspects of the project. The project set out to investigate how academics’ writing practices across three disciplines were changing in response to wider changes in higher education.

One of the latest outputs from the project is an article called Hobson’s choice: the effects of research evaluation on academics’ writing practices in England, which is available via open access here. As the name suggests, the article examines the influence of research evaluation policies, namely the REF, and institutions’ interpretations of this on academics’ writing practices across three disciplines.

One of the key findings reported in this article is that academics’ ability to succeed in their career is closely tied to their ability to meet quantitative and qualitative targets driven by research evaluation systems. For example, early career academics on probation are often required to produce a specified number of scholarly publications within a certain period. Academics often talked about these requirements in terms that echoed the nomenclature of the REF, saying, for instance, “I have to publish two papers at three-star”. Similarly, one head of department told us that the minimum requirement for academics in her department was to publish “one good publication a year”. She explained that this meant a single-authored paper in a good journal. This obviously may constrain the type of collaborations academics feel free to enter into since sole-authored papers are preferred. It also promotes the notion that journal articles are valued over and above, say, monographs. This was a point of contention for many of the historians we interviewed, since it clashes with their traditional view of the scholarly monograph as the gold standard for publication.

Another finding discussed in the Hobson’s choice article is that the effects of research evaluation regimes were unevenly distributed across institutions and age groups. Academics in research-intensive universities experienced extreme pressure to maintain a high level of performance year-in, year-out, with little acknowledgement by institutions of the learning curve that is part and parcel of writing for publication. Those in teaching-focused institutions, on the other hand, were under less immediate pressure because research was often not made a priority. However, one effect of this is that career mobility for academics at these institutions may be curtailed since they are not enabled to fully participate in the scholarly activity that would allow them to move to higher-ranking, research-intensive universities. This is a particular issue for younger academics who may start their careers in less research-intensive institutions with the hope of establishing a research trajectory over time.

The full article can be found here without the need for institutional log in or payment.

by Sharon McCulloch, Lancaster University

Multimodal literacy and assessment – call for papers for special issue of Language and Education

This is posted on behalf of our affiliate member, Dr Lynde Tan, Western Sydney University:

Call for Papers: Language and Education

Special Issue on “Dialogic Inquiry in Multimodal Literacy Education:  Joining the Dots between Assessment and Literacy Curriculum”

In the field of literacy education, there has been a shift in defining literacy as a monolithic term to a pluralistic one. Education research is replete with propositions for multiple literacies. The key tenor of the argument is that different domains of life privilege different literacies and they are politically and culturally influenced. Calling attention to the social, cultural and linguistic diversities, literacy researchers continue to emphasize multiple forms of literacy associated with semiotic modes found in the multiplicity of communication channels and media. The social outcomes of literacy learning are the central thrust of the ongoing debates that justify reforms in literacy curricula. The rallying call for 21st century skills to be embedded in curricula is intensified as heavier emphasis is placed on producing workers, citizens and community members who are able to respond to the rapidly changing working, public and private lives in a technological and globalised world.

While reforms in literacy curricula are taking shape progressively all over the world, less is observed in assessment. As the notion of literacy expands, we need to review how literacies are being assessed. Emerging conceptualizations of literacies raise questions about the lacuna – How do educational authorities reform curricula and assessment in tandem? How do schools cope with assessing 21st century skills when outcome-based curricula tend to foreground assessment of skills that are easier and more convenient to measure? What are the proven ways for implementing multimodal assessments across curriculum? Are there compelling evidence that standardised testing is fulfilling the promise of education to help every child meets his or her potential?

We acknowledge the need to better understand how literacies can be and should be assessed. This special issue intends to examine ways of assessing multiple literacies that are relevant to the 21st century contexts. We are looking for articles that explore ways of assessing literacies that are responsive to, but not limited to, multimodal and digital learning environments.

We welcome contributions that are related but not limited to the following topics:

 

  • Multimodal assessments across content areas
  • Assessing new and /or digital literacies, including 21st century skills
  • Assessment literacies as part of teacher professional knowledge, practice and engagement
  • Challenges and issues in assessing multimodal literacies

 

Contributions should be based on empirical studies that represent an international perspective. Please send an expression of interest to the Guest Editors with a 500-word abstract in a Microsoft Word.doc attachment, outlining the content of the proposed paper.

The Guest Editors will review abstracts and invite authors to submit full papers, approximately 6000 words, for this special issue. Invited authors are advised to comply with the journal’s instructions for authors. Each full paper submitted will be peer blind reviewed by at least one expert in the field.

 

Important Due Dates

Submission of abstract:  31 January 2018

Submission of proposal to journal editors: 15 March 2018

Notification of proposal acceptance by the journal editors: 30 April 2018 (To be confirmed by journal editors)

Notification of paper acceptance to authors/Paper review agreement: 15 May 2018

Submission of full papers: 31 July 2018

Blind reviews to authors: 15 September 2018

Revised manuscript due: 31 October 2018

Expected publication: 2020 (To be confirmed by journal editors)

 

Special Issue Guest Editors

Lynde Tan

Email: lynde.tan@westernsydney.edu.au

Website: https://www.westernsydney.edu.au/staff_profiles/uws_profiles/doctor_lynde_tan3

 

Katina Zammit

Email: k.zammit@westernsydney.edu.au

Website: https://www.westernsydney.edu.au/staff_profiles/uws_profiles/doctor_katina_zammit2

 

Jacqueline D’Warte

Email: J.D’Warte@westernsydney.edu.au

Website: https://www.westernsydney.edu.au/staff_profiles/uws_profiles/doctor_jacqueline_dwarte2

 

 

 

 

 

 

 

 

Tribute to Brian Street from Lancaster

Brian was a dear friend and supportive colleague to members of the Literacy Research Centre at Lancaster, going right back to an early seminar here in 1983 where we were building bridges between literacy theory, research and the practice of adult literacy in the UK.

He took part in many key events we organised in Lancaster, including the ‘Worlds of Literacy’ conference in 2000 and ‘Worlds of Literacy revisited’ in 2014. His convivial presence at these events and his steady support in what was, at the beginning, a fragile new movement, was very important to us. Along with so many other areas of literacy research and action, Brian’s work on academic literacies was an inspiration to us. Thanks partly to him, academic literacies became established as an important field in the Lancaster Literacy Research Centre. As editor of the Benjamins series ‘Studies in written language and literacy’ he encouraged and helped Roz to publish Writing and Identity in 1998, which was the springboard for much of our academic literacies research in the years since then. Roz Ivanic says “Thank you, Brian, for those meticulous editorial comments in your much-loved jiggly handwriting.”

He was always ready to discuss ideas with us and as one person recalls “his gentle and friendly manner with everyone, no matter the state of their knowledge and experience, always made me feel part of his discussions.”

Though he held strongly to the principles of critical, rigorous scholarship he wanted his ideas to enter the worlds of literacy policy and practice and he participated with great energy and good humour, in conferences, research and policy forums around the world, patiently articulating the theory of literacy as social practice and searching for ways to apply it.

Brian was a true internationalist and as President of the the British Association for Literacy in Development (BALID) he fostered thoughtful critique and interaction across an incredible diversity of adult literacies teaching and learning, research and other practices across the globe. He worked hard to connect this wider vision to literacy in the UK as well, through his work with the Research and Practice in Adult Literacies group (RaPAL), as a member of its editorial board and writing for its journal.

For many of our students, Brian’s work was key to the theoretical underpinnings of undergraduate study, professional development courses and research degrees. He fundamentally shaped the understanding of a generation of scholars and practitioners in school and adult literacies. His capacity to return the local to its rightful place in educational discourse shifted many people’s worldview and his words are a powerful legacy.

We miss him sorely and his influence will continue for many years to come.

 

Revisiting Other floors, Other Voices: rereading Swales’ textography of a small university building, first published in 1998.

Many years ago I was in the habit of recommending John M. Swales’ 1998 volume to students whenever I had the opportunity. In print I even suggested it was a linguistics book “you could enjoy on the beach” (Gillen and Petersen, 2005).  But even strong passions fade and it has been a long time since I opened it.  When I decided to reread it, this was with some sense of purpose.  Since what are quaintly known as “new technologies” have become so pervasive, and the landscape of literacy research has changed so much, what would it be like to reread Swales?  More precisely, would his research, conducted when the internet was in its infancy, have much to contribute to the sociomaterial background I see language as entwined with?  How would Swales’ understanding of the relationships between textual artefacts and educational institutions resonate? More generally, how would the book stand up?  Would I want to recommend it to students again?

book-cover

Other Floors Other Voices book cover

Rereading the book was a surprise.  I recalled delight experienced from its detailed descriptions of diverse academic pursuits, whether of the intricacies of the botanist’s life, or the creation of language testing items.  I had a vague recollection of the computer helpdesk on the ground floor, in the infancy of IT in Higher Education.  I remembered that Swales combined some ingenious methodologies and contributed a lot to the notion of “discourse community” in particular, which had afterwards been much criticised. I recalled that I thought this was one of the books, like some work by Gordon Wells for example, where a linguist could offer some tools and insights that could be of interest to any sociocultural researcher probing any area of social activities.  I thought that it would be particularly interesting to reread the book as from the early/mid 90s, when it was still possible to remember a world before the internet, having reflecting on computers entering homes and other spaces in that period (Gillen, 2014).

As so often, my memory was right in some ways, approximate in others and just plain wrong or forgetful in others.  Or, to put it another way, a rereading has been as provocative and lively an experience for me as I like to think the original was.  While continuing to frame this as a personal response, it might be worthwhile to continue this long blog/short essay by focussing on three issues that might be of interest to others working in a broadly sociocultural perspective.  The first two concern controversial matters Swales addresses.  The third is my claim as to what disciplinary territory this book might now be appropriated for, no matter where it was earlier located.

Textography vs Ethnography?

Swales explicitly disowns his work as an ethnography, “not a label I am comfortable with” (Swales, 1998: 1[henceforward page numbers alone refer to this book]).  At the risk of wild, unproveable generalisation, I would assert that this is more of an ethnography than 90% of the work that calls itself that. It was conducted over three years and combines a great number of different methods in the service of arriving at collaboratively constructed understandings.  I had forgotten the multimodality of the enterprise, in particular the use made of photographs.  These are not mere illustrations, but rather elements of sociohistorical data that are discussed with participants.  At a textual level, there is indeed wonderful attention to such matters as script, orthography and the materiality of texts, asking and answering questions arising from data such as “Why is this kind of text N, more important than text type O, always presented in a smaller font?”  The arrangement of rooms is forensically studied; the status of a half-open door as semiotic carefully probed. Most impressively of all is an openness to change during the time of study. So many of us might see “change as a way of life” (p. 38) as a great inconvenience, to be airbrushed out of our descriptions rather than investigated for what might be uncovered about what people are thinking through and choosing to act on.

This is surely an ethnography par excellence, combining observations, interviews, the study of artefacts and so on, all to gain access to the opinions, attitudes and experiences of a huge swathe of people concerned in some way with the building, whether immediately or at a distance.  This is sensibly handled, in the only practical way for writing up an ethnography, in a combination of carefully selected individualised aspects of experience, against a well described backcloth of contextualised interactions.

Why does Swales abjure the term ethnography?  I am not entirely sure.  There is the relatively familiar denial that this is a full ethnography, but many others would surely argue that such an aim is impossible.  Swales is interested in our “textually-mediated social world” (Barton, 2001) and has chosen for examination a setting in which people do a great deal of writing and reading.  Literacy practices are at the centre of this ethnography and of course for me that is part of the appeal.  I will write a little more about that later.

Discourse community – still a useful concept?

 Swales’ rejection of the ethnography label was against the background of a then current dispute, and the questioning around the usefulness of the discourse community occupies a similar territory of contestation.  But whereas I thought I had remembered that the debate around the term took fire after Swales’ book, I had that wrong.  Swales identified already past as well as contemporary arguments as to whether the idea of discourse community stands up.

It seems to me that part of the still ongoing difficulty with the term appertains as to whether or not one focusses on the impossibility of defining any community, let alone a discourse community, in terms of what it borders.  If a border cannot be distinguished, if members cannot be clearly identified then some will, understandably, suggest that a term without clear definition is not worth its salt.  This is an extension of the idea of category in psychology: that it can only work if it delineates perceptible characteristics that are not shared. But my interpretation of Swales’ position would be to say let us accept extremely fuzzy borders and the empirical impossibility of resolving membership disputes.  This is consonant with the acceptance in agential realism drawn from quantum physics and the philosophy of science that even at the most microscopic level phenomena including acts of perception are primary rather than pre-existing entities (Barad, 2007; Suchman, 2007).

Swales presents textual evidence and related analyses, certainly, but these are always in the service of probing interactions between people on differing spatial and temporal scales.  For example he reproduces a letter from one botanical institute to another, requesting a loan of some samples (p.48).  His detailed analysis of the genre includes linguistic features to situate this letter within a well established practice of such letters, that include such conventions as who writes to whom, how the specimen requested is described, what uses can and cannot be made of it, and perhaps most surprising of all to the neophyte reader, the economics of the practice.  Considerable money is spent on preparing the specimen and packaging it off to the other institution although it might not be returned for decades if ever.  The benefit of examining the specimen might be evidenced in part through an exclamation mark, which in this field has a particular conventionalised meaning.

It is in the fine unpicking of such matters, through the examination of textual trajectories, interviews and external commentaries (journal articles and so on) that Swales is at his finest.  A convention is not a mere templated feature to be imitated, but rather has a rich historical background and affordances for today that are multiple and complex to those in the know.  Such people do not acquire such “craft-knowledge” (p. 51) overnight; it is rather a through a rich apprenticeship involving changes in practice, identity and expression.  Swales appropriately adopts here Lave and Wenger’s (1991) notion of legitimate peripheral practice and reminds us of some of its explanatory power.

So the usefulness of the idea of discourse community, in Swales’ account understood as often stretched across distance and time, in part enabled through online communications, is powerfully demonstrated.  My memory was at fault in recalling the book as largely or solely situated within a single physical location.  Rather, the internet was already embraced by many to new effect within their social context: Swales witnessed computerised search algorithms, an online catalogue of mycology specimens and even GIS.  For all, email was enabling faster communication across networks.

A classic work?

 The jacket of Swales book claims this to be a “robust new genre – textography.”  I would not want to disrespect that claim, but suggest it can be complementary to others.

First then I have argued that this is a very fine ethnography.  It could be taken as a lively methodological handbook, offering up lots of suggestions to other researchers about multiple methods that might be adapted to the study of any social group or milieu.  There are matters to criticise here: for example although the voices and activities of lower status individuals such as temporary graduate assistants in the computer centre are presented, these are given less status than people with more prestigious academic positions.  But the combination of different kinds of evidence and how these are synthesised in presentation to the reader is exceptional.  I cannot think of another book which does such a good and entertaining job in preserving the voices of its participants while also offering plenty of the author’s own analyses.  One key asset here is the diversity of tools, including those of linguistic analysis that he deploys; another is the ethnographer’s crucial and explicit reflexive attention to his own role.

Second I see this book as contributing to Literacy Studies, and indeed Linguistic Ethnography in an era of turning to sociomaterial theory. (See for example Hamilton, 2009; Bhatt and de Roock, 2013; also Tusting, 2013 on the alignment between Literacy Studies and Linguistic Ethnography).  The relationships between humans and technologies require more nuanced treatments than calling this tool use, as in what we might term classical sociocultural work, such as that by Vygotsky (1987), writing originally in the early 1930s.

I am guilty here of seeking to define a book as contributing to a very different discipline, from that the author intended, –  “comparative rhetoric” in Swales’ case.  Indeed I can plead prior form in thaving claimed Hutchins’ (1998) work on distributed cognition as also capable of being regarded as a foundational text in Literacy Studies (Gillen, 2014).  Both authors seek to comprehend how humans learn, on timescales from the moment to the duration of cultural shifts, sometimes over decades.  They place language at the heart of such processes, while denying the implicit claim of so much work in linguistics that all that matters is spoken language (e.g. Podesva and Sharma, 2014).  Language as encountered in use is always in part shaped by the characteristics and circumstances of its materiality, whether a sign above a door in a building, an email, or a spontaneous aside during an interview.  The tools of linguistics can help in analytical work, but it is the combination with the ethnographic commitment that so impresses me still in Swales’ unparalleled study.

Finally, a book “for the beach”??  Well, I love Swales’ sense of humour and hope you enjoy it too, if you have not had that pleasure already.

 

Barad, K. 2007. Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning. Durham, NC: Duke University Press.

Barton, D. 2001. ‘Directions for literacy research: analysing language and social practices in a textually-mediated world. ‘ Language and Education15, 92-104.

Bhatt, I and de Roock, R. 2013. ‘Capturing the sociomateriality of digital literacy events. ‘ Research in Learning Technology 2013, 21: 21281.

Gillen, J. 2014. Digital Literacies.  London: Routledge.

Gillen, J. and Petersen, A. 2005. ‘Discourse Analysis’ in B. Somekh and C. Lewin (eds.). Research Methods in the Social Sciences. Thousand Oaks, CA/London: Sage Publications.

Hamilton, M. 2009. ‘Putting words in their mouths: the alignment of identities with system goals through the use of Individual Learning Plans.’ British Educational Research Journal 35 2 221-242.

Hutchins, E. 1998. Cognition in the Wild. Cambridge, MA: MIT Press.

Podesva, R. and Sharma, D. 2014. (eds.). Research Methods in Linguistics.  Cambridge University Press.

Suchman, L. 2008. Human-Machine Reconfigurations: Plans and Situated Actions

(2nd edn). Cambridge: Cambridge University Press.

Swales, J. 1998. Other Floors, Other Voices: A Textography of a Small University Building.  Mahwah, NJ: Lawrence Erlbaum Associates.

Tusting, K. 2013. Literacy studies as linguistic ethnography.  Working Papers in Urban Language and Literacies. King’s College London.  Online at http://www.kcl.ac.uk/sspp/departments/education/research/ldc/publications/workingpapers/the-papers/WP105-Tusting-2013-Literacy-studies-as-linguistic-ethnography.pdf

Vygotsky, L.S. 1987. ‘Thinking and Speech’ in Rieber, R. and Carton, A. (eds.).  The Collected Works of L.S. Vygotsky.  vol. I Problems of General Psychology, including the volume ‘Thinking and speech’ trans. N. Minick. New York: Plenum Press. 37-285