Kill The REF in Complex Circumstances

This is personal opinion. If citing please attribute.
drbill.cooke@gmail.com
KILL THE REF IN COMPLEX CIRCUMSTANCES

Bill Cooke: in a personal capacity
There are rumours going round British universities that some are making explicit promotion, teaching workload, hiring, and firing decisions based on the Research Excellence Framework, the REF. This is odd, because the REF hasnt actually happened, and there is a huge and expensive clustering of committees, rules, and guidelines, never mind a process of detailed assessments designed by HEFCE that have yet to take place. One might think that if decisions on the outcome of the REF of such import to individual careers can competently be taken before it actually happens, then the REF per se isnt needed. Or, conversely, that actually, these decisions cant be competently taken. What I go on to show here is that on HEFCEs own terms, they cant be taken ethically, and perhaps legally either; but also that the REF itself is similarly broken. The stated intention of the REF is to distribute money from HEFCE to universities for research, on the basis of their research performance hitherto. Past performance is taken as an indicator of the future. As institutions get money on this basis, it should be a self fulfilling prophecy. Unlike OFSTED inspections though, there is no assessment of what an institution has made of the resources it has had (no value add measure). To make this resource
allocating judgement, university research is compared by various disciplinary Units of Assessment (UofA). What gets compared are, largely, research outputs mainly, but not exclusively peer reviewed journal articles and monographs, with some variations between disciplines. For HEFCE, institutions comprise agglomerations of their individual research staff, deemed research active. Assessment is per capita; each research active person is counted, and each persons person 4 best publications counted up and each of the four is evaluated by being given a score from 1 (bad) to 4 (best). Only category 3 and 4 work attracts institutional funding. You might think that having your work categorised as being of quality that is recognised internationally in terms of originality, significance and rigour puts you on a solid basis. Sadly, if that is all your work is, it is awarded a 2. To get a 3 work needs to be of quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence. A 4, the holy grail, is quality that is world-leading in terms of originality, significance and
rigour. I know that these categories are not mutually exclusive, or logically related, but I have to go on to my bigger point. There is window dressing about the research environment, but this is essentially how it works. As those of us who teach organization studies try to explain, there are ways of assessing organizational performance other than agglomerating the performance of individual organizational members, and there are choices about which is best depending on context. The example I use is professional football, which is apparently highly individualized, and highly competitive. It is tautological, but despite this competitive individualism, the success of a team can only be assessed by its success as a team, that is, by its collective achievement. Were it assessed on the organizations-as-individuals logic employed by the REF, each team members performance would have to be assessed on necessarily individualizable criteria for passes completed, or time in possession of the ball. Then which team is best would be judged by summing up each players counts for the chosen criteria. Indeed, a couple of days ago, in the UEFA Euros tournament (I write this the day after England beat Ukraine), the Guardian did just this, headlining data which showed in England v France, England players completed a smaller percentage of their passes than the French players. The England the team was therefore facing serious problems, and the French were the better side. But the actual result of that game was that England and France drew: neither could claim to be the better team on the only criterion that mattered, the number of goals scored. And, at the time of writing, England have topped their UEFA group mini-league, with more points than France. When it came to doing what they were supposed to do, England were the better team. HEFCE and the REFs primary mode of analysis is simplistic for its context, and causes unanticipated problems. How could such clever people at HEFCE make such an elementary mistake ? The general answer is the tendency of technocratic managerial experts to hubris, as if there is nothing that cannot be tamed by them. The specific answer is that HEFCEs, and university administrators eyes got too big for their managerial bellies. It became apparent that the REF could be used to shift the balance of power between administrators who the Times Higher now styles university leaders and managers and those who actually do the work of research and teaching. Academics were famously difficult to manage. Much of their work was private, in the classroom, the study, the laboratory or the archive; so much of
2
what they did was heterogeneous, intangible, and politically inconvenient; and so much of its significance takes years to be realized. In addition, academics also didnt want to be managed, and more than an insignificant few didnt do the work for which they collected salaries way above any national average. Along with customer satisfaction surveys for lectures, an individualized, a per capita REF enables (it seems) the atomization of universities intellectual endeavour, and, the performance of each faculty member to be exposed, measured and judged, on its own terms, and against that of others. This is helped by the techniques of human resource management (HRM) probationary periods, target setting, annual performance appraisals, and performance related pay. HRM is also associated with disciplining, by both its proponents, and its Foucauldian critics. But against those who see mutual cause and effect in the simultaneous rise of HRM and Thatcherism, I counterpose my anecdotal experience of its practices being developed and realized in public sector by the political left. When I worked for Leeds City Council in the 1980s, advocates of what now are seen as everyday HRM tools job descriptions, person specifications, annual appraisal presented them as objective methods to counter racism and sexism. It was the stroppy, brave people from councils and health authorities equal opportunities units who drove the HRM movement, as much as it was privatising, union-busting right-to-managers. When I left Leeds for my first proper HRM job at British Telecom in London, I found most of my colleagues developing its new entrepreneurial culture had a similar public sector background.
This ambiguity, of processes which are managerialist, anti-collective, even oppressive, and at the same time potentially progressive, pervades both the REF, and academics engagements with it. The whole evaluation of submissions is by panels of disciplinary peers, resulting from the collective, if tacit, understanding that it is better done like this than any other way. Of course, this peer review is not anonymous the reviewers know who has written what. How panel members solved the puzzle of doing the actual quantity of reading would be of interest to Booker judges. In the last RAE, in my UofA, 18 people apparently read and ranked 12,590 research outputs. Most journals have a word limit of 8,000 words (often exceeded), and each piece really should be difficult to grasp on one reading. That
panel membership is taken on despite this personal strain demonstrates how important it is felt to have us on the inside of the REF. This has carried through into the preparation for the REF. Senior and respected scholars have taken responsibility for preparing REF returns. Others have acted as external assessors of other university departments draft returns, me included. The silver for so doing does come in handy; and there is broader self interest for some academics in maintaining the REF. One argument I used to offer was that it did actually insulate academics from capricious managerial decisions. A good REF, enables promotion, and better terms and conditions, irrespective of whether the boss likes the actual content of ones publications. It means personal leverage, so some can also be more blas about teaching, if that way inclined. Individual outputs are portable. I changed jobs, and was promoted, moments before the last RAE. I know my colleagues in the department wanted me for the work I had done, not for its potential RAE score. But it certainly helped. It is not so good this time round. As I enter my mid-50s, meaningless REF scoring isnt turning me on as much as it did. Its taking me longer to reach my peak, and I am not sure if I will get there in time, if at all. And its all become so mechanical. At the very least, when I did get my current job, I was able, and indeed required to explain why my work mattered, the point of an obsession with ante-bellum slavery, the organizational consequences of the Cold War, and of FBI/CIA surveillance, in management school terms. This time round it would be much easier. Business and Management schools undergoing the REF are now ruled by something called the ABS List. The ABS, Association of Business Schools, has no connection to HEFCE, and is an association of Deans. Being what it is, this was the apogee of managerializing the intellectual, until HEFCEs latest move with complex circumstances. Again, with a methodology that claims rationality and fairness, but which is actually very hard to work out, the ABS list ranks every journal in which business school faculty might publish, from 1 to 4. Some of this, it is claimed, is based on reverse engineering the outcomes of the last RAE, on citations, and on meta-analysing other lists. The outcomes are a bit surprising, but, as is becoming a theme here, there is sufficient sugar to make the shit barely palatable. In this case, in a field dominated by hegemonic US positivist methodologies and journals, a number of more intellectually open-minded, and, the consensus is, good, European journals have also made the 4 grade for example
4
Human Relations, Organization Studies, Business History, British Journal of Management (BJM). This must have needed some post-hoc rationalization; and it is to me odd, for example, that BJM is a 4, but the journal Organization, an obligatory and innovative point of reference in sociologically orientated management scholarship, is a 3. BJM is the more orthodox house journal of the British Academy of Management, whereas Organization publishes critical scholarship. And because BJM is a 4, the collective preference will be to publish there, rather than Organization. Faculty will not even need to be told that that is what is required. The prophetic ranking becomes self-fulfilling; but in so doing, submitters to BJM will know to tone down the critique of management they may have included for Organization. It is also the case that some of us publish in journals outside those associated with our discipline. I do, in Development Studies journals. No matter, the ABS is qualified to judge this UofAs journals too, and finds for this whole discipline that none of its journals are of 4 quality. This is unfortunate for the many of us who believe Business Schools should be paying more attention to Development Studies core concerns of global poverty alleviation, social justice and sustainability. (It is also the case perhaps so obvious to go without saying that the ABS list, and the REF dont consider publications not in English). It is unfortunate for my career, too, because the numbers now really matter. A narrative explanation of the significance of ones work in interview and appraisal is now replaced by simple calculation. Some job candidates vitae now add a little 3 or 4 at the end of each publication listed. It is rumoured, some even dont include publications in lesser 2 or 1 journals, for fear of appearing dilettante. The total is then divided by four, for the number of outputs required to be submitted, to produced a grade point average (GPA). The GPA also figures in discussions of cut off points that is, below what score will it be decided to exclude people, so that number in itself becomes a measure of machismo. University X, it is rumoured requires a GPA of 3.75, another 3.5. That these numbers seem small as penissizes may explain why there is some reversion to a more macho un-averaged total: you gotta have at least a fourteen to be there, or even with a twelve you are not going to get in. The perversity of all this is that the REF panel for our UofA, genuinely demonstrating the value of its peer constitution, has repeatedly stated it will take no account whatsoever of journal rankings. While such rankings may not explicitly exist for other disciplines, it is certainly the case that size matters in all of them, and judgements about inclusion and
exclusion will be being made on more or less explicit gradations of where and how an output is published, rather than its intrinsic content. Such judgements, it is universally accepted, will affect promotion. We still have not quite reached the self-immolatory nadir of REF madness. It can be found in something called complex circumstances, aka special circumstances and exceptional circumstances. I have tried to show what has enabled the REF to get through the campus gates is its discourse of fairness and objectivity. It can claim to be a rational, transparent, peer-run way of allocating funding, and of judging performance in passing, which bypasses old-school, old-boy Oxbridge snobberies. It is actually none of these. But having claimed this terrain, it needs to deal with the fact that, as the clich has it, life isnt fair. Initially, feminist scholars pointed out that the requirement to submit four pieces of work from a given five year period disadvantaged those who have taken maternity leave within it, and might actually be illegal. Once this point is recognized by HEFCE it is quite easily addressed. HEFCE tells REF panels to reduce the requirement pro-rata per period of maternity leave. Who has taken maternity leave is a matter of institutional record, and public workplace knowledge, and so applying this reduced tariff is a simple HRM exercise. An important precedent is now established, however. Exceptional circumstances (like giving birth, yes, I know) can affect individuals performances in the REF, which in turn will affect career prospects, so there needs to be a way of recognizing and managing this.
So, to illustrate, the complex circumstance of adopting a child into a family, rather than being a birth-parent can be dealt with administratively. Those who take adoption leave can have it taken into account in a tariff. But some complex circumstances are more complex, and, in fairness, there needs to be a method of ensuring equivalence of treatment across their range. In the spirit of reason, HEFCE has hired the Equalities Challenge Unit to detail 25 fictional exemplars which set out the way in which different special circumstance can be recognized as a tariff of discounts. Doctor Monroe, who takes maternity leave and then comes back for a period of half time work is entitled to a discount of 2 (from 4) submissions. Dr Jenkins, whose maternity leave was followed by post-natal depression and the prescription of anti-depressants, is entitled to a discount of 3 submissions. Professor Allsop and his partner adopted a child in 2006, but then the partner died. Discount: 1 submission.
The diagnosis of Professor Obabanjos wife with multiple sclerosis in 2010 entitles him to a discount of 1 submission. Dr Childs mother has Alzheimers, and is no longer able to care for her father who had a series of strokes, and was admitted to a nursing home in January 2007. Again, a discount of 1 submission. Doctor Adnam had symptoms of Chronic Fatigue Syndrome, and was off work for thirteen months. Discount: two submissions. Dr Woodrow was depressed, then diagnosed as bipolar, then her medication affected her thyroid: discount, 1 submission. Dr Tsang had bowel cancer, which was treated and then recurred: discount, 3 submissions; but Dr Fullbrooks brain tumour, 2 submissions. Professor Moores gender reassignment: 1 submission. Dr Wilson being anonymously bullied because of his sexuality, 1 submission.
How humane is all this? I would guess that HEFCE, and the exemplar tariffs authors think they are being incredibly so, in their recognition that there are such complex circumstances, and stipulating they absolutely must be taken into account if the REF is to operate fairly. In contrast, I find the assumption that it is possible to produce a tariff of suffering particularly bizarre. That the unit of currency against which the ordeals of colleagues are to be measured, compared, and compensated is peer-reviewed journal articles is particularly repellent. It is an exemplar of not what it claims, but the dark places to which the managerial hubris I keep mentioning leads. But, being forensic scholars, we must look particularly closely at that which we find repulsive. How is complex circumstances tariff put into operation? Inevitably, the process requires it to be found out whether people do have complex circumstances, and the only way for that to happen is for people to disclose that they do. To help universities facilitate this, HEFCE provides a model self-disclosure form for people to fill in, with separate boxes according to whether one wants to claim a discount, for, for example Disability (including conditions such as cancer and chronic fatigue) Mental health condition; Other caring responsibilities (including caring for an elderly or disabled relative) Gender reassignment and Other exceptional and relevant reasons, not including teaching or administrative worki. And, continue onto a separate sheet of paper if necessary. And then, HEFCE says to universities, use something like this form to
implement this tariff. Like those who will sell me a litre of brandy every day, but also tell me to be drinkaware, they say please recognize peoples need for confidentiality, and also the
requirements of the Data Protection Act. On the website, even, there is a link to the webpages of the Information Commissioner to tell you what and what isnt allowed. Now we have reached the real nub of the problem, the place where the REF has become unimplementable. Morally, none of this should be allowed. Lets begin with completing the disclosure of complex circumstances form, and the ECUs cases. If Professor Allsop adopted his child as a baby, does that child yet know he or she was adopted ? Professor Allsop, to claim his due tariff reduction has to reveal this, perhaps to a parent of a child with whom his has playdates, or an anonymous evaluation committee. Perhaps Professor Allsop adopted an older child, who now is 16 years old. Does that child have any say about whether their adoption should be made quasi-public grist to the REF mill in the tight knit academic community in which they live ? Professor Obabanjos poor wife, suffering MS. Who chooses to make her condition known - her or the Prof ? If she sees a difficult financial future for her and her family because of her MS, perhaps she will feel obliged to let her husband disclose, whether she would really wish that or not. It would not be unusual for her to be a Professor too, perhaps at another university. Possibly she hasnt disclosed her condition there yet, and doesnt want to. What is Professor Mr Obabanjo to do ? Dr Wilson, who has been getting anonymous, abusive emails because he is gay. He wants to disclose that? What if he wonders whether the anonymous emails come from the people collecting the forms ? And what of those colleagues with some form of mental illness? Should they disclose, knowing that it can be seen as problematic ? But what if this illness, or any other, precludes them from judging what is in their best interest at this moment ? The REF, it
appears, is so much the centre of the bureaucrats world that basic social mores about privacy or decency are to be disregarded. While perhaps attempts might be made to provide some kind of anonymization, there is no way out of this indecent coercion. People are put in a position where they are forced to reveal what they might want to keep private, or others might want to be kept private, and this revelation, connect to and impact on judgements about their REF performance. As HEFCE have made clear, otherwise, careers and livelihoods will be damaged. Then wonder, what happens to the forms when they are completed. There are to be panels to consider and compare cases to weigh Dads stroke versus Sons autism. To compare, perhaps, the consequences of bladder cancer treatment requiring catheters inserted into
8
the penis on multiple occasions with the transgender operation that sees the penis downgraded or upgradedii, and come up with a comparative measure in journal articles. What will the panels expertise be, and how, if pressed will they justify decisions? How will the appeal process work ? How will HEFCE monitor consistency between Universities ? Who, specifically will actually get to know about all we disclose about ourselves and others cancers, bereavements, ill health, chronic fatigue, anti-depressants, Alzheimers? Quite a number, for the model disclosure form requires the discloser to tick a box authorising this statement (p7): I realise that it may be necessary to share information with the UK funding bodies REF team, who may make the information available to REF panel chairs, members and secretaries and/or the Equality and Diversity Advisory Panel. [Delete if not applicable: I recognise that if a joint submission is made, information may be shared with another institution.] Where permission is not provided [insert name of institution] will be limited in the action it can take If gay Dr Wilson wasnt fully out, he is now. What he is expected to do is not an amelioration of his harassment, but an extension of it. Less likely to have happened if he was straight, of course. Maybe we should just post our complex circumstances on facebook. But perhaps HEFCE can advice me, should Mums hip replacements go on my wall or hers? And then, finally, wonder how Universities deal with non-disclosure. The HEFCE/ECU guidelines speak of persuasion to disclose there can be no obligation. Simultaneously time, the whole point of complex circumstances because that non-inclusion in the REF can lead to punitive consequences, so unless people do self-disclose they will be treated unfairly. Any academic knowing the research ethic of free, full and informed consent should know too that the choice between demotion and spilling on Hubbys colon cancer is no choice at all. Some people will inevitably keep quiet. Some universities, to cover themselves legally (so they think) might be tempted to compel people to disclose, or otherwise fill in a form that says they will not disclose, or have nothing to disclose. Think again of the Professors Mr and Mrs Obabanjo in this case. We are at the point the point of the Loyalty Oath here, and it is no coincidence that its era saw the emergence of game theory. Should one take the oath and swear one and ones life-partner, parents, and/or kids - has not, and
never has had complex circumstances? If that is a dishonesty, and it gets found out, will that be an offence in itself ? What if I refuse the oath, but others dont? The REF is on the brink of cruel and unusual punishment. And, importantly, those universities who try and anticipate the outcomes of the REF, have painted themselves into the corner where HEFCE say that complex circumstances must inform every judgement; but the reality is there is no way of ethically, perhaps legally, making that judgement. If universities are making career limiting decisions on the basis of anticipated REF outcomes without considering circumstances, then people are being punished for having cancer, MS, depression, bereavement, and/or aging parents. Given our record of collusion in the REF, and the pathetic state (for which we are all responsible) of our union, UCU, I am not sure whether even the horror of complex circumstances will lead to mass resistance. Indeed it is a real test of academia, whether we have been so institutionalized by the REF that we dare not escape it. Perhaps, like some Dr Who monster, we might find that the REF that scares us so really is a creation of our own minds. Whatever its ontological status turns out to be, I think there can be no option but for it to cease to be. The REF may be against the law that the Information Commissioners Office is meant to enforce, the Data Protection Act (DPA). A colleague who specializes in surveillance and information, suggests that the DPA requires that universities would only be allowed to process disclosed data with the individuals' consent. Moreover, they would not be allowed to make decisions in any way against individuals who withhold the information or the consent. Individuals should also be provided with a privacy statement (i.e. how data will be handled, how long it will be kept, who will see it, how the data will be destroyed), and a transparent account of how data will be used. They will also need to provide a statement explaining why this collection and use of data complies with data protection principles of necessity and proportionality, i.e. that this type of data is needed for the task and that the intrusion into people's lives is justified on the basis that it is essential to the task. One can complain to the ICO, and indeed I have. Dispiritingly, nothing back since May 14th (its now June 17th) but a pro-forma response letter of acknowledgement. Nonetheless, whatever the ICO say, by standards of common decency, HEFCEs behaviour is unnecessary and disproportionate. I have shown here how HEFCE recognizes, that, because it has chosen to assess individuals one by one, difficult personal circumstances must be
10
taken into account. But, ethically, (and maybe legally, who knows), it has no right to intrude to the extent it desires. What HEFCE says is necessary for its REF to function cannot in actuality be done. Therefore, it has to stop. It is not, though, that the REF must be killed, though it must. It is that the REF has killed itself. There is a catalogue of teaching cases showing managerialism gone bad. Then theres Milgram. Zimbardo. Bay of Pigs and groupthink. The Poll Tax and more groupthink. IBM and the Holocaust. Managerialisms debt to Atlantic slavery. Henry Ford and the Nazis. Chinese Brainwashing in the Korean War morphing into organisational culture change. The Challenger Disaster. To this roll of
dishonour must we now add the 2014 Research Excellence Framework ? But of course, it is only about Mrs Professors Obabanjos MS being grist to its mill. We mustnt take it too seriously, huh ?
This last caveat is particularly disingenuous. Most universities I know have a workload model, which forces faculty to account for every hour of their time in comparison with a notional figure representing the total number of working hours in a year (1650 in my university). There are fixed allowances for different forms of teaching and administration, and an amount for research. In most universities if you are below the annual hours you are given more work. If you are above, no-one says this is impinging on your research hours, stop this extra non-research work immediately. HEFCE, while saying it wants to know about chronic fatigue and your Zoloft intake, really doesnt want to know about you being overworked. ii Lodge- Bradbury Syndrome: The particular obsessions of middle aged male academics like me.
11

Kill The REF in Complex Circumstances

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Kill The REF in Complex Circumstances

Uploaded by

Copyright:

Available Formats

This is personal opinion. If citing please attribute.