The First International Conference on Critical Management Studies

Refereed Paper Stream : Cultural Industry

___________________________________________________________

MAKING A MEAL OF ARTS EVALUATION:

Can Social Audit offer a more balanced approach?

by

Claire Cohen and Maldwyn Pate, Brunel University

____________________________________________________________________

Dr Claire Cohen Maldwyn Pate

Division of Management Studies Division of Management Studies
Brunel University Brunel University
UXBRIDGE UXBRIDGE
UB8 3PH UB8 3PH
Middlesex Middlesex
England England

Tel: + 44 (0)1895/ 274 000 x 3521 Tel: +44 (0)1222/ 759 232
E-mail: claire.cohen@brunel.ac.uk E-mail: mal.pate@btinternet.com

1
 MAKING A MEAL OF ARTS EVALUATION:
Can Social Audit offer a more balanced approach?

by

Claire Cohen and Maldwyn Pate, Brunel University

ABSTRACT
Statistical performance measures are increasingly used to evaluate the
output of arts organisations. These inevitably measure such economic
indicators as user take-up, and earned income/subsidy gearing. The
consequent diminishing attention paid to the artistic work has meant
considerable frustration for arts organisations (Pate, 1998)1 who feel
that the true breadth of their work is not being taken into
consideration.

Thus the inevitable question arises of whether there may be more

appropriate ways of evaluating arts organisations that will not only
take into account those elements that are important to artists but will
also satisfy funders.

The approach taken by the Social Audit (SA) movement offers a

direction which could be usefully explored. SA seeks to encompass
not only financial accounts but takes a broader perspective, by also
examining social and economic impact. It can simultaneously
embrace .aesthetic, cultural, economic and social values (Matarasso,
1997: 3)2.

But can SA fully comprehend arts organisations? Artistic production

is unique, sometimes characterised by an inversion of the fundamental
principles of all ordinary economies(Bourdieu, 1993: 39)3. Since SA
deliberately challenges ordinary economic principles, it should be
tested to see whether it is capable of enabling an understanding of this
upside-down economic worldof the arts.

The paper argues that, if SA is to offer a well-rounded approach to

understanding arts organisations, a redefinition is required, so that it
can embrace those elements that artists feel are important creativity,
innovation, boldness of vision, edifying aesthetic experiences for
viewers and audiences, and so on.

1
Pate, M. 1998, Business Practice in the Arts: How small arts organisations in Wales are coping with
official pressure to adopt practices & procedures from commercial business, MBA dissertation,
August 1998, University of Glamorgan, Pontypridd.
2
Matarasso, F. 1997, Use or Ornament?: The Social Impact of Participation in the Arts, Stroud:
Comedia.
3
Bourdieu, P. 1993, The Field of Cultural Production: Essays on Arts and Literature, Cambridge:
Polity Press.
2
Introduction
Franz Kafkas short story A Hunger Artist (Kafka, 1983: 268-278) concerns a
man who regularly starves himself almost to death in public. For years,
audiences come to gaze at him as he starves; his is a popular show, and there
is immense interest in his bodily suffering. He experiences frustration,
however, as his audience and promoter never allow him to go beyond forty
days starvation; the show has to stop at this point, since if it went on any
longer his audience would begin to lose interest. But as the years go by his
popularity wanes, and finally he is reduced to starving himself in a cage beside
a menagerie; people only take a glance at him as they hurry to see the animals.
Eventually he is abandoned in his cage and, left to himself, he pursues his art
with zeal, starving himself until he is near to death. As he dies, he confesses
that he only became a hunger artist because he couldnt find the food [he]
liked; with the right food, he would have eaten his fill like anyone else. He
dies, is quickly buried, and a young panther, put into his cage, attracts much
more interest.

Kafkas story may be interpreted on many levels; here we would like to

discuss it as an attempt to describe the artists relationship to the world and
the public. The hunger artists art is the expression of an urge which
derived from the worlds incapacity to feed him. But this urge to starve is
checked by the fact that the audiences interest in his work only lasts for a
particular duration (forty days - a hint that Kafka is drawing a parallel between
the artist and Christ). Without the audiences fickleness and predictable
cessation of interest, the artist would almost certainly have starved himself to
death much earlier in his career; thus the art is maintained and ensured through
that tension between inner urge and outer (via the audience) check upon the
urge. The art, therefore, needs the world to starve the artist; the art (to be
maintained) requires that tension between inner urge and outer check.
Kafkas parable therefore asks uncomfortable questions about how we
stimulate the arts, how we appreciate them, how we behave towards our
artists, and, in general terms, how we nourish and check (or control) them.
Further, the parable reminds us that there exist in society deeply equivocal,
problematic, and often fickle attitudes towards artists and the arts: like the
Hunger Artist, they are, sometimes even simultaneously, cherished and
disliked, revered and repudiated, adored and outcast.

Taking this last point, we can say that equivocal attitudes towards the artist are
expressed in the many contradictory gestures of administrations which,
variously, have offered support to the arts, or have withdrawn support, or have
honoured artists, or vilified them, or condemned and even killed them. It has
been said (Pick, 1986: 13) that such ambivalence stems from a concern that the
artist is an outsider whose artistic vision may be challenging: dangerous,
3
violent and shocking to society. Hauser (1982: 540) has argued that there exists
a precarious armistice between the arts and a society which harbours towards
them an often ill-concealed ... suspicion and anger. Such suspicion, we might
conceive, could explain why administrations - despite their equivocal attitudes -
have tended not to disregard the arts; rather, they have tried to manage the
arts, to take hold of them, to make them operate along certain lines and accept
particular responsibilities.

This paper assumes that evidence of the precarious armistice between

government and the arts can be discerned in the various attempts by
governments to evaluate arts organisations, attempts which, we will argue,
have tended to misunderstand and misrepresent both the artistic impulse and
the world in which artists operate. We will ask whether the recent approach of
the Social Audit movement might provide valuable guidance in designing a
more appropriate means by which society may understand the work of arts
organisations. We ask whether this approach might better reflect the needs of
artists and arts organisations, and whether it might offer a more positive and
mutually accepting relationship between government and the arts.

Should governments evaluate the arts?

Poetry is indispensable ... if only I knew what for

(Cocteau, in Fischer, 1986: 1)

As we shall see later, in this century successive UK governments have shifted in

their stance towards the arts, and especially over the question of how to
evaluate the arts. However an underlying assumption of governments has been
that they should be involved, at some level, in the arts, and indeed that arts
evaluation should be an aspect of that involvement. These assumptions are
generally based upon the belief that the arts, if they are to be supported either
through subsidy or tolerance, ought to give something to society.

The question of what the artist should give is controversial. For Oscar Wilde
(1890: 175), the value of the artist lay in his or her dreaming and thinking,
and Wilde was aggrieved that society misunderstood this value to such an
extent that it insisted artists should give productive labour to society (Wilde,
1890). But Dewey (1971: 94) brought the notions of an artists productive
labour and dreaming closer together in his statement that the arts are a
remaking of the experience of the community in the direction of greater order
and unity. The arts, Dewey argued, should lead the way towards an
achievement of societys collective goals through a demonstration of what
might be possible. But according to Read (1949) and Collingwood (1971), the
idea of the artist having some responsibility to give to society is entirely
immaterial and extraneous to artistic production. Read (1949: 190) says the
true artist has a will-to-form which is wholly indifferent to the materials
and conditions through which the artist achieves self-expression. And for
Collingwood (1971: 26-50) artistic expression is born of self-exploration, which
means that materials and techniques cannot be imposed upon the artist, nor can
an outsider assume them to be appropriate before that self-exploration or mental
4
journey is complete. Such views imply that any overarching or government-
motivated conception of the arts which informs public policy and the
distribution of resources to artists and their organisations, and even some sense
that artists have a responsibility to give to society, can be seen as
fundamentally immaterial, inconsequential, and in fact damaging to the arts,
since it is deleterious to the artists self-expression. Hence it can be argued that
if an arts evaluation judges art-work according to what it gives to society, it
may be so foreign to the impulses which produced it, and potentially so
damaging to the artistic impulse itself, that it is dangerous to pursue notions of
evaluation at all; the consequence of such pursuit could be the destruction of
art.

Yet in response it can be argued that an approach to the arts which denies the
relevance of materials and circumstances, which assumes that the arts should
not give to society, and which finds government involvement in the arts and
evaluation of the arts to be entirely inappropriate, offers little recognition of the
audience or the society which must support - in one way or another - artistic
endeavour. Indeed, this purist approach conflicts with the view (e.g. Hauser,
1982) that a work of art is never created in a vacuum, for society and social
and material conditions will inevitably affect the result. Sheppard (1987: 24)
notes that Collingwoods invective against government involvement in the arts
ignores the very craft of art, the fashioning and manipulation of material
which exists in the real world and requires on-going sourcing and maintenance.
To this argument for the grounding of artistic production in a social context,
and hence the evaluation of arts according to some social benchmark, we can
also admit the relevance of Friedrich Schillers (1982: 153) claim in his
Aesthetic Letters that a test of the excellence of art is its effect upon the
audience - preferably giving them a sense of release, freedom and elevation.

Differing perceptions of what the artist should give, and indeed whether the
artist should be expected to giveanything at all, are representative of the many
opinions which exist on what role the arts should play in society. Pick (1987)
notes that this confusion of opinions is reflected in the differing ways in which
the arts are regarded today: [as] an economic or political movement, a
psychological tenet or an educational idea - or all of these things at once. (Pick,
1987: 9) It is mainly this confusion, Pick suggests, which has allowed political
(and therefore governmental) understandings and expectations of the arts to
dominate. Thus in the UK, public support for the arts, most notably through the
Arts Councils, has come under the thumb of politicians, and the arts have been
funded and judged according to political needs and expectations (Pick, 1987).
Indeed, although the Arts Council of Great Britain (ACGB) was established in
1946 as an independent body under a Royal Charter expressly to ensure that it
was separate from the direct influence of government, it is now generally seen
as an arm of government. And speaking of the Canadian experience, Helwig
(1980) points to political control over the arts which is achieved, in part,
through the substitution of the word culture for the arts - a substitution
which conveniently subsumes all artistic endeavour under that vaguer, more
politically malleable label. In many other countries, state support for the arts is
deployed directly through ministries of culture and other government
5
departments. Thus, in global terms, it appears that notions of what the arts should
do, how they should be judged, and what resources should be allocated to them,
have been dominated by a political conception of the arts and a politically-
motivated approach to the treatment of the arts (Pick, 1987: 67).

It might be expected that artists would be opposed to government-dominated

control of the arts and to government-sponsored methods of evaluation.
Certainly, many artists believe the arts are fundamentally different from other
aspects of production and public service and that they should not be subject to
the same types and methods of evaluation; Donoghue (1982) contends that the
arts are at their healthiest when they are in a state of conflict with society. Yet
such a belief would not have been supported by all artists throughout history,
for in many cases artists have sought a fruitful harmony with patrons and the
paying public (see Gowrie, 1995: 2 on Shakespeare, and Lampert, 1986: 137 on
Rodin). Taking a pragmatic view of the artists desires and expectations, the
ideal situation, said Stravinsky (paraphrasing Ezra Pound) was that the artist
should be in a position where he can work as he likes, or waste time as he likes,
never lacking provision (Stravinsky, 1962: 222). Such an attitude could of
course be interpreted as irresponsible, a mere soaking up of what society has to
give without any sense that the artist should give to society. It is perhaps a
sense of that irresponsibility or rather a sense of concern that such feelings
could exist amongst artists (whose popular image has often been Bohemian,
outr, unconventional) which has reinforced the precariousness of the
armistice (Hauser, 1982) between successive UK governments and the arts
world, and has emphasised to governments the need to take hold of the arts, to
manage them, to make them somehow responsible to society.

Perhaps surprisingly, Pate (1998) has found that publicly funded arts
organisations tend broadly to support the idea that artists should be publicly
accountable - and that there should be some system to evaluate them. Indeed,
the main bone of contention is not whether their work should be evaluated but
how they can most appropriately be evaluated. Pates finding should perhaps be
viewed in the context of Bourdieus (1993) caution that the world of artistic
production inverts the fundamental principles of all ordinary economies
(Bourdieu, 1993: 39) and arts organisations can reap considerable rewards - in
both material and moral terms - if they ignore or fail to comply with social,
political or governmental expectations, and merely pursue art for arts sake.
The world of the arts, Bourdieu argues, is a singular upside-down economic
world ... those who enter it have an interest in disinterestedness. (Bourdieu,
1993: 40). Bourdieu cautions, then, that although arts evaluation may find
acceptance amongst artists today, it may provoke opposition and non-
compliance tomorrow. The arts world, he implies, is both fickle and self-
regarding, and it is difficult to predict the reaction of that world to initiatives
which seek to control it.

Trends in the UK Governments Understanding of Arts Evaluation

Government approaches to the arts and arts evaluation should be understood
within the larger context of a changing social and industrial context which,
especially in this century, has placed increasing emphasis upon fast and efficient
6
processes. Within that context the production or performance of a work of art
may appear laboured and unacceptably costly:

Human ingenuity has devised ways to reduce the labor

necessary to produce an automobile, but no one has yet
succeeded in decreasing the human effort in a live
performance of a 45-minute Schubert quartet much below
three man-hours.(Baumol and Bowen, 1966: 164)

Most people would agree that it is a nonsense to compare a performance of

Schubert with the manufacture of a car. But although notions of conveyor-
belt production have been widely criticised when they are applied to areas of
society outside industry, we live in an age when many types of service and
production have a tendency towards predictability, calculability and
homogenisation (Ritzer, 1996). It should therefore not be entirely surprising if
the work of art and the arts organisation were subject to a kind of questioning
which expected them to have something in common - perhaps similar goals
and processes - with industrial manufacture. However, the notion of adopting
measurement tools from the commercial business world to measure the
effectiveness of arts organisations would appear quite inappropriate.
Conducting a Value Chain Analysis, for instance, on a string quartet would be
absurd. What, after all, would be the point of charting its inbound and
outbound logistics?

When the Arts Council of Great Britain (ACGB) was established immediately
after the Second World War, its first Chairman, John Maynard Keynes,
declared that the artist was by nature individual and free, undisciplined,
unregimented and uncontrolled. The artist follows where spirit and inspiration
leads, he cannot be told his direction (Keynes, 1945: 31). For Keynes, the
arts were genuinely different from other aspects of production; consequently
they should be treated differently. For a long time, artists expected to be, and
were indeed, treated differently from other government provisions. Minihan
(1977: x) wrote that state involvement in the arts has usually differed from
bureaucratic efforts to enforce sanitary codes, to run railways, or to organise
the coal industry. Various studies of state patronage of the arts (Harris, 1970;
Jenkins, 1979; Baldry, 1981; Gowrie, 1995; Rees-Mogg, 1985; Pick, 1988;
Lewis, 1990; Peacock,1993; Sawers, 1993; Williams, 1971) also demonstrate
that successive post-war governments have accepted that the arts should be
handled differently from other aspects of government activity.

Nonetheless, thirty years after its establishment, Hugh Jenkins, Minister for the
Arts (1974-76), attempted to oblige the ACGB to adopt a more systematic
approach to its grant allocation procedures. Concerted efforts were made so
that the ACGB would employ an objective, transparent points scoring system
when considering grant applications from arts organisations and artists.
Delaying tactics were used by the ACGB to resist this, culminating in an
eventual flat refusal by its Chairman, Lord Gibson, who stated artistic
judgements cannot be measured. (Jenkins, 1979: 189-200). Indeed, and as
stated earlier, the British Government had established the ACGB as an
7
independent body, therefore implying that government should be kept at arm's
length from the ACGB, and that the arts should be free from political influence.
There are anecdotes from the early days of the ACGB of how officers were
given considerable freedom to act, to initiate exciting creative projects and to
engage with artists in the development of art forms. There are other stories of
how departmental budgets often overran (by as much as 100% in some cases!)
but that it was considered to be no problem. The ACGB culture, in essence,
could be said to reflect the culture of the arts world itself: it was populated by
people who were passionate about the arts, were given the freedom to
experiment (and to fail) and to grasp opportunities as they arose. But
successive administrations (and especially under Margaret Thatcher) which
introduced increasingly strong central government controls over public
spending, did not fail to examine closely the monies distributed to the arts by
the ACGB. During the 1980s this scrutiny gradually extended to the policies
and practices of the ACGB itself , and the belief that the arts should be treated
differently from other areas of activity was subsumed under the new notion that
the arts should represent value for money and should be accountable. In
1985, the Chairman of ACGB, Sir William Rees-Mogg, wrote that the
qualities required for survival (of the arts) in this age will be the qualities of the
age itself ... they include self-reliance, imagination, a sense of opportunity,
range of choice and the entrepreneurial action of small professional groups.
(Rees-Mogg, 1985:8). These words echo the UK governments belief at that
time that arts organisations were not inherently different from other areas of
business and entrepreneurship. Pick (1988) noted that

the key shift has been to describe the arts no longer

in their traditional language, which includes aesthetic
judgement, private satisfaction and spiritual benefit, but
as a purely commercial entity, to be justified by its
economic benefit. (Pick, 1988: 90)

In the 1970s and 1980s, increasing pressures upon arts organisations for
efficiency, effectiveness, transparency, and accountability (Fitzgibbon & Kelly,
1997: 3) brought increasing and widely articulated resistance from artists who
feared that these new priorities for the evaluation of the arts would not only
inevitably lead to the adoption of business practices which would stifle artistic
creativity, but also to an embrace of profit-orientated business objectives (Pate
1988). In particular, the enthusiasm amongst arts funders that the arts should
demonstrate effectiveness, accountability (and so on) was expressed in the
general exhortation by politicians, the popular press, and some sections of the
public, that government-supported artists should aim to appeal to a wider
public. Arts events were obliged to incorporate some indicator of the extent of
public attendance - a trend which artists feared could lead them into populism
and an erosion of quality. Such fears have, it should be said, a long history, and
we could simply ascribe them to a traditional elitism within the arts (Pick,
1988: 30). Yet the concerns of artists today are surely sharpened by the
existence of evaluative measures which attempt to demonstrate an arts
organisations success in winning an audience, and by the rhetoric of arts
accountability which, in recent years, has been imbued with the phraseology of
8
 business, competition, and quantitative measurement. It has been found (Pate,
1998) that a major concern of arts organisations today is the sense that their
artistic work which they see as their raison dtre is gradually being
relegated by funding bodies to a position of secondary importance behind their
capacity to put bums on seats and to demonstrate (through a range of
statistical performance indicators) their efficiency and value for money.

A Brief Overview of Current Appraisal Procedures

Although this paper has so far used the term evaluation in respect to the
judgement of the worth or quality of art-work, appraisal is the term used by
arts funding bodies for the formal process of assessing the work of arts
organisations in receipt of annual core funding (recurring revenue funding,
as distinct from one-off project funding). For arts organisations, the appraisal
is a vital component of the process that leads to the awarding of grant-in-aid
and it therefore has a real bearing on the organisations future viability.
Consequently, we concentrate here on revenue-funded arts organisations
which, typically include such concerns as art galleries, dance companies, theatre
venues, orchestras, drama companies, publishers, opera companies and
community arts groups.

Last year, as part of the governments Comprehensive Spending Review, the

Department of Culture, Media & Sport (DCMS) proposed to establish a tough
new watchdog which would not only monitor and appraise the performance of
arts bodies which were in direct receipt of government funding (such as the arts
councils), but also those organisations which were second-tier funded bodies
(arts organisations)4.

Current arrangements for the appraisal of arts organisations5 typically involve

periodic visits (roughly once every three years) to the organisation, each lasting
between one and five days. The visit is performed by an appraisal team which
usually consists of arts council officers and members, a representative of any
local authority funding partners, and sometimes outside consultants who may
be needed to provide particular expertise - such as management consultants,
those with expertise in disability matters, architects or engineers6. In
preparation for the visit, the arts organisation is required to submit
comprehensive review and planning documentation which covers six areas:
Artistic Programme, Marketing, Financial Performance, Management,
Education/Outreach, and Equal Opportunities. In addition the organisation
must submit a record of income and expenditure during the last three years and
a forecast of the same for the next three years. The meetings held with the
appraisal team cover 10 key areas, such as Artistic Policy & Achievement;
Operating Environment; Performance Indicators & Peer Group Comparisons;
Financial Management & Control. Only the first of these areas (Artistic Policy
and Achievement) deals directly with the organisations artistic vision, and this

4
Department of Culture, Media & Sport 1998, Consultative paper The Comprehensive
Spending Review: A New Approach to Investment in Culture, 24 July.
5
Arts Council of Wales, Appraisal of Arts Organisations, May 1997.
6
Ibid
9
 is itself broken down into ten sub-agenda headings, only three of which relate to
the quality of the work already performed. The other sub-agenda headings
include such items as: Commitment to New Work, Cultural Diversity,
Broadening Public Access. And in case these headings and sub-headings fail to
convince the arts organisation of the tenor of the appraisal, the stated purpose
of the appraisal7 includes such objectives as: to assist the organisation in
reviewing the effectiveness of its operation; to provide an authoritative
document useful in discussions with other sources of finance, and to assist in
demonstrating proper accountability for the use of public funds.

An important argument, and a common complaint against such appraisals is

that they have nothing to do with aesthetic judgement, that they understand the
worth or merit of the arts organisation primarily in terms of such matters as
revenue generation, compliance with legislation and the organisations attempts
to market its work and bring in an audience. However, even if we leave aside
such an important objection, it appears that an unintended consequence upon
arts organisations of the current type of appraisal is a sense of perplexity as to
how they have done and how their organisation should move forward. This
perplexity derives from the fact that the appraisal process often produces
nothing more revealing than a catalogue of the organisations activities over the
past period, offering information about which, usually, the organisation is
already well aware. Furthermore, the whole complicated system of appraisal
allows the appraised to challenge any criticisms by the appraiser, and this can
culminate in what may be called a neutral appraisal - where any judgement of
worth or merit (Guba & Lincoln, 1981, 1989) is avoided. Artists have a further
concern about the current system of appraisal. Increasing importance is being
placed upon performance indicators, upon which artists feel that funding bodies
rely overwhelmingly (and inappropriately) when they make their decisions
(Pate, 1998). This heavy reliance upon quantitative data has probably
developed because, for the funders, they are easy to obtain, easy to use and can
appear authoritative (Matarasso, 1996).

Not only do individual artists have concerns about current forms of arts
organisation appraisal; even large private sector organisations are beginning to
feel that the performance indicators which form such an important part of the
appraisal system do not tell, either in social or in economic terms, the full story
of their performance. They also fear that such headline messages as are often
derived from data supplied by quantitative performance indicators may lead to a
reductionist judgement by anecdote (Raynard, 1997) and hence crass
calculations (Blake, Frederick & Myers, 1976: 42). Perhaps the greatest
danger with performance indicators, from the artists point of view, is that they
are generally based on data that speak very little of that which is important to
the artist creativity, innovation, boldness of vision, and so on. Instead, these
indicators tend to measure the market response to the artist's work rather than
evaluate the artistic qualities of the work itself. And although, as stated earlier,
artists in receipt of public monies acknowledge the importance of public
accountability (Pate, 1998), they also feel that any appraisal procedure should

7
Ibid
10
take into consideration the imperatives of the creative process and the aesthetic
qualities of the art they generate, which they produce in response to creative
forces rather than market forces. For the artist, the processes of creation
revolve around the producer, the artist, and the intrinsic value of the product.
Cultural production is not seen as a process of commodity exchange at all.
(Lewis, 1990: 141) Kao (1989) remarks that the creative process is inner-
directed and even the notion of doing market research to validate a creative
vision is often anathema to people on the creative side of a business(Kao,
1989: 17).

In Bourdieus terms, the artists objection to performance indicators is a

reaction against the emergence of the work of art as a commodity (Bourdieu,
1993: 114). This reaction, according to Bourdieu, would be expected to
provoke an equally strong dissociation of art-as-commodity from art-as-
pure-signification (Bourdieu, 1993: 114). We would therefore anticipate that
artists dislike of number-crunching appraisals would strengthen the
conviction that art has a purely symbolic intent (Bourdieu, 1993: 114).
Historically, of course, there has been a strong tendency for the arts to be
regarded as concerned with the non-material and the spiritual, [looking]
beyond machinery(Matthew Arnold, 1869: 209). And yet, as we saw earlier in
our synopsis of Kafkas Hunger Artist, it is possible that a neglect of the
material can cause the artist her/himself to suffer, and may bring about a
cessation of the art itself. The hunger artist who disregards his promoter and
is insensitive to his audiences desires, merely following his own urge to create
a spectacle that, in his terms, is genuine pure art, ultimately engineers his own
death (and therefore the death of the art which he produces). It is possible to
assert, therefore, that if artists were to reject appraisal on the grounds that it
did not reflect their aesthetic concerns, their best interests might not be served.
That many artists may recognise this, is suggested by the recent research by
Pate (1998) which discovered that most artists have a profoundly pragmatic
approach to the maintenance of their livelihood. But the Audit Commission has
itself expressed the concern that evaluation must not emphasise the
measurable at the expense of the immeasurable (Thornton, 1992): a hint,
perhaps, of self-criticism, or the offer of an opening to, and dialogue with,
those who find current arts appraisals unpalatable and irrelevant to the work of
artists and arts organisations.

An Alternative Approach to Arts Appraisal: Social Auditing

Social Auditing takes a broader social and economic perspective upon the
organisation than the traditional financially-based audit. It even reaches beyond
the organisation itself to examine its social and economic impact upon its
stakeholders, including attention to many variables which may not be amenable
to quantification or measurement in monetary terms (Geddes, 1992). Social
Auditing seeks to determine whether an organisation succeeds in satisfying its
various stakeholders - that is, those who affect the conduct of an organisation
and those who are affected by its operations. Since no two organisations will
have the same structure, objectives, stakeholders or environment, the exact
parameters of a Social Audit will tend to differ from organisation to
organisation.
11
Consequently, the key initial stages in the process of Social Audit entail a listing
of the organisations objectives, then an identification of its stakeholders.
Following from this, the Social Audit tries to determine, in conjunction with the
various stakeholding groups, the indicators by which the organisation should be
measured. This so-called scoping exercise sets the stage for the audit-proper
which involves such elements as further stakeholder consultation (focus group
meetings, interviews and questionnaires), internal and external document
review, preparation of social accounts and reports, external verification, and
then publication. A possible audit cycle is shown in Figure 1 below.

Figure 1: A Possible Social Audit Cycle

(Source: New Economic Foundation 1998)

Social Auditing is gaining momentum, even amongst businesses that are

conducted for profit. Although the first organisations to embrace Social
Auditing tended to be non-profit organisations with explicit social objectives
such as Traidcraft and Shared Earth (Pearce, Raynard & Zadek, 1998) and
private sector businesses with stated social agendas such as the Body Shop and
Ben & Jerrys Ice Cream (Johnston, 1996), more recently some of the worlds
largest corporations have issued Social Audit accounts alongside their financial
accounts. These include Royal Dutch Shell (Corzine, 1997) and British
Telecom (Raynard, 1997). The popularity and importance of Social Auditing
are exemplified by the fact that major accounting firms, such as KPMG, are
now offering Social Auditing services to their clients (Zadek, 1997). It has
been said that Social Auditing offers organisations a way of reporting which
discloses more than mere financial viability and which reflects the
organisations own objectives (Pearce, Raynard & Zadek, 1997). A Social
Audit is, typically, submitted alongside the organisations financial accounts,
12
therefore it does not so much replace the financial accounts as present a more
complete picture of the organisations effectiveness in meeting its objectives
and satisfying its stakeholders.

The Social Audit approach, we might assume, could be of great interest to arts
organisations, for it could simultaneously embrace their aesthetic, cultural,
economic and social values (Matarasso, 1997: 3). It could seek to address the
issue of quality (as opposed to quantity) which tends to be obscured or
undermined by the preoccupation with financial performance and quantitative
indicators (Turok, 1990). In fact, various organisations experience of Social
Audit suggests that adopting this approach could offer several benefits to arts
organisations. For example, arts book-keeping and arts accounting would be
an on-going process conducted by the organisation itself and would enable
evaluation of those aspects of an organisations performance that are not
amenable to quantitative measurement. This approach would ensure that both
the artistic and financial performance of an organisation are taken into account
during the process of evaluation. And, like financial accounts, a Social Audit
can be made subject to verification by one or more persons with no vested
interest, and would be subject to disclosure.

Yet, bearing in mind artists deep concerns about current appraisal methods,
Social Audit should not be accepted without a closer look.

Social Audit and the Arts

In considering the value of Social Audit to the arts, we can draw upon an
existing body of knowledge of Social Audit in certain organisations in other
sectors (such as the non-profit sector, the public sector and Non-Departmental
Public Bodies (NDPBs)). And in recent months, the Arts Council of England
has begun conducting a pilot Social Audit of one of its larger clients. At first
sight the Social Audit does appear to provide an opportunity for arts
organisations to be evaluated on the full extent of their work - including, most
importantly, those aspects that are important to them. Informal discussions
held by the authors of this paper with artists who lead funded organisations
would suggest that they would be very interested in exploring this kind of
approach. However various issues have emerged as warranting careful
consideration before Social Audit can be considered for adoption (or most
probably adaptation) in the arts.

First, to improve on the appraisal procedures currently practised by arts funding

bodies, any new arrangement would need to cut down significantly on the
bureaucracy of appraisal - the sheer time, effort and paperwork that are now
involved. This bureaucracy entails weeks of preparation, the production of
large amounts of documentation (business plans, marketing plans, affirmative
action plans and quantities of statistical data) as well as Income and
Expenditure projections, and cash-flow forecasts, for the next three or five
years. These are then discussed in meetings that can take several days to
complete. There is no doubt that Social Audit can likewise involve a heavy
work-load which includes a lengthy procedure to identify the various
stakeholders, to convene stakeholder group meetings, to determine stakeholder
13
objectives with each of these groups, to administer questionnaires, to analyse data,
and to write up comprehensive reports. Social Audit, in its customary form,
therefore, may not cut down on the bureaucracy of appraisal, and, indeed,
might represent an even greater burden than the traditional form of appraisal:
recent experience with the Arts Council of England pilot Social Audit
demonstrated that stakeholder focus groups were difficult to convene, and
some were cancelled altogether when it became apparent that only one or two
individuals were prepared to attend. In some of the large non-arts businesses
that have attempted Social Audit, separate well-staffed units were established
within the company to oversee the whole Social Audit process - an expense
which the vast majority of arts organisations simply could not contemplate.

Second, Social Audit does not resolve the question of how, if no two
individuals will react to a work of art in entirely the same way (since individuals
have different mentalities, knowledge and experience (Santayana, 1896)), it is
possible to evaluate an organisation's artistic work in a way that is fair.
Usually, the notion of fairness is equated with objectivity - a difficult equation
when we consider reactions to a work of art that are necessarily subjective.
This issue is related to the quantitative/qualitative debate. Social Audits have
tried to address this debate by translating qualitative responses into quantitative
measures. A Social Audit might chart, for instance, the proportion of an
audience that claims to have enjoyed a play or was moved by a piece of
music; these data are then used to determine whether the organisation has met
certain of its objectives. But what proportion of an audience should claim its
satisfaction with an art-work before the objective may be deemed to have been
achieved? An approach to evaluation that is organised around objectives and
evaluated against those objectives has been criticised as not leading to explicit
judgement of worth or merit (Guba & Lincoln, 1981, 1989), and it is easy to
recognise the possibility that objectives, in this Social Audit sense, would
appear to most artists to be wholly irrelevant to the process of artistic creation.
Admittedly, such objectives may be appropriate to some non-creative aspects
of an arts organisations programme, such as the number of new productions
per year, the number of touring weeks, and so on. It is possible that artists
would be more amenable to exploring goal-free evaluation models (Scriven,
1973; Guba & Lincoln, 1981,1989; Lincoln & Guba, 1985).

Third, by whom might Social Audit be carried out? If specialist consultants

were employed, the arts organisation would incur a major cost (probably
similar to accountants' fees). If employees of the arts organisation were used,
there would be both training and opportunity costs, as well as the start-up costs
for initiating the Social Audit programme. Nevertheless, it is likely that any
comprehensive and appropriate system of evaluation for arts organisations
would include costs. In any event, the appraisal procedures currently
employed are, themselves, extremely costly in terms of the significant time and
effort devoted to such exercises - both by funding body officers and artists.

Conclusion
Given that successive recent administrations, including the present government,
have reaffirmed their commitment to ensuring greater accountability for the
14
spending of public monies, it is clear that the concept of evaluating publicly
funded organisations is likely to continue to be significant. Indeed, in recent
research we have found that both the DCMS (re: Arts Council of England) and
the Welsh Office (re: Arts Council of Wales) believe that the importance of
evaluating the performance of arts organisations has increased considerably
following the 1997 announcement of the Chancellors Comprehensive Spending
Review.

For arts organisations, Social Audit appears an attractive alternative to

traditional auditing, and we may regard it as representing at least the first stage
in the evolution of an evaluation procedure which embraces those aspects of an
organisations work that are relevant to the artists creative will-to-form
(Read, 1949). Yet in championing any development of Social Audit with
reference to arts organisations, we must not forget the tensions of which Kafka
(1983) writes so chillingly in his Hunger Artist, in which the artist, in a torment
of physical and mental suffering, produces a spectacle which, though crucially
important and life-threatening to him, is finally of little interest to a fickle
public. The standards, then, by which the outside world judges the artist may
be far removed from those standards which are internal to the artist her/himself.
Further, the artists creativity is ignited by compulsions of which the outside
world has little knowledge, and in which it has little interest. What value, then,
does even a Social Audit have for the artist - enlightened though it may appear
in contrast to the quantitatively-based, financially-interested traditional audit?
The Social Audit, for all its advantages over the traditional audit, may still leave
entirely undisturbed the tensions and the precarious armistice (Hauser, 1982)
between the artist and a public which is largely ignorant of the foundations of
artistic creativity and its complex relationship with society.

-oOo-

REFERENCES

Arnold, M. 1869, Culture and Anarchy in Culture & Society in Britain,

1850-1890, ed. Golby, J.M., Oxford University Press.

Baldry, H. 1981, The Case for the Arts, London: Secker & Warburg.

Baumol, W. J. & Bowen, W. G. 1966, Performing Arts: The Economic

Dilemma, New York: Twentieth Century Fund.

Blake, D. H., Frederick, W. C. & Myers, M. S. 1976, Social Auditing:

Evaluating the Impact of Corporate Programs, New York: Paeger.

Bourdieu, P. 1993, The Field of Cultural Production, Oxford: Polity Press.

Collingwood, R.G. 1971, Art as Expressionin Hospers, J. (ed.), Artistic

Expression, New York: Appleton-Century-Crofts (The Meredith Corporation).

15
Corzine, R. 1997, Shell to Answer Ethics Queries from Investors, Financial
Times, 2 April.

Dewey, J. 1971, The Act of Expressionin Artistic Expression, ed. Hospers,

J., California: Appleton-Century-Crofts.

Donoghue, D. 1982, The Arts without Mystery, London: BBC, cited in

Fitzgibbon, M. & Kelly, A. (eds), p. 1 Op. Cit.

Drucker, P. F. 1990, Managing the Non-profit Organisation Practices and

Principles, Oxford: Butterworth-Heinemann.

Fischer, Ernst 1986, The Necessity of Art (Trans: Anna Bostock),

Harmondsworth: Penguin.

Fitzgibbon, M. & Kelly, A. (eds.) 1997, From Maestro to Manager Critical

Issues in Arts & Culture Management, Dublin: Oak Tree Press.

Geddes, M. 1992, The Social Audit Movementin Owen, D., (ed.) Green
Reporting: Accountancy and the Challenges of the Nineties, London:
Chapman & Hall.

Gowrie, Earl of 1995, The Philosophy of Cultural Subsidy, London: Arts

Council of England.

Guba, E.G. & Lincoln, Y.S. 1981, Effective Evaluation: Improving the
Usefulness of Evaluation Results through Responsive and Naturalistic
Approaches, London: Jossey-Bass.

Guba, E.G. & Lincoln, Y.S. 1989, Fourth Generation Evaluation, London:
Sage.

Harris, J. S. 1970, Government Patronage of the Arts in Great Britain,

London: University of Chicago Press.

Hauser, A.. 1982, The Sociology of Art (Trans. K.J. Northcott), London:
Routledge.

Helwig, D. (ed) 1980, Love and Money: The politics of culture, Canada:
Oberon Press.

Hospers, J. (ed.), 1971 Artistic Expression, New York: Appleton-Century-

Crofts.

Jenkins, H. 1979, The Culture Gap An Experience of Government and the

Arts, London: Marion Boyars.

Johnston, J.-A. 1996, Social Auditors: The New Breed of Expertin Business
Ethics, March/April.
16
Kafka, F. A Hunger Artist, trans. Willa and Edwin Muir, in The Penguin
Complete Short Stories of Franz Kafka, 1983, ed. N. Glatzer,
Harmondsworth: Allen Lane.

Kao, J. 1989, Entrepreneurship, Creativity & Organisation, Englewood

Cliffs, NJ: Prentice Hall.

Keynes, J. M. 1945, The Listener, 12 July, p. 31, quoted in Harris, J. S. 1970,

Op. Cit.

Lampert, C. 1986, Rodin, Sculpture & Drawings, London: Arts Council of

Great Britain.

Lewis, J. 1990, Arts, Culture & Politics: The politics of art and the cultural
industries, London: Routledge.

Lincoln, Y.S. & Guba, E.G. 1985, Naturalistic Inquiry, London: Sage.

Matarasso, F. 1996, Defining Values: Evaluating Arts Programmes, Stroud:

Comedia.

Matarasso, F. 1997, Use or Ornament?; The Social Impact of Participation

in the Arts, Stroud: Comedia.

Minihan, J. 1977, The Nationalisation of Culture The Development of State

Subsidies to the Arts in Great Britain, London: Hamish Hamilton.

Pate, M. 1998, Business Practice in the Arts: How small arts organisations in
Wales are coping with official pressure to adopt practices and procedures
from commercial business, MBA dissertation, University of Glamorgan,
Pontypridd.

Peacock, A. 1993, Paying the Piper, Edinburgh University Press.

Pearce, J., Raynard, P. & Zadek, S. 1998, Social Auditing for Small
Organisations, London: New Economics Foundation.

Pick, J. 1986, Managing the Arts? The British Experience, London:

Rhinegold.

Pick, J. 1988, The Arts in a State, Bristol Classic Press.

Raynard, P. 1997, Pulling Together, Charity World, July/August.

Read, H. 1949, The Meaning of Art, Harmondsworth: Pelican Books.

Rees-Mogg, W. 1985, The Political Economy of Art, London: Arts Council

of Great Britain.
17
Ritzer, G. 1996, The McDonalidization of Society, California: Pine Forge
Press.

Santayana, G. 1896, The Sense of Beauty, partly reprinted in Hospers, J.

(ed.), Artistic Expression, New York: Appleton-Century-Crofts.,Chapter 8, pp.
160-176.

Sawers, D. 1993, Should the Taxpayer Support the Arts?, London: Institute
of Economic Affairs.

Schiller, F. 1982, On the Aesthetic Education of Man, Wilkinson, E.M. and

Willoughby, L.A.(eds.), Oxford: Clarendon Press.

Scriven, M. 1973, 'Goal-free Evaluation', in E. R. House (Ed.), School

Evaluation: the politics and process, Berkeley, CA; McCutchan.

Sheppard, A. 1987, Aesthetics: An Introduction to the Philosophy of Art,

Oxford: Oxford University Press.

Stravinsky, I. 1962, Stravinsky in Conversation with Robert Craft,

Harmondsworth: Pelican Books, quoted in Peacock, A. , p. 24, Op. Cit.

Thornton, C. 1992, London Evaluation Scheme: Final Report, London: Law

Centres Federation, quoted in Matarasso, F. 1996, p.15, Op. Cit.

Turok, I. 1990, Evaluation and Accountability in Spatial Economic Policy: a

review of alternative approaches, Scottish Geographical Magazine, quoted in
Geddes, M. 1992, Op cit.

Wilde, O. 1890, The True Function of Criticism, cited in Goodwin, M.,

(ed.), 1951, Nineteenth Century Opinion, Harmondsworth: Pelican Books.

Williams, R. 1971, Culture and Society, 1780-1950, Harmondsworth:

Penguin.

Zadek, S. 1997, Beyond Profit and Lossin Briefing, quarterly newsletter of

the Prince of Wales Business Leaders Forum, Spring.

-oOo-

18