Professional Documents
Culture Documents
293
ntroduction
books: an
experiment to
measure the
impact of open
access publishing
Ronald SNIJDER
Amsterdam University Press
ABSTRACT. This article describes an experiment
to measure the impact of open access (OA)
publishing of academic books. During a period of
nine months, three sets of 100 books were
disseminated through an institutional repository, the
Google Book Search program, or both channels. A
fourth set of 100 books was used as control group.
OA publishing enhances discovery and online
consultation. Within the context of the experiment,
no relation could be found between OA publishing
and citation rates. Contrary to expectations, OA
publishing does not stimulate or diminish sales
figures. The Google Book Search program is
superior to the repository.
Ronald Snijder
LEARNED PUBLISHING
VOL. 23
NO. 4
OCTOBER 2010
Ronald Snijder
294
the experiment
consists of
creating four
equal sets of
100 titles
lead to more usage? This question has several aspects, described below:
Does OA publishing of academic books
lead to more online consultations or
downloads? Disseminating scientific
output is the main goal of academic
publishers and OA publishing may
enhance that.
Does OA publishing of academic books
lead to higher citation rates? An important part of the discussion on OA
publishing of articles focuses on citation
advantage. The question is whether this
effect also can be measured for books
within the experimentation period,
which is very short for citation analysis.
Does OA publishing of academic books
lead to higher sales figures of their paper
counterparts? At this moment, most
readers seem to prefer a paper book
over an e-book for extensive use.
Which channel is best suited to promote
discovery of academic books published as
OA? Traditionally, OA publishing of articles is associated with repositories. For
books, the Google Book Search program
may be used as an alternative dissemination channel.
Does a multichannel approach lead to a
higher discovery rate of academic books
published as OA, compared to using a
single channel? As scholars in the humanities and social sciences tend to use
several channels for finding information,
making OA publications available through
more than one channel may stimulate
their usage.
Setup of the experiment
The experiment consists of creating four
equal sets of 100 titles, where each title is
placed in one of four sets. The different sets
are defined using two variables: accessibility
and channel. Each set is disseminated using
a specified channel and accessibility settings.
For a period of 9 months, starting in April
2009, the effect on discovery, online consultation, sales, and citations is measured.
Discovery and online consultation are measured using the number of views and
downloads from the respective channels. As
all titles are accessible in the Google Book
Search program, the usage statistics help to
determine the interest by readers. Among
the usage statistics are book visits
measuring the number of times the webpage
of the book is accessed and page views,
which measure the number of pages opened.
Furthermore, the AUP repository can deliver
its own usage statistics: the number of times
a record is accessed and the number of
downloads of the digital books stored in the
repository. Sales figures are provided by
Set 1
Set 2
Set 3
Set 4
No
No
No
Yes
Yes
Yes
Yes
No
Set 1: Available as usual. An electronic version of almost all books by AUP is submitted to the Google
Book Search website. By default, AUP allows a user of Google Book Search to see only 10% of the books
contents. The full content of each book is indexed by the Google search engine. The titles in this set are
not uploaded into the AUP repository. The accessibility of this set is the lowest.
Set 2: Freely available via the repository; visible for 10% in Google Book Search. The titles of this set are
uploaded in the AUP repository. For each title, a record is created in the repository database containing
metadata and an electronic version of the book. The visibility settings of Google Book Search are not
changed and remain at 10%.
Set 3: Visible for 100% in Google Book Search and freely available via the repository. The titles of this set are
uploaded in the AUP repository, and the visibility settings of Google Book Search are set to 100%. The
titles in this set are fully accessible through both channels. The accessibility of this set is the highest.
Set 4: Visible for 100% in Google Book Search; not available via the repository. For this set, the visibility
settings of Google Book Search are set to 100%. The books are not placed in the AUP repository.
LEARNED PUBLISHING
VOL. 23
NO. 4
OCTOBER 2010
The profits of free books: an experiment to measure the impact of Open Access publishing
295
Removal of bias
Considerable effort has been put into the
removal of bias. This experiment operates
using four sets of books; therefore these sets
must be as equal as possible. Each of the 400
books is compared using the following criteria: subject, type of work, language, expected
sales, and publication date. To address most
of these criteria the book series could be
used. All titles within one series should contain the same properties: the same language,
type of document, sales expectations, and
subject. This seemed a promising way to
proceed, as more than 70% (283) of the
selected titles are part of a series. Unfortunately, not all series are centred on a broad
subject. For instance the series LUP
Dissertaties, UvA Proefschriften, Amsterdam
Academic Archive, LUP Academic or
Atheneum Boekhandel Canon cover a wide
range of subjects, while language and document type are similar. Other series such as
Film Culture in Transition, Changing Welfare
States, Tekst in Context or Studies over Politieke
Vernieuwing also share a main topic. In order
to overcome this, all titles from a subject-based series were assigned the same
subject codes.
In the AUP database each title is assigned
several subject codes describing the content.
In order to create relatively large groups
with the same subject, the number of subject
codes was reduced. The same principle was
applied to the expected sales. In order to
measure this, the print run of a title was
used. While each individual title may have a
different print run from 0 for printon-demand titles to 6,500 an amount
rounded up to the next 500 was used. This
created again relatively large groups, which
could be evenly divided over the four sets. In
addition, the publication year was taken into
account for the distribution of the titles.
Research results
The research questions were translated into
hypotheses. The experiments data was analysed using ANOVA (analysis of variance)
in order to test the various hypotheses. The
results are summarized in Table 2.
As described before, each set was created
using several properties of the 400 titles in
LEARNED PUBLISHING
VOL. 23
NO. 4
OCTOBER 2010
considerable
effort has been
put into the
removal of bias
Ronald Snijder
296
Result
LEARNED PUBLISHING
VOL. 23
NO. 4
OCTOBER 2010
The profits of free books: an experiment to measure the impact of Open Access publishing
Citations
Page views
Downloads
Step 1
0.12 (0.11)
0.04 (0.02)
0.80 (0.79)
0.14 (0.11)
0.08
0.06
0.06
0.00
0.07
Book visits
0.06
0.07
0.02
PrintRun
0.31**
Year
0.09**
0.02
0.87**
0.10
0.02
0.05
0.04
0.03
0.01
0.09
0.01
0.02
0.01
0.09
0.08
RepositoryViews
0.09**
0.13*
0.11*
0.01
0.01
0.04
0.36**
Step 2
0.27 (0.18)
0.12 (0.02)
0.82 (0.80)
0.30 (0.13)
0.07
0.06
0.00
0.05
0.01
0.09**
0.07
0.06
0.09**
Book visits
0.06
0.15*
0.07
0.13*
0.00
0.08
0.02
0.01
0.02
0.84**
0.08
0.11
0.11*
PrintRun
0.28**
0.10
0.01
0.09
Year
0.14*
0.01
0.02
0.02
0.08
0.06
RepositoryViews
Art_History
0.30**
0.02
0.02
Culture
0.02
0.05
0.02
Culture_History_Culture
0.01
0.02
0.02
0.06
0.05
0.07
0.01
0.02
Dutch_Language_Study
0.01
0.03
0.02
0.00
Dutch_Literature
0.05
0.04
0.02
0.01
0.20*
0.05
0.01
0.02
Dutch_Language
Dutch_Literature_History
Dutch_Literature_Education
0.36**
0.09
0.03
0.02
0.03
Economics
0.03
0.00
0.01
0.01
Education
0.05
0.01
0.02
0.01
History
0.08
0.06
0.02
0.06
Information_Technology
0.02
0.01
0.04
0.01
0.03
0.02
0.01
Japan_Culture_History
0.15*
LEARNED PUBLISHING
VOL. 23
NO. 4
OCTOBER 2010
297
Ronald Snijder
298
Sales
Page views
Downloads
Law
0.03
0.03
0.01
0.02
Law_History
0.03
0.11
0.08
0.02
Literature
0.03
0.02
0.02
0.02
Mathematics
0.01
0.01
0.03
0.02
Medicine
0.05
0.02
0.03
0.02
Motion_Pictures
0.04
0.03
0.03
0.05
Music
0.04
0.00
0.07*
0.02
Philosophy
0.05
0.01
0.01
0.02
Political_Science
0.03
0.13
0.05
0.07
Political_Science_Law
0.03
0.02
0.00
0.03
Political_Science_Public_Administration
_Sociology
Psychology
0.14
0.07
0.03
0.01
0.06
0.15
0.01
0.03
Public_Administration
0.02
0.01
0.02
0.00
Public_Administration_Economics_Law
0.04
0.00
0.03
0.01
Public_Administration_Political_Science
0.04
0.01
0.01
0.01
Public_Administration_Political_Science
_Sociology
Public_Administration_Sociology
0.11
0.07
0.02
0.04
0.09
0.03
0.01
0.01
0.05
0.01
0.02
0.05
Religion
Science
Sociology
Theatre
*
Citations
0.00
0.27*
0.01
0.05
0.09
0.06
0.02
0.02
0.03
0.02
0.01
0.03
LEARNED PUBLISHING
VOL. 23
NO. 4
OCTOBER 2010
The profits of free books: an experiment to measure the impact of Open Access publishing
LEARNED PUBLISHING
VOL. 23
NO. 4
OCTOBER 2010
299
Ronald Snijder
300
while online
usage is higher
for fully
accessible titles,
it was not
translated in
higher sales
figures. The
reasons for that
remain unclear
LEARNED PUBLISHING
VOL. 23
NO. 4
OCTOBER 2010
The profits of free books: an experiment to measure the impact of Open Access publishing
4.
5.
6.
7.
Limitations
For this particular experiment, no models
were available; therefore no best practices
could be assessed for guidance. The sample
(n = 400) is relatively large and in order to
remove bias the titles were placed in different sets using publication year, subject, print
run and language. The properties publication year, subject and print run fall within a
wide range, ensuring that one aspect does
not dominate the results. Nevertheless, all
titles were published by one academic publisher. If certain aspects of a publisher such
as reputation or marketing budget influence the results, these could not be tested in
this experiment. Furthermore, for citation
analyses, the experimentation period is relatively short and the number of titles used is
low. In citation analysis, the period usually
encompasses several years instead of nine
months, while the number of titles within
one scientific field is usually higher than
100. In addition, at this moment no established standard for the scientific impact of
academic books exists, such as set by the
Institute for Scientific Information.
References
1.
2.
3.
8.
Ronald Snijder
Project Supervisor Digital Publications
Amsterdam University Press
Herengracht 221
1016 BG Amsterdam, The Netherlands
Email: r.snijder@aup.nl
www.aup.nl
Ronald Snijder (1970) joined AUP in 2007, where he is
responsible for developing digital publications, combined
with IT management. Before that, he has worked in
several profit and not-for-profit organisations as an IT and
information management specialist.
Corrigendum
After reanalysis of the data by Philip Davis
of Cornell University it was found that there
is no interaction between Google Books and
the AUP repository (on any response variable), meaning that the observation that
single channel distribution is superior to
multi-channel distribution is incorrect.
All other findings of the experiment
were reconfirmed: Open Access Publishing
enhances discovery and online usage; no
relation could be found between OA Publishing and sales or citations; as a dissemination
channel, the Google Book Search Program
performs better than the AUP repository.
R. Snijder
LEARNED PUBLISHING
VOL. 23
NO. 4
OCTOBER 2010
301
if certain
aspects of a
publisher
such as
reputation or
marketing
budget
influence the
results, these
could not be
tested in this
experiment