0 views

Uploaded by Aslam Aly Saib

Extension

- god bless you
- Case 12
- Tutorial 3
- Hospital Supply, Inc Case 16-1
- Ho Truong Phuong Thao
- 06. Learning Curve
- 10.1.1.720.6093
- Misallocation in Cina e in India
- labor essay
- 7-24
- answerset 2std
- Product Costing Explained ANA
- Application and Results of a Skilled Labor Demand Forecast Model for the US Construction Industry
- Encyclopedia of Management _ Service Process Matrix
- Case 4-6 Grand Jean
- weekly lesson plans-long 5
- Wacquant, Loic - Urban Marginality in the Coming Millennium.pdf
- 10.0000@www.researchgate.net@46524173.pdf
- Reinstatement Executory Even if Decision is Under Appeal
- Academic Word List

You are on page 1of 65

Efficiency: Final Report

Dr Andrew Street

Research Report

No 788

Efficiency: Final Report

Dr Andrew Street

The views expressed in this report are the authors’ and do not necessarily reflect those of the Department

for Education and Skills.

ISBN 1 84478 788 5

Contents

Introduction ............................................................................................................................. 3

1 The concept of efficiency ................................................................................................ 4

Organisational efficiency ...........................................................................................4

Weight restrictions and cross-efficiency models .......................................................9

Decomposing efficiency ..........................................................................................13

Returns to scale ........................................................................................................15

The cost function......................................................................................................15

Analytical techniques: SFA and DEA .....................................................................16

Unit of analysis ........................................................................................................19

Specifying inputs and outputs..................................................................................20

Labour inputs .......................................................................................................21

Capital inputs.......................................................................................................22

Environmental constraints .......................................................................................23

Practical challenges..................................................................................................25

Conclusions..............................................................................................................26

4 Regression analysis ....................................................................................................... 33

Modelling strategy ...................................................................................................33

Model specification..................................................................................................33

Estimation results.....................................................................................................36

Model specification..................................................................................................41

Estimation results.....................................................................................................42

The dataset ...............................................................................................................44

Stochastic frontier analysis and model specification ...............................................45

Endogeneity .............................................................................................................45

Multilevel modelling................................................................................................48

Concluding comments .............................................................................................49

Acknowledgement ................................................................................................................. 49

References .............................................................................................................................. 50

Appendix 1: Regression results ............................................................................................ 52

Full sample...............................................................................................................52

Schools with 6th forms .............................................................................................55

Schools without 6th forms ........................................................................................58

1

2

Introduction

The Department for Education and Skills (DfES) commissioned us to deliver a report

detailing how current techniques used to model multi-input, multi-output production

process, such as Data Envelopment Analysis (DEA) and Stochastic Frontier Analysis

(SFA), can be applied to measure allocative and technical efficiency in the education

sector. We were asked to:

• provide a critical appraisal of the different methods that can be used to

calculate technical and allocative efficiency, by reviewing the current

literature and assessing its value to the school context.

• apply the techniques of DEA and SFA to analyse the relative performance of

secondary schools in England.

• make recommendations about the priorities for future research by the DfES.

This report examines the efficiency of English secondary schools in academic year

2003-04. It uses data from 2,928 schools to develop and test models of efficiency

using standard parametric and non-parametric statistical methods.

Section 1 examines the issues involved in measuring the efficiency of public service

organisations such as schools. Section 2 then discusses the elements of the efficiency

model, such as the units of analysis, the choice of inputs and outputs, and the impact

of external constraints on levels of attainment. For those familiar with the economic

ideas of efficiency and productivity, sections 1 and 2 can be safely skipped.

Section 3 describes the data used in this study. Section 4 describes the development of

parametric models of school efficiency, using the methods of ordinary least squares

regression analysis and stochastic frontier analysis. Section 5 describes the

development of analogous non-parametric models, based on data envelopment

analysis. Section 6 comments on the findings and discusses the priorities for future

work.

3

1 The concept of efficiency

The pursuit of efficiency has become a central objective of policy makers within most

public services. The reasons are manifest. In developed countries, expenditure on such

services amounts to a sizeable proportion of gross domestic product. Policy makers

need to be assured that such expenditure is in line with citizens’ preferences,

particularly when many sources of finance, such as tax revenues, are under acute

pressure.

when – as in most public services – there exist conceptual challenges, multiple

objectives and great scope for measurement error. To address this complexity there

has developed a flourishing research discipline of organisational efficiency analysis.

Following pioneering studies by Farrell (Farrell, 1957), statisticians, econometricians

and management scientists have developed tools to a high level of analytic

sophistication that seek to measure the productive efficiency of organisations and

systems. This report examines their specific relevance to secondary schooling in

England.

confusion in both popular and professional discussion about what is meant by

efficiency. In this section we first discuss the reasons for wishing to measure

efficiency, and then define the concepts of organisational efficiency deployed in this

report. Throughout the report we take the view that educational objectives are known

and agreed. In practice, we acknowledge that objectives and priorities may be highly

contested. However, we set aside such debates for the purposes of this analysis.

Organisational efficiency

referred to as a decision-making unit (DMU). In this study the DMU is assumed

usually to be the school, although there may be circumstances when the Local

Authority (LA) is the organisation under scrutiny. The DMUs consume various costly

inputs (labour, capital, etc.) and produce valued outputs. Efficiency analysis is

centrally concerned with measuring the competence with which inputs are converted

into valued outputs. In general, it treats the organisation as a black box, and does not

seek to explain why it exhibits a particular level of efficiency (Fried et al., 1993).

The terms ‘productivity’ and ‘efficiency’ are often used interchangeably, which is

unfortunate since they are not precisely the same thing. Productivity is the ratio of

some (or all) valued outputs that an organisation produces to some (or all) inputs used

in the production process. Efficiency incorporates the concept of the production

possibility frontier, which indicates feasible output levels given the scale of operation.

Thus the concept of productivity may embrace but is not confined to the notion of

efficiency.

A starting point for examining the basic notion of efficiency is shown in Figure 1.1,

which illustrates the case of just one input and one output. The line OC indicates the

simplest of all technologies: no fixed costs and constant returns to scale. A technically

efficient organisation would then produce somewhere on this line, which can be

4

thought of as the production possibility frontier. Any element of inefficiency would

result in an observation lying strictly below the line OC. For an inefficient

organisation located at P0, the ratio X0P0/ X0P0* offers an indication of how far short

of the production frontier it is falling, and therefore a measure of its efficiency level.

C

Output

*

P0

P0

X0 Input

0

Many other technologies are possible. For example, the curve OV indicates a frontier

with variable returns to scale. Up to the point P0*, the ratio of output to input

decreases (increasing returns to scale), thereafter it increases (decreasing returns to

scale).

The notion of a production frontier can be extended to multiple outputs and a single

input (say, costs). Figure 1.2 illustrates the case with two outputs. For the given

technology, the isocost curve CC gives the feasible combination of outputs that can be

secured for a given input. At a higher level of costs the isocost curve moves out to

C'C'. These curves indicate the shape of the production possibility frontiers at given

levels of input. An inefficient DMU lies inside this frontier. We define the marginal

rate of transformation to be the sacrifice of output 2 required to produce a unit of

output 1, indicated at any particular point on CC by the slope of the curve − ( P2 P1 ) .

It is usually assumed that – as in this Figure – for a given level of input the slope

becomes steeper as the volume of output 1 produced increases.

5

Figure 1.2: The case of two outputs

Output 2

C'

C

P2

P1 C C' Output 1

Likewise in input space we examine the case of two inputs and one output, as in

Figure 1.3. The isoquant QQ indicates the feasible mix of inputs that can secure a

given level of output, with inefficient DMUs lying beyond this curve.

Input 2

Q

Input 1

Extending the analysis to the general case of multiple inputs and multiple outputs, we

define the overall efficiency eff0 of organisation zero to be the ratio of a weighted sum

of outputs to a weighted sum of inputs. Mathematically, if organisation 0 consumes a

6

vector of m inputs X0 and produces a vector of s outputs Y0, its overall efficiency is

measured by applying weight vectors U and V to yield:

S

∑U s Y s0

eff 0 = s=1

M

(1.1)

∑V

m=1

m X m0

where:

Ys0 is the amount of the sth output produced by organisation 0;

Us is the weight given to sth output;

Xm0 is the amount of the mth input consumed by organisation 0;

Vm is the weight given to the mth input.

The weights U and V indicate the relative importance of an additional unit of output

or input. On the input side, the weights V might reflect the relative market prices of

different inputs. It is often the case that – with the notable exception of capital inputs

– these can be measured with some accuracy. Then, if the actual input costs incurred

by organisation 0 are C0, the ratio:

M

∑V m X mi 0

Ceff 0 = m =1

(1.2)

C0

indicates the extent to which the organisation is purchasing its chosen mix of inputs

efficiently (that is, the extent to which it is purchasing its chosen inputs at lowest

possible prices).

However, the organisation may not be using the correct mix of inputs. This can be

illustrated using a simple two input model. For some known production process, the

isoquant QQ in Figure 1.4 shows the use of minimum inputs required to produce a

unit of a single output. The points P1 and P2 lie on the isoquant and therefore – given

the chosen mix of inputs – cannot produce more outputs.

When the unit costs of inputs are known, it is possible to examine the input price (or

allocative) efficiency of the two units. Suppose the market prices are V1* and V2*.

Then the cost minimizing point on the isoquant occurs where the slope is -V1*/V2*

(shown by the straight line BB). In Figure 1.4 this is the point P1, which is input price

efficient. However, although technically efficient, the point P2 is not efficient with

respect to prices, as a reduction in costs of P2P2* is possible. The price efficiency of P2

is therefore given by the ratio OP2*/OP2.

7

Figure 1.4: Allocative efficiency with two inputs

Q

Input 2 B

P1

P2

*

P2

Q

B

Input 1

0

organisations in output space. Figure 1.5 illustrates the case where a single input is

used to produce two outputs. If the relative values U1 and U2 of the outputs are known,

and the production possibilities are given by the curve CC, then organisation P1 is

producing at its allocatively efficient point while organisation P2 exhibits some

allocative inefficiency.

Output 2

P1

P2

C Output 1

0

mix of inputs or producing the wrong mix of outputs, we have so far explored only

those organisations that lie on the frontier of technical production possibilities.

However, it is likely that, particularly in a non-market environment, many

organisations are not operating on the frontier. That is, they also exhibit an element of

technical inefficiency (also referred to as managerial inefficiency or X-inefficiency).

8

This is illustrated in Figure 1.6 by the points P3 and P4. Organisation P3 purchases the

correct mix of inputs, but lies inside the isoquant QQ. It therefore exhibits a degree of

technical inefficiency, as indicated by the ratio OP1/OP3. Organisation P4 both

purchases an incorrect mix of inputs and lies inside the isoquant QQ. Its technical

inefficiency is indicated by the ratio OP2/OP4. Thus its overall level of inefficiency

OP2*/OP4 can be thought of as the product of two components: technical inefficiency

OP2/OP4 and allocative inefficiency OP2*/OP2.

Q

Input 2 B

P3

P1

P4

P2

*

P2

Q

B

Input 1

0

There are important questions relating to the objectives encompassed by any index of

efficiency. Is it legitimate for the central policy maker to attach a uniform set of

objectives to all organisations? If so, is it further legitimate to apply a uniform set of

weights to these objectives? If so, how should they be chosen? If not, what is the

extent of legitimate variation, and who should choose? These are fundamental issues,

the answers to which determine whether or not creating a single measure of

organisational performance is warranted. In our view, organisations can be ranked in

this way only if the policy maker may legitimately (a) set objectives and (b) attach

weights to those objectives.

When considering organizations working in the public sector, the set of output

weights ought to reflect societal values. However it is not a simple matter to derive

such weights, particularly when organisations face multiple objectives and there is

disagreement as to organisational priorities. Ultimately the selection of objectives in

the public services is a job for the politicians charged with reconciling conflicting

claims on public resources. The main role for analysts is to clarify the choices

required of policy makers, to provide evidence on popular preferences, and to develop

measurement instruments that most faithfully reflect the chosen objectives. Note that

9

policy makers are effectively attaching a zero weight to any output that is excluded

from the efficiency index.

In principle, the set of weights to be used in an efficiency index could be derived from

a range of sources, such as economic studies of willingness to pay or conjoint analysis.

However, rather than being externally agreed upon, in most efficiency analysis studies

the weights are generated as a by-product of the statistical estimation process. Indeed,

some see this as an attractive feature of the methods (Cooper et al., 2000).

multiple outputs. Approaches include the creation of a single index of outputs,

estimation of a cost rather than production function, or the use of distance functions

(Coelli and Perelman, 2000, Lothgren, 2000, Shephard, 1970). Irrespective of the

approach, the estimated magnitude of the weight for each output usually corresponds

to the value implicit in the sample mean cost of producing an additional unit of output.

Using a linear model, the weight attached to an output indicates the value of an

additional unit of that output, which remains constant for all levels of attainment of

the output. If a logarithmic model is used, the weight indicates the percentage increase

in attainment implied by a one percent increase in the output. Hence the econometric

approach is conservative in the sense that it implies that the existing output choices of

organisations (on average) reflect the values placed by society on the outputs. If this is

not true, the estimated weights will not be appropriate.

In conventional DEA both input and output weights are allowed to vary freely, so that

each organisation is evaluated in the best possible light. Indeed one quite frequently

finds in unconstrained DEA that the highest efficiency score for an organisation can

be secured simply by assigning a zero weight to one or more outputs on which it

performs poorly. In order to understand the role played by weights in DEA it is

useful to examine the simple two input model (Figure 1.7). For some known

production process, the isoquant QQ shows the use of minimum inputs required to

produce a unit of output, assuming constant returns to scale. The points P* and PT lie

on the isoquant and are therefore efficient in Farrell's technical sense.

10

Figure 1.7: Fixed and variable weights

Input 2 Q

B

P*

P

PT

PA Q

B

O

Input 1

However, when the unit costs of inputs are known, it is possible in addition to

examine the input price (or allocative) efficiency of the two units. Suppose the costs

are C1* and C2*. Then the cost minimizing point on the isoquant occurs where the

slope is -C1*/C2* (shown by the straight line BB). In Figure 1.7 this is the point P*, so

that point is also allocatively efficient. However, the point PT is not efficient with

respect to prices, as a reduction in costs is feasible. The price efficiency of PT is

therefore given by the ratio OPA/OPT. The DMU represented by point P also exhibits

technical efficiency, indicated by the ratio OPT/OP.

weights, and so allows a DMU's efficiency to be assessed using the weights most

favourable to its own circumstances. It therefore yields a measure of technical

efficiency. In this example, therefore, DMU P will be evaluated using weights that

place the allocatively efficient point of the frontier at PT rather than PA. These optimal

weights C1P and C2P for DMU P are represented by the slope of the tangent at the

point PT, indicated by the dotted line with slope -C1P/C2P in the diagram. Clearly very

‘extreme’ DMUs would favour extreme weights, attaching a zero weight to one or

other of the inputs. (Note that the weights can be multiplied by an arbitrary scalar, so

it is only the ratio of the weights – the slope of the dotted line – in which we are

interested.)

This ‘weight flexibility’ gives rise to a conundrum. If the inputs or outputs given zero

weight in the analysis for a particular DMU are important, then the measure of

efficiency must surely be deficient in placing no consequence on them. If on the other

hand they are not important, then why were they included in the analysis in the first

place? It is concerns of this sort that have led to the development of various schemes

for placing limits on variations in weights.

11

By placing constraints on weights, the region of search for those weights is reduced,

and so a DMU's efficiency assessment cannot increase, and may decrease. At the

extreme, if no flexibility in weights is allowed, DEA becomes classical ratio analysis,

in which a unit's efficiency is measured as its ratio of weighted outputs to weighted

inputs, with weights equivalent to the known input and output prices. In Figure 1.7

this measure is equivalent to ‘overall’ efficiency, measured relative to other DMUs.

Thus, as increasingly severe constraints are placed on weights, so the measure of

efficiency derived moves from one of relative technical efficiency to one of relative

overall efficiency.

Here the dotted lines represented the maximum and minimum ratio of weights the

analysts feels are acceptable. For DMUs with optimal weights between these two

limits, there is no change to the DEA assessment – the units continue to be assessed

with respect to the isoquant QQ between points M1 and M2. However, for DMUs with

optimal weights outside this range, efficiency is assessed with respect to the dotted

tangent lines representing the weight limits. For example, the efficiency of point R is

assessed relative to point RR on the tangent rather than point RQ on the frontier.

Imposing weight restrictions has the effect of limiting the ability of DMUs to ‘choose’

extreme weights, and therefore reduces the assessed efficiency of DMUs with extreme

optimal weights, whilst leaving unchanged the efficiency of DMUs with less extreme

choices.

Input 2 Q

RQ

RR

M1

P

PT

M2

Q

O

Input 1

There has been some attention to rules for restricting the flexibility of weight

variations (Allen et al., 1997), but the efforts to date have been poorly informed by

economic theory, and mainly confined to technical considerations (Pedraja-Chaparro

et al., 1999). The lack of a single set of weights implies that it is never appropriate to

12

rank DEA efficiency scores in a conventional ‘league table’ format. The fundamental

difficulty with applying weight restrictions is the same as encountered whenever

seeking to choose any set of weights – who should choose the restrictions, and how

should they choose?

A number of approaches have been suggested as mechanisms for addressing the lack

of knowledge about the magnitude of weights, and the arbitrary efficiency scores that

DEA can yield for ‘extreme’ DMUs. Amongst the most informative is the calculation

of ‘cross-efficiencies’ (Doyle and Green, 1994). For each DMU 0 a set of efficiencies

{e0j} is calculated, where j ranges from 1 to N, the number of DMUs in the sample.

Efficiency e0j indicates the efficiency score for DMU 0 if the optimal weights selected

by DMU j are applied. Thus, e00 is the conventional DEA score for DMU 0, and

because this seeks to cast DMU 0 in the best possible light, none of the remaining e0j

scores can be higher than e00. Cross-efficiencies offer valuable insights into the

‘robustness’ of a DMUs unconstrained efficiency score.

However, it is worth noting that – once DEA moves beyond simple models with just a

handful of inputs and outputs – there are often multiple solutions for optimal weights

that offer identical efficiency measures. That is, there is often not a unique set of

weights that yields the DMU’s maximum efficiency level. This offers challenges for

the systematic application of the cross-efficiency principle.

Decomposing efficiency

Figure 1.9 illustrates the various notions of efficiency that can be examined in the

case of just two inputs and a single output, with constant returns to scale. The figure

shows the inputs required to produce a given level of output. The isoquant QQ shows

the production frontier, in the sense that for any level of input 1 it shows the minimum

level of input 2 required to produce the output. All DMUs must therefore lie on or to

the north east of the isoquant.

13

Figure 1.9: Production possibilities with two inputs, one output

Input 2 Q

B

P*

x2 P

PT

PA Q

B

O

x1 Input 1

Consider a DMU P that consumes physical inputs x1 and x2 to produce the output. It is

technically inefficient because it does not lie on the isoquant. The traditional radial

measure of inefficiency is OPT/OP, the proportion by which it is feasible to reduce all

inputs and still produce the required output.

because – given the relative prices of the two inputs w1 and w2 – the output can be

produced at lower total cost. The line BB has slope -w1/w2 and passes through the

allocatively efficient point P*. It indicates the minimum cost point of production.

The overall inefficiency of DMU P is therefore indicated by the ratio OPA/OP which

can be written as the product of OPT/OP and OPA/OPT. The first ratio is the technical

inefficiency, the second is allocative inefficiency.

This analysis assumes that, although the DMU makes an incorrect choice of inputs,

and uses them inefficiently, it nevertheless purchases them at the correct market

prices w1 and w2, with total cost W* = x1w1 + x2w2. It is however possible that it

purchases inefficiently, at a higher total cost WP<W*. This suggests a third component

of inefficiency, indicated by the ratio W*/WP. Total inefficiency can now be

decomposed into:

• Technical inefficiency OPT/OP;

• Allocative inefficiency OPA/OPT;

• Cost inefficiency W*/WP;

This analysis suggests a need to calculate inefficiency using three different types of

input measure:

• Disaggregated physical inputs

• Physical inputs aggregated at market prices

• Actual costs.

14

If DEA is used, this implies a need to run the model first with disaggregated physical

inputs and then with the inputs aggregated at market prices in order to estimate the

ratios OPT/OP and OPA/OP. The cost inefficiency can be calculated trivially as the

ratio W*/WP without resort to DEA.

Returns to scale

We have so far assumed constant returns to scale. That is, the production process is

such that the optimal mix of inputs and outputs is independent of scale of operation.

In practice there exist important economies and diseconomies of scale in most

production processes, so an important influence on eff0 (from equation 1.1) may be

the chosen scale of operation. This is illustrated in Figure 1.10 for the case of one

input and one output. The production frontier is illustrated by the curve OV, which

suggests regions of increasing and decreasing returns to scale. The optimal scale of

production is at the point P* where the ratio of output to input is maximised. Although

lying on the frontier, the points P1 and P2 secure lower ratios because they are

operating below and above (respectively) the scale efficient point of production. They

therefore exhibit levels of scale inefficiency given by:

OY1 OY2

OX 1 OX 2

Seff1 = *

and Seff 2 = *

(1.3)

OY *

OY *

OX OX

Output

P2 V

Y2

Y* *

P

Y1 P1

X1 X* X2 Input

0

is the production function or its counterpart the cost function. The production function

15

models the maximum (single) output an organisation could secure, given its mix of

inputs.

The cost function simplifies the input side of the production process by deploying a

single measure of the inputs used, rather than a vector. It indicates the minimum cost

that an organisation can incur in seeking to produce a set of valued outputs. Using the

notation introduced above, a cost function can be written in general terms as C0* =

f(Y0).

The cost function combines all inputs into a single metric (costs), and does not model

the mix of inputs employed, or their prices. In practice, as noted above, the costs

incurred by an organisation might be higher than those implied by the cost function

for three reasons. First, it may purchase inputs at higher than market prices (cost

inefficiency). Second, given prevailing prices, it may employ an inefficient mix of

inputs (allocative inefficiency). And third, it may not secure the maximum output

attainable given its inputs (technical inefficiency). However, if no measures of

physical inputs are available, and only aggregate measures of costs are available, it is

impossible to distinguish between these causes of deviation from the cost function.

Therefore, notwithstanding its practical usefulness, a cost function offers little help

with detailed understanding of the input side of efficiency.

those predicted by the cost function (or the extent to which its output falls short of that

predicted by the production function).

to be derived indirectly after taking account of observable phenomena. This very

broadly involves the following process:

1. Observable phenomena are measured such as outputs and inputs (costs).

2. Some form of relationship between these phenomena is specified. If a

parametric method is used and differences in cost are the focus of the exercise, a

production or cost function is estimated. If a non-parametric method is used, an

efficiency frontier is derived.

3. Efficient behaviour is then predicted on the basis of the definition of technical

efficiency.

4. The difference between each DMU’s observed data and the maximum

achievable is then calculated, as predicted by the frontier.

5. The difference (or some portion of it) is defined as inefficiency.

A number of analytic techniques have been developed to estimate the form of cost and

production frontiers and the associated inefficiency of individual organisations (Coelli

et al., 1998). These are covered in the subsequent sections of this report, and can be

divided into two broad categories: parametric methods, which use econometric

methods to estimate the parameters of a specific functional form of cost or production

function, and non-parametric methods, which place no conditions on the functional

form, and use observed data to infer the shape of the frontier.

16

The pre-eminent form of parametric method now in use is stochastic frontier analysis

(SFA). This is similar to conventional regression analysis, but decomposes the

unexplained error in the estimated function into two components: inefficiency (which,

in the case of a cost function will always be positive), and the more conventional two-

sided random error. Cost functions are used extensively in parametric analysis of

efficiency because the alternative strategy - estimating models with both multiple

inputs and multiple outputs - is methodologically challenging and demanding in terms

of data requirements. In contrast, single input cost functions and single output

production functions can be readily estimated using standard econometric methods, or

straightforward variants.

Most non-parametric methods take the form of data envelopment analysis (DEA) and

its many variants. These were stimulated by the pioneering work of Farrell (Farrell,

1957), later generalised by Charnes, Cooper and Rhodes (Charnes et al., 1978). DEA

uses linear programming methods to infer a piecewise linear production possibility

frontier, in effect seeking out those efficient observations that dominate (or ‘envelop’)

the others. In contrast to parametric methods, DEA can handle multiple inputs and

outputs without difficulty.

estimating the frontier – to provide an estimate of how far each observation falls short

of the estimated frontier. This emphasis on the residual for each observation marks an

unusual departure from conventional statistical and econometric analyses, which are

in the main preoccupied with estimated coefficients (that is, relationships displayed by

the population of observations as a whole) rather than individual observations. This

novel focus gives rise to important methodological complications.

17

2 Issues in model construction

An organisation’s efficiency is considered to be the ratio of the value of outputs it

produces to the value of inputs it consumes. Figure 2.1 summarises the principles

underlying this viewpoint. The organisation consumes a series of M physical

resources, referred to as inputs, and valued in total as X by society. Some

transformation process takes place, the details of which do not immediately concern

us. This leads to the production of S outputs, which society values in aggregate as Y.

Our summary measure of ‘efficiency’ is the ratio of Y to X – what might be more

accurately referred to as cost-effectiveness. This is analogous to the DfES’s schema

which depicts the process as comprising three phases, termed economy, efficiency

and effectiveness.

Input 1 Output 1

Decision

Costs Input 2 Output 2 Benefits

Making

X : : Y

Unit

Input m Output s

Procurement of inputs at Use of inputs to produce Choice of output mix to

minimum costs maximum output produce maximum impact

on outcomes

outputs. Central to the calculation of Y is therefore the relative weight Us attached to

each output s. These weights reflect the relative importance attached to an additional

unit of production of each output, and allow us to calculate for organisation 0 the

S

valuation of outputs Y = ∑U s Y s 0 as discussed in section one. In the same way, when

s=1

there are multiple inputs, the relative weight Vm attached to input m allow us to

M

calculate the valuation of inputs X =∑V m X m 0 . If we have secure information on the

m=1

magnitudes of U and V we can readily compute the efficiency as the ratio Y/X. In

particular, in competitive markets, both U and V might be readily observed as prices.

In such circumstances there may be no need to use the analytic techniques of SFA and

DEA. Instead, comparative efficiency can be readily computed using the exogenously

observed weights. However, a special feature of public services such as schools is that

such prices are rarely observed, particularly on the output side. It is in such

circumstances that the analytic techniques can be deployed in order to furnish

evidence on the weights U and V.

18

Although beguilingly simple, numerous complex issues are raised when seeking to

use this framework to develop operational models of organisational efficiency. The

complexity involved in developing an operational framework reflects the complexity

of the production process. The production of the majority of public service output

rarely conforms to a production-line type technology, where a set of clearly

identifiable inputs are used to produce a standard type of output. This means that the

production process is much less clearly defined and there is the potential for

considerable heterogeneity in how and what outputs are produced. Contributions to

public service outcomes are often made by multiple agents or organisations, a service

package may be delivered over multiple time periods and in different settings, and the

responsibilities for delivery may vary from place to place and over time.

• What is the appropriate unit of analysis?

• What are the system outputs?

• What value should be attached to these outputs?

• What inputs are used in the production of these outputs and how should these be

valued?

• What environmental constraints are faced?

constrained by the scope and nature of data availability.

Unit of analysis

It is important that the boundaries of any efficiency analysis should be clearly defined.

A fundamental question to ask is: what is the unit of organisation in which we are

interested? Any efficiency analysis should have a clear idea of the entity it is

examining, but should also recognise that its achievements are likely to be influenced

by the actions of other organisations or factors beyond its immediate control. This is

especially likely when multiple agencies or organisations are involved in joint

production. Three criteria should guide the choice of analytical unit:

• First, the unit of analysis should capture the entire production process of interest.

This may entail defining artificial units of analysis if there is variation among

organisations in how the production process is divided between them.

• Second, they should be ‘decision-making units’ (DMUs). In a strict sense this

requires that their function is to convert inputs into outputs, and that the DMUs

have discretion about the technological process by which this conversion takes

place. But a weaker definition of DMUs requires only that they play an

organising function, establishing the rules and conditions to which producers

have to adhere. This definition would allow government bodies to be considered

as DMUs.

• Third, the units comprising the analytical sample should be comparable,

particularly in the sense that they are seeking to produce the same set of outputs.

In the current context the choice of analytical unit is fairly straightforward, with the

focus on school performance. Nevertheless, there may be elements of the process of

the production of education that are provided outside the school (eg by parents) or that

19

are organised differently from one school to the next (for instance, some schools may

be in a better position to take advantage of educational opportunities provided by

other agencies in their area).

Central to the development of any efficiency model is the specification of the inputs

consumed by the DMU and the valued outputs it produces. Indeed, although

efficiency analysts usually refer to the ‘outputs’ of the production process, regulators

and other decision-makers are more usually interested in the ‘outcomes’ of education

services, in terms of the impact they have on individual and social welfare.

Throughout this report we adhere to convention by referring to the ‘outputs’ of the

schools. However, it should be understood that – wherever possible – the analyst

should seek to measure outputs in terms of their ultimate outcomes on pupils and

society.

product. Of course, even in a reasonably homogeneous market, such products (say, a

refrigerator) can vary enormously in various dimensions of ‘quality’, such as

reliability, looks and temperature range. The quality of the product is intrinsic to its

social value, but that value can be readily inferred by observing the price people are

prepared to pay. Usually, therefore, there is no need explicitly to consider the ultimate

‘outcome’ of the product, in terms of the value it bestows on the consumer.

In many parts of the economy, however, not only do prices not exist, but outputs are

difficult to define. In particular this is true for many of the goods and services to

which government spending is devoted (Atkinson, 2005). In the context of efficiency

analysis, two fundamental issues need to be considered. How should the outputs of

the public services sector be defined? And what value should be attached to these

outputs?

Public service organisations rarely collect routine information about the outcomes

they produce. More commonly the analyst is forced to rely on comparing

organisations in terms of the quantity and type of activities they undertake. For

example, in the educational sector, numbers of pupils taught are readily measured.

However, in schooling we also have available an important intermediate measure of

outcome, expressed in the form of examination results. Although these do not

measure the eventual outcomes achieved, in terms of intellectual development,

citizenry and ‘life chances’, they are acknowledged to offer a useful indicator of

success in some important dimensions of school attainment.

The input side of efficiency analysis is usually considered less problematic than the

output side. Physical inputs can often be measured more accurately than outputs, or

can be summarised in the form of a measure of costs. However, even the specification

of inputs can give rise to serious conceptual and practical difficulties in efficiency

analysis. This section briefly summarises some of the major issues.

be specified. At one extreme, a single measure of aggregate inputs (in the form of

20

total costs) might be used. The efficiency model then effectively becomes a cost

function. This approach assumes that the organisations under scrutiny are free to

deploy inputs efficiently, taking account of relative prices. Any failure to do so (price

inefficiency) will be reflected in a lower estimate of measured efficiency. Use of a

single measure of costs therefore adopts a long-run perspective as it assumes – for

example – that organisations can freely adopt an optimal mix of capital and labour.

of the input mix are considered beyond the control of the organisation. In these

circumstances, it becomes necessary to disaggregate the inputs in order to capture the

different input mixes that organisations have inherited. In particular, disaggregation of

labour and capital may be required. We consider these two classes of input in turn.

Labour inputs

Labour inputs can usually be measured with some degree of accuracy, often

disaggregated by skill level. An important issue is then how much aggregation of

labour inputs to use before pursuing an efficiency analysis. Unless there is a specific

interest in the deployment of different labour types, it may be appropriate to aggregate

into a single measure of labour input, weighting the various labour inputs by their

relative wages. A central contribution of efficiency techniques is to offer evidence on

efficiency when there is no direct information on the relative value placed on inputs or

outputs. Where such evidence does exist, as for example on the market price of inputs,

there may be little merit in disaggregation unless there is a specific interest in input

allocative efficiency. Aggregation leads to a more parsimonious model, thereby

allowing the analyst to focus attention on aspects of the production process where

there is less secure evidence on weightings.

However, there may be an interest in the relationship between efficiency and the mix

of labour inputs employed. Under such circumstances, a model using measures of

labour input disaggregated by skill type may be valuable. This might yield useful

policy recommendations about (say) substituting some types of labour for another.

It may be that labour inputs are measured in either physical units (hours of labour) or

costs of labour. Which should be used in an efficiency analysis depends on context.

Use of physical inputs will fail to capture any variations in organisations’ wage rates.

This may be desirable (for example if there are variations in pay levels beyond the

control of organisations) or undesirable (if there is believed to be input price

inefficiency, in the form of different pay levels for identical workers).

Organisations such as schools may vary in the extent to which they ‘buy in’ certain

services, rather than directly employ labour. For example, some DMUs may purchase

cleaning services from independent contractors and have little idea of the associated

labour inputs. Cleaning services might therefore appear as a ‘goods and services

purchased’ input rather than labour and capital inputs. If other DMUs directly employ

cleaning personnel, it may be the case that such inputs are not being treated on a

strictly comparable basis. In such circumstances, the analyst may need to resort to a

single measure of inputs, in the form of total costs.

21

Capital inputs

This is partly because of the difficulty of measuring capital stock and partly due to

problems in attributing its use to any particular period.

Measures of capital are often very rudimentary, and even misleading. For example,

accounting measures of the depreciation of physical stock usually offer little

meaningful indication of capital consumed. Indeed, in practice, analysts may have to

resort to very crude measures. Furthermore, public services often invest in important

non-physical capital inputs, such as staff development.

In principle, an efficiency model should use the capital consumed in the current

period as an input to the production process. But capital is by its nature deployed

across time. On the one hand, contemporary output may rely on capital investment

that took place in previous periods, whilst on the other hand, some current activities

are investments that are intended to contribute to future rather than contemporary

outputs.

Year t-1 Year t Year t+1

Year t-1 Year t Year t+1

Year t-1 Year t Year t+1

The accounting difficulties capital inputs give rise to can be summarised in the form

of a diagram, as shown in Figure 2.2. In each period, inputs are consumed and direct

outputs emerge from the organisation. A rudimentary efficiency analysis will examine

the ratio of outputs to inputs for only a single time period t. Yet crucially the

organisation in year t has enjoyed the benefits of past investments. And it also leaves

an ‘endowment’ for future periods in the form of investments undertaken in this and

preceding periods. The endowment might be in the form of real capital (buildings or

personnel) or investment in social ‘capital’ of some sort. In principle the endowment

may be an important aspect of both the inputs to and outputs of the system. In practice

it can be very difficult to measure.

22

A central issue in the treatment of capital is the extent to which short-run or long-run

efficiency is under scrutiny. In the short run, it makes sense for organisations to take

advantage of whatever investment infrastructure they have available, and optimise

subject to any constraints imposed by that infrastructure. So, for example, short run

efficiency should be judged in the light of the capital configuration that a DMU has

available. Yet in the longer run one might expect the DMU to reconfigure its capital

resources when this can bring about efficiency improvements.

Environmental constraints

classify as the ‘environmental’ determinants of performance. These are exogenous

influences on the organisation’s production function, beyond its control, that reflect

the external environment within which it must operate. In particular, many of the

outcomes secured by public service organisations are highly dependent on the

characteristics of the population group they serve and the public bodies to which they

are responsible. For example, in education:

• levels of achievement may be related to the inherent ability of pupils. Ability at

intake may vary from school to school because of local socio-economic

conditions or because of the operation of selection processes;

• attainment is likely to be partly due to the efforts of the pupil’s parents or

guardians, and these efforts may vary systematically across the country;

• schools have differing freedoms and constraints, as they are answerable to both

central government and their Local Authority.

‘controllable’ by the DMUs under scrutiny. For example, to what extent should

engagement with parents be considered a controllable influence on school

performance? The analyst’s choice is likely to be heavily dependent on whether the

purpose of the analysis is short run and tactical, or longer run and strategic. In many

circumstances it will be appropriate to undertake both short run and long run analysis.

such as other public services, the voluntary sector or private families. In principle we

should recognise this when modelling efficiency, and there may be a difficulty in

identifying the element of outcome that is attributable specifically to the school. The

danger is either (i) a school’s contribution towards some outcomes is ignored in the

analysis (under-attribution) or (ii) the contribution of other external agencies towards

outcome is ignored (over-attribution). Whether these external efforts should be treated

as exogenous may depend on the extent to which the behaviour of external agencies is

amenable to influence by the DMU under scrutiny.

Broadly speaking, there are three ways in which environmental factors can be taken

into account in efficiency analyses:

• restrict comparison only to organisations within a similarly constrained

environment;

• model the constraints explicitly, as being analogous to ‘factors’ in the production

process;

• undertake risk adjustment.

23

The first approach to accommodating environmental influences is to cluster

organisations into similar ‘families’, using techniques such as cluster analysis (Everitt

et al., 2001). The intention is then to compare only like with like. Of course this begs

the question as to what criteria are to be used to create the families. Statistical

examination of the link between putative exogenous influences and performance is

often ruled out because efficiency too is correlated with performance. Unless

exogenous influences are known to be entirely independent of efficiency levels, it

becomes impossible to determine whether a correlation with performance is reflecting

exogenous influences or efficiency variations. The analyst will therefore often have to

resort to informed judgement in choosing the basis for creating families.

A further problem with the use of families is that it will reduce sample size, as it rules

out extrapolation of performance of one type of organisation as a basis for comparison

with another type. This may be appropriate to ensure robust comparisons, but with

parametric methods can seriously affect the confidence with which efficiency

judgements can be made, for example by leading to smaller samples and larger

standard errors. Furthermore, of course, useful lessons that organisations might learn

from being compared to a more heterogeneous sample will be lost. In this report we

employ the clustering approach when analysing separately schools with and without

sixth forms.

production model, often treating them as an exogenous ‘input’ analogous to labour or

capital. This approach effectively generalises the clustering approach by allowing

extrapolation from one class of organisation to another. For example, an

environmental factor might be included as an explanatory variable in the parametric

modelling approach or as an input or output in the non-parametric approach. Whilst

leading to a more general specification of the efficiency model than the clustering

approach, the direct incorporation of environmental factors into the efficiency model

leads to new modelling challenges that are considered further later in this report. We

also employ this approach when we introduce variables such as the proportion of

pupils entitled to free school meals into the models.

The final method to control for variation in environmental circumstances is the family

of techniques known as ‘risk adjustment’. These methods adjust organisational

outputs for differences in circumstances before they are deployed in an efficiency

model, and are – where feasible – often the most sensible approach to treating

environmental factors. In particular, they permit the analyst to adjust each output for

only those factors that apply specifically to that output, rather than use environmental

factors as a general adjustment for all outputs. For example, as in this report, the

outcomes secured by pupils (examination results) are adjusted for some measure of

prior attainment.

The methods of risk adjustment are often a highly efficient means of controlling an

output measure for a multiplicity of environmental factors. The risk-adjusted output

can then be entered into the efficiency model without any further need to enter

environmental factors. The methods of risk adjustment have been developed to a high

level of refinement (Iezzoni, 2003). However, it must be noted that risk adjustment

usually has demanding data requirements, in the form of information on the

24

circumstances of individual pupils. And even when adequate data do exist it is often

difficult to secure scientific consensus on the most appropriate way of undertaking the

risk adjustment.

consider the factor further in any efficiency analysis. It therefore considerably

simplifies the efficiency modelling process. Yet it is important to bear in mind that

conventional risk adjustment usually requires the assumption that the sample average

outcome for a particular population group is a suitable benchmark against which

individual or organisational performance can be measured. If for some reason certain

population groups receive systematically poor services, it may be inappropriate to

include such groups in the risk adjustment process, particularly if the intention is to

highlight such underperformance and promote better performance amongst providers

of services for such groups.

One particularly important situation arises when organisations have been allocated

funds to deliver some standard level of care, for example when a school funding

formulae adjusts the ‘per pupil’ allocation for certain pupil characteristics. The

purpose of such formulae is to adjust for legitimate environmental circumstances, so

that in principle all organisations are operating on a ‘level playing field’. In these

special circumstances there may be no need explicitly to consider any exogenous

variables at all on the input side. Efficiency analysis should focus solely on variations

in organisational outputs, as all inputs have been equalised. This argument, of course,

presumes that the funding formulae have been correctly designed, which is rarely the

case. It is nevertheless important to note the link between funding mechanisms and

models of efficiency.

Practical challenges

section, it is often the case that the most serious difficulties for the analyst arise from

the scope and nature of the available data sources. Numerous difficulties can arise:

performance, most notably the service outocmes. This is especially problematic in

efficiency analysis, when the ambition is to present a rounded view of

performance.

• Where measurement instruments are available, they may be incomplete or highly

imperfect metrics with large random elements.

• Time series are often short, or interrupted by structural changes.

• As always, there may be missing data for some observations, leading to difficult

technical choices as to whether to broaden the scope of the model (and so

effectively reduce sample size) or use a more circumscribed model with a larger

sample.

• More generally, given the complexity of the production process in many of the

public services, sample sizes may often be too small to draw secure inferences on

the nature of the preferred model.

analysis. However, they can become particularly problematic in efficiency analysis

25

because of the great attention placed on unexplained variations from the estimated

frontier. Results may be highly sensitivity to model specification and data errors.

This suggests a need for great attention to sensitivity analysis, experimentation with

different model specifications, and caution about drawing definitive judgements on

the efficiency of individual organisations. It is important to bear in mind that a great

deal of conventional uncertainty analysis merely models uncertainty in the data, and

not the implications of wrongly specifying the underlying model (Smith, 1997).

Conventional measures of uncertainty may therefore seriously overestimate the

degree of confidence in efficiency scores, and should be viewed with caution.

In short, data constraints will often circumscribe the ability to answer the questions of

public service regulators in their entirety. Instead, the analyst must adopt pragmatic

solutions to otherwise infeasible modelling demands. Under these circumstances, the

imperative is to state what technical assumptions have been made, and to

communicate clearly the limitations of the analysis.

Conclusions

and methodological problems. Setting aside the obvious measurement difficulties, the

structural problem can be illustrated as in Figure 2.3. Naïve efficiency analysis

involves examining the ratio of outputs to inputs (the shaded boxes). Yet system

inputs should also incorporate previous investments (endowments) and exogenous

inputs (such as other organisations and pupil characteristics). And system outputs

should also include endowments for the future, joint outputs and outputs not directly

related to the service under scrutiny, such as enhanced economic productivity arising

from public service performance.

Research, training Year t Productivity

Etc. Etc.

Year t-1 Year t Year t+1

Other organizations Policy constraints

Population characteristics Inputs Physical constraints

Etc. Year t Etc.

26

It will never be feasible to accommodate all the issues summarised in Figure 2.3 into

an efficiency analysis. Rather, the analyst should be aware of which factors are likely

to be material for the application under consideration, and seek to offer guidance on

the implications of serious omissions from the efficiency model.

27

3 Data and descriptive analysis

The variables used to analyse the performance of English secondary schools are listed

in Table 3.1, with summary statistics provided in Table 2. The total sample available

for analysis comprises 2,928 schools, out of a possible 2,997. Sixty-nine schools are

omitted because of either missing data (n=55) or extreme – probably erroneous -

values (n=14) for one or more variables.

Variable | Description

teachpp | Number of teachers per 1000 pupils

x1m learnsuppp | Number of learning support staff per 1000 pupils

adminclpp | Number of administrative and clerical staff per 1000 pupils

eLearnITpp | Expenditure on Learning and ICT resources per 1000 pupils

e_exp | Total expected expenditure

x2

nfsm | % pupils not eligible for free school meals

zk nsen | % pupils without special education needs

nsenwout | % pupils without special education needs statements

eal | % pupils with English as additional language

eal2 | eal squared

eal3 | eal cubed

cat_sixth | If 1, school has a sixth form

f

cat_religi~s | If 1, religious domination school

dd cat_rural | If 1, school is rural

cat_eicclu~s | If 1, school in excellence in city programme

cat_specia~s | If 1, specialist school

cat_bursar | If 1, school has a bursar

avftepupils | Average number of full time equivalent pupils

s

The key performance measure (y) is the GCSE uncapped value added score

(gcsevauncap), measured for each school in academic year 2003-04. This captures

the "value added" attainment of pupils as it compares their key stage 4, GCSE results,

against the average level that they would be expected to attain given their key stage 3

attainment scores (sat 2 years earlier). The measure is termed “uncapped”, because it

captures the total sum of all GCSE’s that a pupil sits as opposed to the “capped”

measure which only includes a pupil’s top 8 GCSE results. This measure is

standardised around 100, as it is measured as deviations from the average expected

attainment plus 100 to avoid negative numbers. A unit increase in the measure is

equal to one GCSE grade on this scale. As table 3.2 shows, the measure is subject to

wide variation, ranging from 88.5 to 128.2.

Variable | Obs Mean Std. Dev. Min Max

teachpp | 2928 62.22643 5.568889 46.59661 94.8949

x1m learnsuppp | 2928 15.98078 6.73287 .5590622 57.53579

adminclpp | 2928 7.850698 2.568798 1.227646 30.54874

eLearnITpp | 2928 237.4525 93.42784 38.37225 1040.047

e_exp | 2943 2476.203 340.7276 1129.734 11487.96

x2

nfsm | 2928 84.75535 13.28694 18.93 99.89

zk nsen | 2928 83.6953 9.297719 38.77 100

nsenwout | 2928 97.51585 1.644732 87.76 100

eal | 2928 9.023672 16.82746 0 99.65301

cat_sixth | 2928 .5717213 .4949138 0 1

f

28

cat_religi~s | 2928 .1653005 .371515 0 1

dd cat_rural | 2928 .1410519 .3481345 0 1

cat_eicclu~s | 2928 .0502049 .2184048 0 1

cat_specia~s | 2928 .3934426 .488597 0 1

cat_bursar | 2928 .8295765 .3760687 0 1

avftepupils | 2928 1033.487 335.7743 174 2622.5

s

A number of factors might have influence on the attainment of pupils in each school

and explain this variation in GCSE results, one of which is the relative efficiency of

secondary schools. The purpose of the analysis is to identify these influences. We

separate observable factors into two broad categories: inputs of production and

exogenous constraints upon production.

Inputs (x’s) include staffing and resources such as teaching materials and equipment.

Note that no measures of capital inputs are included. This means that the analytical

focus is on the efficiency of how schools utilise their recurrent funding streams, not

their capital budgets.

technical efficiency, or by taking account of their relative value, in order to estimate

allocative efficiency.

units (x1), measuring the number of teachers (teachpp), the number of learning

support staff (learnsuppp), the number of administrative and clerical staff

(adminclpp), and expenditure on learning resources (including ICT learning

resources) (ave19e20pp, renamed eLearnITpp). All of these variables are measured

per 1,000 pupils in each school. The source of the data on staff input is the PLASC

return, and information is averaged over academic years 2002-2003 and 2003-2004.

Expenditure on learning and IT learning resources is derived from the Consistent

Financial Reporting (CFR) returns. On average, a school employs 62 teachers, 16

learning support staff and 8 administrative and clerical staff per 1,000 pupils.

( x2 ). Total expected expenditure is calculated by weighting staff numbers by their

respective wage rate, and summing across input categories (including eLearnITpp).

National wage rates are listed in table 3.3. By taking account of the relative price of

inputs insight can be gained into the extent to which schools might be mis-allocating

resources across input types.

Average salary Notes

Secondary teachers £30,000 FTE average salary of regular qualified

classroom teachers in maintained secondary

schools, 13 March 2004. Source: DTR

Admin & Clerical £13,932 FTE average annual salary for school

secretaries. Source: ASHE 2004, code 4213

Education Support £11,602 FTE average annual salary for educational

assistants. Source: ASHE 2004, code 6124

Figure 3.1 plots the relationship between GCSE results and each physical input and

expenditure on learning resources. There is no visually obvious relationship between

these measures, although the correlation coefficients reported in Table 3.4 suggest a

29

positive, but weak, relationship between GCSE results and the number of teachers

(r=0.1) and expenditure on learning resources (r=0.16).

(a) GCSE score and teaching staff

130 (b) GCSE scores and learning support staff

130

120

120

gcsevauncap

gcsevauncap

110

110

100

100

90

90

40 60 80 100 0 20 40 60

teachpp learnsuppp

(c) GCSE scores and admin/clerical staff (d) GCSE scores and expenditure on learning &

ICT learning resources

130

130

120

120

gcsevauncap

gcsevauncap

110

110

100

100

90

90

adminclpp ave19e20pp

| gcseva~p teachpp learns~p adminc~p eLearn~p nfsm eal nsen

-------------+---------------------------------------------------------------------

gcsevauncap |

teachpp | 0.1004*

learnsuppp | 0.0036 0.3188*

adminclpp | 0.0125 0.2167* 0.4156*

eLearnITpp | 0.1698* 0.2627* 0.2497* 0.2246*

nfsm | -0.1053* -0.4319* -0.4594* -0.1930* -0.2645*

eal | 0.2833* 0.3410* 0.1360* 0.0798* 0.1644* -0.5638*

nsen | 0.0248 -0.3191* -0.5374* -0.2481* -0.1577* 0.5916* -0.2271*

nsenwout | 0.0769* -0.2282* -0.5149* -0.1861* -0.0656* 0.3195* -0.0361 0.5049*

Schools face a number of constraints in helping pupils to attain high GCSE results.

We take account of some of these constraints, by considering measured characteristics

of pupils in each school. These variables are expressed in a form such that an increase

in their value is expected to lead to an increase in GCSE results (a characteristic

termed “positive isotonicity”, which is a required condition for Data Envelopment

Analysis). The variables are:

• The percentage of pupils in school who are not eligible for free school meals

(nfsm). It is hypothesised that the higher this percentage, the better the GCSE

30

scores are likely to be. Figure 3.2a and a weakly negative correlation coefficient

(r=-0.1) suggest this hypothesis might be misplaced however.

• The percentage of pupils that speak English as an additional language (eal). The

relationship with GCSE results is difficult to anticipate. Initially, such children

might be constrained by the language barrier, and schools might need to devote

additional resources to these pupils, implying that lower levels of attainment are

achievable the higher the proportion of such pupils. Once the barrier has been

overcome, the attainment of such children should be similar to that of other

children. Within schools, attainment may even be above average, the reason being

that pupils with English as an additional language are often concentrated in

particular schools that offer a different – and perhaps more conducive - learning

environment. If so, the direction of influence of this variable on attainment might

be positive. Figure 3.2b and the significantly positive correlation coefficient

(r=0.28) suggest that schools with a higher percentage of pupils with English as

an additional language are likely to have above average value added GCSE results.

In our regression analyses, we explore a flexible functional form to capture the

relationship between attainment and this variable, by including higher order

powers of eal. A F test is conducted to test the significance of the relationship.

This flexibility does not prove important for schools without 6th forms, but

improves the model of technical efficiency for schools with 6th forms.

• The percentage of pupils that do not have Special Education Needs (nsen) nor

Special Education Needs with statements (nsenwout). The higher the proportion

in each category, the better the value added GCSE scores are likely to be. To see

what these two variables capture and how they are related, consider figure 3.3

below. nsen captures an average of 85% of pupils, represented as area A, who are

not considered a constraining factor on achievement. nsenwout captures an

average of 96% of pupils, represented as area A+B, and this variable represents a

“medium” constraining factor. The omitted (reference) group, area C, are pupils

with SEN statements. nsen and nsenwout are correlated, though not strongly so

(r=0.5). Correlation is to be expected, given that nsen is a subset of nsenwout.

whether or not they have a 6th form, are in rural or urban locations, or are specialist

schools. These take the form of dummy variables, with a value 1 if the school has the

characteristic in question, 0 otherwise.

31

Figure 3.2: Relationship between GCSE scores and pupil characteristics

(a) GCSE scores and % no free school meals (b) GCSE scores and % EAL

130

130

120

120

gcsevauncap

gcsevauncap

110

110

100

100

90

90

20 40 60 80 100 0 20 40 60 80 100

nfsm eal

(c) GCSE scores and % no SEN (b) GCSE scores and % no SEN without statement

130

130

120

120

gcsevauncap

gcsevauncap

110

110

100

100

90

90

40 60 80 100 85 90 95 100

nsen nsenwout

A B C

with

statements

without

Pupils:

SEN

SEN

SEN

% 85 96 100

32

4 Regression analysis

Modelling strategy

Any statistical analysis requires a modelling strategy in order to guide the choice of

recommended model. In this study, numerous choices had to made, including:

• Whether to examine the entire sample, or to model separately schools with and

without sixth forms;

• What type of functional form to employ;

• Which explanatory variables to include;

• Whether to use ordinary least squares or stochastic frontier methods;

• Whether to model the hierarchical organisation of schools within local authorities.

A summary of our modelling strategy is presented in figure 4.1. This summarises the

various steps taken in analysing the data, and the basis for selecting among modelling

choices.

Model specification

Our baseline model (Model 1) is formulated to explain the variation in GCSE results

among the i schools as a function of the type and amount of staffing and learning

resources, controlling for pupil characteristics. This model takes the form:

M K

yi = α + ∑ β m x1mi + ∑ β k z ki + γ f i + ε i (4.1a)

m =1 k =1

where yi is the GCSE uncapped value added score; x1mi is a vector of m physical

input variables, z ki is a vector of k exogenous variables, and fi is a dummy variable

indicating whether or not the school has a 6th form. All variables appear in linear form

and the model is estimated by ordinary least squares in Stata/SE 8.2.

We performed the Ramsey RESET test, to test for functional form and omitted

variables. A significant test statistic of p<0.05 suggests that the model is misspecified

in some way (Ramsey, 1969). This test indicated that different models should be used

for schools with sixth forms and those without, the main reason being the relationship

between attainment and the percentage of pupils with English as an additional

language. For schools with 6th forms, the relationship between these variables is not

straightforward, and a flexible functional form is required, so higher powers of eal are

included (eal2, eal3). For schools without 6th forms, modelling the relationship in

linear form was sufficient to satisfy the RESET test.

stochastic frontier model by maximum likelihood in Limdep (Greene, 1995). This

procedure partitions the error term into two components: vi which captures random

(stochastic) error and ui which captures inefficiency in production.

M K

yi = α + ∑ β m x1mi + ∑ β k z ki + γ f i + vi − ui (4.1b)

m =1 k =1

33

Figure 4.1: Schema of modelling strategy

variable

form: Skew test of OLS residual

linear, logarithmic,

translog

variables

powers for EAL?

OLS estimation of

models

M1, M2, M3, M4

estimation of models

M1, M2, M3, M4

Authority effects multiplier test

random noise and measurement error: in other words, there is no evidence of

inefficiency in the sample (Schmidt and Lin, 1984, Wagstaff, 1989).

34

Model 2 replaces the vector x1mi with total expected expenditure x2i , in which the

number of staff in each category is weighted by the average rate of pay.

K

yi = α + x2i + ∑ β k z ki + γ f i + ε i (4.2a)

k =1

K

yi = α + x2i + ∑ β k z ki + γ f i + vi − ui (4.2b)

k =1

Model 3 drops all input variables, and includes the vector z ki , with the specification

taking the form:

K

yi = α + ∑ β k z ki + γ fi + ε i (4.3a)

k =1

K

yi = α + ∑ β k z ki + γ fi + vi − ui (4.3b)

k =1

This parsimonious specification includes only those variables that are considered

exogenous constraints on production. It has been argued that, in efficiency analysis,

all variables that are endogenous to the production decision should be excluded

(Giuffrida et al., 2000). So, the choice about the technical production process, such as

what level and mix of staffing to employ, should be excluded from the model on the

grounds that these are decided by the organisation. The analytical task is to construct

a behavioural function that reflects the feasible production possibilities within a

constrained environment (Smith, 1981). Of course, the choice of what constitute

exogenous variables and over what timeframe these constraints are binding is likely to

be contentious. Recognising this, we specify a fourth model, in which school size s is

introduced, along with a vector of dummy variables d d capturing various school

characteristics:

K D

yi = α + ∑ β k z ki + γ f i + δ1 si + δ 2 si2 + δ 3 si3i + ∑ β d d di + ε i (4.4a)

k =1 d =1

K D

yi = α + ∑ β k z ki + γ f + δ1 si + δ 2 si2 + δ 3 si3i + ∑ β d d di + vi − ui (4.4b)

k =1 d =1

After omitting outliers, these models were estimated in linear form for the full sample

of schools, and separately for schools with 6th forms and for schools without 6th forms.

There are sound theoretical arguments for estimating production functions specified in

Cobb-Douglas or translog form (Christensen et al., 1973, Intriligator, 1978). These

functional forms entail transforming variables into logarithmic form and including

35

higher order powers of the explanatory variables. These specifications may provide

additional insights into the relationship between GCSE results and the explanatory

variables.

However, transformation in this way is less likely to yield models that are estimable

as Stochastic Frontier models, because the composite error ε i is less likely to be

skewed, for two reasons. First, logarithmic transformation tends to normalise the

variables that are transformed and, by implication, the unexplained residual will also

approach normalisation (Cook and Weisberg, 1982). Second, the more explanatory

variables included in the model and the more flexible their functional form, the less

unexplained variance there is to be captured by the error.

Finally, we consider the hierarchical nature of the data, whereby schools are clustered

within Local Authorities. School performance may be partially influenced by the LA

in which it is based, because of the influence that the LA has over school funding and

policy. Schools, therefore, are likely to share closer similarities to other schools

within the same LA, than to schools elsewhere. Importantly for estimation, this

clustering implies that schools cannot be considered independent observations. To

account for this inter-dependence among schools, we estimate a two-level fixed

effects model, where yij represents the GCSE uncapped value added score of the ith

within the jth LA and κ j captures the LA fixed effect:

M K

yij = α + ∑ β m x1mij + ∑ β k z kij + γ f ij + κ j + ε ij (4.5)

m =1 k =1

clustering is important. A significant χ 2 statistic implies that the fixed effects

estimator is preferable to OLS.

Estimation results

OLS results for the full sample are shown in the first set of columns in Table 4.1

below. This model passes the Ramsey RESET test, suggesting that the model is not

misspecified – ie there are no omitted variables. There are a number of notable

features:

• The number of staff, of whatever category, per pupil does not explain variation

in GCSE results among schools. This may be due partly to the limited

variation among schools in staffing levels (the standard deviations reported in

Table 3.2 are relatively small). It may also be indicative of endogeneity

between performance and staffing levels. On the one hand, pupils in schools

that are better resourced may have higher levels of achievement – so resource

levels explain achievement. On the other hand, schools with low achievement

may be provided with extra resources to help them improve their performance,

in which case achievement explains the level of resource.

36

• Higher expenditure on learning resources (including ICT learning resources,

eLearnITpp) is significantly related to higher GCSE results.

• Higher GCSE results are associated with a higher proportion of pupils who do

not receive free school meals (nfsm), who speak English as an additional

language (eal), and who do not have special education needs (nsen,

nsenwout).

• Schools with 6th forms achieve lower GCSE results than those without.

• The residual was not skewed. This has two implications. First, it rules out

Stochastic Frontier analysis because, in the absence of a skewed error term,

decomposition is unfeasible, and the entire error must be interpreted capturing

random noise. In other words, under this specification, there is no evidence of

differential efficiency among schools. Second, given that the primary interest

is in extracting estimates of efficiency rather than the parameter estimates,

exploration of logarithmic or translog functional forms need not be pursued.

All schools With 6th forms Without 6th forms

Coeff SE t Coeff SE t Coeff SE t

teachpp .027 .018 1.49 .038 .023 1.64 .006 .028 0.20

learnsuppp .009 .017 0.51 .005 .024 0.22 .017 .024 0.71

adminclpp -.035 .037 -0.96 -.074 .050 -1.49 -.013 .054 -0.25

eLearnITpp .007 .001 6.76 .007 .001 5.59 .006 .002 3.87

nfsm .027 .011 2.36 .017 .018 0.94 .035 .014 2.45

eal .082 .030 2.75 .097 .042 2.31 .100 .009 10.59

eal2 -.001 .001 -0.79 -.002 .002 -1.29

eal3 .000 .000 1.31 .000 .000 1.54

nsen .030 .014 2.10 .043 .018 2.42 .016 .022 0.72

nsenwout .170 .066 2.59 .218 .096 2.26 .109 .090 1.22

cat_sixth -.390 .181 -2.15

_cons 74.8 6.371 11.74 69.0 9.306 7.42 82.2 8.92 9.21

R2 0.117 0.098 0.146

p RESET 0.661 0.658 0.582

p F test eal, eal2, eal3 0.000 0.000

OLS results for schools with sixth forms are presented in the second set of columns in

Table 4.1. Compared to estimation results for the full sample, the following is worth

noting:

• For schools with 6th forms, the proportion of pupils that do not have free

school meals does not explain variation in GCSE results

OLS results for schools without 6th forms are shown in the final set of columns of

Table 4.a. The main difference between these results and those for the full sample is

that:

• eal appears in linear form only.

37

• There no longer appears to be a relationship between GCSE results and the

proportion of pupils who do not have special education needs (nsen,

nsenwout)

Rather than reproducing all of the regression results from the other models in the main

text, a summary of the significant findings is contained in Table 4.2 below, in which

the symbols can be interpreted as:

• 0 : non-significant relationship (p>0.05)

• + : significant positive relationship (p<0.05)

• - : significant negative relationship (p<0.05)

• F tests are performed to ascertain if the size variables are jointly significant

(p<0.05)

• RESET p / f : pass / fail Ramsey RESET test

• SFA no : ε i normally distributed, so stochastic frontier analysis not possible

Full results for these models are provided in Appendix 1.

Full sample 6th form No 6th form

Eq 1 2 3 4 1 2 3 4 1 2 3 4

teachpp 0 0 0

learnsu 0 0 0

admin 0 0 0

eLearnIT + + +

e_exp + + +

nfsm + + 0 0 0 0 0 0 + + 0 0

eal,2,3 + + + + + + + +

eal + + + +

nsen + + + 0 + + + 0 0 0 0 0

nsenw/o + + 0 + + + + + 0 0 0 0

6thform - - 0 0

size,2,3 0 0 0

religio + + +

rural + + 0

excel 0 0 0

special + + +

bursar 0 0 0

RESET p f f p p f f f p p p p

SFA No No No No No No No No No No No No

• For schools with 6th forms, all but Model 1 appear misspecified.

• The choice between x1 ji (Model 1) and x2i (Model 2) has no qualitative

impact on the results.

• Omission of the input vector x1 ji (Model 3) results in a misspecified model for

the full sample, and for both the full sample and for schools without 6th forms,

the significance of nfsm and nsenwout is now eliminated. Qualitative results

for schools with 6th forms are unaffected.

38

• Inclusion of additional school characteristics d d (Model 4) eliminates the

significance of nsen and, for schools without 6th forms, nsenwout.

• Size is not a significant explanator of GCSE results.

• Under Model 4, religious and specialist schools are associated with higher

GCSE results. Rural schools also appear to perform better than urban schools,

but this holds only for the full sample and schools with 6th forms, where this

model appears misspecified.

• None of the specifications produces a skewed residual, ruling out the

possibility of stochastic frontier analysis.

Finally, we report results of estimating equation 4.5 for schools with and without 6th

forms in table 4.3 below. This clustering is clearly important for schools with sixth

forms, as indicated by the significance of the Breusch-Pagan test, p < 0.000 . The LA

effect appears less important for schools without sixth forms, where p = 0.046 . The

apparent differential importance of the LA effect may partly explain why the OLS

models fail the RESET test when applied to schools with 6th forms, but not appear

misspecified for schools without 6th forms.

• around 25% of the variation in the data is due to variation among Local

Authorities (rho=0.266 and 0.246)

• nsen and nsenwout no longer appear significant explanators of performance

for schools with 6th forms. This could be taken as an indication that schools

are compensated adequately to overcome these additional education needs, as

discussed earlier (page 25).

• estimation results for schools without 6th forms are not qualitatively different

from those reported in Table 4.1.

With 6th forms Without 6th forms

Coeff SE t Coeff SE t

learnsuppp .004 .027 0.13 .043 .028 1.57

adminclpp -.043 .052 -0.84 -.023 .056 -0.41

eLearnITpp .008 .001 5.51 .004 .002 2.33

nfsm .041 .019 2.12 .086 .019 4.41

eal .027 .044 0.61 .092 .012 7.75

eal2 -.001 .001 -0.99

eal3 .000 .000 1.98

nsen .026 .020 1.32 -.018 .021 -0.86

nsenwout .137 .110 1.24 .111 .108 1.02

_cons 76.0 10.60 7.17 80.95 11.04 7.33

R2 within 0.064 0.061

R2 between 0.073 0.209

R2 overall 0.080 0.117

sigma u 2.460 2.429

sigma e 4.087 4.248

rho .266 .246

p Bruesch-Pagan test 0.000 0.046

39

5 Data envelopment analysis

Data envelopment analysis is based on similar economic principles to the parametric

methods described above, but employs a dramatically different estimation method,

using linear programming methods. In effect, it draws an ‘envelope’ around the most

extreme (best performing) schools, and estimates the performance of all other schools

relative to that envelope.

DEA assesses efficiency in two stages. First, a frontier is identified based on either

those organisations using the lowest input mix to produce their outputs or those

achieving the highest output mix given their inputs (the input or output orientation).

Second, each organisation is assigned an efficiency score by comparing its

output/input ratio to that of efficient organisations that form a piece-wise linear

‘envelope’ of surfaces in multi-dimensional space. If there are M inputs and S outputs,

then the production frontier becomes a surface in M+S dimensional space. The

efficiency of a DMU is the distance it lies from this surface - the maximum extent by

which it could improve its outputs given its current level of inputs (or reduce its

inputs given its current level of outputs).

Efficiency in DEA is therefore defined as the ratio of the weighted sum of outputs of a

DMU divided by a weighted sum of its inputs. A separate linear programme is

estimated for each DMU. It seeks out for DMU0 the set of output weights us and input

weights vm that maximizes the efficiency of DMU0, subject to the important constraint

that – when they are applied to all other DMUs – none can have an efficiency greater

than 1. The weights can take any non-negative value, and in general a different set of

weights is computed for each DMU. Thus, the weights us and vm are a central feature

of DEA. They are chosen to cast the DMU in the ‘best possible light’, in the sense

that no other set of weights will yield a higher level of efficiency.

In creating the efficient frontier, DEA selects a subset of ‘extreme’ DMUs that are

deemed efficient. The frontier is merely a linear combination of these efficient DMUs,

so that every inefficient DMU is assessed relative to some linear combination of

efficient units. The chosen efficient units are referred to as its ‘peers’.

DEA also yields specific input or output targets for each DMU, depending on whether

the input or output orientation has been used. For example, under input orientation,

these indicate the specific amounts by which a particular DMU should be able to

reduce its consumption of particular inputs without reducing output. DEA preserves

the input-output mix of DMU, which is therefore compared to a linear combination of

efficient peers that uses similar or identical input-output ratios, but at more efficient

levels.

One final detailed point is that in its original form DEA assumes all inputs and

outputs are measured in ‘absolute’ units, reflecting the size of the DMU. However, in

this application, inputs and outputs are measured as ratios (e.g. staff per pupil), which

are independent of school size. As explained by Hollingsworth and Smith

(Hollingsworth and Smith, 2003), it is important under such circumstances always to

use the modification of DEA advocated by Banker, Charnes and Cooper (Banker et al.,

1984).

40

Model specification

Three DEA models were run using software developed and used by Peter Smith over

a twenty year period. This software is written in FORTRAN, and employs a

Numerical Algorithms Group (NAG) optimization routine. It has been successfully

used in numerous academic and commercial DEA applications since 1984, although it

has not been marketed commercially. Where direct comparison has been possible, the

results secured in this study agree completely with the PIMSOFT results.

All three DEA models included the single output, gcsevauncap, but differed

according to how inputs were specified. These models are directly analogous to the

regression models described in section 4 (equations 4.1, 4.2 and 4.3).

Models 1 and 2 were also estimated as both output-oriented and input-oriented models.

An output-orientation implies that inputs are fixed, and the objective is to maximise

output with these fixed inputs. The input-orientation is analogous to a situation in

which a school has to achieve a target level of attainment, and the task is to minimise

the inputs utilised in meeting this target. The estimates of inefficiency for schools

below the frontier will be sensitive to choice of orientation.

The first model includes the vector of x1 ji is a vector of j physical input variables, and

the hypothesised constraints on production, represented as vector z k , as follows:

gcsevauncap

(5.1)

teachpp,adminpp,learnpp,eLearnITpp, zk

reference to whether the optimal mix of inputs is being deployed by the schools.

The second DEA model uses total expected expenditure (e_exp) as an input. This is

calculated by aggregating information about physical resources, using relative factor

prices reported in Table 3 as weights.

gcsevauncap

(5.2)

Expected expenditure, zk

The purpose of this model is to identify the ‘overall’ efficiency of the school, given

the inputs it consumes. It will be the product of technical and allocative efficiency.

The third DEA model estimates attainment, controlling for production constraints.

This can be estimated assuming an output-orientation only, given that these

constraints are assumed fixed, in the sense of being beyond the control of the school:

gcsevauncap

(5.3)

zk

This model therefore indicates the extent to which schools secure the best possible

results given their external environments.

41

Estimation results

These models were estimated for schools with and without 6th forms. This generated

the following results:

Model 1 1 2 2 3

Orientation input output input output output

mean 0.884 0.907 0.832 0.876 0.855

min 0.582 0.764 0.535 0.738 0.738

100% 196 189 60 60 34

mean 0.899 0.911 0.855 0.879 0.861

min 0.646 0.761 0.525 0.761 0.761

100% 198 192 62 62 32

Model 1 indicates the extent to which schools are technical inefficient. Referring back

to Figure 1.9, the ratio OPT/OP for the average school with a 6th form amounts to

0.884 – in other words the average school is operating 88.4% of the level of efficiency

achieved by the most efficient schools.

Model 2 indicates the total extent of (technical and allocative) inefficiency, OPA/OP

in Figure 1.9. School specific estimates of allocative inefficiency can therefore be

derived as the ratio of efficiency estimates from model 2 and model 1, which captures

the distance OPA/OPT. Taking the input orientation for schools with 6th forms, average

allocative efficiency is calculated as 0.832/0.884=0.941 - thus, the average level of

allocative efficiency among these schools is estimated as being 94.1%. 60 schools

with 6th forms are allocatively efficient and one school has an estimated level of

allocative efficiency of 75.4%, illustrating the most inefficient mix of inputs in this

sub-sample. Summarised details for the two samples of schools under input and

output oriented models are provided in table 5.2.

Allocative efficiency

Orientation input output

mean 0.941 0.966

min 0.754 0.794

100% 60 81

mean 0.952 0.965

min 0.674 0.802

100% 62 115

42

The allocative efficiency results may depend on the weights attached to each of the

physical inputs. We have adopted what we consider to be reasonable weights that

reflect estimates of the national average unit cost of each input (predominantly the

different categories of staff) – as shown in Table 3.3. The results are not sensitive to

minor perturbations of these weights. However, if radically different weights were

adopted, these could alter estimates of allocative efficiency.

For example, purely for illustrative purposes, we adopted a weighting system that

assumed equal weights for all categories of staff, and zero weight for non-staff inputs.

For the input-oriented model, this resulted in some quite large changes in estimates of

allocative efficiency. For schools with a sixth form, the average level of allocative

efficiency dropped a modest 0.9%. However, changes for individual schools ranged

from an increase of 17.4% to a decrease of 14.5%. The equivalent average change in

allocative efficiency for schools without sixth forms was from 95.2% to 92.2%, with

individual changes varying from an increase of 14.8% to a decrease of 16.0%. We

would not suggest that such a radical change in weights could ever be justified. But

these results do indicate sensitivity amongst some schools to the choice of weights.

Finally, we can consider the impact that choices about input use or expenditure have

on efficiency, over and above the set of exogenous constraints. This involves

comparison of the estimates from models 3 and 1 and models 3 and 2.

mean 0.943 0.976

min 0.755 0.814

100% 55 303

mean 0.946 0.980

min 0.769 0.791

100% 85 526

Taking schools with 6th forms as an example and focusing on the impact of

expenditure, the results confirm the crucial importance of the external constraining

factors in explaining efficiency. Adding expenditure to the model improves average

efficiency by only 2.4% (1-0.976).

43

6 Commentary and next steps

This report has laid out the foundations for efficiency analysis and described the

cross-sectional data for 2003-04 available for all secondary schools in England. We

have analysed these data using regression methods and data envelopment analysis. An

important focus of the work was a desire to explore levels of technical and allocative

efficiency. However, in addition to addressing these issues, the analysis has brought

to light a number of fundamental modelling issues that have an important impact on

results.

The regression analysis provides insight into the influence of explanatory variables on

GCSE results. We describe the numerous modelling choices that must be made in

developing such models, and present only a subset of possible specifications. These

are deliberately chosen to be parsimonious, and are estimated in linear form only. It

was not possible to undertake Stochastic Frontier Analysis of these models, because

the composite residual was not skewed.

Data envelopment analysis has been conducted using our own in-house programme.

The results of our analyses will be compared to those produced by the PIMSoft

package, the development of which has been supported by the Department for

Education and Skills. Reassuringly, results for the models compared to date are

identical.

The analysis offers estimates of technical and allocative inefficiency in the secondary

school sector. It suggests that levels of allocative efficiency are relatively high, and

that – on average – the more important element of school inefficiency is technical

inefficiency.

our view deserve further analysis: enhancing the data set; the use of SFA; endogeneity;

and multilevel modelling. We conclude by discussing these in turn.

The dataset

As is always the case, the variables available for this analysis are limited in scope.

Clearly more robust and persuasive results could be secured by expanding the dataset.

Most notably, we had available no measure of capital input, and this may explain

some of the variations in efficiency we identify.

Furthermore, we had available only a cross-section of data from one academic year.

For the parametric modelling, extension of the dataset to a longer series of years

would yield more robust estimates, although it would also introduce new modelling

challenges, for example requiring decisions about how to model the changes in

efficiency over time.

If suitable time series data are available, they can also be used to explore productivity

gains of schools over time. In particular, Appendix 2 illustrates how DEA can be used

to decompose changes of efficiency over time into technological progress and

efficiency gains, employing a Malmquist index approach. This report has

concentrated on the role of efficiency models in offering insights into the efficiency of

44

individual schools. However, the Malmquist approach may be a useful focus for

future work if there is also a policy interest in efficiency gains for the whole

secondary school sector.

We have been unsuccessful in being able to develop SFA models, because none of the

models examined exhibited the requisite asymmetric error term required to model

‘inefficiency’ in SFA.

When undertaking SFA from a regulatory perspective, a necessary condition is that all

variables included as regressors are indicators of environmental factors beyond

organisational control (Giuffrida and Gravelle and Sutton, 2000). This contrasts with

the traditional approach to statistical model building, where the aim is to select a set

of explanatory variables that best explain variation in the dependent variable. It is

quite likely that many potential explanatory variables are indicators of both

environmental effects (which we would wish to include in the model) and policy or

efficiency effects (which we would wish to exclude from the model). Most statistical

modelling of efficiency relies on traditional variable selection devices to test whether

an environmental variable should be included in the model.

This approach cannot be used in SFA, because the analyst must make a joint decision

regarding the variables to include and the model’s error structure. Unexplained

variation from predicted output is decomposed into two parts: symmetric random

error and one-sided inefficiency. Suppose therefore that we wish to test an

environmental variable for inclusion in a SFA model. It will be a candidate for

inclusion if it ‘explains’ a material proportion of the overall residual and therefore

exhibits what is conventionally termed a statistically significant model coefficient.

However, the attribution of a statistically significant effect to the additional

explanatory variable may be for one or both of the following reasons:

1. It explains some of the random error. This implies that the original model suffered

from omitted variable or functional form misspecification.

2. It explains some of the inefficiency error. This implies that the variable is

correlated with the original estimates of inefficiency.

Whether the new variable should be included depends on whether it is judged to

measure an unavoidable hindrance to reaching the estimated frontier (in which case it

would be included) or some potentially controllable characteristic of inefficient

organisations (in which case it would be omitted). There is no scientific guide to

making this judgement. It can be made only with reference to the context of the

specific production process under scrutiny.

Endogeneity

causality from inputs to outputs, and few published efficiency studies have examined

the implication of assumptions other than this simple relationship between inputs and

outputs. Yet, although common sense suggests an impact of inputs on outputs, it is

equally plausible, at least in the public sector, to suggest that there is a simultaneous

feedback from outputs to inputs. When output is measured by well-publicized

performance indicators such as examination pass rates, such feedback is likely to be

45

an intrinsic part of any model of performance. The feedback may arise, for example,

because of the importance attached amongst public sector DMUs to securing national

targets.

In particular, schools for which published examination results are poor are likely to

come under pressure - from residents, central government and other political

sources - to promote improved performance. One manifestation of a high political

priority being given to schooling is likely to be increased resource provision. Thus, as

well as the common sense positive causality from inputs to outputs, there is also

likely - using the terminology of systems dynamics - to be a negative feedback from

outputs to resources. This leads to one or more input variables being dependent on

output measures, and therefore being endogenous. Other things being equal, this

process will lead to a tendency for educational output levels to converge, as the desire

for equity leads to poorly performing schools securing more resources, even if such

activity is not allocatively efficient. Smith calls this process negative political

feedback (Smith, 1995).

When using econometric methods, the failure to model endogeneity may have

profound policy implications in two important areas. First, it may result in the

misspecification of any model which is employed to estimate the production function.

As a consequence of this, faulty inferences may be made about the importance of

input variables as determinants of those output possibilities which are of particular

interest. And second, the incorrect specification of the production function may lead

to errors in assessments of the managerial efficiency of individual organizations.

Econometricians have developed various approaches, such as instrumental variable

methods, to address such issues, although there have been few applications in the

productivity literature.

In contrast, it has usually been assumed that endogeneity causes no material problems

in DEA, since it is a boundary technique that merely models feasible relationships

between inputs and outputs. Orme and Smith challenge the assumption that

endogeneity is always irrelevant to DEA (Orme and Smith, 1996). They introduce the

notion of sparsity bias in DEA efficiency assessments, and suggest that this may

become problematic when the number of observations is small.

The implications of endogeneity for regression analysis are illustrated in Figure 6.1,

which shows the use of x (resources) and s (environmental input) for a given level of

output. The truly efficient frontier is indicated by the isoquant FF. All observations

lie within the frontier, but, because of the link between inefficiency and resource

inputs, there is a tendency for DMUs with low levels of input x to be closer to FF than

those with higher resource input levels. Instead of being parallel to FF, therefore,

OLS statistical methods yield a misleading estimate RR of "average" performance.

46

Figure 6.1: True frontier FF and OLS frontier RR with endogeneity

The analogous implication for DEA is that, while the method will certainly indicate

feasible improvements in output, the targets set may be more demanding for DMUs

adopting low levels of resource inputs than for those using high levels of resource

inputs. This phenomenon arises because, as the preceding argument suggests, at high

levels of x there is likely to be a smaller proportion of DMUs close to the true frontier

FF than at lower levels of x. Thus simultaneity introduces into DEA the possibility

that efficiency estimates (and therefore output targets) may be biased, in the sense that

different parts of the estimated DEA frontier are comprised of DMUs with different

levels of true efficiency. Efficiency improvement targets yielded by DEA in the

presence of endogeneity may therefore be unfair, in the sense that inefficient DMUs

using low levels of resources x may be asked to move closer to the truly efficient

frontier than equally inefficient units (as measured by ε) using higher levels of

resources.

Figure 6. 2 illustrates the effect of such endogeneity on DEA. Because of the sparsity

of efficient DMUs at high levels of x, the estimated isoquant EE is further from the

true isoquant FF there than at low levels of x. Thus two DMUs Ph and Pl with similar

levels of efficiency with respect to FF are indicated as having very different levels of

efficiency according to DEA, with Ph being deemed efficient and Pl inefficient. We

term the distorted view of relative efficiency this gives rise to sparsity bias.

47

Figure 6.2: True frontier FF and DEA frontier EE with endogeneity

Smith and Orme examine the relatively favourable treatment of DMUs with high

levels of x using Monte Carlo simulation. They find that endogeneity in DEA can be

problematic when using a small number of observations, but that its importance

declines as the sample size increases. In the context of this schools example,

endogeneity is unlikely to be a major problem when using DEA.

Multilevel modelling

therefore seems to be natural incorporate the hierarchical structure into the parametric

modelling, using multilevel modelling techniques. We have demonstrated the impact of

this modelling choice in section 4, and find important local authority effects on

attainment levels.

Whether the multilevel formulation should be used may depend on the purpose of the

analysis. For example, the use of a simple ‘fixed effects’ multilevel specification

effectively removes the local authority effect from estimates of a school’s efficiency.

This may be useful if the intention is to examine variations within a local authority.

However, it may be less helpful if the intention is to compare schools across jurisdictions.

The fixed effects model also yields estimates of the ‘fixed’ local authority effect on

efficiency across all schools within its responsibility. This may be useful information if

the interest is in examining local authority (rather than school) efficiency. In practice, we

suspect that there will be policy interest in models both with and without the multilevel

structure.

48

Concluding comments

In the light of this discussion, our strongly held view is that – in the absence of

unequivocal evidence on the nature of environmental constraints on schooling – the

use of SFA is premature. The development of parametric models with asymmetric

effort terms is not in our judgement a priority at present, and instead we would

recommend attention is focused on:

• Expanding the data set to embrace more inputs and outputs, covering a greater

number of years;

• Developing statistically robust regression models based on symmetric errors;

• Examining in more detail the issue of endogeneity, and testing whether it is a

material influence on results;

• Developing multilevel models of performance that reflect the hierarchical

nature of school organisation in England, as a complement to the models

described here.

DEA does not suffer from the same potential for endogeneity problems as regression

methods. It also produces useful, school-specific information on levels of technical

and allocative efficiency, and is relatively easy to justify and explain. In the current

context, its main weaknesses are sensitivity to the inclusion of high attainment

outliers, and the lack of tests to identify whether a model is correctly specified. We

should therefore recommend that DEA can form a useful basis for inter-school

benchmarking, but that in doing so the limitations of data and modelling assumptions

need to be very carefully explained. If additional years’ data become available, DEA

could also serve as a basis for examining productivity gains, using the Malmquist

index approach.

Acknowledgement

We should like to thank Darren Pigg for his constructive help and thoughtful

comments throughout the project.

49

References

Allen, R., Athanassopoulos, A., Dyson, R.G., Thanassoulis, E., 1997, Weight

restrictions and value judgements in data envelopment analysis: evolution,

development and future directions, Annals of Operations Research 73, 13-34.

Atkinson, T., 2005, Atkinson Review: Final Report. Measurement of Government

Output and Productivity for the National Accounts (Palgrave Macmillan,

Basingstoke).

Banker, R.D., Charnes, A., Cooper, W.W., 1984, Some models for estimating

technical and scale inefficiencies in data envelopment analysis, Management

Science 30, 1078-1092.

Charnes, A., Cooper, W.W., Rhodes, E., 1978, Measuring the efficiency of decision

making units, European Journal of Operational Research 2, 429-444.

Christensen, L.R., Jorgenson, D.W., Lau, L.J., 1973, Transcendental logarithmic

production functions, Review of Economics and Statistics 55, 28-45.

Coelli, T., Perelman, S., 2000, Technical efficiency of European railways: a distance

function approach, Applied Economics 32, 1967-1976.

Coelli, T., Rao, D., Battese, G., 1998, An Introduction to Efficiency and Productivity

Analysis (Kluwer Academic Publishers, Boston).

Cook, R.D., Weisberg, S., 1982, Residuals and Influence in Regression (Chapman

and Hall, London).

Cooper, W.W., Seiford, L.M., Tone, K., 2000, Data Envelopment Analysis: a

comprehensive text with models, applications and references (Kluwer, Boston).

Doyle, J., Green, R., 1994, Efficiency and cross-efficiency in DEA: derivations,

meanings and uses, Journal of the Operational Research Society 45, 567-578.

Everitt, B., Landau, S., Leese, M., 2001, Cluster analysis (Arnold, London).

Farrell, M.J., 1957, The measurement of productive efficiency, Journal of the Royal

Statistical Society Series A 120, 253-290.

Fried, H.O., Lovell, C.A.K., Schmidt, S.S.e., 1993, The Measurement of Productive

Efficiency (Oxford University Press, Oxford).

Giuffrida, A., Gravelle, H., Sutton, M., 2000, Efficiency and administrative costs in

primary care, Journal of Health Economics 19, 983-1006.

Greene, W.H., 1995, Limdep Version 7.0 User's Manual. Castle Hill: Econometric

Software, Inc.).

Hollingsworth, B., Smith, P.C., 2003, The use of ratios in data envelopment analysis,

Applied Economics Letters 10, 733-735.

Iezzoni, L.I., 2003, Risk adjustment for measuring health outcomes (Health

Administration Press, Baltimore).

Intriligator, M.D., 1978, Econometric Models, Techniques and Applications

(Prentice-Hall Inc, Englewood Cliffs, New Jersey).

Jacobs, R., 2001, Alternative methods to examine hospital efficiency: data

envelopment analysis and stochastic frontier analysis, Health Care Management

Science 4, 103-116.

Lothgren, M., 2000, Specification and estimation of stochastic multiple-output

production and technical inefficiency, Applied Economics 32, 1533-1540.

Orme, C., Smith, P.C., 1996, The potential for endogeneity bias in data envelopment

analysis, Journal of the Operational Research Society 47, 73-83.

50

Pedraja-Chaparro, F., Salinas-Jimenez, J., Smith, P., 1999, On the quality of the Data

Envelopment Analysis model, Journal of the Operational Research Society 50,

636-644.

Ramsey, J.B., 1969, Tests for specification errors in classical linear least squares

regression analysis, Journal of the Royal Statistical Society 31, 350-371.

Salinas-Jiménez, J., Pedraja-Chaparro, F., Smith, P.C., 2003, Evaluating the

introduction of a quasi-market in community care: assessment of a Malmquist

index approach, Socio-Economic Planning Sciences 37, 1-13.

Schmidt, P., Lin, T., 1984, Simple tests of alternative specifications in stochastic

frontier models., Journal of Econometrics 24, 349-361.

Shephard, R.W., 1970, Theory of cost and production functions (Princeton University

Press, Princeton).

Smith, P.C., 1995. Performance indicators and control in the public sector, in: Berry,

A.J., Broadbent, J. Otley, D., eds., Management Control: Theories, Issues and

Practices, (Macmillan, Basingstoke) 163-178.

Smith, P.C., 1997, Model misspecification in data envelopment analysis, Annals of

Operations Research 73, 233-252.

Smith, V.K., 1981, Elasticities of substitution for a regulated cost function, Economic

Letters 7, 215-219.

Wagstaff, A., 1989, Estimating efficiency in the hospital sector: a comparison of three

statistical cost frontier models, Applied Economics 21, 659-672.

51

Appendix 1: Regression results

Full sample

. * Equation 4.2 full sample

F( 8, 2919) = 29.91

Prob > F = 0.0000

R-squared = 0.1062

Root MSE = 4.3197

------------------------------------------------------------------------------

| Robust

gcsevauncap | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

e_exp | .0021147 .0003995 5.29 0.000 .0013314 .0028979

nfsm | .0253548 .0112963 2.24 0.025 .0032053 .0475043

eal | .0708859 .0298966 2.37 0.018 .0122654 .1295065

eal2 | -.0006092 .0010647 -0.57 0.567 -.0026969 .0014784

eal3 | .000011 9.24e-06 1.19 0.232 -7.08e-06 .0000292

nsen | .0368197 .0141206 2.61 0.009 .0091323 .064507

nsenwout | .2251183 .0638767 3.52 0.000 .0998703 .3503662

cat_sixth | -.3832179 .178845 -2.14 0.032 -.733893 -.0325427

_cons | 66.99142 6.118748 10.95 0.000 54.99392 78.98892

------------------------------------------------------------------------------

( 1) eal = 0

( 2) eal2 = 0

( 3) eal3 = 0

F( 3, 2919) = 44.85

Prob > F = 0.0000

. ovtest

Ho: model has no omitted variables

F(3, 2916) = 1.25

Prob > F = 0.2895

52

. * Equation 4.3: exogenous variables full sample

. reg gcsevauncap nfsm eal eal2 eal3 nsen nsenwout if outlier==0, robust

F( 6, 2921) = 33.47

Prob > F = 0.0000

R-squared = 0.0954

Root MSE = 4.3442

------------------------------------------------------------------------------

| Robust

gcsevauncap | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

nfsm | .0061649 .0104098 0.59 0.554 -.0142463 .0265761

eal | .0754249 .0300698 2.51 0.012 .0164648 .1343851

eal2 | -.0007394 .001074 -0.69 0.491 -.0028453 .0013665

eal3 | .0000122 9.34e-06 1.30 0.193 -6.15e-06 .0000305

nsen | .0246316 .0139762 1.76 0.078 -.0027726 .0520357

nsenwout | .1535022 .0636668 2.41 0.016 .0286659 .2783385

_cons | 81.60338 5.73392 14.23 0.000 70.36044 92.84631

------------------------------------------------------------------------------

( 1) eal = 0

( 2) eal2 = 0

( 3) eal3 = 0

F( 3, 2921) = 47.00

Prob > F = 0.0000

. ovtest

Ho: model has no omitted variables

F(3, 2918) = 0.77

Prob > F = 0.5124

53

. * Equation 4.4: exogenous and categorical variables

. * full sample

. reg gcsevauncap nfsm eal eal2 eal3 nsen nsenwout avfte* cat* if outlier==0,

> robust

F( 14, 2912) = .

Prob > F = .

R-squared = 0.1499

Root MSE = 4.218

------------------------------------------------------------------------------

| Robust

gcsevauncap | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

nfsm | .0019275 .0107489 0.18 0.858 -.0191487 .0230037

eal | .0909546 .0294639 3.09 0.002 .0331824 .1487268

eal2 | -.0012839 .0010441 -1.23 0.219 -.0033312 .0007634

eal3 | .0000164 9.05e-06 1.81 0.070 -1.35e-06 .0000341

nsen | .0164977 .0137138 1.20 0.229 -.010392 .0433874

nsenwout | .1469254 .0621325 2.36 0.018 .0250974 .2687535

avftepupils | -.0057575 .0038721 -1.49 0.137 -.01335 .0018349

avftepsq | 4.00e-06 3.24e-06 1.23 0.218 -2.36e-06 .0000104

avftepcu | -8.24e-10 8.57e-10 -0.96 0.336 -2.50e-09 8.56e-10

cat_religi~s | 1.258029 .2245764 5.60 0.000 .8176845 1.698374

cat_rural | .4247131 .2324907 1.83 0.068 -.0311499 .880576

cat_eicclu~s | .054268 .3817073 0.14 0.887 -.6941756 .8027116

cat_specia~s | 1.986463 .1711574 11.61 0.000 1.650861 2.322065

cat_bursar | -.3067835 .2077901 -1.48 0.140 -.7142139 .1006469

cat_sixth | -.2719149 .1861522 -1.46 0.144 -.6369182 .0930884

_cons | 85.05352 5.664218 15.02 0.000 73.94725 96.1598

------------------------------------------------------------------------------

( 1) eal = 0

( 2) eal2 = 0

( 3) eal3 = 0

F( 3, 2912) = 49.24

Prob > F = 0.0000

( 1) avftepupils = 0

( 2) avftepsq = 0

( 3) avftepcu = 0

Constraint 3 dropped

F( 2, 2912) = 2.06

Prob > F = 0.1273

. ovtest

Ho: model has no omitted variables

F(3, 2909) = 1.76

Prob > F = 0.1527

54

Schools with 6th forms

. * Equation 4.2 schools with sixth form

. reg gcsevauncap e_exp nfsm eal eal2 eal3 nsen nsenwout if outlier==0 & cat_s

> ixth==1, robust

F( 7, 1666) = 14.14

Prob > F = 0.0000

R-squared = 0.0854

Root MSE = 4.2728

------------------------------------------------------------------------------

| Robust

gcsevauncap | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

e_exp | .0023284 .0005357 4.35 0.000 .0012776 .0033792

nfsm | .0171531 .0184079 0.93 0.352 -.0189519 .0532582

eal | .0859107 .0425679 2.02 0.044 .0024186 .1694029

eal2 | -.0018642 .0017006 -1.10 0.273 -.0051997 .0014712

eal3 | .0000231 .0000165 1.40 0.162 -9.30e-06 .0000555

nsen | .0528169 .018281 2.89 0.004 .0169607 .0886731

nsenwout | .2941143 .0865426 3.40 0.001 .1243706 .4638581

_cons | 58.7303 8.283696 7.09 0.000 42.48275 74.97785

------------------------------------------------------------------------------

( 1) eal = 0

( 2) eal2 = 0

( 3) eal3 = 0

F( 3, 1666) = 13.53

Prob > F = 0.0000

. ovtest

Ho: model has no omitted variables

F(3, 1663) = 0.67

Prob > F = 0.5684

55

. * Equation 4.3 schools with sixth form

. reg gcsevauncap nfsm eal eal2 eal3 nsen nsenwout if outlier==0 & cat_sixth==

> 1, robust

F( 6, 1667) = 13.59

Prob > F = 0.0000

R-squared = 0.0738

Root MSE = 4.2987

------------------------------------------------------------------------------

| Robust

gcsevauncap | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

nfsm | .0014568 .0182116 0.08 0.936 -.0342631 .0371768

eal | .0960775 .0428984 2.24 0.025 .0119372 .1802178

eal2 | -.0020952 .0017034 -1.23 0.219 -.0054363 .0012458

eal3 | .0000249 .0000165 1.51 0.132 -7.50e-06 .0000572

nsen | .0442916 .0182441 2.43 0.015 .0085078 .0800754

nsenwout | .2316594 .0860164 2.69 0.007 .0629479 .4003708

_cons | 72.63389 7.727842 9.40 0.000 57.47659 87.79119

------------------------------------------------------------------------------

( 1) eal = 0

( 2) eal2 = 0

( 3) eal3 = 0

F( 3, 1667) = 14.87

Prob > F = 0.0000

. ovtest

Ho: model has no omitted variables

F(3, 1664) = 1.77

Prob > F = 0.1511

56

. * Equation 4.4 schools with sixth form

. reg gcsevauncap nfsm eal eal2 eal3 nsen nsenwout avfte* cat* if outlier==0 &

> cat_sixth==1, robust

F( 13, 1659) = .

Prob > F = .

R-squared = 0.1316

Root MSE = 4.1724

------------------------------------------------------------------------------

| Robust

gcsevauncap | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

nfsm | .0009442 .0178281 0.05 0.958 -.0340237 .035912

eal | .1022575 .0416528 2.45 0.014 .0205599 .1839551

eal2 | -.0022498 .0016387 -1.37 0.170 -.005464 .0009643

eal3 | .0000256 .0000157 1.63 0.104 -5.26e-06 .0000565

nsen | .0319112 .0177897 1.79 0.073 -.0029815 .0668039

nsenwout | .2048671 .0846562 2.42 0.016 .0388228 .3709113

avftepupils | -.0083306 .0060696 -1.37 0.170 -.0202356 .0035743

avftepsq | 5.66e-06 4.72e-06 1.20 0.231 -3.60e-06 .0000149

avftepcu | -1.17e-09 1.17e-09 -1.00 0.317 -3.46e-09 1.12e-09

cat_religi~s | 1.4176 .2943041 4.82 0.000 .840353 1.994846

cat_rural | .6028934 .2904641 2.08 0.038 .0331787 1.172608

cat_eicclu~s | -.6115292 .6147881 -0.99 0.320 -1.817371 .5943129

cat_specia~s | 1.848931 .2147227 8.61 0.000 1.427774 2.270087

cat_bursar | -.1675917 .2800421 -0.60 0.550 -.7168648 .3816814

cat_sixth | (dropped)

_cons | 79.06749 7.787543 10.15 0.000 63.79304 94.34194

------------------------------------------------------------------------------

( 1) eal = 0

( 2) eal2 = 0

( 3) eal3 = 0

F( 3, 1659) = 16.16

Prob > F = 0.0000

( 1) avftepupils = 0

( 2) avftepsq = 0

( 3) avftepcu = 0

Constraint 3 dropped

F( 2, 1659) = 1.53

Prob > F = 0.2176

. ovtest

Ho: model has no omitted variables

F(3, 1656) = 3.08

Prob > F = 0.0266

57

Schools without 6th forms

. * Equation 4.2 schools without sixth form

. reg gcsevauncap e_exp nfsm eal nsen nsenwout if outlier==0 & cat_sixth==0, r

> obust

F( 5, 1248) = 33.00

Prob > F = 0.0000

R-squared = 0.1372

Root MSE = 4.3748

------------------------------------------------------------------------------

| Robust

gcsevauncap | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

e_exp | .0016778 .0006008 2.79 0.005 .0004991 .0028564

nfsm | .0324484 .014124 2.30 0.022 .004739 .0601577

eal | .0981797 .0093498 10.50 0.000 .0798366 .1165228

nsen | .018164 .0216753 0.84 0.402 -.0243601 .0606881

nsenwout | .1392957 .0905176 1.54 0.124 -.0382878 .3168792

_cons | 77.11407 8.942679 8.62 0.000 59.56972 94.65841

------------------------------------------------------------------------------

. ovtest

Ho: model has no omitted variables

F(3, 1245) = 1.12

Prob > F = 0.3382

58

. * Equation 4.3 schools without sixth form

. reg gcsevauncap nfsm eal nsen nsenwout if outlier==0 & cat_sixth==0, robust

F( 4, 1249) = 37.47

Prob > F = 0.0000

R-squared = 0.1302

Root MSE = 4.3906

------------------------------------------------------------------------------

| Robust

gcsevauncap | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

nfsm | .0174858 .0131488 1.33 0.184 -.0083103 .0432819

eal | .0992881 .0092913 10.69 0.000 .08106 .1175163

nsen | .0052605 .0209965 0.25 0.802 -.0359318 .0464527

nsenwout | .0757416 .0901768 0.84 0.401 -.101173 .2526563

_cons | 89.67447 8.307904 10.79 0.000 73.37548 105.9735

------------------------------------------------------------------------------

. ovtest

Ho: model has no omitted variables

F(3, 1246) = 0.66

Prob > F = 0.5753

. reg gcsevauncap nfsm eal nsen nsenwout avfte* cat* if outlier==0 & cat_sixth

> ==0, robust

F( 11, 1241) = .

Prob > F = .

R-squared = 0.1845

Root MSE = 4.2651

------------------------------------------------------------------------------

| Robust

gcsevauncap | Coef. Std. Err. t P>|t| [95% Conf. Interval]

-------------+----------------------------------------------------------------

nfsm | .0053815 .0132991 0.40 0.686 -.0207097 .0314728

eal | .0966751 .0089221 10.84 0.000 .079171 .1141792

nsen | -.0026198 .0212028 -0.12 0.902 -.0442171 .0389775

nsenwout | .0569815 .087895 0.65 0.517 -.1154577 .2294207

avftepupils | -.0046562 .0097066 -0.48 0.632 -.0236994 .0143871

avftepsq | 2.91e-06 .0000107 0.27 0.786 -.0000181 .0000239

avftepcu | -2.87e-10 3.75e-09 -0.08 0.939 -7.65e-09 7.08e-09

cat_religi~s | 1.002352 .3473405 2.89 0.004 .3209124 1.683791

cat_rural | .3718704 .3846203 0.97 0.334 -.3827073 1.126448

cat_eicclu~s | .2663331 .4918984 0.54 0.588 -.6987112 1.231377

cat_specia~s | 2.197616 .2819458 7.79 0.000 1.644473 2.750759

cat_bursar | -.5423945 .3058863 -1.77 0.076 -1.142506 .0577169

cat_sixth | (dropped)

_cons | 94.50605 8.49998 11.12 0.000 77.83013 111.182

------------------------------------------------------------------------------

( 1) avftepupils = 0

( 2) avftepsq = 0

( 3) avftepcu = 0

Constraint 3 dropped

F( 2, 1241) = 0.86

Prob > F = 0.4222

Ramsey RESET test using powers of the fitted values of gcsevauncap

Ho: model has no omitted variables

F(3, 1238) = 1.72

Prob > F = 0.1613

59

Appendix 2: The Malmquist index of productivity change

The three indices used most frequently to measure changes of productivity are the

Törnqvist Index, Fisher's Ideal Index (the geometric mean of the Laspeyres and

Paasche indices) and the Malmquist Productivity Index. The first two require the

calculation of both the amounts and the prices of all inputs and outputs. In contrast,

the Malmquist Index has the advantage that no information is needed on the prices of

inputs and outputs. Furthermore, calculation of the Malmquist Index requires no

restrictive assumptions regarding whether the units under analysis are benefit

maximizers or cost minimizers. These two characteristics make the Malmquist Index

a particularly suitable tool for the analysis of productivity change in the public sector,

where output prices are not in general available. A further advantage of the

Malmquist approach is that it decomposes productivity into three parts that capture

changes in the level of technical efficiency, changes in the level of scale efficiency,

and changes due to technical progress.

The usual approach uses the non-parametric methods of data envelopment analysis

(DEA) to develop a series of production frontiers under different assumptions. The

analytic framework is described in detail in the Appendix. It can be illustrated

graphically by means of Figure 1, which seeks to explain the Malmquist indices in

intuitive form for a technology exhibiting variable returns to scale with just one input

x and one output y. There are two time periods, t and t+1. The variable returns to

scale (VRS) technology estimated by DEA in period t is represented by the frontier

StVRS, while the notional constant returns to scale (CRS) technology is indicated by

the line StCRS. The unit of interest consumes input xt and produces output yt in year t.

Then we can examine the Malmquist Index as comprising three elements, M = (P x S)

x T, as follows.

(Oe/Oq)

P= .

(Od/Op)

This simply indicates the change in the unit’s distance from the current technically

efficient frontier from one year to the next. The change in scale efficiency S is given

by

(Oc/Oq)

S = (Oe/Oq) ’

(Ob/Op)

(Of/Op)

whilst the change in the scale efficient technology indicated by the CRS frontiers is

estimated by

(Og/Oq) (Ob/Op)

T= . .

(Oc/Oq) (Oa/Op)

In this simple graphical example, the two components of T are identical, but this will

not in general be the case.

60

Output

St+1CRS StCRS St+1VRS

y

StVRS

(xt+1,yt+1)

! ! ! ! !

(xt,yt)

! ! ! ! !

Figure A1: Illustration of productivity change with one input, one output

(Oc/Oq)

(Oe/Oq) (Oe/Oq) (Og/Oq) (Ob/Op)

M= .

(Ob/Op) (Oc/Oq) (Oa/Op)

. . .

(Od/Op)

(Of/Op)

That is, the unit’s productivity change is expressed as the product of pure efficiency

change, scale efficiency change, and an estimate of technological progress.

In order to make the index operational, it is necessary to consider its four constituent

distance functions. This can be achieved using parametric or non-parametric

approaches. In our view, the non-parametric approach (DEA) offers distinct

advantages for analyzing public sector units. It requires fewer assumptions than

parametric methods regarding the functional form of the production technology; it can

readily handle multiple inputs and outputs; it can yield useful results with small

numbers of observations; and it is less vulnerable than parametric methods to

statistical biases, such as endogeneity and omitted variables. We therefore recommend

implementation of these principles using successive formulations of DEA models.

example is the examination of productivity change in hospitals (Jacobs, 2001) and

social care (Salinas-Jiménez et al., 2003). It should be noted that – whilst such

methods offer interesting insights into progress over time – the estimates they yield

can be quite volatile and sensitive to model specification. Modelling therefore

requires careful implementation and interpretation.

61

Copies of this publication can be obtained from:

DfES Publications

P.O. Box 5050

Sherwood Park

Annesley

Nottingham

NG15 0DJ

Fax: 0845 60 333 60

Minicom: 0845 60 555 60

Online: www.dfespublications.gov.uk

Ref No: RR788

www.dfes.go.uk/research

- god bless youUploaded byMahendra Naidu
- Case 12Uploaded byAjay Devnani
- Tutorial 3Uploaded byBhag Chand Meena
- Hospital Supply, Inc Case 16-1Uploaded bymouse1965
- Ho Truong Phuong ThaoUploaded byTiểu Hồ Ly
- 06. Learning CurveUploaded byDinesh Raghavendra
- 10.1.1.720.6093Uploaded byYudith Vega Paramitadevi
- Misallocation in Cina e in IndiaUploaded bymichelerovatti
- labor essayUploaded byapi-263740683
- 7-24Uploaded byriza147
- answerset 2stdUploaded byakram1978
- Product Costing Explained ANAUploaded byDhanush Kumar
- Application and Results of a Skilled Labor Demand Forecast Model for the US Construction IndustryUploaded byinventionjournals
- Encyclopedia of Management _ Service Process MatrixUploaded byDarshan N Gangolli
- Case 4-6 Grand JeanUploaded byanindya_himmah
- weekly lesson plans-long 5Uploaded byapi-316651663
- Wacquant, Loic - Urban Marginality in the Coming Millennium.pdfUploaded byIrina Lovely Smile
- 10.0000@www.researchgate.net@46524173.pdfUploaded byvanny
- Reinstatement Executory Even if Decision is Under AppealUploaded byyurets929
- Academic Word ListUploaded bykangchu
- CHAPTER 2Uploaded byChii Lala
- Tushar Labor VlawUploaded byPushpendra Jangid
- Bhaduri & Marglin (1990)Uploaded bymarceloscarvalho
- CAUSES FOR THE COLLAPSE OF CAPITALISM AND TASKS FOR THE REVOLUTION.docxUploaded byNemo Starem
- 31 Production and Materials ManagementUploaded byAnkush Gupta
- Econ111_235_405_Tubera_RJUploaded byRj Tubera
- 8639_princ-ch02-presentation_6.pptUploaded byShaira Unggad
- 2009520223911853Uploaded bySanghoon Jeong
- Industrial Employment Standing Orders Act 1946 114[1]Uploaded bysearchshri
- 18524899Uploaded byTanya Kanodia

- ShawUploaded byAslam Aly Saib
- SuffahFoundation - Did the Sahaba Celebrate Milad-un-NabiUploaded byAslam Aly Saib
- Option PricingUploaded byAslam Aly Saib
- 9 Binomial WaysUploaded byHendy Kurniawan
- syllabusUploaded byAslam Aly Saib
- syllabusUploaded byAslam Aly Saib
- GoodUploaded byAslam Aly Saib
- Numerical ExamplesUploaded byAslam Aly Saib
- Oliveira Freportsm10Uploaded byAslam Aly Saib
- Arabiyyah Bayna Yadayk Book 1Uploaded byAslam Aly Saib
- Naat by SahabasUploaded byAslam Aly Saib
- ExtensionUploaded byAslam Aly Saib
- with a TA.pdfUploaded byAslam Aly Saib
- with a TA.pdfUploaded byAslam Aly Saib
- Aims and ObjUploaded byAslam Aly Saib
- Jwahar-ul-Bahar Fi Fazail-ul-Nabi-ul-Mukhtar 2 by - Allama Yusuf Bin Ismail NabhaniUploaded byMarfat Library
- Renewal 3Uploaded byAhmad Muhammad
- Renewal 3Uploaded byAhmad Muhammad
- Black SeedUploaded byAslam Aly Saib
- Should We Fear DerivativesUploaded byJ. Perry Stonne
- Financial Risk ManagementUploaded byRajashekar Gatla
- hw1_2Uploaded byAslam Aly Saib
- Applied Probability and Stochastic ProcessesUploaded byAslam Aly Saib
- sas programmUploaded byDavid Vivas
- ViewerUploaded byAslam Aly Saib
- PizzaUploaded byAslam Aly Saib
- ) Sfeeha...pdfUploaded byAslam Aly Saib
- Iron DisordersUploaded byAslam Aly Saib
- 6 Regulations for First Degree Dec 2015 1Uploaded byAslam Aly Saib

- How to Configure User Accounts to Never ExpireUploaded byAshutosh Mayank
- AssignmentUploaded byPramiti Parashar
- ASTM D6758-08 Standard Test Method for Measuring Stiffness and Apparent Modulus of Soil and Soil-Aggregate In-Place by Electro-Mechanical Method.pdfUploaded bygtddy
- Turbines and GoverningUploaded byKVV
- Catalog HIDUploaded bysawan_iit84
- katharine resumeUploaded byapi-383593925
- Production ForecastingUploaded bymanish.7417
- FRACP Short Case NotesUploaded byleh.mo9315
- HorizonCompactPlus Backup Restore v1.00.Build.4.InstructionsUploaded byGilberto Martinez
- Tablet FriabilityUploaded byJean Florencondia
- JDMProSystemArchitecture Boeing Jeppesen JDMUploaded bytechgeekvn
- AugmentinUploaded byHeba_Al_Khozae
- Jesus--Name Above All Names_ Re - Joyce MeyerUploaded byDhugo Lemi
- PhD Course Work DrKhannaUploaded bykalyankpy
- uobj_paper_2014_51927543Uploaded byEvelyn Deza Guzmán
- ConstitutioUploaded bySubrahmanya Sringeri
- lm3 clear accountabilities and monitoringUploaded byapi-287200371
- practicum reflection 1Uploaded byapi-252317894
- QUESTION 13-17 Power and PoliticsUploaded byainonlela
- Edwards Vacuum PumpUploaded byแล้วแต่ จะคิด
- Webleaflet Eng A5 Combo 180523Uploaded byRoxana Birtum
- Response 2000 IntroductionUploaded byRory Cristian Cordero Rojo
- Tutorial Saphir 32Uploaded byAndre Yudhistira
- Automatic Water Tap Using ArudinoUploaded byShreyansh Gupta
- Newsletter 23rd Mar 2008Uploaded bydonbarton
- MIS_ACCUploaded byEsheeta Ghosh
- The use of intramuscular benzodiazepines and antipsychotic agents in the treatment of acute agitation or violence in the emergency departmentUploaded bytomokai
- Fieldwork Guide (R. Butler)Uploaded bySergio
- F5 VarianceUploaded bynomno999
- IJRET20150401009.pdfUploaded byNilesh Lal