Urban Studies http://usj.sagepub.

com/

An Indicator-based Approach to Measuring Sustainable Urban Regeneration Performance: Part 2, Empirical Evaluation and Case-study Analysis
Lesley Hemphill, Stanley McGreal and Jim Berry Urban Stud 2004 41: 757 DOI: 10.1080/0042098042000194098 The online version of this article can be found at: http://usj.sagepub.com/content/41/4/757

Published by:
http://www.sagepublications.com

On behalf of:
Urban Studies Journal Limited

Additional services and information for Urban Studies can be found at: Email Alerts: http://usj.sagepub.com/cgi/alerts Subscriptions: http://usj.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations: http://usj.sagepub.com/content/41/4/757.refs.html

>> Version of Record - Apr 1, 2004 What is This?

Downloaded from usj.sagepub.com at Serials Records, University of Minnesota Libraries on September 24, 2011

Urban Studies, Vol. 41, No. 4, 757772, April 2004

An Indicator-based Approach to Measuring Sustainable Urban Regeneration Performance: Part 2, Empirical Evaluation and Case-study Analysis
Lesley Hemphill, Stanley McGreal and Jim Berry
[Paper rst received, January 2003; in nal form, October 2003]

Summary. This paper examines the sustainability of current urban regeneration practice, through the application of weighted indicators and a points scoring framework. The analysis applies the hierarchical model discussed in the preceding paper of this journal issue to case studies of waterfront areas and cultural quarters in three European cities: Belfast, Dublin and Barcelona. The evaluation permits performance comparisons to be made between the case studies regarding the sustainability of regeneration areas and projects, variations on an indicator set basis and the sensitivity of scores. Conclusions are drawn concerning regeneration practice, the extent to which sustainability principles are adhered to, potential policy benets and the applicability of the model.

Introduction Sustainable development and urban regeneration have become increasingly recognised as complementary goals, but there remains only limited evidence of performance evaluations that investigate the extent to which regeneration projects have achieved sustainable outcomes (Carley and Kirk, 1998; DETR, 1998a; OECD, 2000). Likewise, the dissemination of what constitutes good sustainability practice as discussed in the accompanying paper and how this can be benchmarked remains underinvestigated. What is becoming increasingly obvious is the prominent place given in UK policy to the issue of sustainability and ultimately how this can be best achieved (DETR, 1998b, 1998c, 1998d, 1999a, 1999b, 2000). However, despite this policy drive to instigate sustainable development, the role of urban regeneration in fullling sustainability objectives remains in its infancy. Sustainable Development: The UK Strategy (DOE, 1994) recognised the importance of urban regeneration in contributing to sustainability by developing areas in an efcient way whilst making them more attractive places in which to live and work. In a similar vein, the Urban Task Force in its Towards an Urban Renaissance report (DETR, 1999b) advocated the need for a comprehensive package of regeneration measures to address both the physical regeneration of an area and the economic and social needs of the local population (p. 133). This is further supported by the UK governments Urban White Paper Our Towns and

Lesley Hemphill, Stanley McGreal and Jim Berry are in the Centre for Research on Property and Planning, School of the Built Environment, University of Ulster, Shore Road, Newtownabbey, County Antrim, Northern Ireland, BT37 0QB. Fax: 028 9036 6826. E-mail: la.hemphill@ulster.ac.uk; ws.mcgreal@ulster.ac.uk and jn.berry@ulster.ac.uk. 0042-0980 Print/1360-063X On-line/04/04075716 2004 The Editors of Urban Studies DOI: 10.1080/0042098042000194098
Downloaded from usj.sagepub.com at Serials Records, University of Minnesota Libraries on September 24, 2011

758

LESLEY HEMPHILL ET AL.

Cities: The FutureDelivering an Urban Renaissance (DETR, 2000) which identied bringing browneld land and buildings back into constructive use as a key objective to exploit their potential in contributing to the quality of urban life. The Sustainable Development Strategy for the UK: A Better Quality of Life (DETR, 1999a) lists regeneration as one of a number of aspects contributing to sustainable development whilst indicating that building sustainable communities remains a priority for government action. However, Imrie and Thomas (1995) consider that the proliferation of new urban policy initiatives since the early 1980s has not been matched by systematic monitoring and evaluation. Likewise, Turok (1989) states that the scope and form of evaluative research have often been illogical, weak and increasingly concerned with a narrow range of quantitative evaluative criteria. Such a focus on the quantitative measurement of policy impacts has failed to recognise the importance of the subjective, experiential life-world of human well-being (Burns, 2000). Indeed, it is this failure to appreciate the human aspects of urban policy through the use of purely quantitative methods of evaluation that has often led to the disquiet expressed over urban policy evaluation. Hakim (2000) argues that too often the secondary analysis of an existing data-set is proposed at the expense of qualitative research. Similarly Wong (2002) points towards the need for more qualitative research to measure satisfactorily intangible issues, such as community identity and institutional capacity. Perhaps of greater signicance is the potential to combine both quantitative and qualitative techniques within any given evaluation to ensure that their respective strengths can be capitalised on. Indeed, qualitative research can be used in combination with virtually all other types of study, informing the interpretation of more impersonal statistical data (Hakim, 2000). The research reported in this article builds upon the concepts developed in the preceding paper in this issue and draws upon a combination of techniques as a means of

gathering data appropriate for the indicators. Expert opinion plays a major part in the empirical investigation which seeks to assess the measurement of urban regeneration practice in light of sustainability principles. A detailed performance analysis of six independent case studies is presented utilising the 52 indicators discussed in the accompanying paper. The performance of each case-study area is compared for each of the ve indicator groups in the model and against an overall aggregated score to provide an indication of relative performance. Sensitivity analysis is applied to each indicator to assess different options. The paper provides the basis for the use of the model for evaluation and policy purposes, notably in the identication of the issues where present urban regeneration practice is not fully embracing the sustainability agenda. Conclusions are drawn on the respective case-study performances and the successes or shortcomings of present regeneration policy in creating sustainable outcomes, potential policy benets and applicability of the model.

Sustainable Regeneration Benchmarking

Performance

The subject of measuring sustainability performance has been assessed in detail in the accompanying paper; however, an equally important aspect of measurement is how to benchmark this performance. Benchmarking can be considered an effective means of helping to deliver better services by comparing performance against an accepted norm and learning from other areas/organisations (Audit Commission, 2000). This is all the more evident in todays society with headline (DETR, 1999a) and best value (ODPM, 2002) indicators providing two examples of circumstances where indicators have been used to measure and subsequently benchmark performance. Benchmarking enables the comparison of quantitative performance data, identies how performance differs from other areas, how it has changed over time and whether it can act as the spur for im-

Downloaded from usj.sagepub.com at Serials Records, University of Minnesota Libraries on September 24, 2011

MEASURING URBAN REGENERATION, PART 2

759

provement or an example of how to achieve good practice (Audit Commission, 2000). Practitioners need to be able to establish the baseline economic and social conditions against which they can measure the progress and improvement resulting from regeneration projects (Bristow, 1999). Likewise, it is acknowledged that local authorities and developers require practical tools to measure the sustainability of developments (both buildings and infrastructure) at site or estate level (Brownhill and Rao, 2002) to ensure that they meet planning requirements and contribute to wider sustainability objectives. Similarly, a crucial part of benchmarking and the use of indicators is to help guide appropriate action through the setting of suitable targets (European Commission DGXI, 1996). The Audit Commission (2000) identies the key elements of the benchmarking process namely, select service/project, identify resources, identify partners, dene and collect measures, compare performance, nd best practice/targets, plan change, implement and monitor outcomes. This paper adopts a similar benchmarking strategy to the measurement of sustainable urban regeneration and utilises the model developed and discussed in this journal issue. Although the points scoring parameters reect current legislative or policy standards, it is important to stress the dynamic characteristics of the model and its exibility to permit change as and when new technology or information renders current standards as insufcient to meet sustainability requirements. Evaluative Model Overview The preceding paper provides a detailed resume of the methodology underpinning the research identifying a four-phase process (Figure 1). This paper deals primarily with the application of the evaluative model at case-study level. In particular, the focus is upon phase fourthe case-study application and sensitivity analysis. The model is based on 52 individual indicators which have been weighted into ve main groups: Economy

and work (21.5 per cent); Resource use (17.5 per cent); Buildings and land use (18.9 per cent); Transport and mobility (22.1 per cent); Community benets (20 per cent). Further discussion of the indicators, the points scoring framework and how they were selected and tested is provided in the accompanying paper. Case-study Selection In testing the validity of the model and seeking to achieve a meaningful set of results, rigorous selection criteria were drawn up to ensure that the same basic characteristics were present in each case-study area thereby enabling a full comparative evaluation between individual schemes. These criteria include: previous track record of the case-study area, the hybrid of contacts available, indicator coverage, completed schemes, size and scale of scheme, and the occurrence of waterfront and cultural regeneration projects within prospective locations. The selection of the most appropriate casestudy locations, as governed by these criteria, identied a number of potential urban regeneration schemes embracing good practice. However, the requirement for differing schemes including both cultural and waterfront examples reduced the range of possibilities. A further consideration was the number of case-study areas. Literature sources suggest that the signicance and robustness of research ndings increases with the number of sites in which a survey is conducted, although the largest single gain occurs when the number of sites is increased from one to two (Sudman, 1976). This was supported by Yin (1984) who indicated that survey results can be strong with a small number of case studies provided that they are carefully selected. Based upon these considerations, three case-study cities were selected with each providing two individual studies to reect the waterfront and cultural themes underpinning the designated regeneration area. It is considered that the six case studies provided a sufcient and diverse range of examples to allow the indicators to be tested.

Downloaded from usj.sagepub.com at Serials Records, University of Minnesota Libraries on September 24, 2011

760

LESLEY HEMPHILL ET AL.

Figure 1. Methodological approach.

Belfast provides examples of mainstream UK-based initiatives including the development corporation model, Laganside Corporation, the last remaining development corporation excluding the recent revival of the model in Thames Gateway and extensive use of gap-funding through urban development grant. In Belfast, two key schemes Laganside and Cathedral Quarter t the wider selection criteria of waterfront and cultural quarters respectively (Table 1). The former has a proven national and international track record. Laganside schemes received recognition from the RICS in 1998 by winning their Reclamation of Contaminated Land Award and were the subject of an OECD evaluation study (OECD, 2000). The

Cathedral Quarter, formerly Northside, on the northern city-centre fringe of Belfast, although possessing many key physical attributes, has been a difcult area to regenerate with many different initiatives (Berry and McGreal, 1993) the latest of which focuses upon its ethos as a cultural location. The second case-study city, Dublin, provides many contrasts with Belfast from a policy perspective (Berry and McGreal, 1992). Both of the selected schemes in Dublin (Table 2), the Docklands area and the cultural quarter of Temple Bar have beneted signicantly from the use of taxation breaks. Temple Bar is widely accepted as being a good model of cultural regeneration with the area promoted as the agship

Downloaded from usj.sagepub.com at Serials Records, University of Minnesota Libraries on September 24, 2011

MEASURING URBAN REGENERATION, PART 2

761

Table 1. Belfast case studies: summary LagansideWaterfront Cathedral QuarterCultural

140 ha site situated 1 km from Belfast city centre 12.1 ha site located on the northern fringe of the city centre Area rst designated in 1987 with the scheme Area designated in 1998 with the scheme prodetails highlighted in the Laganside Concept posals highlighted in the Cathedral Quarter RePlan generation Strategy (previously Northside) Area falls under the control of the Laganside Area falls under the control of the Laganside Corporation, a form of development corporation Corporation, but treated as a distinct area separset up to oversee the development of the area ate from the main Laganside site Area consists of 6 key commercial sites and 6 apartment developments, as well as environmental improvements to River Lagan and the introduction of a cross-harbour road and rail bridge Area (will) consist of 2 apartment developments; a hotel; various cultural and arts developments; building refurbishments; a multi-storey car park; environmental improvements to streetscape

Table 2. Dublin case studies: summary Dublin DocklandsWaterfront Temple BarCultural

526 ha site located in the northern area of the 12 ha site located in the heart of Dublin city traditional dockland; 85 ha available for develop- centre ment and a further 10 ha for open space Area rst designated in 1986 although an ofcial Area designated in 1991 with development carMaster Plan was not adopted until 1997 ried out in line with the Temple Bar Development Programme 1996 Area falls under the control of the Dublin Docklands Development Authority (DDDA) which replaced the Custom House Docks Development Authority (CHDDA) in 1997 Area consists of 6 main development areas each of which have various objectives ranging from promoting residential use, developing landscaping, promoting public transport, promoting tourism/hotel facilities, protecting natural habitats, supporting cultural development and the expansion of enterprise uses Area fell under the control of 2 independent companiesTemple Bar Renewal Ltd (responsible for deciding funding applications) and Temple Bar Properties Ltd (responsible for carrying out the development work) Development carried out in 2 main phases: 199196 and 19961999. Area consists of an Irish Film Centre, infrastructural improvements, apartment developments, retail lettings, building refurbishments, provision of innovative archaeological component, promotion of area as a cultural quarter

project to mark Dublins year as European City of Culture in 1991 (Montgomery, 1995). In a similar vein, the Docklands area has been widely advocated as a success (Gahan, 1993) and long-term sustainability is one of the objectives of the Dublin Docklands Master Plan making the area an ideal case study in which to apply the indicators (DDDA, 1997). In developing the contrast to Belfast, the Dublin case studies provided the opportunity to compare and contrast the differing

approach taken to the stimulation of private investment between tax-based incentives and grant-based mechanisms as applied in Belfast. The selection of Barcelona (Spain) widens the research to a European perspective, facilitating comparison with the UK and Ireland dimension. Barcelona also provided two very different regeneration schemesnamely, the Ciutat Vella (Old City) as the cultural example and the Olympic Village as the water-

Downloaded from usj.sagepub.com at Serials Records, University of Minnesota Libraries on September 24, 2011

762

LESLEY HEMPHILL ET AL.

Table 3. Barcelona case studies: summary Olympic VillageWaterfront Ciutat VellaCultural

47 ha site located between the Cuitadella Park 386 ha site located in the old historical centre of and the centre of the Poblenou areapart of a the city and constitutes one of the 10 districts zone of coastal facilities (130 ha) into which Barcelona is divided Decision to regenerate the area coincided with the announcement that Barcelona was to be the venue for the 1992 Olympic Games. In 1989, various changes were made to the Plan General to allow a change in the function of the area Area fell under the control of an ad hoc body called Holding Olimpico SA (HOLSA) who were responsible for co-ordinating the developments Area consists of various 5-storey apartment blocks with commercial and retail facilities below, the twin Olympic Towers (Hotel and Ofce); and high-quality public space Area designated as an Area of Integrated Rehabilitation (AIR) in 1986 with 4 special plans for internal reform (PERI) to cover the key areas in the old city: El Ravel, Casc Antic, Barceloneta and Barri Gotic Area fell under the control of a publicprivate company called Promocio Ciutat Vella SA (PROCIVESA) to carry out or co-ordinate the main works envisaged in the special plans The main elements of PERI plans included land clearance, redevelopment or refurbishment of property, creation of public open space and the provision of modern infrastructure

front example (Table 3). The decision to use Barcelona as a case-study location was based on a number of counts, but primarily the international reputation that the city has achieved since the Olympic Games. Indeed, the Urban Task Force report (DETR, 1999b) charted some of the regeneration work undertaken as being a good example of the way forward for future practice. Furthermore, the urban design and architectural quality of what has been developed have been documented in journals, culminating in the RIBA awarding the area with a gold medal for the outstanding quality of the transformations (DETR, 1999b). Case-study Results Accessing information to enable each of the indicators to be assessed required the utilisation of a combination of highly varied methods, applied consistently across each of the case studies. Some provided hard quantiable data that could be directly applied to the points scoring framework, whilst others provided more qualitative information requiring interpretation prior to the allocation of points. The sources of information utilised reect the complexity and variety of indica-

tors and include: government departments/ agencies and other development organisations, literature/ofcial reports, other research, end-user surveys of occupiers/ employers, consultation with a range of professionals in the built environment (architects/property developers/estate agents), questionnaire surveys (employees/residents) and site survey and investigation. The ve separate indicator setsnamely, Economy and work, Resource use, Buildings and land use, Transport and mobility and Community benets for each case study were evaluated on both an individual and collective basis in accordance with the points scoring criteria and rationale highlighted in the associated paper. The case-study results are considered under three headings: indicator set analysis, benchmark analysis and sensitivity analysis. Indicator Set Analysis This analysis initially reviews the scoring on an indicator basis, followed by the overall points for each indicator group, supplemented by a brief overview of the relative performance of the individual case studies in delivering sustainable regeneration out-

Downloaded from usj.sagepub.com at Serials Records, University of Minnesota Libraries on September 24, 2011

MEASURING URBAN REGENERATION, PART 2

763

Table 4. Economy and work scoring performance Economy and work indicators Number of jobs created per 1000 square metres Net jobs created Percentage of new enterprises still operating after 3 years Quality of jobs created Leverage ratios Performance of incentive mechanisms Partnership structure performance Effectiveness of exit strategy Incorporation of training programmes End-user scheme satisfaction Total score Percentage score Weighted score (weighting factor 21.5) Ranking LS 8 6 10 4 4 6 6 7 8 8 CQ 10 4 8 10 4 1 5 5 7 5 DD 6 6 10 4 4 6 8 8 7 6 TB 4 4 6 8 4 2 5 2 7 6 OV 4 8 10 6 2 6 8 8 6 8 CV 2 8 8 8 2 8 8 7 6 7

67 59 65 48 66 64 67 59 65 48 66 64 1440.5 1268.5 1397.5 1032.0 1419.0 1376.0 1 5 3 6 2 4

comes. Derivation of these indicators has been discussed in detail in the accompanying paper. Their richness stems from the number and comprehensive range of indicators employed across the ve component areas and the extensive nature of the evidence. Economy and work. This constitutes a subset of 10 indicators formulated to evaluate the performance of economic strategies as well as the tangible outcomes of the investment process in terms of the employment generated. For this indicator set, the three waterfront localities of Laganside Belfast, the Olympic Village Barcelona and Dublin Docklands achieved the highest scores reecting the economic emphasis initially needed in these areas to stimulate investment (Table 4). Of the individual indicators, the percentage of new enterprises still operating after 3 years showed the best performance across all case-study areas, supporting the need for longevity of businesses in the longterm sustainability of regeneration schemes. The 2 Barcelona case studies are the only examples to embrace signicantly the indicator relating to the percentage of net jobs created: both provide over 30 per cent of their workforce from their local area. The Cathedral Quarter Belfast had notable suc-

cess in terms of the number of jobs created and the quality of those jobs with both indicators scoring highly. However, the relatively poor performance of the incentive mechanisms indicator, within the Cathedral Quarter and Temple Bar areas, seems to indicate that incentives had little inuence on the decision of businesses to move to these locations. This also reects a high number of in situ end-users. From the sustainability perspective, the ndings of the Economy and work indicators show that in most of the regeneration schemes partnership structures, exit strategies and training programmes are important, although the low scores awarded to the leverage ratios suggest a continuing dependency on public-sector money to secure long-term success. Resource use. The Resource use indicators relate to the effective use of the physical resources whilst dealing with environmental issues such as the reclamation of materials, waste minimisation, energy efciency and conservation. This indicator group produced a similar set of results across 5 of the casestudy areas with only 7 per cent covering the top 5 performers (Table 5). The Barcelona case studies lead on these indicators with the Olympic Village scoring 64 per cent and the

Downloaded from usj.sagepub.com at Serials Records, University of Minnesota Libraries on September 24, 2011

764

LESLEY HEMPHILL ET AL.

Table 5. Resource use scoring performance Resource use indicators Reclamation of building materials Retention of environmental features Household waste disposal Firms undertaking waste minimisation Energy efciency: building lay-out/design Energy efciency: building materials/construction Conservation of built heritage Incorporation of environmental design Performance of environmental management Total score Percentage score Weighted score (weighting factor 17.5) Ranking LS 2 6 2 10 8 6 6 4 8 52 58 910.0 4 CQ 2 6 2 8 7 4 10 4 6 49 54 857.5 6 DD 2 4 2 10 8 8 2 10 5 51 57 892.5 5 TB 6 2 2 10 9 8 10 6 1 54 60 945.0 3 OV 2 2 6 10 8 6 6 10 8 CV 8 6 6 8 5 5 10 4 5

58 57 64 63 1015.0 997.5 1 2

Ciutat Vella 63 per cent. The Cathedral Quarter Belfast provided the poorest performance in this group (54 per cent) reecting cost constraints involved in improving the energy efciency of older buildings. All case studies scored well in the waste minimisation indicator demonstrating a very high percentage of rms actively undertaking waste audits. However, this positive treatment of waste did not extend to household waste disposal, with the Barcelona case studies the only areas to perform above the less than 20 per cent bandindicating that in the UK and Ireland households are falling well short of the 40 per cent recycling target. As expected, the three cultural case studies substantially outperformed the waterfront areas regarding the conservation of built heritage, but conversely were lower ranked in terms of incorporating environmental design into the building stock. With the exception of Ciutat Vella and Temple Bar, the case studies illustrated that little or no reclamation of building materials occurred. This nding is somewhat surprising in sustainability terms, given the high levels of land lling of such waste. Buildings and land use. The best overall performance in ve of the case studies is for the Buildings and land use indicator group, reecting the emphasis of these particular

regeneration schemes upon property and development outcomes. The top-ranked locations are the Olympic Village (78 per cent), followed by Ciutat Vella (74 per cent) with Laganside, Cathedral Quarter and Dublin Docklands all scoring 73 per cent (Table 6). These results place the Barcelona case studies collectively as the best performers in this sector corresponding with their well-recognised position at the forefront of urban design and the creation of buildings with architectural distinction. Contrasts are apparent in terms of the individual indicator performances. The analysis demonstrates that the three waterfront case studies demonstrate a high ratio of open space to built form, whilst the cultural case studies balance this with having a much higher ratio of converted/redeveloped buildings to new build. Nevertheless, all case studies illustrate high density, a balance of uses (apart from Dublin Docklands) and the complete removal of any contaminated land. The ofce rental indicator, with the exception of Temple Bar, implies that favourable rents are achievable with relatively high levels of occupancy in areas that previously were classied as weak market locations. The performance of the property variables reects the acceptable levels of design and quality of the nal built product. However, the main vulnerability from a sustainability

Downloaded from usj.sagepub.com at Serials Records, University of Minnesota Libraries on September 24, 2011

MEASURING URBAN REGENERATION, PART 2

765

Table 6. Buildings and land use scoring performance Buildings and land use indicators Ratio of open space: built form Ratio of converted buildings: new build Reclamation of contaminated land Density levels in relation to plot size Balance of uses Occupancy levels Ofce rental v. CBD rents Quality of nal product Design quality Quality of public space Usage of public space Quality of private space Total score Percentage score Weighted score (weighting factor 18.9) Ranking LS 10 2 10 8 8 8 8 8 8 6 6 6 88 73 1663.2 3 CQ 6 10 10 10 6 6 8 7 6 6 8 4 87 73 1644.3 4 DD 10 4 10 10 4 10 8 8 7 5 6 5 87 73 1644.3 4 TB 4 10 10 8 10 8 2 6 9 4 6 3 80 67 1512.0 6 OV 10 2 10 8 8 6 10 8 9 8 8 7 94 78 1776.6 1 CV 6 6 10 10 10 10 8 7 7 5 6 4 89 74 1682.1 2

perspective, shown by the rankings, is the perception of poorer quality private space in four of the case studies, the exceptions being the Olympic Village and Laganside. Transport and mobility. Transport and mobility indicators encapsulate matters pertaining to infrastructure improvement, travel habits in relation to a range of activities and car parking provision. This indicator set produced a varied performance across the case studies, with only the Olympic Village (73 per cent) and Temple Bar (70 per cent) producing high scores reecting their excellent internal accessibility and proximity advantages within their respective city centres (Table 7). The Dublin Docklands was ranked third (63 per cent) ahead of Cathedral Quarter, Belfast (62 per cent). Laganside (56 per cent) had the lowest score for Transport and mobility, closely followed by Ciutat Vella (58 per cent), making the Belfast case studies the weakest collective performers. Given that a gap of 17 per cent separates the top and bottom performers, there is ample evidence to suggest that the importance placed upon accessibility and discouraging private car use differs greatly between case-study locations. Those indicators measuring public trans-

port links and travelling habits in relation to leisure activities perform well. However, with the exceptions of Temple Bar, the Olympic Village and to lesser degree in Laganside, the land allocation dedicated specically for pedestrian space and movement is ranked low with the converse that too high a proportion of land is devoted to roads. For residential car parking provision, the scores reect that fewer spaces are allocated within schemes, whereas commercial car parking provision points towards an overprovision with the result that the public transport option becomes less lucrative. This raises particular concern in meeting sustainability objectives. Clearly, further deliverability is required across the case studies in integrated land-use and transport planning. Community benets. Community benets capture a highly variable range of indicators, with the performance of Temple Bar (78 per cent) signicantly ahead of the next-highestranked case studiesnamely, Ciutat Vella (67 per cent) and Olympic Village (66 per cent). The high score associated with Temple Bar reects attempts to foster close links with the community (Table 8). In comparison, the Dublin Docklands (54 per cent) had

Downloaded from usj.sagepub.com at Serials Records, University of Minnesota Libraries on September 24, 2011

766

LESLEY HEMPHILL ET AL.

Table 7. Transport and mobility scoring performance Transport and mobility indicators Land devoted to roads Land devoted to pedestrians Road improvements Work travelling habits Leisure travelling habits Public transport links Car-parking provisionresidential Car-parking provisioncommercial Integration of land use and public transport Total score Percentage score Weighted score (weighting factor 22.1) Ranking LS 4 6 8 4 6 8 6 4 5 51 56 1127.1 6 CQ 6 2 5 4 10 8 8 8 5 56 62 1237.6 4 DD 8 2 4 6 10 10 8 4 5 57 63 1259.7 3 TB 4 8 5 6 10 7 8 8 7 63 70 1392.3 2 OV 6 8 7 6 10 10 8 4 7 CV 4 2 5 6 10 8 6 4 7

66 52 73 58 1458.6 1149.2 1 5

Table 8. Community benets scoring performance Community benets indicators Access to open space Access to leisure facilities Access to retail facilities Access to educational needs Access to medical facilities Access to entertainment facilities Access to cultural facilities Access to housing On-site retail facilities LA21 effectiveness Community ownership Community group involvement Total score Percentage score Weighted score (weighting factor 20.00) Ranking LS 8 8 8 4 2 8 4 6 4 7 4 3 66 55 1320.0 5 CQ 8 6 10 8 6 8 4 4 5 4 5 1 69 58 1380.0 4 DD 8 8 8 4 4 6 4 6 5 6 5 1 65 54 1300.0 6 TB 10 10 10 8 4 10 10 6 8 6 6 5 93 78 1860.0 1 OV 10 10 4 6 10 8 4 7 5 8 5 2 CV 10 10 6 8 6 6 6 7 8 6 6 1

79 80 66 67 1580.0 1600.0 3 2

the lowest score followed by the two Belfast examples of Laganside (55 per cent) and Cathedral Quarter (58 per cent). For Dublin Docklands and Laganside, this in part reects the predominant emphasis upon a physical development programme particularly in the early stages of the regeneration process. The scores indicate that more attention needs to be placed upon improving community relations at the outset of regeneration schemes rather than being slowly addressed in later phases.

On an individual indicator basis, there is a low level of community group involvement and, at best, only partly developed community ownership of the schemes across all case studies reecting that, with the exception of the Ciutat Vella, none is a truly established residential community area. In terms of accessibility, the outcome is variable. Certain facilities such as retailing, leisure and entertainment perform at reasonably encouraging levels in terms of sustainability principles, although more alarmingly

Downloaded from usj.sagepub.com at Serials Records, University of Minnesota Libraries on September 24, 2011

MEASURING URBAN REGENERATION, PART 2

767

Table 9. Overall points scoring summary Case-study location Laganside Cathedral Quarter Dublin Docklands Temple Bar Olympic Village Ciutat Vella Weighted score 6460.8 6387.9 6494.0 6741.3 7249.2 6804.8 Percentage score 62 62 63 65 70 66 Scaling Good Good Good Good Goodexcellent Good Overall ranking (5) (6) (4) (3) (1) (2)

others such as education and medical facilities produced a less satisfactory result apart from Olympic Village. Likewise, the fact that accommodation in these areas has become sought after and hence is highly priced can exclude lower-income households. This is apparent in the variable performance of the housing accessibility indicator and raises social equity issues concerningfor example, housing affordability. This grouping of indicators shows that out-turn performance is skewed more towards the remunerative deliverables, whereas the community components lag considerably with the result that sustainability objectives are not being fully embraced. Benchmarking Analysis The overall weighted scores, the summation of the product of the total points times the weighting factor, permit an analysis of each case study against the theoretical maximum possible score of 10 382. The median score of 5191 is taken to represent average performance with the weighted points band (4152.8 to 5078.18) in the decile below the median considered to be of below-average performance. A case study with a weighted total below 40 per cent of the maximum possible score is rated as poor in relation to sustainability practice. However, as all six case studies were selected on the basis of recognised or potential achievements in regeneration, it is important to have a better discrimination of weighted scores above the median. Again, deciles permit a rational categorisation of scores into nominal groups with the band 5059 per cent (weighted score

5191.0 to 6125.38) indicating above-average performance, 6069 per cent (6229.2 to 7163.58) classied as good, 7079 per cent (7267.4 to 8201.78) goodexcellent and above 80 per cent (weighted score 8305.6) excellent. However, the spread of scores and associated banding have sufcient distinction also to enable application of the model to areas considered to be failing from a regeneration perspective. Each of the six case studies received high total scores and can be classied as falling within the good category, with the Olympic Village Barcelona achieving the distinction of falling within the goodexcellent category. A relatively small percentage (8 per cent) separates the top and bottom performers, reecting the exemplar status of these three cities in terms of regeneration. On the basis of this analysis, the top-ranked casestudy location is the Olympic Village with 70 per cent (weighted score 7249.2) and Ciutat Vella has the next-best performance with 66 per cent (weighted score 6804.8). Temple Bar ranks third with 65 per cent (weighted score 6741.3). Outside the top three case studies in terms of sustainability performance, the difference between the scores is small. Dublin Docklands (63 per cent, weighted score 6494) just edges out Laganside (62 per cent, weighted score 6460.8) for fourth place, whilst the Cathedral Quarter (62 per cent, weighted score 6387.9) although in the bottom position is still classied in sustainability terms as good (Table 9). It is the weighted score that an area or project receives that is important. The ndings illustrate that the model is capable of discriminating between compara-

Downloaded from usj.sagepub.com at Serials Records, University of Minnesota Libraries on September 24, 2011

768

LESLEY HEMPHILL ET AL.

tive overall performance levels on a casestudy basis. Signicantly, each of the case studies conrmed that sustainability is becoming a more integral part of regeneration practice, with none of the areas below the good category. Furthermore, no noticeable pattern was established between those case studies undertaken in waterfront locations compared with the more culturally based projects; rather, differences are more related to the city than dened locations within a city. The case studies that have been completed or those nearing completion provided the best performances, notably the Olympic Village (70 per cent), Ciutat Vella (66 per cent) and Temple Bar (65 per cent), as these areas benet from more established markets and greater investor, resident and employee condence. This suggests that the performance levels of the other areas should improve as schemes mature. In this respect, Dublin Docklands (63 per cent), Laganside (62 per cent) and Cathedral Quarter (62 per cent) have performed well given these constraints and just lag the performance levels set by the top-ranked case studies. Sensitivity Analysis The regeneration characteristics evaluated by the indicators are complex, with a degree of qualitative evidence from specic expert groups as discussed in the accompanying paper. Hence, sensitivity analysis is deployed to allow for any uncertainty and to analyse the effect that a change in the scores may have on the overall result. Sensitivity simulations are undertaken whereby higher and lower variations of the nal scores of several individual indicators were chosen to test what impact, if any, this would exert on the overall nal points scoring. This demonstrates how sensitive the results are to a change in the individual indicator scores and indicates whether or not a particular case study should be reallocated to another performance level on the benchmark scale. The lower score variation was taken from alternative pessimistic responses received in the

data collection phase, substituting the original score with the lower score where any difference occurred, with the converse occurring in the case of the higher variation. Both sets of scores provide an alternative view of performance and represent the extreme of the distribution of scores. The results of the sensitivity analysis illustrate the effect that a change in the points allocated to any indicator would have on the weighted scores and relative ranking of the case studies. In terms of the low variation, some case-study areas proved to be more sensitive to a change in points total than others (Table 10). For example, the Olympic Village shows the smallest percentage drop (4 per cent) in the overall performance level indicating that it is the least sensitive to a fall in points total. Further conrmation of top ranking for Olympic Village (66 per cent) is illustrated by an increase in the gap (to 6 per cent) to the next-highest-scored location, Ciutat Vella (60 per cent), on the basis of lowest score variation. The largest drop in points was experienced by Laganside and Cathedral Quarter (8 per cent), followed by Temple Bar (7 per cent), resulting in both Belfast case studies displaying a similar performance level (54 per cent of possible weighted score) and conrming their relative status as the weakest performers, although still above average. The utilisation of the low score variation also shows a narrowing of the gap between the Dublin case studies to 1 per cent, demonstrating that the Temple Bar results were more sensitive to change than the Docklands (Table 10). Perhaps the most signicant result of applying the low score variation is the effect on the overall ranking. In all cases, the application of the low score variation saw a reallocation of the case studies from the good category to the lower above average tier of the benchmark scale and the top-ranked Olympic Village area (6823.9) was reallocated to the upper half of the good category. Of the case studies reallocated to the above average category, Ciutat Vella (6195.7), Temple Bar (5982.8) and the Dublin Docklands (5948.3) are in the upper

Downloaded from usj.sagepub.com at Serials Records, University of Minnesota Libraries on September 24, 2011

MEASURING URBAN REGENERATION, PART 2

769

Table 10. Comparison and ranking of sensitivity analysis (low variation) Case-study location Laganside Cathedral Quarter Dublin Docklands Temple Bar Olympic Village Ciutat Vella Weighted Percentage Percentage score score difference 5588.5 5485.5 5948.3 5982.8 6823.9 6195.7 54 54 57 58 66 60 -8 -8 -6 -7 -4 -6 Scaling category Averagegood Averagegood Averagegood Averagegood Good Averagegood Original ranking (5) (6) (4) (3) (1) (2) Low ranking (5) (6) (4) (3) (1) (2)

Table 11. Comparison and ranking of sensitivity analysis (high variation) Case study location Weighted Percentage Percentage score score difference Laganside Cathedral Quarter Dublin Docklands Temple Bar Olympic Village Ciutat Vella 7003.9 7164.7 6788.0 7115.8 7849.9 7251.3 68 69 65 69 76 70 6 7 2 4 6 4 Scaling category Good Good Good Good Goodexcellent Goodexcellent Original ranking (5) (6) (4) (3) (1) (2) High ranking (5) (3) (6) (4) (1) (2)

half, whilst Laganside (5588.5) and Cathedral Quarter (5485.5) are placed in the middle to lower part (Table 10). The analysis suggests that the overall scores awarded for the case studies are sensitive to a downward change in points total, with all six experiencing a reallocation to the next lower category. Concerning the high score variation, some case studies demonstrated a greater sensitivity to points change than others (Table 11). The Dublin Docklands showed the smallest increase (2 per cent) in points total, followed by Temple Bar and Ciutat Vella (4 per cent), illustrating that the Docklands is the least sensitive to a rise in points total from the high score variation. Cathedral Quarter (7 per cent), the Olympic Village and Laganside (6 per cent) experience the greatest increase in points. The analysis based on a high score variation establishes Cathedral Quarter (69 per cent), demonstrating its latent potential, as the third-ranked case-study area behind the Olympic Village (76 per cent) and Ciutat Vella (70 per cent), which consolidate their positions at the top. The application of the high score variation moved Ciutat Vella (7251.3) to the margins

of the goodexcellent tier with the other case studies consolidating their position within the respective categories, notably the Olympic Village (7849.9), Cathedral Quarter (7164.7), Temple Bar (7115.8) and Laganside (7003.9). The Dublin Docklands experienced little change in weighted points total, with only a slight improvement to the mid position of the good category (Table 11). The overall scores awarded for the case studies have proved to be less sensitive to an upward change in points total, suggesting that respondents may have adopted optimistic assessments of regeneration areas in their original response to particular indicators. As none of the case studies experienced greater than a 7 per cent change in the points total allocated by selecting either the high or low score variations, it can be concluded that the original scores awarded can be treated with a high degree of condence. Furthermore, the overall rankings developed from the sensitivity analysis showed very little movement, with the exception of the Cathedral Quarter case, which experienced a major upward change attributable to its inherent potential. The outcome offers con-

Downloaded from usj.sagepub.com at Serials Records, University of Minnesota Libraries on September 24, 2011

770

LESLEY HEMPHILL ET AL.

siderable support to the scores allocated and the rankings developed as well as the reliability of the data sources used. Conclusions Within urban regeneration policy and practice, the application of performance indicators is becoming increasingly signicant in measuring sustainability outputs. However, the use of methodologies to benchmark and assess the performance of sustainability remains largely underresearched. Weighted indicators and a points scoring framework developed in the preceding paper are applied to both waterfront-led and culture-led regeneration schemes in three European cities to measure achievements in the delivery of sustainable developments/environments. The study demonstrates that the more established and mature regeneration locations (Olympic Village, Ciutat Vella and Temple Bar) deliver better sustainability performance than those areas which are less well developed, highlighting the importance of timing in regeneration areas. The use of sensitivity analysis to test the various options, particularly for those indicators relying on qualitative evidence, indicates that the points score allocated to the various case studies in the empirical investigation can be treated with condence. Signicantly, the analysis shows that both of the Barcelona case studies, the Olympic Village and Ciutat Vella, and Temple Bar, Dublin, display a high degree of adherence to sustainability principles. A key parameter of the hierarchical model developed in the accompanying paper and tested in this analysis is its inclusiveness in terms of 5 indicator sets in the components tier and 52 separate indicators. The robustness of the model is demonstrated by its application in different regeneration typologies (waterfronts and cultural quarters) and in different urban contexts in terms of city characteristics, institutional structures and policy frameworks. Development of the model through the use of expert opinion is a key aspect of the methodological approach, the comprehensive nature of which allows

reliance to be placed upon the weights ascribed to the components tier. In addition, the model has exibility in that parameters can be changed to reect new standards arising from shifts in policy or technological developments, thereby contributing to its effectiveness in the evaluation of mechanisms in creating sustainable urban environments. The weightings applied to the respective components enhance the utility of the model as a comparative tool in the analysis of sustainability practice and outcomes. In this respect, the model is capable of application across regeneration areas and projects of differing size, scale, level of perceived success/ delivery and at different phases of a particular initiative. Indeed, the model is useable as a tool to test at different stages the degree to which a particular regeneration project is adhering to the principles of sustainability. Thus, at strategic points over the life cycle of the project, the model could be utilised to inform and inuence strategy in a corrective capacity by highlighting any signicant divergence from sustainability objectives. Ideally, a project/regeneration area should be surveyed before mid term to ensure that sufcient time is available to implement change if scores are low implying that the scheme that is unlikely to achieve sustainability targets. In theory, resurvey could potentially occur at several stages over the period of a project as well as forming part of the overall evaluation at the end of the initiative. Potential policy benets stem from the models holistic approach and its applicability across different regeneration mechanisms. In assessing the extent to which regeneration areas are conforming to sustainability criteria, the model provides a valuable tool in policy analysis. Application, as demonstrated by the case studies, illustrates where good practice is occurringnotably, in the Barcelona examples and particularly the Olympic Village. An added feature and potentially of greater signicance is the ability of the model to operate at a components level which, from a policy perspective, discriminates between those aspects where the regen-

Downloaded from usj.sagepub.com at Serials Records, University of Minnesota Libraries on September 24, 2011

MEASURING URBAN REGENERATION, PART 2

771

eration initiative is working well and those matters which require further policy intervention. From the analysis contained in this paper, the strong performance of Laganside (Belfast) in terms of Economy and work is highlighted, but equally apparent is the weaker performance in relation to Community benets and Transport and mobility. Hence, policy priorities in Laganside need to be redirected to the latter issues, whereas in the case of Temple Bar (Dublin) the model highlights a converse situation. The capacity of the model to drill down and examine components is important from a policy and decision-making perspective, particularly in the allocation and targeting of funds and other resources by allowing policy-makers to identify areas requiring action. This in turn offers the possibility to inuence future performance and guide policy and practice. On a wider perspective, the applicability of performance indicators allows policy approaches to be assessed on their ability to deliver sustainable development and assess whether partnership structures are effective. Regeneration performance outputs are increasingly expected to show joined-up holistic thinking across programmes by promoting physical, social, economic and environmental criteria. The indicator-based model advocated and developed in these two papers allows an evaluation of the extent to which integration is being achieved and sustainability objectives delivered within urban regeneration.

References
AUDIT COMMISSION (2000) Getting Better All the Time: Making Benchmarking Work. London: HMSO. BERRY, J. N. and MCGREAL, W. S. (1992) Fiscal mechanisms and inner city development, Journal of Property Finance, 3(4), pp. 543552. BERRY, J. N. and MCGREAL, W. S. (1993) Public sector initiatives in the regeneration of Belfast, in: J. N. Berry, W. S. MCGREAL and W. DEDDIS (Eds) Urban Regeneration, Property Investment and Development, pp. 193214. London: E&FN Spon. BRISTOW, H. (1999) Developing the tools to mea-

sure success, Urban Environment Today, 62(21 January), p. 13. BROWNHILL, D. and RAO, S. (2002) A Sustainability Checklist for Developments: A Common Framework for Developers and Local Authorities. Watford: CRC Ltd. BURNS, R. B. (2000) Introduction to Research Methods. London: Sage Publications Ltd. CARLEY, M. and KIRK, K. (1998) Sustainable by 2020? A Strategic Approach to Urban Regeneration for Britains Cities. Bristol: The Policy Press, for the Joseph Rowntree Foundation. DDDA (DUBLIN DOCKLANDS DEVELOPMENT AUTHORITY) (1997) Dublin Docklands AreaMaster Plan 1997. Dublin: DDDA. DETR (DEPARTMENT OF THE ENVIRONMENT, TRANSPORT AND THE REGIONS) (1998a) Sustainable Regeneration Good Practice Guide. London: HMSO. DETR (1998b) Planning for the Communities of the Future. London: HMSO. DETR (1998c) Planning for Sustainable Development: Towards Better Practice. London: HMSO. DETR (1998d) Sustainability Counts. London: HMSO. DETR (1999a) Sustainable Development Strategy for the UK: A Better Quality of Life. London: HMSO. DETR (1999b) Towards an Urban Renaissance: Final Report of the Urban Task Force. London: DETR. DETR (2000) Our Towns and Cities: The Future Delivering an Urban Renaissance. London: HMSO. DOE (DEPARTMENT OF THE ENVIRONMENT) (1994) Sustainable Development: The UK Strategy. Cm 2426. London: HMSO. EUROPEAN COMMISSION DGXI (1996) European Sustainable Cities. Expert Group on the Urban Environment. Luxembourg: European Commission. GAHAN, J. (1993) Fiscal measures in promoting urban regeneration: Custom House Docks, Dublin, in: J. N. BERRY, W. S. MCGREAL and W. DEDDIS (Eds) Urban Regeneration, Property Investment and Development, pp. 95107. London: E&FN Spon. HAKIM, C. (2000) Research Design: Successful Designs for Social and Economic Research. London: Routledge. IMRIE, R. and THOMAS, H. (1995) Changes in Local Governance and their Implications for Urban Policy Evaluation, in: R. HAMBLETON and H. THOMAS (Eds) Urban Policy Evaluation: Challenge and Change, pp. 123138. London: Paul Chapman Publishing. MONTGOMERY, J. (1995) The story of Temple Bar: creating Dublins cultural quarter, Planning Practice and Research, 10(2) pp. 135172.

Downloaded from usj.sagepub.com at Serials Records, University of Minnesota Libraries on September 24, 2011

772

LESLEY HEMPHILL ET AL.

ODPM (OFFICE OF THE DEPUTY PRIME MINISTER) (2002) Best Value Performance Indicators 2002/2003. London: ODPM. OECD (ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT) (2000) Urban Renaissance: Belfasts Lessons for Policy and Partnership. Paris: OECD. SUDMAN, S. (1976) Applied Sampling. New York: Academic Press.

TUROK, I. (1989) Evaluation and understanding in local economic policy, Urban Studies, 26, pp. 587606. WONG, C. (2002) Developing indicators to inform local economic development in England, Urban Studies, 39(10), pp. 18331863. YIN, R. K. (1984) Case Study Research: Design and Methods. London: Sage.

Downloaded from usj.sagepub.com at Serials Records, University of Minnesota Libraries on September 24, 2011