International Journal of Architecture, Engineering and Construction Vol 1, No 4, December 2012, 199-212

A Framework for Performance Assessment of Organizations in the Construction Industry

Tarek Zayed1, , Emad Elwakil2 , Mohammad Ammar3
1

Building, Civil, and Environmental Engineering Department, Concordia University, Montreal, Quebec H3G 1M8, Canada

Department of Civil Engineering and Construction Management, California State University, Northridge, CA 90802, United States

Structural Engineering Department, Faculty of Engineering, Tanta University, Tanta, Egypt

Abstract: Competition and customer needs forced construction companies (organizations) to measure their performance beyond the nancial aspects. Success, the main driver of any organization, depends mainly on a variety of factors that impact organizations performance. Predicting the performance of a construction organization helps in identifying the weak points and in searching solutions to improve its performance and increase prot. Due to the diversity and complexity of construction organizations, it is intricate to adopt a scientic strategy that measures their success. Previous research works showed a lack of attention towards modeling organizations non-nancial performance while focusing on measures of project success. Therefore, the objective of the present research is to identify and study the success factors and to develop performance prediction model(s) for construction organizations. The potential success factors are collected from literature and practitioners through a questionnaire that is prepared and sent to evaluate the eect of these potential success factors on organizational performance. The collected data are analyzed using articial neural network (ANN) to determine the most signicant (critical) success factors. Two performance prediction models are developed using ANN and regression analysis, which show robust results when veried and tested. The analysis shows that the developed models are sensitive to the identied critical success factors. Keywords: Construction Industry, organization performance, critical success factors, articial neural networks, regression analysis DOI: 10.7492/IJAEC.2012.022 increase prot. Kaplan and Norton (1993) concluded that it is critical to measure and control new strategies Globalized competition and customer needs forced after they have been put to work at the operational construction companies to assess their performance phase of construction organizations. In measuring the beyond the nancial measures, i.e., protability; performance of a business on executive monitoring systurnover, etc (Isik et al. 2010). Prot and success tem, Holophan (1992) proved that critical success facare considered the main drivers of any organization. tors (CSFs) are the best methodology to develop an Achieving success depends on many factors which have executive monitoring system to contain corporate-wide direct eect on the performance of organizations. In indicators of success, which is the main goal of any orthe construction industry, it is even more dicult to ganization. Many research eorts have been done to adopt or maintain a scientic strategy to measure suc- determine the success factors. However, many of these cess due to the diversity and complexity of construc- studies have been done at the project level rather than tion organizations (Abraham 2002). Predicting perfor- at the organizational level. mance of construction organizations helps dene weak The determination of success factors was approached points in order to improve their performance and to through various methods. The most commonly utilized *Corresponding author. Email: zayed@encs.concordia.ca
199

1 INTRODUCTION

Zayed et al./International Journal of Architecture, Engineering and Construction 1 (2012) 199-212

methods are questionnaires and interviews with technical persons and professionals. Most of the construction organizational success factors are qualitative in nature rather than quantitative. To accomplish success of a construction organization, there is a need to determine these success factors, which can then be used later to predict and improve organizational performance. Modeling the performance of construction organizations from nancial prospective are extensively researched, however, modeling the performance considering non-nancial aspects does not receive sucient attention from researchers. Therefore, the scope of the present study is to investigate the most signicant organizational success factors with focus on construction organizations and develop a framework for predicting organizational performance. This paper is organized as follows: a comprehensive literature review on success factors followed by a discussion of the previous modeling work that has been carried-out. Extensive eorts are done to collect data from various construction organizations. The collected data are used to develop two organizational performance models using articial neural network (ANN) and regression analysis. The developed models are tested and veried.

2 RESEARCH OBJECTIVES
The objectives of the present study are summarized as follows: 1. Identify and study the success factors for the construction industry at organizational level. 2. Determine the signicant factors (i.e., critical) that mostly contribute to the organization performance.d bad weather. 3. Develop organizational performance model(s)/ framework(s) based on these critical success factors.

3 BACKGROUND
Organization as a term is dened, according to the American Heritage College Dictionary (1993), as: An organization is a structure through which individuals cooperate systematically to conduct business. Aldrich (1979) described the organization as a series of Goal directed boundary-maintaining activity systems. The denition of success has been changed and more developed to include quality as an indication of success. The project success diers from one person to another and the success criteria change according to the project itself. However, success is mostly dened as the overall achievement of project/organization goals and expectations (Part and Sanvido 1993). Manufacturing, communication, and other industries used the revenue growth rate over a period of time as a factor

to determine success. The previous denitions of success are project oriented. Moreover, success factors, quantitative or qualitative, are not easily measurable. Success factors as a concept and their applications to business are not new; it dated back to 1960s by Daniel (1961). Rockart (1982) described the principles of success factors to make a systematic way of identifying the needed information for executives. Organizational performance comprises the actual output of an organization as measured against its intended/planned outputs (i.e., goals and objectives). In addition, the success of organizations is less dependent on attractiveness of its industry or countrys environment; however, it is more reliant on rm-specic factors that control its competitive advantage (Hawawini et al. 2004). Companies need to expand their thinking beyond national borders when it comes to competition, capabilities, and customers (Wethyavivorn et al. 2009). Globalization methodologies aect the construction industry in many ways. It can facilitate a markets expansion from a local market to an international one. A single company can have projects in dierent countries; for example, Venezuela, Algeria, China and United Arab Emirates. As a consequence of this expansion, construction organization faces new competitors from dierent countries (Shurchuluu 2002). Rockart and Bullen (1981) dierentiated between the goals and success factors through the traditional strategic planning and management, where goals and objectives are fairly well known but success factors denition is much less clear. However, the organization goals are dened to be the targets established to achieve the organizations mission. They are very specic and well known what to accomplish, when to be achieved, and by whom in a quantitative measurable approach (Richard 2004). Rockart (1979, Rockart (1982) identied four prime sources of success factors for any organization working in any industry, which are: (1) structure of the industry, (2) competitive strategy, industry position, and geographic location, (3) environmental factors, and (4) temporal factors. Ahmad and Dye (1994) addressed the common essential attributes of business organizations throughout an extensive literature. These attributes should be taken into consideration while analyzing organization performance. The organizational performance criteria and their factors are summarized by Caballero and Dye (1999). They include several main criteria, such as business experience, personnel, and nancial. The factors that impact each criterion are also listed, such as the number of years in business and the type of business. There are also several factors that impact personnel criterion, such as the number of full time employees, average length of time employed, the ratio of supervisors to workers, the level of training for supervisors, etc. In addition, annual revenues, liquid assets, dollar value of lines of credit, and aging of receivables mainly

200

Zayed et al./International Journal of Architecture, Engineering and Construction 1 (2012) 199-212

aect the nancial criterion. Several nancial and nonnancial success factors are identied and reviewed in this study.

3.1 Success Factors in Construction Organizations

According to Jaselskis et al. (1996), safety was considered a success factor to achieve excellence in construction. Abraham (2002) adopted the approach of Pollalis and Frieze (1993) of combining the latest strategic management theory: the seven guiding principles of strategic management for civil engineering (Chinowsky and Meredith 2000) with the latest success factors methodology for IT organizations (Rockart et al. 1996). Abraham (2002) added a third dimension and incorporated information from organizational behavior theory, specically the characteristics of an organization, with the two knowledge domains. Camp (1995) provided a list of the most important internal business processes that might be considered when evaluating rms performance against other competitors. The list includes: market management, product design and engineering, product operations, customer engagement, logistics and inventory management, product maintenance, business management, information and technology management, nancial management, and human resource management. The rst six factors are considered operational business processes while the rest are support business processes. In practice, managers are encouraged to select measures from three categories: customer, internal business processes and learning and growth in order to assess organization performance (Kaplan and Norton 1996).

In the construction industry, performance indicators are used to establish a new conceptual framework for performance management in construction (Kagioglou et al. 2001), to conduct performance measurement of construction rms in developing countries (Luu et al. 2008), and to measure proper performance in construction (Bassioni et al. 2004; Bassioni et al. 2005). According to Abraham (2002), a proposed list of success factors for construction organizations was developed from the characteristics of organizations, organizational behavior theory, and the seven guiding principles of strategic management utilizing the Pollalis and Frieze (1993) approach for the IT organizations. The methodology followed by Pollalis and Frieze (1993) was quantitative (survey-based research) while Rockart et al. (1996) and Martin (1997) used a qualitative methodology (interviews with top executives in the industry). The objective of quantitative methodology was to isolate categories before the study was undertaken as precisely as possible. However, the qualitative one depicted the nature and the denition of categories in the course of a project McCracken (1988). Then, Abraham (2002) combined the two methodologies together in order to benet and capture the advantages of both. Chinowsky (2001) identied seven guiding principles of strategic management for civil engineering industry according to the results of interviews with civil engineering, construction, and public agency executives; which included Vision, Mission, Goals, Core Competencies, Knowledge Resources, Education, Finance, Markets, and Competition. The list was not inclusive for all construction organizations; however, organizations should consider continuous updates to success

Table 1. Suggested success factors for construction originations and sample of the collected raw data
Category Administrative and Legal Success Factors Responses (Scale: 1-5) Sample #1 1. Clear Vision, Mission and Goals 2. Competition Strategy 3. Organizational Structure 4. Political Conditions 5. Number of Full Time Employees Technical 6. Usage of International Aspects (ISO) 7. Availability of knowledge 8. Usage of IT 9. Business Experience (no. of years) 10. Product Maintenance Management 11. Employee Culture Environment 12. Employee Compensation and Motivation 13. Applying Total Quality Management 14. Training Market and Finance 15. Quick Liquid Assets 16. Feedback Evaluation 17. Research and Development 18. Market Conditions/Customer Engagement Overall Company Performance (%) 5 3 5 4 5 3 4 5 4 2 5 5 3 3 3 4 5 5 70 Sample #2 5 5 5 4 5 4 4 5 4 3 4 4 4 4 4 4 5 5 80

201

Zayed et al./International Journal of Architecture, Engineering and Construction 1 (2012) 199-212

Figure 1. Typical example of a simple ANN architecture factors as they can be changed with time according to environmental issues, transformation of industry, and variations in competitive strategy. Abraham (2002) revealed eleven success factors for construction organizations. These factors comprised: competitive strategy, market conditions, political environment, organizational structure, technical applications, employee enhancements, process benchmarking, feedback and evaluation, inter organizational relationships, environmental factors, and management skills and relationships. Based on the above review of literature and focusing on construction organizations, the most related success factors to construction industry are selected as shown in Table 1. These factors are considered in the present study. the network, the weights for nodes are generated randomly. BPNNs support a contribution factor module which produces a number for each input parameter called a contribution factor that is a rough measure of the importance of such variable in predicting networks output, relative to the other input parameters in the same network. The higher the number, the more the variable is contributing to prediction or classication. The ANN analysis considers the non-linear interaction relationship among input variables (Zayed and Halpin 2005a). Based upon the above literature review, it is noticed that assessing the construction organization from nonnancial perspectives does not receive much attention from researchers in the eld. Most research focused on the performance in the project level not the organization level leaving a research gap in this area, which pave the way to the present research objectives.

3.2 Articial Neural Networks

Articial Neural Network (ANN) attempts to model the brain learning, thinking, storage, and retrieval of information, as well as associative recognition. The ANN consists of neurons usually organized into a sequence of layers with full or partial connections among successive layers (Moselhi et al. 1991). Figure 1 shows typical three-layer network architecture. Similar to the human thinking and decision making, an ANN takes previously trained problems to build a system of neurons that makes new decisions, classications and forecasts. The ANN can learn patterns that are being presented during the training or learning phase. Throughout the course of training, ANN develops the ability to generalize, which correctly classify new patterns or to make forecasts and predictions (Zayed and Halpin 2005b). Back Propagation Neural Networks (BPNNs) are one of the most common ANN algorithms, as they are simple and eective and have found home in a wide assortment of machine learning applications. BPNNs start as a network of nodes arranged in three layers; input, hidden, and output (Zayed et al. 2005). The input and output layers serve as nodes to buer input and output for the model, respectively, while the hidden layer provides a means for input relations to be represented in the output. Before any data has been run through

4 RESEARCH METHODOLOGY
To achieve the objectives of the present research, several steps are accomplished as shown in the schematic diagram (Figure 2). These steps are summarized as follows: 1. Review success factors in general and specify the most common used ones for construction industry at the organizational level. Data regarding success factors and organization performance are collected from literature, practitioners, and various construction companies. 2. Determine the most signicant factors (i.e., critical) that contribute mostly to the organization performance through the opinion of experts in the construction industry who are working in top organizational level. Thus, critical success factors (CSFs) are identied and their eect on the success of construction organization is qualitatively/quantitatively studied and analyzed. 3. Develop two organizational performance predictors (models) based on the CSFs using both ANN and regression analyses. These analysis methods

202

Zayed et al./International Journal of Architecture, Engineering and Construction 1 (2012) 199-212

Figure 2. Schematic diagram of research methodology are selected because they are data oriented techniques that require the values of input and output variables to be available in order to accommodate their success in predicting organization performance. The collected data include the values of input and output variables. These data are then prepared to be ready for ANN and regression analyses. Organizational performance models are then developed and ready for testing. 4. Both models are tested and veried in order to check their robustness in assessing the performance of a construction organization. Sensitivity analysis is conducted to test the eect of changing the input variables on the organization performance. fect construction organization. The questionnaire was designed to identify factors that aect company performance and then, to predict this performance in an abstract approach. It had two parts where the rst part (I) was asking decision makers in construction organizations to ll-in questionnaires reecting their experience and company information. Part II was asking the experts, using a specied 5 point subjective scale as shown in Figure 3, to assess the eect of identied success factors on organization performance. In addition, the decision-maker was asked to assess the overall success of his/her construction organization using a value out of 100. One hundred and fty questionnaires were sent to top and middle management decision makers in construction organizations worldwide, i.e., Canada, Egypt, France, Greece, Germany, USA, Saudi Arabia and United Arab Emirates. The response rate was immense because sixty three responses were received with a response rate of 42%, which was sound in the construction industry. They were scattered as follows: Egypt (31 questionnaires), Canada (21 questionnaires), and other countries (11 questionnaires). Sample of the raw data of two responses obtained from the questionnaire is given in Table 1. It is noticed from these two responses that several factors are agreed upon to be the

5 DATA COLLECTION
After identifying the potential success factors that may aect the organization performance, a questionnaire was prepared to assess the eect of each factor on performance. These factors are collected from various literatures and from experts and practitioners. The factors are perceived by top managers to mostly af-

203

Zayed et al./International Journal of Architecture, Engineering and Construction 1 (2012) 199-212

Figure 3. Performance scale for success factors most signicant ones, such as Clear Vision, Mission & Goals, Structure, No. of Employee, Usage of IT, Research & Development , and Market Conditions. It is also noted that the experts evaluation for these two companies are 70 and 80 out of 100 for sample 1 and 2, respectively. These values are assessed based upon the given criteria and factors as well as experts judgment. might depend on the results of the present study to focus on the most critical factors. Therefore, the present study will be considered as an overview, an outline, or a framework that guides future researchers to the starting point(s) of detailed research. In other words, the present research assists in developing an overall framework/roadmap for research in this area in an abstract approach where detailed research is however required to study each CSF and its eect on organization performance.

Given the diversity of cultures present in each of these countries, the manner with which they make decisions and run their business diers, as does their evaluation of these aspects. However, due to the scarcity 6 DETERMINATION OF CRITICAL of available data and to the lack of information related SUCCESS FACTORS to companies internal records, the present study does not consider the diversity. This assumption can be There are several methods to assess the signicance treated vigorously in future studies where researchers of independent factors aecting the performance of a Table 2. Ranking and relative signicance of success factors
Rank 1* 2* 3* 4* 5* 6* 7* 8* 9* 10 11 12 13 14 15 16 17 18 Success Factors X1: Availability of knowledge X2: Clear Vision, Mission & Goals X3: Organizational Structures X4: Feedback Evaluations X5: Business Experience X6: Political Conditions X7: Research & Development X8: Employee Culture Environments X9: Competition Strategy Market Conditions/Customer Engagement Training No of Full Time Employees Product Maintenance Usage of IT Quick Liquid Assets Applying TQM Usage of International Aspects (ISO) Employee Compensation and Motivation Contribution Percentage 7.13 7.02 6.73 6.63 6.55 6.48 6.45 6.25 6.14 5.34 5.12 5.01 4.82 4.75 4.37 4.06 3.99 2.79 Dierence in Contribution Percentage 0.114 0.282 0.101 0.082 0.07 0.029 0.199 0.117 0.794 0.225 0.105 0.193 0.07 0.378 0.308 0.07 1.199

* Critical Success Factors (CSF).

204

Zayed et al./International Journal of Architecture, Engineering and Construction 1 (2012) 199-212

dependent criterion. In the present research, ANN is used to assess the most signicant success factors using Neuroshell software package. This is because the ANN software provides the contributing weight of each factor after completing the training process. Therefore, in the following sub-sections, details regarding these contributing weights are presented.

variable will have a high contribution percentage. In Table 2, the contribution percentage (relative signicance) of the 18 success factors are given. The success factors are ranked according to their contribution percentages. The dierences in contribution percentage between each two successive success factors are also given in Table 2. It is obvious that the dierence in contribution percent between success factors No. 9 and 10 is 0.794 which is about seven times the immediate 6.1 Articial Neural Network Training preceding dierence in contribution percent (0.117). Before the ANN is trained, training criteria must be Therefore, only the rst nine factors can be considspecied in advance, which include maximum and minered more signicant (i.e., critical) factors. These nine imum absolute errors and number of training cycles CSFs are utilized to develop two models that predict without improvements. The data space is divided into the overall performance of a construction organization. three sets: training, test, and production. The training set is used to train the network where the error with respect to the training cycles is calculated. The test set is 7 DEVELOPMENT OF used to test the network during development/training ORGANIZATION PERFORMANCE and to continuously correct it by adjusting the weights MODELS of network links in order to reduce the error. The production set is the part of data which is used to validate Both ANN and regression analysis are used to develop two prediction models for the performance of constructhe model(s). tion organizations. The two models are developed based on the previously specied/selected CSFs. They 6.2 Ranking of Input variables are developed in order to select the best model that The purpose of establishing a rank of input variables might represent the collected data. is to determine the relative importance of the variables and to identify those mostly aect organizational 7.1 ANN-Based Organization Performance performance. In this study, estimates of the contriModel bution of eighteen input (independent) variables are done. The contribution percentages are derived from Data on the selected CSFs are only used to train a new the analysis of weights generated from the trained ANN in order to obtain an ANN-based organization ANN. The higher the number, the more that variable performance model. Similar training criteria are used is contributing to the classication and/or prediction. (i.e., maximum and minimum absolute errors and numEvidently, if a certain variable is highly correlated, the ber of training cycles without improvements). Data

Figure 4. The ANN architecture for the developed model

205

Zayed et al./International Journal of Architecture, Engineering and Construction 1 (2012) 199-212

are divided into two randomly selected sets: training (84%) and validation (16%). The training data set is used to train the network, using Neuroshell software package, while validation data set is used to test the output of the trained models response to the inputs of this data set. In other words, the inputs of validation data set cases are introduced to the trained model in order to generate the predicted output. This output is then compared to the actual collected output values from experts. If they are similar or approximate, then the model is valid and vice versa. The selection of input and output variables greatly aects the ANN architecture. The selection of these variables depends basically upon the nature of the problem. As shown in Figure 4, the input variables include the previously selected CSFs, which are 9 main CSFs. Each variable is represented by one articial neuron in the networks input layer. The ANN has only one output neuron that represents the performance of a construction organization. Hence, to build the ANN architecture, there should be nine neurons in the input layer and one neuron in the output layer. The hidden layer relies on the available model building data set and the nature of outputs. Several iterations are used to generate the optimal number of neurons in the hidden layer. The ANN achieves the required goal (0.05) when the training network has 30 neurons in the hidden layer, learning rate of 0.05, and 2000 number of epochs. Therefore, the best architecture for ANN is 9 input, 30 hidden, and 1 output neurons. After the ANN is trained, it can be recalled to predict the organization performance for any given values of the CSFs. The training and testing processes are performed successfully with reasonable results in which the R2 of the ANN model equals to 0.81, the mean square error (MSE) equals to 0.147, and the mean absolute error equals to 0.196. These results show the robustness of the developed model using real world data, which are typically noisy.

7.2 Regression-Based Organization Performance Model

MINITAB is used to build a regression model for construction organizations performance as a function of the previously selected CSFs. MINITAB is a general purpose statistical package which provides a wide range of basic and advanced data analysis capabilities, such as analysis of variance, basic statistics, correlation and regression, and multivariate analysis (MINITAB 2006). Step-wise regression analysis is utilized to select the best number of variables to be included in the model. Four selection criteria are used to distinguish between dierent proposed models. These criteria are R-square, adjusted R-square, MSE, and Mallows Cp . The best model that can represent the collected data set is selected according to the largest R-square and adjusted R-square, the minimum MSE, and the closest Cp to the number of independent variables. Hence, if the selected model has only ve independent variables, the best model is the model with Cp value close to ve. All these criteria have been considered when selecting the best model to achieve the above requirements. Therefore, the selected model has the highest R2 of 0.79 and adjust R2 of 0.783, the Cp value of 8.2 close to 9 (i.e., number of variables), and the minimum MSE value of 2.2524. Organizational performance [Y ] is measured as the actual output of an organization measured against its intended/planned outputs (i.e., goals and objectives). The best obtained formula describing the organization performance [Y (%)] as a function of CSFs is given by Eq. (1): Y (%) = 62.7 + 1.93 X1 + 4.33 X2 2.02 X3 + 1.19 X4 + 3.83 X5 3.87 X6 + 1.83 X7 + 1.24 X8 2.68 X9

(1)

The dependent variable (Y ) denotes the organization performance expressed as a percentage and Xs denote

Table 3. Analysis of variance for the developed regression model

Source Regression Residual Error Total Factor X 1 2 3 4 5 6 7 8 9 DF 9 263 272 SS 9550.4 5337.3 14887.6 Predictor Availability of knowledge Clear Vision, Mission & Goals Organizational Structure Feedback Evaluation Business Experience Political Conditions Research & Development Employee Culture Environment Competition Strategy MS 1061.2 20.3 F 52.29 P 0.000

Coecient 1.9332 4.3319 -2.0214 1.1871 3.8339 -3.8647 1.8312 1.2373 -2.6753

P -value 0 0.028 0.686 0 0 0 0 0.694 0

206

Zayed et al./International Journal of Architecture, Engineering and Construction 1 (2012) 199-212

could be a weak evidence of null hypothesis for these particular coecients. However, due to large number of predictors in the model and due to satisfactory results of other statistical diagnostics, these results are considered acceptable for the model as a whole. In other words, the entire model has robust statistical diagnostic analysis with P -value of the F -test equals to zero. If there are several model parameters, which are not individually signicant, will not aect the prediction capabilities of the entire model. Attempts are conducted in order to remove X3 and X6 from the model and build a new model with only seven variable; however, this leads to unsatisfactory results statistically and logically. Therefore, the presented model is selected/considered although it fails the t-tests for two variables because the entire model is sound. Based on statistical tests, the designed regression model performs well in representing current data. Looking at the coecients of the regression model in To determine P (F ) for the entire model, a hypoth- Eq. (1), it is quite clear that the factor Clear Viesis test is carried out. The null hypothesis (H0 ) as- sion, Mission & Goals has the highest positive imsumes that all regression coecients, 0 , 1 ... p1 pact on company performance with a coecient value equal to zero, i.e., 0 = 1 = p1 = 0 and the al- of 4.33 followed by Business Experience with a coefternate hypothesis (Ha ) assumes that not all of them cient value of 3.83. Conversely, Political Conditions equal to zero. Based on the Minitabs output, shown has the highest negative impact on organization perforin Table 3, the P -value for the F -test is 0.00, which mance with a coecient value of -3.87 followed by the means that the null hypothesis is rejected. Similarly, complexity of Organizational Structure, which has a to determine the validity of regression coecient in- coecient value of -2.02. These values appear reasondividually, t-tests are performed separately for the able in which more details are discussed in the sensiindividual coecients: 0 , 1 ... p1 . In case of 0 , tivity analysis section of this paper. the null hypothesis (H0 ) of t-test assumes that 0 = 0; while alternative hypothesis (Ha) assumes that 0 = 0. 7.3 Validation of Developed Organizational Similarly, the other null hypothesis assumes that 1 = Performance Models 0 and vice versa. The results of these tests, for X1 , X2 , X4 , X5 , X7 , X8 , and X9 indicate that the P -value for The validation process is an important step to guarX1 is 0.028 and 0.00 for the rest as shown in Table 3. antee that the developed models best t the available As a result, alternative hypothesis is accepted with 95% data. All models are tested statistically, logically, and condence. However, the P -value for X3 and X6 are practically to determine whether they are ecient in 0.686 and 0.694, respectively. These results have P - predicting real world results. The collected data are values greater than 0.05, which indicates that there divided into two data sets, model building (80%) and Table 4. Validation results of the developed models
Case No. Company Performance (%) 1 2 3 4 5 6 7 8 9 10 80 70 87 82 90 80 75 75 75 80 ANN Model Predicted Performance (%) 82.2 74.38 83.23 82.8 80.17 82.36 77.54 82.78 75.89 75.02 AIP (%) = AVP (%) AIP 0.03 0.06 0.04 0.01 0.11 0.03 0.03 0.1 0.01 0.06 4.94 94.24 Regression Model Predicted Performance (%) 88.96 75.27 90.01 87.01 85.82 85.67 80.71 82.37 69.8 77.33 AIP (%) = AVP (%) AIP 0.11 0.08 0.03 0.06 0.05 0.07 0.08 0.1 0.07 0.03 6.77 93.23

the CSFs specied in Table 3 and the subscript refers to their numbers in the table. For example, X7 denotes CSF number 7 which is Research & Development. The developed model is checked for their statistical validity. The main diagnostics in this regard are R2 (coecient of multiple determination), F -test, and t-test for model coecients. The R2 measures the proportional variation in company performance explained by the nine CSFs. The coecient of multiple determinations (R2 ) is the most commonly used measure, which applies to multiple regression analysis. A perfect t would result in an R2 value of 1, a very good t near 1, and a very poor t near 0. The R2 value corresponding to Eq. (1) is 0.79, which shows a good degree of total variability considering the noisy in real world data. The R2 adjusted (0.783) accounts for the number of predictors in the model. Both values indicate that the model ts the data well.

207

Zayed et al./International Journal of Architecture, Engineering and Construction 1 (2012) 199-212

Table 5. Summary of sensitivity analysis results for the developed models

No. Critical Success Factor ANN Model Performance (%) Min. Max. Range value value (1) (2) (3)=(2)-(1) 81.08 79.89 87.33 78.82 81.02 77.95 81.79 76.32 73.34 84.15 91 91.45 87.67 85.56 79.03 87.87 86.23 74.98 3.07 11.11 4.13 8.85 4.54 1.08 6.08 9.91 1.64 Regression Model Performance (%) Min. Max. Range value value (4) (5) (6)=(5)-(4) 74.25 67.05 76 76.47 68.55 72.3 74.55 76.32 74.68 83.9 88.7 86.1 82.42 87.7 91.65 83.7 82.52 88.08 9.65 21.65 10.1 5.95 19.15 19.35 9.15 6.2 13.4

1 2 3 4 5 6 7 8 9

X1: X2: X3: X4: X5: X6: X7: X8: X9:

Availability of knowledge Clear Vision, Mission & Goals Organizational Structures Feedback Evaluations Business Experience Political Conditions Research & Development Employee Culture Environments Competition Strategy

validation (20%). The validation data set, which is selected randomly, is kept away while modeling the ANN and regression analysis. After building both models, the validation data set is utilized to test the ability of the developed models to predict the organization performance. The developed models are validated by comparing the predicted results with the actual values of the validation data set. Eqs. (2) and (3) are used to validate the developed models (Al-Barqawi and Zayed 2006) as follows:
n

both models, it is quite clear that they are comparable; however, the ANN model is more precise and generates better results, i.e., closer to real cases. Therefore, the ANN model is utilized to draw the performance curves in Figures 5 and 6 based upon sensitivity analysis results.

7.4 Sensitivity Analysis

To test the eect of each CSF on the organization performance, a sensitivity analysis is performed using the developed performance prediction models. Each factor is allowed to change within its scale limits (1-5) followed by predicting the organization performance using the developed models. The maximum and minimum values as well as the range of change in organization performance corresponding to the change in each CSF are given in Table 5. It is obvious form Table 5 that the organization performance is greatly aected by the change in the value of CSF number 2 (Clear Vision, Mission & Goals) for both ANN (range = 11.11%) and regression (range = 21.65%) models because it has the maximum range. On the other hand, the CSFs, which have the least eect on organization performance include CSF number 6 (Political Conditions) in ANN model (range = 1.08%) and CSF number 8 (Feedback Evaluations) in regression model (range = 5.95%). Based upon the sensitivity analysis of ANN model, performance curves are developed in order to measure the variation of the company performance versus the various CSFs as shown in Figures 5 and 6. Figure 5 shows the curves of various CSFs, which are directly related to company performance, i.e., the more the CSF value, the more the performance will be. On the other hand, Figure 6 shows the CSF, which are inversely related to the company performance. For example, Figure 5a shows the eect of CSF X1 (Availability of Knowledge) on company organization. It shows a positive eect in which the performance equals almost 84% when the value of knowledge equals 2 (moderately ineective) and 87% when the X1 value is 4 (moderately eective). The developed curves in Figure 5 show

AIP =
i=1

[1 (Ei /Ci )] 100/n

(2)

AV P = 100 AIP

(3)

where AIP is the Average Invalidity Percent, AVP is the Average Validity Percent, Ei is the ith predicted value, Ci is the ith actual value, and n is the number of observations. Eq. (2) expresses the average invalidity, which indicates the prediction error, while Eq. (3) presents the average validity percent. The AVP values for the developed performance prediction models using the ANN and regression are 94.24% and 93.23% as shown in Table 4, respectively. These values indicate that the obtained results are satisfactory. Table 4 also shows that the prediction of ANN model is closer to the actual data than that of regression model outputs. This is because the ANN model predicts the performance in 6 cases (i.e., organization numbers 1, 2, 4, 6, 7, and 9) closer to the actual performance than regression model outputs. However, the regression model predicts the performance of only 3 cases (i.e., organization numbers 3, 5, and 10) closer to the actual performance than the ANN model outputs. There are few organizations where the ANN model predicts the performance with discrepancy in which AIP equals to 11% and 10% for organization numbers 5 and 8, respectively. The case is similar in the regression model where it has discrepancy of AIP equals to 11% and 10% for organization numbers 1 and 8, respectively. Comparing the results of

208

Zayed et al./International Journal of Architecture, Engineering and Construction 1 (2012) 199-212

Figure 5. Factors that are directly related to company performance (positive slope) robust equations with R2 values above 95%. Similarly, Figure 6a shows the eect of CSF X3 (Organizational Structures) on company organization. It shows a negative eect in which the performance equals almost 89% when the value of knowledge equals 2 and 81% when the X3 value is 4. The developed curves in Figure 6 show robust equations with R2 values above 98%. The developed curves show that organization performance is directly related to Availability of Knowledge, Clear Vision, Mission & Goals, Feedback Evaluation, Business Experience, Research & Development, and Employee Culture Environment. It makes sense that the organization performance increases when increasing, for example, the availability of knowledge to organizations employees or system. It also makes sense to increase performance when the companys vision, mission, and goals are very clear to all management levels and employees. Similarly, the organization performance increases by increasing experience, percentage of research and development, and continuous accommodation of the feedback from employees and customers. On the contrary, the organi-

209

Zayed et al./International Journal of Architecture, Engineering and Construction 1 (2012) 199-212

Figure 6. Factors that are inversely related to company performance (negative slope) zation performance is inversely related to Organizational Structure, Political Conditions, and Competition Strategy. This also makes sense because if an organization accommodates a complex structure, attains unstable political conditions, and has many competitors or defective strategy, this negatively impacts its performance. Therefore, based on the aforementioned tests and discussions, the developed curves are statistically and logically sound. Based upon the above discussion, it is concluded that the developed models are robust, realistic, and practical in predicting construction organization performance and its trend towards various CSF. The CSF can be used as indicators of the organization performance and the developed charts in Figures 5 and 6 can be utilized as gauges for performance improvement/drop. In addition, the developed models can be implemented as a conceptual assessment tool for the entire performance of a company, then, detailed investigation is warranted when needed. More details of the critical success factors as well as extra factors should be vigorously researched and analyzed in future research activities on organization performance. The developed models are deemed essential to top management of construction organizations given that they provide them with the
210

means to assess their organizations performance and measure/gauge how to improve such performance.

8 STUDY LIMITATIONS
The presented study in this paper comprises several limitations as follows: 1. Only nine main factors are considered when developing the intended models to predict the overall performance of a construction organization. Many other sub-factors are not also considered in this study. 2. Construction organization characteristics and structures are not considered in this study as it is in the exploration stage of this area. 3. The collected data rely only on experts opinion without considering quantitative data related to the performance of construction organization. This mainly results from the scarcity of available data and condentiality of such data. 4. The risk for bias is not measured nor considered when dealing with experts who are evaluating their organization. The experts might be pro or against their organization which will aect their

Zayed et al./International Journal of Architecture, Engineering and Construction 1 (2012) 199-212

assessment. This bias is not treated in the current study. The abovementioned limitations will be considered for future studies in order to improve the developed performance assessment models. Therefore, the presented study is only considered as a discovery for the area of construction organization performance, its limitation, research gaps, diculties, and problems that need solutions.

9 SUMMARY AND CONCLUSIONS

Construction is a highly competitive industry nowadays. Achieving success depends on many factors which aect the performance of construction organizations. It is dicult to measure the non-nancial performance of construction organizations due to their diversity and complexity. A multi-dimensional study on performance of construction organizations has been conducted using sixty three surveys obtained from various construction organizations worldwide. The collected data are analyzed using ANN to determine the relative signicance of various success criteria (i.e., critical success factors). These specied CSFs are used in turn to develop ANN and regression-based performance models in order to predict the performance of construction organizations. The developed models are validated and compared using the average validity percent where the results found were satisfactory, i.e., AVP = 94.24% and 93.23% for ANN and regression models, respectively. The developed models benet both researchers and practitioners because they provide academics with robust ANN and regression models or frameworks that advance the state of the art of predicting construction organization performance. They also provide practitioners with the various essential factors aecting organization performance and the techniques with which it can be assessed or predicted, i.e., performance assessment and prediction tools.

REFERENCES
Abraham, G. (2002). Identication of Critical Success Factors for Construction Organizations in the Architectural/Engineering/Construction (A/E/C) Industry. Ph.D. Thesis, Georgia Institute of Technology, Atlanta, United States. Ahmad, I. and Dye, J. (1994). Development of a Database of MBE/DBE Firms and Decision Model to Determine their Capacity for the Florida Construction Industry. Technical Report No. 120, Department of Construction Management, Florida International University, United States. Al-Barqawi, H. and Zayed, T. (2006). Condition rating model for underground infrastructure sustainable water mains. Journal of Performance of Constructed Facilities, 20(2), 126135.

Aldrich, H. (1979). Organizations and Environments. Organization Studies, Englewood Clis, New Jersey, United States. American Heritage College Dictionary (1993). Houghton Min Organization. Boston, United States. Bassioni, H., Price, A., and Hassan, T. (2004). Performance measurement in construction. Journal of Management in Engineering, 20(2), 4250. Bassioni, H., Price, A., and Hassan, T. (2005). Building a conceptual framework for measuring business performance in construction: An empirical evaluation. Construction Management and Economics, 23(5), 495507. Caballero, A. A. and Dye, J. (1999). Comparison of construction rms based on fuzzy sets. Journal of Construction Education, 4(3), 313320. Camp, R. (1995). Business Process Benchmarking: Finding and Implementing Best Practices,. ASQC Quality Press, Milwaukee, Wisconsin, United States. Chinowsky, P. (2001). Strategic management in engineering organizations. Journal of Management in Engineering, 17(2), 6068. Chinowsky, P. and Meredith, J. (2000). Strategic Corporate Management for Engineering. Oxford University Press, New York, United States. Daniel, D. (1961). Management information crisis. Harvard Business Review, 39(5), 111116. Hawawini, G., Subramanian, V., and Verdina, P. (2004). The home country in the age of globalization: How much does it matter for rm performance. Journal of World Business, 39(2), 121135. Holophan, J. (1992). Use of executive information system in measuring business performance. Journal of Information Technology, 7(3), 177186. Isik, Z., Arditi, D., Dikmen, I., and Birgonul, M. T. (2010). Impact of resources and strategies on construction company performance. Journal of Management in Engineering, 26(1), 918. Jaselskis, E. J., Anderson, S. D., and Russell, J. S. (1996). Strategies for achieving excellence in construction safety performance. Journal of Construction Engineering and Management, 122(1), 6170. Kagioglou, M., Cooper, R., and Aouad, G. (2001). Performance management in construction: A conceptual framework. Construction Management and Economics, 19(1), 8595. Kaplan, R. S. and Norton, D. P. (1993). Putting the balanced scoreboard to work. Harvard Business Review, 71(5), 3440. Kaplan, R. S. and Norton, D. P. (1996). Using the balanced scorecard as a strategic management system. Harvard Business Review, 74(1), 7585. Luu, T., Kim, S., Cao, H., and Park, Y. (2008). Performance measurement of construction rms in developing countries. Construction Management and Economics, 26(4), 373386. Martin, R. (1997). Do we practice quality principles

211

Zayed et al./International Journal of Architecture, Engineering and Construction 1 (2012) 199-212

in the performance measurement of critical success factors. Journal of Total Quality Management, 8(6), 428439. McCracken, G. (1988). The Long Interview. Qualitative Research Methods Series, Volume 13, Sage Publications, Newbury Park, California, United States. MINITAB (2006). Users Manual. Release 14, Mintab Inc., State College, Pennsylvania, United States. Moselhi, O., Hegazy, T., and Fazio, P. (1991). Neural networks as tools in construction. Journal of Construction Engineering and Management, 117(4), 606625. Part, M. and Sanvido, V. (1993). Checklist on critical success factors for building projects. Journal of Management in Engineering, 9(3), 243249. Pollalis, Y. and Frieze, I. (1993). A new look at critical success factors in IT. Information Strategy, The Executives Journal, 10(1), 2435. Richard, A. (2004). The Critical Success Factor Method: Establishing a Foundation for Enterprise Security Management. Technical Report, Software Engineering Institute, Carnegie Mellon University, Pennsylvania, United States. Rockart, J. (1979). Chief executives dene their own data needs. Harvard Business Review, 38(1), 4355. Rockart, J. (1982). The changing role of information system executive: A critical success factors perspec-

tive. Sloan Management Review, 24(1), 313. Rockart, J. and Bullen, C. (1981). A Primer on Critical Success Factors. Center for Information Systems Research, Massachusetts Institute of Technology, Maryland, United States. Rockart, J., Earl, M., and Ross, J. (1996). Eight imperatives for the new IT organization. Sloan Management Review, 38(1), 4355. Shurchuluu, P. (2002). National productivity and competitive strategies for the new millennium. Integrated Manufacturing Systems, 13(6), 408414. Wethyavivorn, P., Charoenngam, C., and Teerajetgul, W. (2009). Strategic assets driving organizational capabilities of thai construction rms. Journal of Construction Engineering and Management, 135(11), 12221231. Zayed, T. and Halpin, D. (2005a). Pile construction productivity assessment. Journal of Construction Engineering and Management, 131(6), 705714. Zayed, T. and Halpin, D. (2005b). Productivity and cost regression models for pile construction. Journal of Construction Engineering and Management, 131(7), 779789. Zayed, T., Halpin, D., and Basha, I. (2005). Productivity and delays assessment for concrete batch planttruck mixer operations. Construction Management and Economics, 23(8), 839850.

212