A Methodology to Evaluate the Organizational Impact of IT on Knowledge Management

A METHODOLOGY TO EVALUATE THE ORGANIZATIONAL IMPACT OF IT ON KNOWLEDGE MANAGEMENT: AN ITALIAN CASE STUDY
Michle Grimaldi, DIMSAT University of Cassino, Cassino, Italy m.grimaldi@unicas.it Pierluigi Rippa, DIEG University of Naples Federico II, Naples, Italy pierippa@unina.it Massimo Ruffolo, EXEURA, Cosenza, Italy ruffolo@exeura.it ABSTRACT The important role played by Information Technology (IT) for Knowledge Management (KM) within many companies has been widely recognized. However, the relationship between IT investment and firm performance is complex and multifaceted and the evaluation of the impact of IT for KM on organizational performance has proved to be a difficult task. This has made it difficult to demonstrate that IT for KM may be used to leverage other firm advantages and opportunities and to justify spending on IT projects in support of KM. This paper provides a methodological framework to analyze the consequences related to the adoption of IT tools for the management of information and knowledge. In particular, we present a four-phase framework that captures the benefits of KM features in meeting business objectives through an examination of organizational processes and actors. The aim of the constructed methodology coincides with the main target pursued by firms: to have the possibility of evaluating IT solutions both ex ante, i.e. at the project/design phase, and ex post, i.e. at the assessment phase. A case study application in an Italian small business is described in detail. KEYWORDS Infonnation Technology; Knowledge Management; Methodological Framework; Evaluation; Case Study INTRODUCTION The current economic context is characterized by ever-growing global competition and complexity. Companies thus need to focus on those exclusive resources that allow the acquisition and the maintenance of a competitive advantage. Because these resources need to be renewed and improved constantly, management of these dynamics adds significant value. This requires making precise plans and suggesting managerial guidelines inside each firm, as a firm's success depends not only on its technologies, resources, organizational functions and core processes, but also on

JITCAR, Volume 10, Number 2

A Methodology to Evaluate the Organizational Impact of IT on Knowledge Management their integration (Ahn and Chang, 2004; Hansen et al., 1999; Liebowitz et al., 2000; March, 1991). For many organizations, knowledge has become a strategic resource. Organizations should develop their capacity to generate, store, share and apply knowledge (Nonaka and Takeuchi, 1995; Tiwana, 2000; Zack, 1999). According to Penrose (1959), a company should develop and use both knowledge resources and knowledge competences in such a way as to improve its performance. Indeed, according the resource-based view, once the acquired knowledge assets are considered as key success factors, they should be included in the business organizational context, by properly applying human and knowledge resources to technologies and organizational processes in an integrated way so as to improve performance (Zack, 1999; Von Krogh et al, 2000). However, theories of dynamic capabilities and the knowledge-based view theories extend the resource-based view by asserting that the competitive advantage should be reached by the development and reconfiguration of knowledge assets and by their dynamic combination and integration with the other available assets (Freiling, 2004; Teece et al., 1997). Knowledge Management (KM) concerns the development and use of intangible knowledge resources in an effective way so as to improve a firm's performance. KM initiatives should be directed towards the achievement of strategic goals of the company by means of the definition of a program supporting companies in introducing the appropriate infrastructure. A strong effort in building the technological infrastructure for the diffusion of knowledge and for the coordination of business processes is of primary importance (Davidow and Malone, 1992). Thus Information Technology (IT) could represent the main knowledge transmission vehicle within the firm. The role of IT is not only to organize data into useful information, but also to support the transformation of individual information into organizational and shared information (Adamides et al, 2006). Several KM tools and techniques support the performance of organizational activities and facilitate the implementation of knowledge processes (Tyndale, 2002). But, even if IT is the key enabler of the implementation of KM processes and the primary support of a KM system (Reyes et al., 2002; Wensley, 2000), it does not guarantee the efficiency and the efficacy of business processes (Davenport and Prusak, 1998; Hansen et al, 1999). IT investments and implementation are important aspects in a firm. They should be carefully justified, measured, controlled and evaluated because of the unpredictability related to some of their characteristics, such as high risks, long-term returns, tangible and intangible benefits, and user resistance to change (Milis and Mercken, 2004). Moreover, it is important for a firm to evaluate the benefits of such investment, since, even though an organization invests a great deal of IT resources, it is questionable whether the expected benefits can be reached (Huang et al, 2006). A methodological approach was developed to evaluate the organizational impact of IT on KM from a Business Process Management (BPM) perspective. This methodology is the result of joint research carried out by the authors and Exeura, an Italian company founded in 2002 specializing in KM process consulting. Exeura's business aim is to act as a link between academic research and industrial production in the IT field. Exeura has a project-based organization. Until 2004, it executed its own projects on the basis of a total quality management system called Exeura

JITCAR, Volume 10, Number 2

A Methodology to Evaluate the Organizational Impact of IT on Knowledge Management Enterprise Management System (EEMS). The purpose of EEMS was to clarify and formalize the strategic and operational processes of the business so as to control and evaluate them in a simple way. In 2004, Exeura managers determined that EEMS no longer supported the aim of the firm, since it had no capability to manage the firm's consultants' knowledge, which Exeura believed to be the driver of its expansion strategy. Before replacing EEMS, the management of Exeura, together with the authors, constructed a methodology to assess the benefits that could be generated by the introduction of a new infonnation system that would support business processes execution incorporating the potential benefits of KM. Managers of Exeura decided to design and develop a new IT solution in order to assist plan, execute and monitor multiple ongoing projects. This IT system, which is named KMS-Plus, has been partially founded by the Italian Ministry of University and Research. Exeura sought, by means of the constructed methodology, to evaluate the potential benefits that KMS-Plus could generate and establish its eventual adoption. In the following sections, we first present an in-depth description of the literature on the relationship between IT and KM. We then describe the proposed methodology and the case study application in Exeura. The methodology has been used to support executives and organizational analysts in evaluating the organizational and economic performance of a specific IT solution for knowledge and infonnation management. Finally, we discuss the results of the research, draw conclusions and suggest future developments. ORGANIZATIONAL IMPACT OF IT ON KM The organizational perspective on IT influences and suggests several approaches for evaluating its impact on organizational performance. On the one hand, IT can be conceived in a 'traditional' way, as is the case when the organization focuses its efforts on improving information systems in order to support management and administration without linking IT to other organizational assets. On the other hand, IT can be 'integrated' with other organizational assets, as is the case when organizations take into consideration IT as a strategic resource and apply and evaluate it in the same way as other available resources. In both cases, strategic alignment influences company performance, the first on a piecemeal basis, the second on a company-wide scale (see Table 1). Table 1 - Theoretical Approaches in the interpretation of IT for KM Approach Assumptions Methodologies Traditional Improvement of Information Technology Broker Systems to support management Intellectual Capital Index Skandia Navigator Balanced Scorecard IT as a source of competitive Intangible Assets Monitor advantage Integrated Development of Coordinated Business Impact of KM (Me Hugh) Technological Systems KM Impact on Business Performance (Carlucci and Schiuma) IT as a Coordination Enabler KM Project Evaluation (Ifthicar and Erikson) KM Configuration (Corso et al.) IT as a strategic resource IMPaCT Assessment (Robinson et al.)

JITCAR, Volume 10, Number 2

10

A Methodology to Evaluate the Organizational Impact of IT on Knowledge Management In the literature, the 'traditional' interpretation of IT is represented by the most widely applied asset evaluation methodologies, such as the Technology Broker (Brooking, 1996), the Intellectual Capital Index (Roos et al, 1998), the Skandia Navigator (Edvinsson and Malone, 1997), the Balanced Scorecard (Kaplan and Norton, 1996) and the Intangible Assets Monitor (Sveiby, 2001). These methodologies consider IT as one of the sources of competitive advantage. But, according to the resource-based analysis, the relationship between IT management and other forms of investment is one of the most important contributors to sustainable competitive advantage (Mata et al, 1995; Sambamurthy et al, 2003). There is strong evidence that companies need to assess their investment in IT and in other organizational assets jointly, as they are complementary. "Integrated" IT methodologies focus on the performance generated by the business integration provided by IT solutions. In this case, IT plays a dual role in an organization: it is used directly as a production technology system, like other types of investments, and, at the same time, it takes the role of a coordination system (Malone et al, 1987). Because of the potential of KM for coordinating organizational activities, we have rejected the traditional approaches to evaluate KM systems in favour of an integrated approach. KM tools can only be understood in the organizational context in which they are used. Thus, the adoption of IT solutions for KM should require the analysis of organizational actors and business processes, and the interpretation and codification of internal and external data (Bloodgood et al, 2001). Therefore, linking IT organizational value to environmental and contextual aspects that are critical to strategic management plays a crucial role when implementing new IT systems for KM (Roberts, 2000). A significant number of papers have focused their attention on specific aspects of the organizational and economic impact of IT investments for KM. Some models aim at the evaluation of the satisfaction of stakeholders of KM initiatives (McHugh, 2002; Carlucci and Schiuma, 2002); other studies have analyzed the interrelationship between KM initiatives and organizational variables, providing methodologies to evaluate their mutual infiuence (Ifthicar et al, 2002; Corso et al, 2002); Robinson et al. (2002) have defined an integrated metric, similar to Return on Investment (ROI), to estimate the profitability of IT investments; Milis and Mercken (2004) have proposed an evaluation process derived from the Balanced Scorecard for the appraisal of IT investments, by defining IT-related metrics for each of the four perspectives; similarly, Huang et al (2006) have investigated the relationship between IT investment and company performance, by determining and quantifying the infiuence of IT infrastructure, IT human resources and IT-enabled intangibles on company performance. All of these methodologies constitute valuable "tools" for evaluating the organizational consequences of the introduction of IT for KM. They all have stressed the difficulties related to this problem that are mainly due to the coexistence and the simultaneous interaction of tangible and intangible assets, and to the mutual influence of both quantitative and qualitative aspects. These approaches, however, do not explicitly address the problem of evaluating IT investment for KM from a business process based point of view and do not involve consideration on how IT systems for KM are concretely deployed, adopted and applied within organizations from a technological perspective. By adopting both process-based and software engineering perspectives, some difficulties in evaluating the impact of IT for KM can be overcome.

JITCAR, Volume 10, Number 2

11

A Methodology to Evaluate the Organizational Impact of IT on Knowledge Management The methodology proposed in this paper aims at linking IT business value to contextual aspects that are critical to strategic management: environmental, strategic, and managerial factors. In particular, our framework defines quantitative and qualitative elements to determine the performance improvement of the knowledge life-cycle in terms of cost and time reduction, profitability, productivity, cost/benefits ratio, usability of IT solutions, and information quality. The innovativeness aspect of the methodology is its combined use of business process management (BPM) and software engineering approaches in the analysis. THE METHODOLOGICAL APPROACH The methodology presented in this paper was developed during several research meetings between the authors and Exeura Management through a critical analysis of the traditional and integrated methodologies discussed above. Our intent was to incorporate the best features of these methodologies while extending them to address explicitly the role of KM in the IT solution. The methodology consists of four phases (see Figure 1). In the following sections, each phase will be described in detail. As previously mentioned, the adoption of IT tools for KM should require an in-depth analysis of business processes, organizational actors, and internal and external data. The methodology has been constructed by taking into consideration the well known BPM tools (Hammer et al, 1993). They have been applied and contextualized referring to the case of adoption of IT solutions for KM. In particular, attention has been focused on some formal and informal processes of information and knowledge sharing.
Phase 1: Definition of business performance

Phase 2: Definition of technological and organizational model Business process cataiog Actors catalog Features cataiog Cost/Benefit Driver Cataiog Int. Asset Driver Cataiog

I
Phase 3: Interdependency evaluation

I
Phase 4: Assessment Figure 1 - The flow chart of the methodology

Phase One: Definition of Business Performance This phase consists in the definition of the organizational areas involved in the evaluation process and the estimation of the IT solution benefits in terms of time and cost reduction, profitability, productiveness, cost/benefit ratio, usability and quality of infonnation shared. In particular, the

JITCAR, Volume 10, Number 2

12

A Methodology to Evaluate the Organizational Impact of IT on Knowledge Management methodology aims at assessing whether the adoption of the new IT solution improves knowledge sharing activities among organizational members, and/or leads to such an increase of skills as to reduce or eliminate some training activities, and/or enhances exchange of capabilities and/or produces organizational benefits in terms of communication activities, customer satisfaction and productivity. As a generalization of the analysis, it is possible to consider the variation of some financial indicators such as the ROI, i.e. the ratio between the net profit and the cost of the investment. Phase two: Definition of Technological and Organizational Model In this phase, technological and organizational representation of the firm is defined. Using software engineering and BPM tools (Unified Modeling Language - UML, Business Process Reengineering - BPR, Activity Based Costing - ABC), a set of four catalogs is developed (1. Business process catalog, 2. Actor catalog, 3. IT feature catalog, 4. Cost/benefit and intangible asset driver catalogs). These catalogs aim at giving an outlined description of the most relevant technological and organizational aspects affecting the evaluation process, such as business processes and their activities, organizational actors, system features, cost/benefit drivers. 1. Business Process Catalog. Business process analysis is aimed at exploiting the processes that should be managed with a new technology. It could be developed following typical BPM techniques, such as ICAM Definition Languages and Unified Model Languages (UML) tools. Applying these methodologies, activities and their mutual relations and interdependencies are defined. Sometimes, firms could use also the process scheme starting from the internal quality control system (where such activities are required). In Table 2, the schema of business process and the relationships between processes and activities is illustrated. This table is a synthesis of the resuits that could be obtained following BPM methodologies. Table 2 - Business Processes Catalog Processes Description Process 1 Activity 1.1 .. Activity l.n Process m Activity m.l ... Activity m.p ...

2. Actor Catalog. Actors represent all the organizational members whose behaviors could be affected by the introduction of the new IT solution. Organizational actors are classified on the basis of their competences, skills and expertise after the organizational structure has been defined (e.g. process oriented, divisional, functional, etc.). The purpose of the actor catalog is to clearly represent the profile of the end user of the new IT solution. Actors can be intemal or external to the organization, and they can also be classified on the basis of their nature, human or not. An organization chart and job description can be used to schematize the actors involved in the organization. Table 3 illustrates the Actor Catalog.

JITCAR, Volume 10, Number 2

13

A Methodology to Evaluate the Organizational Impact of IT on Knowledge Management Typology of actors Employee Client/supplier Automatic (system and machinery used by the actors) Table 3 - Actor Catalog Actors Competences, Skills, Expertises Actor 1 Actor j Actor n ...

...

3. IT Feature Catalog. Features of the IT solution are identified by the Feature Driven Development Agile Software methodology (Highsmith, 2002). This catalog provides taxonomy of the information system features. Requirements of the system are clustered in such a way that each cluster is composed by a set of homogeneous features. Thus, for each group of features, managers can identify end users, their activities and responsibilities, as illustrated in Table 4. Table 4 - IT Feature Catalog Feature Description Cluster 1 Feature 1.1 Number and profile of end user ... Feature 1 .n Cluster m Feature m. 1 Number and profile of end user Feature m.p 4. Cost/benefit and Intangible Asset Drivers Catalog. The cost/benefit ratio evaluates the economic impact of the new IT system. In "Show me the money", Martin (2002) analyzed the organizational and economic impacts of a specific IT solution for KM by selecting and evaluating four critical factors: a. internal expense (the benefits of KM can be directed at reducing internal costs); b. increasing profit (KM can be targeted to improve low company profit margins); c. measuring 'hard' returns (the primary tool used by businesses to measure the value of technology acquisition is the ROI); d. 'soft' benefits and comparative evaluation (a firm can also focus directly on the effectiveness of the investment in generating intellectual capital and other intangible benefits). On the basis of the above considerations, specific metrics have been selected to assess a quantitative evaluation of the economic impact when a new IT solution is implemented (Table 5). These metrics are the cost drivers, revenue drivers and intangible asset drivers. The Intangible Asset Driver component of the Cost/Benefit Driver Catalog has been distinguished for actors and, successively, for combinations of activities, features and actors. This catalog contains expected values and variations of drivers with respect to internal actors, taking into consideration those values and variations which can be modified by the utilization of potential

JITCAR, Volume 10, Number 2

14

A Methodology to Evaluate the Organizational Impact of IT on Knowledge Management external actors. In this way, for example, it is possible to determine how a firm could pay for a specific activity carried out by a person having a certain skill. Broadly, it could be possible to estimate and measure, for each driver, the modification generated by the introduction of the IT solution for KM and the ensuing variation of knowledge sharing, capabilities exchange and training activities. Table 5 - Cost/Benefit Driver Catalog Typology Driver Description Formula ,, , Cost Driver Driver 1 Revenue Driver j . ... Driver Intangible Driver n ... Asset Driver Phase three: Interdependency evaluation At the end of the second phase, all the characterization of the technological and organizational model of the firm is extrapolated/accrued from the business process catalog, actor catalog, IT feature catalog and cost/benefit driver catalog. The Interdependency Matrix (Table 6) shows the set of drivers for a chosen set of activities/features/actors in a concise way. In fact, in the first column of the matrix the business processes are bulleted; in the second column, for each process, the activities are bulleted; finally, in the third column for each activity, IT features, actors and drivers are bulleted. In this way, the Interdependency Matrix aims at identifying areas where work can be saved so as to help determine benefits. Processes Process Process 1 Table 6 - Interdependency Matrix IT Drivers Actors Features Activity Driver 1 Activity 1.1 Feature 1 ... ... Feature n Activity 1.2 Feature e ... 1 Feature 2 ... Feature m ... ... ... ...

...

Driver n

...

... . ...

Phase Four: Assessment In this phase, the interdependency matrix is checked in order to assess whether its structure is able to explain and justify the evaluation goals. Moreover, this matrix is utilized to evaluate the drivers before the adoption of the new IT solution. The same matrix is then utilized to estimate the same drivers on assuming the adoption of the new IT solution. By comparing the value drivers of the 'as is' situation with those of the 'to be' estimation, management is able to

JITCAR, Volume 10, Number 2

15

A Methodology to Evaluate the Organizational Impact of IT on Knowledge Management understand the potential economic benefit generated by the adoption of the new IT solution. Moreover, the measures obtained in the previous phase can be used to detennine the ROI index. FIELD STUDY The Case Study Environment The proposed methodology was tested at Exeura by a group composed of researchers and managers of Exeura. In order to decide whether to adopt the KMS-Plus, the firm wanted to evaluate the organizational benefit deriving from it by using the methodology. Data collection Data collection was achieved through meetings with managers of Exeura. The following sections detail the application of the methodological phases previously described. Phase One: Definition of Business Performance Within the wide range of business performance goals, management of Exeura established ROI maximization as the main goal in the adoption of the KMS-Plus system. Phase two: Definition of Technological and Organizational Model 1. The Business Process Catalog By exploiting the already existing EEMS, which provides a formalized representation of the business processes, it was possible to identify the following activities: Trading (Exeura acts as an advisor for the realization of research and development activities for technological solutions); Consultancy (Exeura acts as an innovation consultant); Research and Development (Exeura acts as an advisor in research and development activities). Table 7 shows the business process catalog for the trading process. Table 7 - Example of business process catalog in Exeura Processes Process Trading Activities Acquisition of requirements/call forbids Analysis of the feasibility of requirements/call for bids Development of technical requirements/call for bids Contract drawn up 2. The Actor Catalog The actor catalog was drawn by using the organization chart and job descriptions. In Table 8, a section of the actors catalog developed in Exeura is shown. specification related to the ... Description

JITCAR, Volume 10, Number 2

16

A Methodology to Evaluate the Organizational Impact of IT on Knowledge Management Table 8 - Example of actor catalog in Exeura Typology Internal Employees Actor Chief officer Project Manager Researcher Prsales Engineer External employees Private customer Public Customer Partner ... ... Description

3. The ITfeature Catalog Features of the new IT solution (KMS-Plus) were drawn from the handbook of KMS-Plus. KMSPlus supports a virtual community paradigm that allows business processes management to represent tacit and explicit knowledge. This solution was designed considering that knowledge is a dynamic combination of skills, individual values and capabilities that flow among organizational members and that only a portion of them is transformed from a tacit to an explicit dimension by means of information systems. The KMS-Plus workspace, when referring to a virtual community, enables several services to support group activities, such as synchronous and asynchronous communication (chat, e-mail, forum, blog, etc.), collaborative work (expert search, knowledge brokering, workflow management), project management (project planning, project execution, and project monitoring). All these features increase efficiency and efficacy of managerial and operational processes, adding value to the production process. In Table 9, a part of the features catalog for Exeura is shown.

JITCAR, Volume 10, Number 2

17

A Methodology to Evaluate the Organizational Impact of IT on Knowledge Management Table 9 - Example of feature catalog for Exeura Feature Group Representation, acquisition, ans usage of knowledge Function Representation of knowledge Representation of processes Semantic representation of actors Semantic management of contents
t

Description

...

4. The cost/benefit and intangible asset driver Catalog Once all the catalogues had been developed, drivers were chosen to estimate the economic impact of KMS-Plus, through the analysis of the balance sheet. Then, for each driver, the most appropriate formula was identified. A section of the cost/benefit and intangible asset driver catalog is shown in Table 10 and 11. Table 10 - Example of cost/benefit driver catalog in Exeura Typology Cost Driver Driver Production hour per person Hour of effective usage of communication system Hour per Training

Description ...

Formula Employees cost = Production hour/Employees*cost per hour Communication service cost = service usage cost per hour * number of hour of effective usage of the service

...

...

Table 11 - Example of cost/benefit driver catalog in Exeura Typology Intangibile Asset Driver Driver Consultant market value Numbers and frequency of employees' complaint Description Formula

KMS-Plus allows consultants to share Intellectual Capital knowledge about technical and = average market organizational aspects value per advisor An higher visibility and control of the variables related to employees's level of satisfaction allow an increasing of the level of participation of each employees ... Satisfaction = number and consistency of employees' compliances ...

JITCAR, Volume 10, Number 2

18

A Methodology to Evaluate the Organizational Impact of IT on Knowledge Management Phase three: Interdependency evaluation In Table 12, the interdependency matrix for the trading process is shown. For each set of processes, features, and actors the most appropriate drivers were selected. Table 12 - Example of the interdependency matrix in Exeura
Processes Process Trading process Activity Acquisition of requirements/ call for bids Feature Knowledge object creation and storage Trading process management Actor Manager Administrative Project manager Prsales engineer Manager Project manager Prsales engineer Private or client customer IT architectural expert Project manager Prsales engineer IT architectural expert Manager Project manager Prsales engineer IT architectural expert Manager Project manager Driver Production hour/ employees Number of hour communication service Training hour

Analysis of the feasibility of requirements/ call for bids

Semantic management of contents

Virtual workspace management

Development of technical specification related to the requirements/ call for bids

Contract drawn up

Human resources management Semantic management of contents Virtual workspace management Trading process management Semantic management of contents

JITCAR, Volume 10, Number 2

19

A Methodology to Evaluate the Organizational Impact of IT on Knowledge Management Phase four: Assessment These drivers were estimated on the basis of the ties identified in interdependency matrix. In Table 13, an example referring to the driver "Production Hours" is illustrated. Once the whole set of drivers has been estimated, it becomes possible to connect the features of the KMS-Plus with the activities defined in the business process catalog.
Table 13 - Example of dnver selection in Exeura Personnel costs Total Production Training Costs Operational annual hour hour per costs hour hour 1670 1670 42 70140 0 2088 1670,4 16704 417,6 10 1000 1000 2000 5010 505 6680 505 20458 800 800 1600 4008 404 5344 404 16700,4 200 200 400 1002 101 1336 101 3757,6 22,7 22,7 22,7 18,8 22,7 14,32 22,7 18160 18160 36320 75350,4 9170,8 76526,08 9170,8 329702,08 Training costs Teaching Training costs costs 0 2600 0 1600 0 2000 0 0 0 6200 0 6776 4540 6140 9080 20837,6 2292,7 19131,52 2292,7 71090,52 400792,6

Typology

Actors

Internal employees

Manager Administrative personnel Project manager Researcher Architecture designer Software designer Quality manager Programmer analyst Prsales engineer Partial total Total

Results of the assessment showed that the introduction of the KMS-Plus technology to KM in the business activities of Exeura could give rise to the following expected benefits: Increased productivity of personnel (employees are able to carry out the same activities saving a considerable amount of time); Training time reduced by 20%; 10% increase in market value per advisor (because of an increase in knowledge circulation and a reduction in training time); Personnel complaints reduced by 20%. The expected results reveal a greater availability of time per person to be added to the business activities. The ROI was estimated to evaluate the economic benefits that could be reached with KMS-Plus. The starting point for this kind of analysis is represented by the income statement, where the revenue value in 2005 was about 660,000 . On the basis of the value revealed by the methodology, this result could have been achieved with 14,195 hours annually instead of the 16,700 effectively used in the production activities. As a result, about 2500 of production hours that could have been used to undertake several additional projects. Extrapolating from this data, we calculate that the firm could have reached a revenue of about 815,200 (14,195 production hours: 662,793 = 17,450 production hours: 815,200 ). Estimating that the total cost of implementing KMS-Plus at 400,000 Euros, Table 14 shows that, by taking into account an increase in revenue of about 150,000 Euros, the KMS-Plus investment has a Pay Back Period of three years.

JITCAR, Volume 10, Number 2

20

A Methodology to Evaluate the Organizational Impact of IT on Knowledge Management

Table 14-Example of driver in Exeura Value () Driver 1" Year r" Year i"' Year 4'" Year Monetary Estimation of 152,440 152,440 152,440 152,440 efficiency Increasing Total 304,880 457,320 609,760

5'" Year 152,440 762,200

On the basis of the results obtained in this analysis, Exeura could expect tangible benefits from the adoption of the KMS-Plus. As a consequence Exeura decided to adopt the KMS+ as an appropriate IT solution for its specific organizational targets. DISCUSSION In this paper, we have proposed a methodology to evaluate the benefits that could be gained by the adoption of an IT solution to support KM activities. The problem, when evaluating IT for KM, is the quantification of benefits from intangible assets. To solve this problem, the methodology uses catalogs that aim to identify the knowledge of the actors and the business processes of the firm, the features of the IT solution, and the drivers of benefit to evaluate the economic impact of the new IT solution. In the case study presented, the methodology has been used to evaluate the impact and the benefit of KMS-Plus technology, an IT solution for improving KM within the firm. Exeura sought to understand whether the adoption of the new solution could have brought about some benefits. The methodology proved useful in deciding to adopt the new solution. The proposed methodological approach can be considered as a valid tool to estimate efforts and economic benefits through the adoption of IT for KM. However, this tool should not be considered alone in this evaluation process. An organizational culture oriented to a knowledge sharing behavior, a strong leadership and commitment by top management, as well as the availability of an appraisal system with incentives towards innovative attitudes are also necessary. CONCLUSIONS AND FUTURE DEVELOPMENT The methodology described in this paper has been adopted to analyze the performance of an IT solution for KM. Results show that the adoption of the IT solution can enhance the performance of the case study firm both from the technological and organizational point of view. From a managerial standpoint, the proposed framework could help managers in understanding the potential benefits deriving from the adoption of IT tools for KM; from a methodological point of view, it adds a new step in the analysis and measurement of the performance of IT tools for KM. To understand the limits of the framework, the next step of the research will be the adoption of the methodology within different types of firms (e.g. large enterprises) and in other business sectors.

JITCAR, Volume 10, Number 2

21

A Methodology to Evaluate the Organizational Impact of IT on Knowledge Management

REFERENCES
1. Adamides, E.D. & Karacapilidis N. (2006). Information Technology support for the knowledge and social processes of innovation management. Technovation, 26, 50-59. 2. Bloodgood, J.M. & Salisbury, W.D. (2001). Understanding the influence of organizational change strategies on information technology and knowledge management strategies. Decision Support Systems, 31, 55-69. 3. Brooking, A. (1996). Intellectual Capital: Core Assets for the Third Millennium Enterprise. London: Thomson Business Press. 4. Carlucci D. & Schiuma G. (2002). How Knowledge Management initiatives impact on Business Performance, Third European Conference on Knowledge Management. Dublin Ireland: Trinity College. 5. Corso M. & Martini A. & Paolucci E. & Pellegrini L. (2002). Knowledge Management configuration in SMEs Results of an empirical analysis. Impresa e Competizione KnowledgeBased, XIII AilG Scientific Conference. 6. Davidow, W.H. & Malone, M.S. (1992). The virtual Corporation. New York: Harper&CoUins. 7. Davenport, T. & Prusak, L. (1998). Working Knowledge: How organization manage what they know. Boston, MA: Harvard Business School Press. 8. Edvinsson, L. & Malone, M. (1997). Intellectual Capital: Realising Your Company's True Value by Finding its ' Hidden Brainpower. New York: Harper Collins. 9. Freiling, J. (2004). A Competence-based Theory of the Firm. Management Revue, 15(1), 2752. 10. Hammer M. & Champy J., (1993). Reengineering the corporation. A manifesto for Business Revolution. New York: Harper Business Essential. 11. Hansen, M.T. & Nohria, N. & Tiemey, T. (1999). What's your strategy for managing knowledge? Harvard Business Review, 77(2), 106-116. 12. Huang S.M. & Ou C.S. & Chen CM. & Lin B. (2006). An empirical study of relationship between IT investment and firm performance: A resource-based perspective European Journal of Operational Research, 173, 984-999. 13. Kaplan, R. S. & Norton, D. P. (1996) Using the Balanced Scorecard as a Strategic Management System. Harvard Business Review, January-February. 14. Ifthicar, Z. & Erikson, I. V. & Dickson G. V. (2002). Devoloping an Instrument for Knowledge Management Project Evaluation. Third European Conference on Knowledge Management. Dublin Ireland: Trinity College. 15. Liebowitz J. & Rubenstein-Montano B. & McCaw D. & Buchwalter J. J. & Browning C. (2000). The Knowledge Audit. Journal of Knowledge and Process Management, 7(1), 3-10. 16. March, J.G. (1991). Exploration and Exploitation in Organizational Leaming. Organization Science,l{\),l\-%1 17. Martin, K. (2002). Show Me the Money - Measuring the Retum on Knowledge Management. LLRX.com. 18. Mata, F. J. & Fuerst, W. L. & & Bamey, J. B. (1995). Information technology and sustained competitive advantage: A resource-based analysis. MIS Quarterly, 19(4), 487-505. 19. McHugh, M. W. (2002). Measuring the Business Impact of KM in Practice, Third European Conference on Knowledge Management. Dublin Ireland: Trinity College. JITCAR, Volume 10, Number 2 22

A Methodology to Evaluate the Organizational Impact of IT on Knowledge Management 20. Milis, K. & Mercken, R. (2004). The use of balanced scorecard for the evaluation of Information and Communication Technology projects. International Journal of Project Management, 22, 87-97. 21. Nonaka, I. & Takeuchi H. (1995). The Knowledge Creating Company New York: Oxford University Press. 22. Penrose, E.T., (1959), The theory of the growth of the firm, Oxford: Basic Blackwell. 23. Reyes, P. & Raisinghani, M. S. (2002). Integrating Information Technologies and Knowledge-based Systems: A Theoretical Approach in Action for Enhancements in Production and Inventory Control. Knowledge and Process Management, 9(4), 256-263. 24. Roberts, J., (2000). From Know-how to Show-how? Questioning the Role of Information and Communication Technologies in Knowledge Transfer. Technology Analysis and Strategic Management, 12(4), 429-443. 25. Robinson, H.S. & Carrillo, P.M. & Anumba, C.J. & Al-Ghassani, A.M. (2002). Evaluting Knowledge Management Strategies: An IMPaCT Assesment. Third European Conference on Knowledge Management. Dublin Ireland: Trinity College. 26. Roos, G., Roos, J. & Dragonetti, N. & Edvinsson, L. (1998). Intellectual Capital: Navigating in the New Business Landscape. New York: University Press. 27. Sambamurthy, V. & Bharadwaj, A. & Grover, V. (2003). Shaping Agility through Digital Options: Reconceptualizing the role of IT in Contemporary Firms. MIS Quarterly, 27(2), 237-263. 28. Sveiby, K. E. (2001). Methods for Measuring Intangible Assets, [online], http://www.sveibv.com/articles/IntangibleMethods.htm. 29. Teece, D.J. & Pisano, G. & Shuen, A. (1997). Dynamic Capabilities and Strategic Management, Strategic Management Journal, \S{7), 509-533. 30. Tiwana, A. (2000). The Knowledge Management Toolkit: Practical Techniques for Building a Knowledge Management System, Prentice-Hall PTR, NJ. 31. Tyndale, P. (2002). A taxonomy of knowledge management software tools: origins and applications. Evaluation and program planning,15, 183-190. 32. von Krough, G. & Ichijo, K. & Nonaka I. (2000) Enabling Knowledge Creation: How to Unlock the Mystery of Tacit Knowledge and Release the Power of Innovation, New York: Oxford University Press. 33. Wensley, A. K. P. (2000). Tools for knowledge management. BPRC Conference on Knowledge Management: Concept and Controversies, 10-11. 34. Zack, M.H. (1999). Developing a Knowledge Strategy. California Management Review, 41(3), 125-145.

JITCAR, Volume 10, Number 2

23

A Methodology to Evaluate the Organizational Impact of IT on Knowledge Management BIOGRAPHIES Michle Grimaldi received his first-class honours degree in 2002 and a Master's in Industrial Engineering in 2003. In 2005 he achieved his PhD in Industrial and Management Engineering at the University of Rome 'Tor Vergata'. He actually works as researcher at the the faculty of Engineering at the University of Cassino (Italy). He has published papers in International Journals and Conference Proceedings. His current research field concerns knowledge management strategies and metrics. Pierluigi Rippa completed his Ph.D in Business and Managerial Engineering at the University of Naples Federico II. He currently works as researcher at the Department of Business and Managerial Engineering, University of Naples Federico II (Italy). His research interests include information technology adoption and organizational interaction, ERP systems development and adoption, Statistical and Fuzzy application in Decision Support Systems. He has published papers in international journals and presented researches in international conferences. Massimo Ruffolo is a Computer Science researcher at the Institute of High Performances Computing and Networking of the Italian National Research Council (ICAR-CNR). He is temporarily posted to Exeura s.r.l., where he fills the position of Senior Researcher. He received the Doctoral degree in System Science and Computer Science from the University of Calabria in 2000 and the Bachelor degree in Managerial Engineering in 1996. He received in 2006 the master degree in enterprise management. He has led or participates in several research and development projects aimed to obtain technologies and infrastructures for knowledge management. He has published several papers in international scientific conferences and journals in the areas of knowledge representation and reasoning, ontology and semantic web, data/text mining, knowledge management and its organizational impact. ACKNOWLEDGMENT An earlier version of this Paper was presented at the GITM Conference in 2007 held in Naples. Referees suggested submitting the paper to JITCAR. We would like to thank the referees for their useful suggestions and appreciation about our work.

JITCAR, Volume 10, Number 2

24