What affects the performance of knowledge workers?

Carolyn Stringer, and Rebecca Polson

University of Otago
Paul Shantapriyan**
University of Tasmania

**corresponding author

What affects the performance of knowledge workers?

Abstract

Purpose. The purpose of this research is to examine pay satisfaction, intrinsic and
extrinsic job satisfaction and performance for employees at a large software
company.

Design. This paper draws on a case study using a survey, open-ended comments,
in a large IT company in Australasia.

Findings. We find that Information Technology (IT) professionals, our knowledge

workers, who are satisfied with pay, have higher job satisfaction. We find support
for intrinsic job satisfaction as a mediating variable in the pay satisfaction to job
performance relationship. Whereas, extrinsic satisfaction did not have a
mediating role. Job Satisfaction, as an overall construct, did have a mediating role
in the pay to performance relationship. The positive association between pay
satisfaction and intrinsic job satisfaction challenges the arguments of the
crowding out theory. Further, pay satisfaction is not the largest driver of
performance, challenges another set of extant literature.

Research implications. For knowledge intensive industries such as IT, intrinsic

job satisfaction is a powerful driver of individual performance. Pay satisfaction
as a lever for improving performance, seems to influence intrinsic satisfaction.
This would suggest further research into extrinsic and intrinsic motivation,
including examining autonomous motivation along the lines of Gagne and Deci
(2005).

Practical implications. Our findings have implications for designers of pay for
performance plans used to align highly skilled and motivated IT professionals
who are at the coal face in creating value for the business and customers.

Novelty. Intrinsic motivation is not crowded out by pay. Pay does not directly
drive performance for the knowledge worker. Rather, pay is a hygiene factor,
increasing intrinsic motivation which in turns drives performance.
Keywords: case study, incentive system, extrinsic job satisfaction, intrinsic job
satisfaction, pay satisfaction, performance.

Introduction
Managing human resources is likely to be one of the most challenging tasks in
organisations (Davila, 2005), especially in knowledge intensive industries such as the IT
industry. Intrinsically motivated IT workers are more likely to perceive their jobs as
meaningful, and valuable to others, thus judge their jobs more positively (Thatcher et al.
2006 p136). Innovation is a crucial aspect of the IT industry and requires employees with
knowledge and creativity to construct new projects Employees are a resource that
produce superior performance (Holton et al., 2008), and who can yield long term
sustainable advantage (Spender, 1996).

From this resource based view of the firm

(Colbert 2004; Lado and Wilson 1994; Prahalad and Hamel, 1990; Wernerfelt, 1995;
Wright & McMahan, 1992), strategic human resource management accounting must
recognize the value of people to guide decision making by management (Flamholtz,
1973). In understanding the need to meet the growing expectations on management
accounting, Kominis and Emmanuel (2005) direct future research to draw upon
organisational psychology and human resource literatures to enable future theoretical
development in management accounting.
Compensation issues are currently being scrutinized worldwide, following the sizeable
compensation received by middle and senior managers at failing finance companies and
banks.

Compensation is given or received in recompense for a specific, worthy

behaviour, designed to encourage that behaviour (Macky 2008), which is important

because compensation seek to align employees interests with the goals of the
organisation (Broadbent and Cullen 2005). Common examples of compensation are cash
bonuses, stock options, fringe benefits (Kominis and Emmanuel 2005) and pay for
performance bonuses (Merchant, Van der Stede, and Zheng 2003; Anthony and
Govindarajan 2007).
Understanding the individual levels of satisfaction and attitudes are important (Pfeffer,
2007), particularly in the IT industry (Burgelman, 2002). People are social animals and
importance is placed on inter personal relationships (Pfeffer, 2007), satisfaction with their
pay (Nelson et al 2008) and their job (Judge et al. 2001).

In the IT industry,

understanding the social and psychological processes is important (Bhal and Gulati,
2007). Pay satisfaction and job satisfaction are critical elements in performance outcomes
of knowledge workers. However, we know little about the relationship between pay

satisfaction and job satisfaction on performance for IT professionals, the knowledge

workers in this study.
Pay satisfaction implies that employees perceive that pay reflects their effort compared to
other employees, and that they have been fairly treated by the organisation. Job
satisfaction is the degree to which a person is satisfied with their job. Intrinsic job
characteristics such as challenge, recognition, autonomy, and the work itself; and
extrinsic job characteristics such as pay, promotion, and job security influence an
individuals overall job satisfaction. Pay has been often mentioned as a motivator for
performance and a determinant of job satisfaction (John and Weitz, 1989; Sager, Futrell
and Varadarajan, 1989, Money and Graham 1999), although pay affects job satisfaction
or performance remains unclear.
The purpose of this study is to examine the linkage between pay satisfaction and job
performance in the context of intrinsic and extrinsic job satisfaction. This paper reports
findings of a survey we sent to employees at a large software company in Australasia. We
find that intrinsic job satisfaction mediates the relationship between pay satisfaction and
performance. However, we do not find that extrinsic or overall job satisfaction mediates
the relationship between pay satisfaction and performance.
This study contributes to the extant literature by providing further support that the case
that extrinsic or external rewards crowd out intrinsic rewards or intrinsic job
satisfaction (Kominis and Emmanuel, 2005, 2007; Stringer, Pillai and Didham, 2011).
This paper argues for the importance of looking at the ways in which pay levels, raises,
benefits, structure and administration may be internalised by the individual. Such an
internalization may necessitate further research into autonomous motivation.

IT

professionals are highly skilled and valuable resources that need to be understood,
motivated and remunerated in a manner to retain and attract them.

Literature Review and hypotheses development

There is extensive literature on whether compensation has an influence on employees
who are satisfied with their jobs and whether that job satisfaction is related to intrinsic
job characteristics, extrinsic job characteristics or both. Job satisfaction is an important
factor in the work environment (Huang and Van De Vliert 2003).
In the IT industry, pay satisfaction has a significant positive relationship with job
satisfaction (Thatcher et al. 2006). However, Smits, Tanner and McLean (1995) found

salary importance and attainment do not have a significant relationship with job
satisfaction for IT professionals.

While these IT specific articles lead to different

conclusions, the balance of the literature shapes a positive relationship between overall
pay satisfaction and job satisfaction (Perry 1993; Ellickson and Logsdon 2001).
Relationship between Job Satisfaction and Job Performance
The association between job satisfaction and job performance has been controversial. On
one hand, there is no positive relationship between job satisfaction and job performance,
generally because some academics see them as separate constructs (Greenberger, Strasser,
Cummings and Dunham 1989; Judge et al. 2001). On the other hand, and on the balance
of the literature, there is a positive relationship between job satisfaction and job
performance (Slocum 1971; Ferris 1977; Petty, McGee, and Cavender 1984; Lawler and
Porter 2000; Currall, Towler, Judge and Kohn 2005; Gagne and Deci 2005).
However, the direction of the relationship delivers the most varying views. Riketta
(2008) supports job satisfaction leading to job performance and not the opposite,
whereas, Petty, McGee, and Cavender (1984) argue for both directions. There is an
extensive meta-analysis which supports the positive association between job satisfaction
and job performance and suggests attitudes toward the job should be related to
behaviours on the job, the most central of which is performance on the job (Judge et al.
2001 p378). Therefore, this study breaks down job satisfaction into the two components;
intrinsic satisfaction (the enjoyment of doing the job) and extrinsic satisfaction (doing the
job for the pay or rewards).
This discussion leads to the following hypotheses:
Hypothesis 1:
The relationship between Pay Satisfaction and Job Performance will be
explained by an indirect effect whereby Pay Satisfaction increases Intrinsic
Job Satisfaction and this in turn enhances Performance.
Hypothesis 2:
The relationship between Pay Satisfaction and Job Performance will be
explained by an indirect effect whereby Pay Satisfaction increases Extrinsic
Job Satisfaction and this in turn enhances Performance.

Hypothesis 3:
When job satisfaction is higher, the positive relationship between pay
satisfaction and job performance is stronger.

The Case Study

This research is focused on the information technology (IT) industry, a dynamic
environment where there are rapidly changing technologies and high risk investments
(Anderson, Banker, and Ravindran 2000). Employees in the IT industry are considered
knowledge workers and there is substantial evidence of the growth and importance in
knowledge work (Benson and Brown 2007). In most firms, particularly IT firms, people
are important assets. The employees are the knowledge and it is crucial to retain and
build on knowledge, especially in such a fast-moving, technical field. Intrinsically
motivated IT workers are more likely to perceive their jobs as meaningful, and valuable
to others, thus judge their jobs more positively (Thatcher et al., 2006 p136). Innovation
is a crucial aspect of the IT industry and requires employees with knowledge and
creativity to construct new projects (Tang 1998).
The Case Company
Comptal1 is an Australasian IT company which specializes in technology research and
innovative software development, with over 30 years of experience in the IT industry.
Comptal solutions are used by over 3,000 organisations across Europe, North America
and Asia Pacific. Comptal has experienced substantial growth which is important to
attract and retain employees. A growing organisation finds it easier to attract and retain
motivated staff (Comptal 2008).

Comptal note the importance of its employees

interests: In software, it is staff who make the instructions that bring the hardware to life
enabling it to do valuable things for our communities (Comptal 2008).
The survey was sent to 257 employees at Comptal and all employees were given the
opportunity to participate. The majority of employees who participated were categorized
as senior manager, manager, system developer, consultant, administrator or systems
engineer.
An online questionnaire was chosen over a traditional mail survey as it was more
convenient and less timely for participants (Sax, Gilmartin, and Bryant 2003). The online
questionnaire was designed so participants were not prompted if they missed a question
1 Called Comptal for confidentiality reasons
5

in order to reduce frustration and to encourage a complete response. A separate web-page

was set up for the researcher to keep track of responses obtained. Several participants
only completed the demographic information which still counted as a complete
questionnaire. Consequently, 17 questionnaires were invalid, resulting in a total of 99
completed questionnaires. There were no significant differences between early and late
respondents. The response rate was 30% as shown in Table 1 below.
Table 1: Response Rate
Total employees at Comptal
257
Respondents who indicated
116
they would take part
Questionnaires returned
99
Usable Questionnaires
81
Response rate of all
30%
employees
Response rate of indicated
69%
respondents

Measures
We use well established instruments. The Minnesota Satisfaction Questionnaire (MSQ)
was used to measure job satisfaction (Weiss, Dawis, England and Lofquist, 1967), Pay
Satisfaction Questionnaire (PSQ) developed by Heneman and Schwab (1985). Job
performance was measured by a model developed by Baddoo et al. (2006) that listed key
dimensions of software developers. The list was adapted into an instrument by adding
two sub-scales for participants to rate the importance and their own performance.
The descriptive statistics for the variables are shows in Table 2, Table 3 shows the
correlation matrix and Table 4 presents the high Cronbach Alphas for all the measures,
along with the correlation matrix.

Table 2 Descriptive Statistics

Pay Satisfaction
PS - Level
PS - Benefits
PS - Raises
PS S&A
Job satisfaction
Intrinsic job satisfaction
Extrinsic job satisfaction
Performance

No.

Minimum

Mean

Maximum

99
99
99
99
99
90
90
89
82

1.27
1
1.33
1
1.17
2.42
2.5
1.5
3.82

3.4
3.48
3.86
3.15
3.1
3.68
3.78
3.39
5.42

4.79
5
5
4.75
4.4
4.63
4.75
4.50
6.88

Standard
Deviation
.67
.96
.90
.85
.67
.46
.44
.65
.58

Table 3 Correlation Matrix

Pay
Satisfaction
Job
satisfaction
Intrinsic
job
satisfaction
Extrinsic job
satisfaction
Performance
Note: * p<.05,

Pay
satisfaction
1.0

Job
satisfaction

Intrinsic job
satisfaction

.678***

1.0

.539***

.915***

1.0

.730***

.855***

.612***

Extrinsic job
satisfaction

Performance

1.0

.249*
.433***
.484***
.237*
1.0
** p<.01, ***p<.001 (one tailed). Cronbach Alphas are reported on the

diagonal.

Table 4 Cronbach alpha

Measure
Pay Satisfaction Overall
Pay Level
Pay Raise
Pay Benefits
Pay Structure & Admin
Job Satisfaction Overall
Intrinsic job satisfaction
Extrinsic job satisfaction
Performance Overall
Importance
Level of performance

Cronbach
alpha
0.937
0.974
0.833
0.838
0.850
0.874
0.811
0.77
0.907
0.851
0.856

Pay Satisfaction
Confirmatory factor analysis distinguished the degree to which a theoretical structure
matched the observed structure (Meyers, Gamst, and Guarino 2006). Pay satisfaction
theoretically has four sub constructs: pay level, benefits, raises, and structure and
administration. Pay satisfaction principal component factor analysis produced a
favourable KMO of 0.892, which Kaiser (1974) suggested was sufficient as a measure of
sampling adequacy. When four factors were extracted, the variance explained by these
four factors was 76%. The resulting factor analysis confirmed the four sub constructs of
pay satisfaction (Heneman and Schwab, 1985).
Job Satisfaction
Job satisfaction has two sub constructs: intrinsic and extrinsic. Principal component
factor analysis with promax rotation as we did not want to impose orthogonality on the
sub scales. This factor produced a favourable KMO of 0.892. However there was low
variance explained of 42%. The confirmatory factor analysis supported the two sub
constructs of job satisfaction (intrinsic and extrinsic).
In examining the intrinsic elements of job satisfaction, the mean was 3.78 while the mean
for extrinsic elements of the job was 3.39. This would indicate that the intrinsic levels of
satisfaction were higher than the extrinsic elements of job satisfaction.
Performance
Job Performance comprises 17 questions which were rated on Importance as well as How
well did you perform. See Appendix for the instrument on job performance.
The core variables used in the empirical tests of the hypotheses include the following:
Pay Satisfaction The mean of the nineteen questions from the PSQ (see Appendix 2).
The theoretic ranges were 1 to 5.
Job Satisfaction

Intrinsic Job Satisfaction

Extrinsic Job Satisfaction

The mean of nineteen questions of the MSQ (see

Appendix 2)
The mean of three questions (The theoretic ranges
were 1 to 5.
The mean of seven questions and had a theoretic
range of 1 to 5

The variables included in the analyses as possible explanatory or control variables

included:
Gender

A dummy variable was used to distinguish between male and

female.
Age
Dummy variables were used to distinguish groups: 15-24;.
Year of service in Industry:
continuous variable
Year of service in Case Company:
continuous variable

Discussion and Analysis of the Findings

We use quantitative and qualitative data to examine the effects of intrinsic and extrinsic
work motivation on the pay satisfaction and job satisfaction.
Path analysis Hypothesis 1
In testing for Hypothesis 1, we examine the relationships between pay satisfaction and
intrinsic job satisfaction and job performance (see Figure 1).

Next, we examine the

relationships between pay satisfaction and intrinsic job satisfaction on performance. We

find that the indirect path term as the path between pay satisfaction and intrinsic
motivation is significant (Hypothesis 1 is supported). Interestingly, the direct relationship
between pay satisfaction and performance is not significant (see Figure 1, Tables 5, 6).
Figure 1. Overall path analysis of Pay Satisfaction and Performance: Intrinsic Job
satisfaction mediating

Intrinsic Job
Satisfaction 2
p21 0.539***
Pay
Satisfaction 1

p32 0.635***

p31 0.132

Note: * p<.05, ** p<.01, ***p<.001

p21 is the path coefficient linking Intrinsic Job Satisfaction and Pay Satisfaction
p31 is the path coefficient linking Performance and Pay Satisfaction
p32 is the path coefficient linking Performance and Intrinsic Job Satisfaction

Performance 3

Table 5: Zero-Order Correlations Among Variables

Zero-Order Effect
Pay Satisfaction/ Intrinsic Job
Satisfaction
Pay Satisfaction/ Performance
Intrinsic Job Satisfaction /
Performance

r = 0.539

p < 0.0001

r = 0.249
r = 0.484

p < 0.05
p < 0.0001

Table 6: Derivation of Path Coefficients p31 and p32

Coefficient

Value

Standard Error

p31

0.132

0.114

1.158

< 0.3

p32

0.635

0.152

4.186

< 0.0001

Adj. R2 = 0.248

F = 5.236

p < 0.0001

Table 7: Decomposition of Effects in Model

Linkage
Pay Satisfaction/
Intrinsic job
satisfaction
Pay Satisfaction/
Performance
Intrinsic job
satisfaction /
Performance

Total Effect
(rij)

Direct
Effect (pij)

0.539

0.539

0.249

0.132

0.484

0.635

10

Indirect
Effect

Spurious
Effect

0.117
-0.151

Path Analysis Hypothesis 2

Figure 2: Overall path analysis of Pay Satisfaction and Performance: Extrinsic Job
Satisfaction intervening

Extrinsic Job
Satisfaction 2
p21 0.730***
Pay
Satisfaction 1

p32 0.137

p31 0.265

Performance 3

Note: * p<.05, ** p<.01, ***p<.001

p21 is the path coefficient linking extrinsic job satisfaction and pay satisfaction
p31 is the path coefficient linking performance and pay satisfaction
p32 is the path coefficient linking performance and extrinsic job satisfaction

We find that the lack of significant results for extrinsic motivation (as a mediating
variable) suggests that a larger degree of performance can be achieved through pay
satisfaction alone (24.9% correlation, Table 9), rather than through pay satisfaction and
extrinsic motivation path. These findings are consistent with such researchers as Cable
and Judge (1994) and Patchen (1961) who argue that compensation (as measured by pay
satisfaction) is in fact a determinant of job satisfaction, as pay satisfaction in our study
has the ability to affect the overall level of satisfaction with ones job.
With regard to Hypothesis 2, the null could not be rejected which indicated that extrinsic
motivation was not an intervening variable. The significance of the pay satisfaction and
the intrinsic job satisfaction warrants closer examination as they are in contrast with
Cognitive Evaluation Theory CET which proposes that performance-contingent
financial incentives erode intrinsic motivation and thereby diminish task performance
(Deci, 1971 cited in Jenkins et al., 1998).

11

Table 8: Zero-Order Correlations Among Variables

Zero-Order Effect
Pay Satisfaction/ Extrinsic
Job Satisfaction
Pay Satisfaction/
Performance
Extrinsic Job Satisfaction/
Performance

r = 0.730

p < 0.0001

r = 0.249

p < 0.05

r = 0.237

p < 0.05

Table 9: Derivation of Path Coefficients p31 and p32

Coefficient

Value

Standard Error

p31

0.265

0.148

1.786

< 0.1

p32

0.137

0.149

0.917

< 0.5

Adj. R2 = 0.074

F = 2.020

p < 0.1

Table 10: Decomposition of Effects in Model

Linkage
Pay Satisfaction/
Extrinsic Job
Satisfaction
Pay Satisfaction/
Performance
Extrinsic Job
Satisfaction /
Performance

Total Effect
(rij)
0.730

Direct
Effect (pij)
0.730

0.249

0.265

0.237

0.137

12

Indirect
Effect

Spurious
Effect

-0.016
0.100

Path analysis Hypothesis 3

Figure 3: Overall path analysis of Pay Satisfaction and Performance: Overall Job
Satisfaction mediating

Job
Satisfaction 2
p21 0.678***

p32 0.629***

Pay
Satisfaction 1

p31 0.078

Performance 3

Note: * p<.05, ** p<.01, ***p<.001

p21 is the path coefficient linking Job Satisfaction and Pay Satisfaction
p31 is the path coefficient linking Performance and Pay Satisfaction
p32 is the path coefficient linking Performance and Job Satisfaction

Table 7: Zero-Order Correlations Among Variables

Zero-Order Effect
Pay Sat/Job Sat

r = 0.678

p < 0.0001

Pay Sat/Performance

r = 0.249

p < 0.05

Job Sat /Performance

r = 0.433

p < 0.0001

Table 8: Derivation of Path Coefficients p31 and p32

Coefficient

Value

Standard Error

p31

0.078

0.129

0.605

< 0.5

p32

0.629

0.174

3.617

< 0.001

Adj. R2 = 0.208

F = 4.380

p < 0.001

Table 9: Decomposition of Effects in Model

Linkage

Total Effect

Direct

13

Indirect

Spurious

(rij)

Effect (pij)

Pay Sat/Job Sat

0.678

0.678

Pay

0.249

0.078

0.433

0.629

Effect

Effect

0.171

Sat/Performance
Job
Sat/Performance

-0.104

This research investigated the role of pay satisfaction and job satisfaction on job
performance (Hypothesis 3). The overall findings suggest that job satisfaction is a
significant explanatory of performance which was consistent with prior literature (Judge
et al 2001; Judge et al 2008). What is curious is that pay satisfaction is not the largest
driver of performance which challenges another set of extant literature (Williams et al,
2007; Tekleab et al, 2005; Sweins et al 2008). We undertake a robustness test to see if
job satisfaction is a moderator as well as break job performance into each of the theoretic
components. To undertake this robustness test, we use the importance of each dimension
of performance as a dependent variable in Table 10 below and the actual performance
along the dimensions as the dependent variable on the extreme right.
Further analysis
Moderated regression analysis is appropriate because it allows the implied interaction
terms to be directly examined (Schoonhoven, 1981; Darrow & Kahl, 1982; Covin &
Slevin, 1989). One of the limitations of using moderated regressions is that the product
cross moment terms are likely to suffer from multicollinearity as the product cross
moment is likely to be strongly correlated with the terms that compose it (Dewar &
Werbel, 1979; Drazin & Van de Ven, 1985; Fry & Slocum, 1984). There are two issues
that counter this criticism. First, multicollinearity is a "question of degree rather than
kind" (Kimenta, 1986, p.431), indicating that there is some kind of multicollinearity
between most regressors. If, however, the multicollinearity is strong with the correlation
between regressors exceeding 0.8 (Lewis & Beck, 1980), or the Variance Inflation
Factors (VIF) scores exceeding 10 (Myers, 1990), there is cause for concern. Second,
this multicollinearity can be reduced if not eliminated by manipulating the origin points
for each of the independent variables (Smith & Sasaki, 1979; Southwood, 1978). Such a
shift in the axes does not affect the significance of the interaction coefficient
(Govindarajan & Fisher, 1990). Furthermore, shifting the axes does not affect the R2 and
F ratio for the whole equation (Govindarajan & Gupta, 1985). An alternate way of
reducing multicollinearity would be to centre each variable of the product cross moment

14

(Cronbach, 1987; Jaccard, Turrisi & Wan, 1990). For example if A.E is the cross product
term of the main effects A and E, then centring would derive the term as (A-)(E-).
While centring each of the components of the cross product terms can reduce
multicollinearity, the interpretation of the cross product moment term is undertaken with
caution. Cohen & Cohen (1983) advise the use of hierarchical regression to test the
significance of the product cross moment term, rather than placing emphasis on the sign
and significance of the regression weight of the product cross moment term. However,
Blanton and Jaccard (2006) do suggest a test of the multiplicative model should therefore
focus on the significance of the regression coefficient for the product term in a model that
includes both the component parts of the product term as well as the product term.
Accordingly, we do a hierarchical model development by developing the first model
where the main effects of pay satisfaction (into the four sub components) and job
satisfaction (into the two subcomponents) are examined. Then a further model with the
multiplicative interaction term is used. Following Cronbach's (1987) opinion, we centered
the independent variables on their means before creating the interaction terms. While
Ehmabadi & Hess (2007) caution that mean centering does not always eliminate
multicollinearity between the main effect variable and the interaction term. We find that
the Variance Inflation Factor is below 4 and therefore multicollinearity is not severe
(Jaccard, Turrisi & Wan, 1990).

However, the interaction term coefficient is not

significant as the F falls as does the adjusted R2.

In each of the robustness tests against the key components of job performance, intrinsic
satisfaction remains a main effect while the interaction between job satisfaction and pay
satisfaction is not significant (see Table 10).

15

Table 10 Hierarchical moderated regression

Performance

Performance

Performance

overall

Importance

ratings

2.492(.538)***
0.129 (.091)
0.154 (0.076)*
-0.138 (0.117)
-0.027 (0.127)
-0.013 (.158)
+0.635 (0.169)***
0.088(0.162)
0.101 (0.111)
-0.012 (0.012)
-0.005 (0.011)
Adj. R2 = .0.258

2.82192(.609)***
0.178 (.105)
0.085 (0.090)
-0.289 (0.140)*
-0.007 (0.143)
0.193 (.185)
+0.471 (0.199)*
-0.060 (0.181)
0.177 (0.123)
-0.004 (0.014)
-0.011 (0.012)
Adj. R2 = 0.172

2.420(.596)***
0.061 (.101)
0.205 (0.084)*
0.021 (0.129)
-0.100 (0.141)
-0.191 (.176)
+0.743 (0.187)***
0.244 (0.180)
0.056 (0.123)
-0.015 (0.013)
0.001 (0.012)
Adj. R2 = 0.245

F = 3.681***

F = 2.684**

F = 3.503***

Hiearchical
Regression 1
Intercept
Pay Satisfaction Level
Pay Satisfaction Benefit
Pay Satisfaction Raises
Pay Satisfaction S&A
Extrinsic Satisfaction
Intrinsic Satisfaction
Genderd
Agee
Service for company
Year of Service industry

Hierarchical Regression 2 with Interaction Term

Interaction Pay Sat x Job

.062 (.219)

-0.274 (0.221)

0.309 (0.240)

Adj. R2 = .0.248

Adj. R2 = 0.178

Adj. R2 = 0.253

F = 3.308***

F = 2.598**

F = 3.365***

Satisfaction

* p<0.05
**p<0.01
***p<0.001

Conclusion
Job satisfaction was the most influential variable on key elements of job performance.
For IT professionals in large Australasian companies, job satisfaction could be a powerful
explanatory of performance, supported by prior literature (Thatcher et al. 2006). Our
findings show that intrinsic job satisfaction has a stronger influence on performance than
pay satisfaction alone (hypothesis 1 is supported). There is no support for hypothesis 2
that extrinsic job satisfaction mediates the relationship between pay satisfaction and
performance. Hypothesis 3 is also not supported. Further testing shows that it is intrinsic
job satisfaction that is driving the results, not overall job satisfaction.
A main contribution of this study is that intrinsic job satisfaction is an important driver of
performance, rather than pay satisfaction. We find that intrinsic job satisfaction has a
significant positive relationship with job performance, and this is supported by prior
literature (Berry 1993; Pool 1997; Lu 1999). Interestingly, this finding challenged the
crowding out effect that arose in CET, which argued there was a trade-off between
intrinsic and extrinsic motivation when intrinsic motivation existed (Osterloh and Frey

16

2000). Therefore, the findings supported the complementary use of intrinsic and extrinsic
motivation (Kunz and Pfaff 2002; Merchant, Van der Stede, and Zheng 2003; Kominis
and Emmanuel 2007).
The practical contributions of this study include that the findings will be useful for
practitioners in terms of designing incentives. The research demonstrated that motivated
employees generally had higher job performance; which was supported by reinforcement
theory, that is, employees who were motivated by incentives, relate prior positive
experiences to good performance (Katzell and Thompson 1990). Intrinsic motivation had
a significant positive relationship with job performance so practitioners could combine
intrinsic variables with monetary reinforcements to increase performance.
The results of this case study were subject to limitations. Reliability and validity are
common limitations to any study; however, a considerable limitation was external
validity. The case study and exploratory nature of this study means that these results
cannot be generalised across other companies.

A second limitation is that the

performance instrument in the questionnaire presented a self-reporting bias, where

participants were likely to be lenient on how they perceived their own performance. The
model used a self-rating method over a superior-rating because it solved the problem of
halo error (Heneman 1974), that is, the tendency to evaluate on only one cognitive
dimension (Kominis and Emmanuel 2007 p59). Also, time and cost constraints meant a
self-rating method was more practical for this research.
Future research on pay satisfaction, job satisfaction and performance is needed in the
management accounting and information technology fields. Most of the literature in this
area has been published in management journals.
The IT industry will continue to be of interest in terms of how to motivate and satisfy
employees in highly innovative jobs.

A specific focus on creativity could present

interesting findings to see if those employees who are motivated creatively, are also
satisfied in their job and exceeding performance expectations on individual projects. This
could also be subject to the type of IT job; speculating, for example, whether software
designers are motivated differently to IT managers.

Bibliography
Amabile, T.M. (1998). How to Kill Creativity, Harvard Business Review, 76 (5), 77-87.

17

Ambrose, M.L. & Kulik, C.T. (1999). Old Friends, New Faces: Motivation Research in
the 1990s, Journal of Management, 25 (3), 231-292.
Anderson, M.C., Banker, R., Ravindran, S.(2000) Executive Compensation in the
Information Technology Industry Management Science 46, 4, pp 530 - 547
Anthony, R. and Govindarajan, V., 2007, Management Control Systems, 12th Edition,
Irwin, Singapore.
Benson, J. & Brown, M., (2007)Knowledge workers: what keeps them committed what
turns them away Work Employment and Society 21, 1, pp. 121 - 141
Berry, W., 1993, HRIS Can Improve Performance, Empower and Motivate Knowledge
Workers, Employment Relations Today 20, 297-297.
Blanton, H., & Jaccard, J. (2006) Tests of multiplicative models in psychology: a case
study using unified theory of implicit attitudes and self concept Psychological Review
113, 1, pp115 - 169
Bodur, S. (2002). Job satisfaction of health care staff employed at health centres in
Turkey, Occupational Medicine, 52 (6), 353.
Broadbent, M. & Cullen, J. (2005). Divisional control and performance, In
Management Control: Theories, Issues and Performance, Berry, A.J., Broadbent, J. &
Otley, D., (Eds), Palgrave Macmillan: Hampshire, 137-166.
Cable, D.M. & Judge, T.A. (1994). Pay preferences and job search decisions: A person
organisation fit perspective, Personnel Psychology, 47, 317-348.
Carr, S.C., McLoughlin, D., Hodgson, M. & MacLachlan, M. (1996). Effects of
unreasonable pay discrepancies for under- and over-payment on double demotivation.
Genetic, Social, and General Psychology Monographs, 122: 475-494.
Champion, R. (2002). Sampling can produce solid results, Journal of Staff
Development, 23 (1), 1-3.
Cimete, G, Gencalp, N.S. & Keskin, G. (2003). Quality and life and job satisfaction of
nurses, Journal of Nursing Care Quality, 18 (2), 151-159.
Coakes, S.J. & Steed, L.G. (2001). SPSS Analysis without Anguish, Wiley and Sons:
Australia, Version 10.0.
Cohen, J., P. Cohen. 1983. Applied Multiple Regression: Correlation Analysis for
Behavioral Sciences 2nd ed. Lawrence Erlbaum Associates, Hillsdale, NJ.
Colbert, B. A. (2004). The complex resource-based view: Implications for theory and
practice in strategic human resource management. Academy of Management Review, 29,
341-358.
Covin, J.G., & Slevin, D.P. (1989), "Strategic Management of Small Firms in Hostile and
Benign Environments" Strategic Management Journal, Vol. 10, 1989, pp. 75 - 87.
Cronbach, L.J.,(1987) Statistical Tests for Moderator Variables: Flaws in Analysis
Recently Proposed Psych. Bulletin. 102(3) 414-417
Darrow, A.L & D.R. Kahl,(1982) " A Comparison of Moderated Regression Techniques
Considering Strength of Effect" Journal of Management, Vol. 8, pp. 35 - 47.
Davila, T. (2005) An exploratory study on the emergence of management control
systems: formalizing human resources in small growing firms Accounting
Organizations and Society 30, pp. 223 - 248
Dewar R., & Werbel, J. (1979) "Universalistic and Contingency Predictions of Employee
Satisfaction and Conflict" Administrative Science Quarterly, Vol. 24, pp. 426 - 448.

18

Deci, E.L. (1971). Effects of externally mediated rewards on intrinsic motivation,

Journal of Personality and Social Psychology, 18, 105-115.
Deci, E. L., & Ryan, R. M. (1985). Intrinsic Motivation and Self-Determination in
Human Behavior, New York: Plenum.
Deming, W.E. (1986). Out of the Crisis, MIT Centre for Advanced Engineering Study,
Cambridge, MA.
Dermer, J.D. (1975). The Interrelationship of Intrinsic and Extrinsic Motivation,
Academy of Management Journal, March, 125-129.
Drazin, R., and A.H. Van de Ven, (1985)"An Examination of Alternative Forms of Fit in
Contingency Theory" Administrative Science Quarterly, (Vol. 30,), pp. 514 - 539.
Ellickson, M., and K. Logsdon, (2001), Determinants of job satisfaction State & Local
Government Review, 173-184.
Echambadi, R., Hess, J.D. (2007) Mean centering does not alleviate collinearity problems
in moderated multiple regressions Marketing Science 26, 3, pp. 438 - 445
Festinger, L. A. (1954). Theory of Social Comparison Processes, Human Relations, 7,
117.
Flamholtz, E., (1973) Human Resource Accounting: Measuring positional replacement
costs Human Resource Management Spring pp 8-16
Frey, B.S. & Osterloh, M. (2002). Successful Management by Motivation, Balancing
Intrinsic and Extrinsic Incentives, Springer-Verlag, Berlin.
Fry, L.W., & J.W.Slocum, (1984)Technology, Structure and Workgroup Effectiveness: A
Test of a Contingency Model" Academy of Management Journal, ( Vol. 27,) pp. 221 246.
Gagne, M., & Deci, E.L. (2005). Self-determination theory and work motivation,
Journal of Organizational Behavior, 26, 331-362.
Govindarajan, V. & J. Fisher, "Strategy, Control Systems and Resource Sharing: Effects
on Business Unit Performance" Academy of Management Journal, (Vol. 33, No. 2, 1990),
pp. 259 - 285.
Govindarajan V. & A.K. Gupta, "Linking Control Systems to Business Unit Strategy:
Impact on Performance" Accounting, Organisations and Society (Vol. 10, No. 1, 1985),
pp. 51 - 66.
Gupta, N., & Mitra, A. (1998). The value of financial incentives: myths and empirical
realities, ACA Journal, 58-66.
Heneman, H.G & Schwab, D.P (1985). Pay satisfaction: Its multidimensional nature and
measurement, International Journal of Psychology, Vol. 20, pp. 129-141.
Heneman, R.L. (1992). Merit pay: Linking pay increases to performance ratings.
Addison-Wesley-Longman: Readings, MA.
Heneman, R.L., (2002). Corporate Business Strategies and Compensation Strategies,
189-211. In Heneman, R.L. (ed.), Strategic Reward Management: Design,
Implementation, and Evaluation, Information Age Publishing: Connecticut.
Heneman, R.L., Eskew, D., & Fox, J. (1998). Using employee attitudes to evaluate a
new incentive program. Compensation and Benefits Review, 18 (1), 40-44.
Heneman, R.L., Fay, C.H. and Wang, Z. (2002a). Compensation Systems in the Global
Context, 5-34. In Heneman, R.L. (ed.), Strategic Reward Management: Design,
Implementation, and Evaluation, Information Age Publishing: Connecticut.

19

Heneman, R.L., Ledford, G.E. Jr. & Gresham, M.T. (2002b). The changing nature of
work and its effects on compensation design and delivery, 35-73. In Heneman, R.L.
(ed.), Strategic Reward Management: Design, Implementation, and Evaluation,
Information Age Publishing: Connecticut.
Herzberg, F. (2003). One more time: How do you motivate employees? Harvard
Business Review, 46, January-February, 53-62.
Holton, E.F. & B. Yamkovenko (2008) Strategic Intellectual Capital Development: A
defining paradigm for HRD? Human Resource Development Review No 7: pp 270 - 291
Houston, D.J. (2009). Motivating Knights or Knaves? Moving Beyond PerformanceRelated Pay for the Public Sector, Public Administration Review, 69 (1).
Huang, X & Van de Vliert (2003) Where intrinsic job satisfaction fails to work: National
moderators of intrinsic motivation, Journal of Organisational Behaviour, 24 (2), 159.
Igalens, J. & Roussel, P. (1999) A study of the relationship between compensation
package, work motivation and job satisfaction, Journal of Organisational Behaviour, 20
(7), 1003 1025.
Jaccard, J. R., R. Turrisi, C. K. Wan. (1990). Interaction Effects in Multiple Regression.
Sage Publications, Newbury Park, CA
Jenkins, G.D., Mitra, A., Gupta, N., & Shaw, J.D. (1998). Are Financial Incentives
Related to Performance? A Meta-Analytic Review of Empirical Research, Journal of
Applied Psychology, 83 5, 777-787.
Kerr, S. (1975). On the Folly of Rewarding A, While Hoping for B, The Academy of
Management Journal, 18 (4), 769-783.
Kmenta, J. Elements of Econometrics (New York: Macmillan Publishing Company)
1986 .
Kohn, A. (1993). Why Incentive Plans Cannot Work, Harvard Business Review,
September-October, 54-63.
Kominis, G., & Emmanuel, C.R. (2005). Exploring the reward preferences of middle
level managers, Qualitative Research in Accounting and Management, 2 (1) p.54-76.
Kominis, G., and C. Emmanuel, 2007, The expectancy-valence theory revisited:
Developing an extended model of managerial motivation, Management accounting
research 18, 49-75.
Kunz, A. & Pfaff, D. (2002) Agency theory, performance evaluation and the
hypothetical construct of intrinsic motivation, Accounting, Organisations and Society,
27 (3), 275.
Lado, A. A., & Wilson, M. C. (1994). Human resource systems and sustained competitive
advantage: A competency-based perspective. Academy of Management Review, 19, 699727
Lawler III, E.E. (1971). Pay and organisational effectiveness: A psychological view,
McGraw-Hill, New York, NY.
Lawler, E. E. III. (1990). Strategic Pay, Aligning Strategies and Pay Systems, JosseyBass Inc: San Francisco.
Lawler, E. E. III. (2000). Rewarding Excellence, Pay Strategies for the New Economy,
Jossey-Bass Inc: San Francisco.
Lee, S-Y, Whitford, A.B. (2007). Exit, Voice, loyalty, and Pay: Evidence from the Public
Workforce, Journal of Public Administration Research and Theory, 18, p.648-671.
20

Lepper, M.R. & Greene, D. (1978). Overjustification research and beyond: towards a
means-ends analysis of intrinsic and extrinsic motivation. In M.R. Lepper & D Greene
(Eds.) The hidden costs of reward: new perspectives on the psychology of human
motivation, (109-148), Hillsdale: Erlbaum.
Lewis-Beck, M.S., Applied Regression Sage University Paper Series on Quantitative
Applications in the Social Sciences, (Beverley Hills and London: Sage Publications ).
1980
Locke, E.A., & Latham, G.P. (1990). A Theory of Goal setting and Task Performance,
Prentice Hall: New Jersey.
Lu, L., 1999, Work Motivation, Job Stress and Employees' Well-being, Journal of
Applied Management Studies 8, 61-72.
Merchant, K.A. (1989). Rewarding Results: Motivating Profit Centre Managers, Harvard
Business School Press, Boston, M.
Merchant, K. A., Van der Stede, W., & Zheng, L. (2003).Disciplinary constraints on the
advancement of knowledge: the case of organizational incentive systems. Accounting,
Organizations & Society, 28, 251-286.
Miceli, M.P. & Heneman, R.L. (2002). Contextual determinants of variable pay plan
design: a proposed research framework. In Strategic reward Management: Design,
Implementation, and Evaluation, Heneman, R.L. (ed.) Information Age Publishing:
Connecticut, 213-231.
Miles, M.B. & Huberman, A.M. (2004). An expanded Sourcebook, Qualitative Data
Analysis, Second edition, Sage Publications: California.
Myers, R.H., Classical and Modern Regression with Applications, (Boston: PWS Kent)
1990
Nadler, D.A. & Lawler, III, E.E. (1977). Motivation: A Diagnostic Approach, McGrawHill, New York, NY.
Oshagbemi, T. & Hickson, C. (2003) Some aspects of overall job satisfaction: A
binomial logit model, Journal of Managerial Psychology, 18 (4), 357-368.
Osterloh, M., and B. Frey, 2000, Motivation, knowledge transfer, and organizational
forms, Organization science, 538-550.
Osterloh, M. & Frey, B.S, (2002). Does pay for performance really motivate
employees? Business Performance Management, University Press, Cambridge, UK.
Ostroff, C., 1992, The relationship between satisfaction, attitudes, and performance: An
organizational level analysis, Journal of Applied Psychology 77, 963-974.
Patchen, M. (1961). The choice of wage compensations, Prentice Hall, Englewood Cliffs.
Perry, L., 1993, Effects of inequity on job satisfaction and self-evaluation in a national
sample of African-American workers, The Journal of social psychology 133, 565
Pfeffer, J. (1998). Six dangerous myths about pay, Harvard Business Review, MayJune, 109-119.
Pool, S., 1997, The Relationship of Job Satisfaction with Substitues of Leadership,
Leadership Behavior, and Work Motivation, The Journal of Psychology 131, 271-283.
Porter, L.W. & Lawler III, E.E (1968). Managerial Attitudes and Performance, Irwin,
Homewood. IL.

21

Prahalad, C. K., & Hamel, G. (May-June 1990). The core competence of the corporation.
Harvard Business Review, 79-91.
Prendergast, C. (1999). The provision of incentives in firms, Journal of Economic
Literature, 37, 7-63.
Ryan R.M., & Deci, E.L. (2000a). Intrinsic and Extrinsic Motivations: Classic
Definitions and New Directions, Contemporary Education Psychology, 25, 54-67.
Ryan, RM, & Deci, EL. (2000b). When rewards compete with nature: The undermining
of intrinsic motivation and self-regulation, 13-54. In Sansone, C, & Harackiewicz, JM
(Eds.), Intrinsic and extrinsic motivation: The search for optimal motivation and
performance, New York: Academic Press.
Ryan, RM, & Deci, EL. (2000c). Self-determination theory and the facilitation of
intrinsic motivation, social development, and well-being. American Psychologist, 55,
68-78.
Sax, L., S. Gilmartin, and A. Bryant, 2003, Assessing response rates and nonresponse
bias in web and paper surveys, Research in Higher Education 44, 409-432.
Schoonhoven, C.B.( 1981) "Problems with Contingency Theory: Testing Assumptions
hidden within the language of Contingency Theory" Administrative Science Quarterly,
(Vol. 26,), pp. 349 - 377.
Skinner, B.F. (1953). Science and human behavior, New York, MacMillan.
Smith, K.W. & M.S. Sasaki (1979) , "Decreasing Multicollinearity: A Method for Models
with Multiplicative Functions" Sociological Methods and Research, (Vol. 8, No. 1,), pp.
35 - 56.
Southwood, K.E., (1978) "Substantive Theory and Statistical Interaction: Five Models",
American Journal of Sociology, (Vol. 83,), pp. 1154 - 1203.
Spender, J. C. (1996). Competitive advantage from tacit knowledge? Unpacking the
concept and its strategic implications. In B. Moingeon & A. Edmondson (Eds.),
Organizational learning and competitive advantage (pp. 56-73). Thousand Oaks, CA:
Sage
Symons, R.T. & Jacobs, J.A. (1995). A total quality management-based incentive system
supporting total quality management implementation, Production and Operations
Management, 4 (3), 228.
Sweins, C., & Kalmi, P., Pay Knowledge Pay Satisfaction and employee commitment:
evidence from Finnish profit-sharing schemes Human Resource Management Journal
18, 4 pp 366 - 385
Tang, H.K. (1998) An integrative model of innovation in organizations Technovation 18
5 pp. 297 - 309
Tekleab, A., Bartol, KM, Liu, W (2005) is it pay levels or pay raises that metter to
fairness and turnover? Journal of Organizational Behaviour 26, pp. 891 - 921
Thatcher, J.B., Liu, Y., Stepina, L.P. Goodman, J.M. Treadway, D.C. (2006) IT worker
Turnover: An empirical investigation of Intrinsic Motivation The DATA BASE for
Advances in Information Systems - Spring-Summer pp. 133 - 146
Vroom, V.H (1964). Work and Motivation. Wiley, New York.
Weiss D.J, Dawis R.V, England G.W, & Lofquist L.H. (1967). Manual for the Minnesota
Satisfaction Questionnaire, Minneapolis: Industrial Relations Center, University.

22

Wernerfelt, B. (1995). The resource-based view of the firm: Ten years after. Strategic
Management Journal, 16, 171-174.
Williams, M., McDaniel, M.A, & Ford, LR, (2007) Understanding multiple dimensions
of compensation satisfaction journal of Business and Psychology 21, 3., pp 429 - 459
Wright, P. M., & McMahan, G. C. (1992). Theoretical perspectives for strategic human
resource management. Journal of Management, 18, 295-320.

23

Appendix 1
Table 11 Factor for Pay Satisfaction
Rotated Component Matrixa
Component
1

Level

.889

.285

.174

.175

Benefits

.551

.068

.277

.500

Raise

.284

.270

.199

.709

Level

.889

.299

.116

.210

Benefits

.058

.097

.921

.165

Raise

.607

-.005

.139

.496

Structure & Admin

.302

.725

.160

.283

Structure & Admin

.233

.429

.486

.345

Level

.875

.318

.141

.185

Benefits

.221

.099

.888

.189

Raise

.386

.332

.208

.662

Structure & Admin

.322

.752

.032

.145

Structure & Admin

.086

.824

.124

.313

Level

.867

.336

.171

.198

Benefits

.181

.128

.916

.099

Structure & Admin

.176

.802

.038

.341

Structure & Admin

.171

.594

.148

-.084

Raise

.140

.225

.142

.748

Pay Level

Pay

Pay

Pay raise

Structure

benefits

& Admin
Extraction Method: Principal Component Analysis.
Rotation Method: Varimax with Kaiser Normalization.
a. Rotation converged in 6 iterations.

24

Table 12 Factor Analysis of Job Satisfaction

Rotated Component Matrixa
Component
1
JobSatisfaction_Q1

2
.334

.299

JobSatisfaction_Q2

.356

.263

JobSatisfaction_Q3

.609

.308

JobSatisfaction_Q4

.277

JobSatisfaction_Q5

.913

JobSatisfaction_Q6

.908

JobSatisfaction_Q7
JobSatisfaction_Q8
JobSatisfaction_Q9
JobSatisfaction_Q10
JobSatisfaction_Q11

.714

JobSatisfaction_Q12

.503

JobSatisfaction_Q13

.422

JobSatisfaction_Q14

.637

JobSatisfaction_Q15

.692

JobSatisfaction_Q16

.762

JobSatisfaction_Q17

.619

JobSatisfaction_Q18

.387

JobSatisfaction_Q19

.335

.631

Extrinsic

Intrinsic

Satisfaction

Satisfaction

Extraction Method: Principal Component Analysis.

Rotation Method: Varimax with Kaiser Normalization.
a. Rotation converged in 3 iterations.

25

Table 13 Job Performance

Rotated Component Matrixa
Component
1

performance on innovation

.511

performance
proactive

on

being

.818

performance
itself

work

driven

.592

3
0.467

high technical competence

.906

resolving complex problems

.824

strong
knowledge
problem domain

of

.642

flexible and adaptable

eagerness
technology

to

try

.584
new

.573

prepared to work long hours

.653

communicate
stakeholders
sharing
team

well

with

.640

knowledge

with

.456

adhering to process

.780

fully documenting your work

.843

meeting time deadlines

.717

meeting budget deadlines

.706

meeting
expectations
performance
dependence

customer
on

.575

self

.502

.711
Profession
al
orientation

Process
and
outcomes

Extraction Method: Principal Component Analysis.

Rotation Method: Varimax with Kaiser Normalization.
a. Rotation converged in 6 iterations.

Extraction Method: Principal Component Analysis.

Rotation Method: Promax with Kaiser Normalization.

26

innovation

Appendix 2 Performance Instrument

Performance
Listed below are some of the key dimensions of performance. Please indicate
how important these items are to you.
How important are these items in doing your
job?
Performance Measures

Unimport
ant

Moderate
ly
important

Extremel
y
importan
t

1 Self-dependence

2 Innovation

3 Being proactive

4 Being driven by the work

itself

5 High technical competence

6 Resolving complex problems

7 Strong knowledge of problem

domain

8 Being flexible and adaptable

9 Eagerness
technology

new

1 Being prepared to work long

0 hours

1 Communicating
1 stakeholders

with

1 Sharing knowledge with the

2 team

1 Adhering to process
3

1 Fully documenting your work

4

1 Meeting time deadlines

5

1 Meeting budget deadlines

6

1 Meeting
7 expectations

to

try

well

customer

27

1
8

Other: (Please specify)

Any comments on what is important in doing your job

(optional):
Please indicate how well you rate yourself on your job
performance:

How well do you perform on these

measures?
Performance Measures

Well
below
expectati
on

As
expect
ed

Well
above
expectati
on

1 Self-dependence

2 3

5 6

2 Innovation

2 3

5 6

3 Being proactive

2 3

5 6

4 Being driven by the work itself

2 3

5 6

5 High technical competence

2 3

5 6

6 Resolving complex problems

2 3

5 6

7 Strong knowledge of problem

domain

2 3

5 6

8 Being flexible and adaptable

2 3

5 6

9 Eagerness to try new technology

2 3

5 6

1 Being prepared to work long

0 hours

2 3

5 6

1 Communicating
1 stakeholders

with

2 3

5 6

the

2 3

5 6

2 3

5 6

1 Sharing
2 team

well

knowledge

with

1 Adhering to process
3

28

1 Fully documenting your work

4

2 3

5 6

1 Meeting time deadlines

5

2 3

5 6

1 Meeting budget deadlines

6

2 3

5 6

1 Meeting customer expectations

7

2 3

5 6

2 3

5 6

2 3

5 6

1
8

Other: (Please specify)

1
9

Overall,
how
performed

have

you

29