An evaluation of a professional portfolio project for

principals in Western Australia

Robert C Dixon
Curtin University of Technology, Perth, Australia
dixonkc@bigpond.com
Kathryn Dixon
Curtin University of Technology, Perth, Australia
k.dixon@curtin.edu.au

Abstract: Principals of the Swan Education District in metropolitan Perth, Western

Australia, under the guidance of the district director, voluntarily participated in a pilot
Professional Portfolio Project that began early in 2002. The project sought to develop a
leadership portfolio underpinned by the draft Education Department of WA competency
framework. This study is an evaluation of the initiative. Its focus was to clarify a number of
issues related to principals perceptions of their experiences in creating a portfolio,
including the adequacy of the training and preparation phase, whether the goals and
objectives of the project had been achieved and what value the portfolio offered to principals
participating in the project. It investigated principals perceptions about the effects that
resulted from creating a portfolio, what further applications principals thought a
professional portfolio might have and if any significant ethical issues arose from the project
Keywords: portfolio; professional portfolio; educational leadership
Introduction
Early in 2002, the director of the Swan Education District in Western Australia took the
initiative for improving the professional growth of the districts principals by developing a
version of the professional portfolio that is currently in the early stages of implementation.
One hundred and twelve principals, both primary and secondary, were invited to participate
in the project.
Principals were in various stages of professional development as leaders; therefore, the
director deemed it appropriate to provide participants with the opportunity to complete a
competency-based model, a targeted competency model or an enquiry-based model
(Eggleston-Hackney, 1999) as the framework for the portfolio. For the initial stage of the
Swan portfolio project, the director decided that the most appropriate strategy would be to
combine elements of all three, according to the experience and achievements of each
principal and to collect feedback from a representative sample of principals regarding the
components and structure of the portfolio. However, there was a general focus on key
competencies.
According to Eggleston-Hackney (1999), the targeted competency model focuses on a goal
or target such as, for example, a community- based project co-ordinated by the principal that

could be used to demonstrate improved planning and communications skills. The enquirybased model of portfolio is more suited to the experienced administrator and is more loosely
composed around professional enquiry such as action research or the study of personal
leadership practice.
The competency framework underpinning the portfolio depends upon the Education
Department of Western Australias Competency framework. This framework is buttressed by
a list of six critical competency dimensions that principals are required to demonstrate. The
competency dimensions are; Leading in the Learning Environment, Supporting Staff
Effectiveness, Resourcing the Learning Environment, Building and Maintaining
Partnerships, Building a Responsive and Inclusive Curriculum, Providing Policy and
Strategic Direction. The current study is an evaluation of the Swan Education Districts
Professional Portfolio Project.
Conceptual framework
A conceptual framework was developed to underpin the structure, final components and
evaluation of the portfolio regardless of which model was chosen as most appropriate by the
sample. As a result, the Leadership Portfolio Model as shown in Figure 1 (developed by
Dixon, 2002) encompasses the competencies approach for the practitioner, incorporating
Quality Teacher Programme (QTP) developments. The model is informed by the concepts of
transformational leadership (Banathy, 2001; Popper & Zakkai, 1994; Sergiovanni, 1990,
1994), critical reflection (Brookfield, 1996) and infocratic leadership (Clawson, 2000;
Sergiovanni, 2000).
The model also allows for the possibility of the creation of future electronic means to present
the portfolio. Barrett (2000) has developed a conceptual framework for an approach to an
electronic portfolio, which presents a practical approach and incorporates some of these
characteristics.
Swan Leadership Portfolios

PURPOSE
Traditional

Format components

Digital/Electronic

Quality Assurance
Monitoring

Process Oriented

EDWA Dimensions

Research Considerations

Transformational
Leadership

Effect of ICT
implementation
on leadership role
emergent crucial
investigation
area

Evaluation Criteria linked to purpose

Emotional
Intelligence
Leadership

Critical Reflection

EDWA Leadership Dimensions (Plan for Government School Education 1998-2000, EDWA)

Figure 1: Swan leadership portfolios (Dixon, 2002)

Infocratic
Leadership

The sample
The initial intention of the director of the Swan Education District was that every principal in
the Swan District would participate in the Professional Portfolio Project. However, the reality
is that, despite all efforts by the researcher, the district director and his staff of the 112
principals in the district, only 28 responded to the questionnaire. It could not be ascertained
whether this was the final number of those who continued from the training and preparation
phase to develop their portfolio or not. All respondents were primary school principals. No
respondents represented secondary, middle school or senior colleges. Of the respondents, 20
were male and 8 were female. The mean length of experience in terms of years as principal
was 10.29 years as shown in Table 1.
Table 1: Years of service as a principal

Variable
A3

Mean
10.29

Std Dev
7.32

Minimum
1.00

Maximum
27.00

N
28

The mean student population in each school for respondent principals was 379.64 and the
mean staff number was 21.5 teachers. Of the principals who returned the questionnaire, 13
had a first degree, 5 had a postgraduate diploma, 9 had a masters degree, and 1 had a
doctorate. Seventeen respondents were currently undertaking formal study of one form or
another, including two at the Doctoral level.
The instrument
The instrument utilised in the research was a three-part questionnaire. Section one comprised
forty-six items in six categories, which were determined by phase two of the evaluation
regarding Training and Preparation, Goals and Objectives, Perceived Value, Perceived
Effects, Further Applications and Ethical Issues associated with their experiences working on
their portfolio. Respondents were asked to answer each question using a Likert scale, which
was graded 5 (strongly agree), 4 (agree), 3 (neither agree or disagree), 2 (disagree) and 1
(strongly disagree).
Section two included three open questions that required participants to comment on the
positive and negative aspects of creating their portfolio. Respondents were encouraged to
make insightful and personal comments. Section three comprised ten items, which were
designed to determine biographical information, for comparison of responses. Table 2
presents alpha coefficients for the reliability of the instrument.
Table 2: A summary of the reliability statistics for scales used in measuring the attitudes of
principals to the Professional Portfolio Project.

Research Variable
Training and Preparation
Goals and Objectives
Perceived Value
Perceived Effects
Further Applications
Ethical Issues

Items

Alpha Coefficient

B 1-8
C 1-8
D 1-8
E 1-8
F 1-9
G 1-5

.8580
.9617
.8809
.7617
.6309
.2463

Alpha coefficients for instrument reliability indicated that most of the variables were valid,
with the exception of ethical issues as shown in Table 2. These results should be read with
caution however, because the sample comprised only 28 principals.
Data analysis
Data was obtained and computed using the Statistics Package for Social Sciences version 6.0
(SPSS, v6.0). Statistics gathered included mean scores, standard deviations, and t-values.
Alpha reliability coefficients were calculated for each of the scales of the instrument.
To determine the most frequently occurring themes, a content analysis of the comments made
in response to the open questions by principals was undertaken. Comments were examined
and divided into categories of responses similar to that suggested by Sax (1979). Two
members of the Faculty of Education, at Curtin University of Technology, confirmed the
reliability of these categories. These members were not involved in the research. An interrater reliability coefficient of 90 percent was calculated using the formula from Huck,
Cormier and Bounds (1974, p. 335):
Total number of agreements x 100
Percentage of agreement =
Total number + total number of agreements
of disagreements

Dividing the comments into categories allowed for the numerical identification of each,
therefore facilitating the examination of frequency of each type of comment.
Findings and discussion
The purpose of the evaluation described in this paper was to determine the adequacy of the
training and preparation stage of the Professional Portfolio Project, to discover the clarity and
adequacy of the goals and objectives set from the beginning of the initiative, and to ascertain
the perceived value by principals of participating in the creation of a portfolio. Furthermore,
it sought to establish the perceptions of principals as to any effects creating a portfolio had on
the discharge of their professional responsibilities, what further applications for the portfolio
might be envisaged and what, if any, ethical issues arose from the project.
Differences between male and female principals were examined to determine if there were
variations on their responses to the questionnaire, as were variations in responses from
principals with longer than ten years experience as opposed to principals with less than ten
years experience. The response of principals with more than 20 staff in their school were
compared with those with less than 20 staff, and the perception of principals with postgraduate qualifications were compared with those with graduate qualification in terms of
their perceptions of the project.
Training and preparation

The purpose of the Professional Portfolio was made clear to participants, as indicated by a
very high mean of 4.53 from item B1of the questionnaire (The overall purpose of the
Professional Portfolio was made clear). The result suggests clear and successful articulation
by the district director of the Swan Education District as to what he intended the Professional
Portfolio to achieve.

Similarly, the training, resource allocation and the goals and objectives for the portfolio were
seen in a most positive light, with mean scores above 4.0. This is contradicted by the content
analysis where approximately 17 percent of respondents indicated that there was a lack of
resources allocated to the project. If a principals time is considered a resource, the
contradiction has substantial implications, with nearly 80% percent of respondents answering
in the negative as to the amount of time they spent creating the portfolio. However, this is
tempered by principal satisfaction with the resulting benefits of the portfolio in the longer
term. It is interesting to imply from this data that principals in this study do not make a link
between resources and time.
Respondents to the survey indicated positive perceptions to the value of collegial sharing in
the implementation phase in order to clarify what was required of them. Personal
preparedness and the amount of training that went into the project were considered to be
positive aspects of the Training and Preparation variable. Less experienced principals, (less
than 10 years) who were female indicated that they were the best prepared for the portfolio
project. Male principals with less than ten years experience were also positive regarding the
preparation phase. It appears that the younger principals, who were also the least
experienced, were more positive towards the Training and Preparation stage than any other
sub-population.
Goals and objectives

All responses to the items concerning the second variable, Goals and Objectives were
moderately positive, with means in a narrow range from 3.86 to 4.14. The consistency in
responses may come from the similarity of both the wording and presentation regarding the
criteria for each of the competency dimensions. Very little variation in response from all subgroups was recorded. It may be that the goals and objectives as articulated in the portfolio
were explained clearly enough and illustrated to avoid confusion. This is further supported by
phase two of the study, which elaborated on the competency dimensions that provided
information as to how these were developed through the ILSSC board in the USA.
Perceived value

All groups recorded very positive responses to items concerning the perceived value of the
project. The most significant positive response was to the promotion of self-analysis by the
portfolio (mean = 4.5) and the fact that the portfolio provided a valuable resource (mean =
4.5). These results support the assertions by Eggleston-Hackney (1999) about the worth of
self-analysis and the value of the self-reflective elements of the portfolio. These findings also
lend weight to Salends (2001) assertions that professional leadership portfolios should be
both process and product oriented with a focus on the collaborative and reflective process of
the teaching and learning experience.
Of the sub-populations measured, principals with less than ten years experience tended to be
most positive about the self-reflective and self-analysis process the portfolio enabled. They
also viewed the portfolio as a more valuable resource than the more experienced principals.
Male principals with less than ten years experience were most positive about the value of the
self-reflective process. Principals without postgraduate qualifications were also more positive
about the self-reflective process, as opposed to those with postgraduate qualifications.
Interestingly, the results point to the fact that principals with a larger staff (more than 20) felt
that the time they spent on their portfolio was more valuable than principals with smaller

staff (less than 20), however principals with a smaller staff were further likely to view the
portfolio as a more valuable resource.
Perceived effects

The outcome of the variable, Perceived Effects shows a moderately positive response with
a narrow range of mean scores between 3.07-3.82. In retrospect, the items in this variable
were asked prematurely considering that the project had been adopted for such a short time.
Questions of improved leadership, more effective planning, streamlining administrative
responsibilities, articulating a vision for a school and facilitating organisational change are
the sort of long term effects one would expect from lengthier participation in the project,
rather than a snapshot of the pilot programme. However, the reasonably positive responses at
this early stage auger well for the future.
Results from observations of the sub-populations indicated that principals with more than 10
years service were more positive about the portfolio assisting in planning processes, whereas
their less experienced cohort were more likely to be positive about the portfolio improving
leadership and were more interested in using a portfolio for performance management of
staff. Females were more likely than males to perceive the portfolio as improving leadership
and planning, whereas males observed more changes in their planning practices through their
use of the portfolio. Principals with postgraduate qualifications were more likely to be
positive to the administrative streamlining effect and the facilitation of organisational change
as a result of the portfolio.
Those principals with a staff larger than twenty were most likely to report improved
leadership and changes to planning practices as opposed to those principals with less than
twenty staff. Those principals with less than 20 staff in their school were more positive about
the portfolios value in effectiveness of planning and articulating a vision for the school and
were therefore more likely to want their own staff to take up the portfolio challenge.
Further applications

The items for the variable which looks at future applications for the portfolio were again
mainly positive, with the exception of a small negative response by respondents to the
applicability if the portfolio (in its current form), to members of their staff.
Principals clearly demonstrated a preference for the portfolio to be linked to the selection
criteria for promotional positions created by The Education Department of WA (EDWA) and
rejected the industry model link. The findings of the survey also found principals were keen
to have their own staff develop a portfolio and showed a strong positive response to the
exploration of an electronic version of a Professional Portfolio. Furthermore, all principals
were positive about their ongoing commitment to the project.
Ethical issues

Overall, principals were trusting of those who had access to the sensitive personal
information contained within their portfolio. They attached strong importance to their critical
friend and reported that they were willing to be honest in their thoughts and reflections in the
portfolio. They were mostly confident that issues of privacy had been dealt with and
generally felt the portfolio was more than a passing trend.
The least experienced principals were more likely to see the portfolio as having a future and
more likely to be honest in their personal reflections. Males were more likely than females to
trust others with access to their personal file and were more likely than women to be honest

in their personal thoughts and reflections. Principals with a smaller staff were also more
confident in those handling their information and happier with issues of privacy. Those
principals with a larger staff considered their relationship with their critical friend as more
important.
Content analysis
The results of the content analysis mainly supported the quantitative data described above. A
majority of the respondents (67.85%) found the self-reflection afforded by the portfolio
process beneficial. They saw the portfolio as an opportunity to map their leadership journey
and to create a leadership profile from which reflection for improvement could occur.
Similarly, content analysis highlighted the positive elements of the creation of a personal
leadership file, which mapped their leadership journey and created a leadership profile from
which reflection for improvement could occur. This is buoyed up by the questionnaire results
for the Perceived Effects variable and the Perceived Value variable, which are strongly
linked.
An overwhelming negative response from principals regarding the amount of time spent on
creating the portfolio was noted in the content analysis, with nearly 80 percent of respondents
making some form of comment on this issue. Yet, analysis of the quantitative data for the
same theme tends to support the fact that the time spent on creating the portfolio was very
positive in terms of the impact it had on their leadership. Principals appear to be
acknowledging that the process is demanding and time-consuming, but overall they seem
satisfied that there is value and worth in embarking upon the construction of the portfolio.
A similar disparity is noted where honesty in reporting was considered reasonably positively
in the closed question survey, but was cited as an issue in the open section by 25 percent of
respondents. It may be that principals differentiated between honesty in writing their personal
thoughts and reflections in the diary section of the portfolio and what they would actually
present to the district director in a more formal context.
Content analysis also highlighted the perceived need by principals to have access to a trained
mentor, someone experienced and neutral who could guide them and provide an active
resource when required. A number of other responses indicated great favour for the project
and praised its initiative, but tempered this with the need for refinement over time.
Conclusions
Piper (1999) asserted that the process of creating a portfolio should be undertaken slowly,
with an aim to seek linkages for stakeholders and to be realistic with designs and
expectations of the portfolio. She recommended that portfolio authors use available models
relevant to their design and use and that ownership of the project needs to be instilled.
Eggleston-Hackney (1999) asserted the creation of the portfolio should take into account the
highly individualised nature of the process with the emphasis on how one examines oneself,
what value is placed on artefacts and their representation of work completed and what
rationale is used to support the artefact as a reflection of personal growth. The findings of
this study tend to support these assertions.
The findings of this study must be read with extreme caution because it was not possible to
ascertain how the number of survey respondents relates to the overall number of participants

beyond the training phase and the sample is small (n =28). However, the findings strongly
support the confidence of the director of the Swan Education District that the Professional
Portfolio Project would bring benefit to principals.
The study provides valuable information regarding the importance of thoughtful
implementation and the collaboration and involvement of stakeholders at all levels of the
project. Its success emphasises the importance of thorough training and preparation. It
underlines the necessity to have clear goals and objectives and the need for thorough
evaluation and feedback for modification and development purposes to ensure the
accomplishment of a project of this magnitude.

References
Banathy, B. H. (2001). Guided evolution of society: A systems view. New York: Kluwer Academic/Plenum
Publishers.
Barrett, H. (2000). Create your own electronic portfolio. Learning and Leading with Technology, 27, (7), 14-17.
Brookfield, S.D. (1996). Becoming a critically reflective teacher. San Francisco: Jossey-Bass.
Claudet, J. G. (1998). Multi-media case simulations for school leader professional growth. Technological
Horizons in Education Journal, 25 (11), 82-86.
Council of State School Officers, (1996). Interstate school leaders licensure consortium standards for school
leaders.
Dixon, R. (2002). The professional portfolio project: Report to the Swan Education District of Western
Australia. (Unpublished manuscript).
Eggleston-Hackney, C. (1999). Three models for portfolio evaluation of principals. School Administrator, 56
(5), http://www.aasa.org/publications/sa/1999_05/foc_hackney.htm.
Huck, S.W., Cormier, W. H. & Bounds W.G. (1974). Reading statistics and research. New York, USA: Harper
and Row.
Morgan, B. (1999). Portfolios in a preservice teacher field-based program: evolution of a rubric for performance
assessment. Education, 119 (3), 416.
Piper, C. (1999). Electronic portfolios in teacher education reading methods courses, no ED 442755 in ERIC,
(Educational Resources Information Centre: Washington).
Popper, M., & Zakkai, E., (1994). Transactional, charismatic and transformational leadership: Conditions
conducive to their predominance. Leadership and Organisational Development Journal. 15 (6), 3-8.
Salend, S. (2001). Creating your own professional portfolio. Intervention in School & Clinic. 36 (4), 195.
Sax, G. (1968). Empirical foundations of educational research. New Jersey, USA: Prentice Hall.
Seldin, P. (2000). Teaching portfolios: a positive appraisal. Academe, 86 (1), 37-44.
Sergiovanni. T. (1994). Building community in schools. San Francisco: Jossey-Bass.
Sergiovanni, T. (2000). The principalship: A reflective practice perspective. (4th Ed.) Needlam Heights, MA.:
Allwyn & Bacon.

Copyright 2003 Dixon & Dixon: The authors assign to HERDSA and educational non-profit institutions a non-exclusive
licence to use this document for personal use and in courses of instruction provided that the article is used in full and this
copyright statement is reproduced. The authors also grant a non-exclusive licence to HERDSA to publish this document in full
on the World Wide Web (prime sites and mirrors) on CD-ROM and in printed form within the HERDSA 2003 conference
proceedings. Any other usage is prohibited without the express permission of the authors.