Evaluating Communication Effectiveness in Organisational Contexts.

Neville Meyers Lecturer Coordinator AMN435 Semester 1 2014.

Our Focus Tonight

Recall our Lecture 2: Principles of Communication? Core focus - today is on evaluating organisational communication and leaders central role. Also, evaluating employee communication. Selective review of what we covered earlier (Lectures 2, 4, 5). Primary focus tonight also on actual research methodologies for communication evaluation. Assistance of tonights methodologies for your final group proposal later application in a BEN910 Project? May also offer linkages for the examination.

So Why Evaluate?

Identify disconnects between leader-managersemployees communication styles and organisational objectives. Evaluate employees own communication effectiveness. Evaluate leadership-manager-employees mediapreferences/actual use of communications media. - confirm preferences/uses are reaching organisational goals - allow corrective action as necessary Implement a gap analysis on Desired existing communication styles/strategies vs. Existing communication styles/strategies. Various methods to do this.

Practical Evaluation Examples

Questionnaires (inc. Communications Audits). Other survey instruments. The Gap Analysis Interviews

Next weeks tutorials.

Readings/1

Albrecht, T. (1988). Communication and personal control in empowering organizations. Communication yearbook, 380-390 . Bernard, H.R. (2013). Social research methods: Qualitative and quantitative approaches. Los Angeles: Sage Publications. Binder, J. (2007). Global project management: Communication, collaboration and management across borders. Aldershot, UK: Gower. See esp. Ch:9 Global Communication Techniques. Dalglish, C., & Miller (2010). Leadership: Understanding its global impact. Prahran, Vic: Tilde University Press. Esp. Ch: 9:Leadership communication. Set Text: Dubrin, A.J. (2013). Leadership: Research findings, practice, and skills. Ch: 12 Communication/Conflict Resolution Skills. Also texts reference/linkages under Communication in Index.

Readings/2

Hargie & Tourish (2000), Handbook of Communication Audits for Organisations (copy on CRC). Jha, S. (2010). The project managers communication toolkit. Boca Baton, FL: CRC Press. Kenworth, T.T. (2002). Leadership effectiveness in virtual teams. J. Management Information Systems, 18(3), 7-40. Meyers, N. (2007). Communication issues and communicative self-efficacy in for sustainable telework: An empirical study. Presented International Communication Association Conference, San Francisco, May 2007.

Communication Focus Recent BEN910 Projects/1

Student L. Alderman (2012, Semester 2). Evaluating Effective Strategies for Todays Fast-paced Teams: An Applied Focus at Virgin Australia. Student M. Howard (2012, Semester 1). Stakeholder engagement: Managing Project Success Through Effective Relation-ship Building: A Pilot analysis. Student N. Cooper (2012, Semester 1). Benchmarking Communications Media Uses for Effective Project Management: An Exploratory Analysis.

Communication Focus Recent BEN910 Projects/2

Nguyen, C.V. (2012, Semester 2). Strategies for Improving Communication on Construction Sites: An exploratory analysis. Gouda, (2013, Semester 2). Communication Policy and Planning to Enhance Staff Competencies in Use of Communications Technologies in the Construction Industry. Shapcott, D. (2013, Semester 2):Effective Leadership Strategies for Conflict Resolution on Construction Sites.

BEN910 Project: Stakeholder Engagement Project Services

Your Future BEN910 Project

Supervisor for these earlier projects: Neville Meyers Other BEN910 projects. Your future project??? Am happy to discuss with you (n.meyers@qut.edu.au).

Leaders, Managers, Employees: Evaluation of Impact of Communication on Relationships

Reasons Why We (and Leaders/Managers-Employees ) Are Concerned with Communication

Communication is often also seen as a frequent driver of:

Employee productivity. Employee job satisfaction. Employee lifestyle satisfaction (i.e. by reduced stress on the job). An open organisational culture. Increased trust. Improved client relationships. Overall organisational effectiveness. Recall our earlier data (Lecture 2) : 31.57% of Australian respondents said: Improve communication! (2002 Study).

Leader/Managers Influence on Communication Processes.very popular area for evaluationby researchers, consultants.

Evaluation Focus: Leaders Communication Competencies : Study Guide Module 2 (Topic 2.2).
Leaders listening and responding skills. Leaders ability to establishing positive interactions. Leaders skills in negotiating/resolving conflicts. Leaders understanding of people. Leaders ability to adapt his/her communication behaviours to suit circumstances. Four types of leadership communicators (Baldoni, 2003 cited in Module): - The Expert. - The Visionary. - The Coach. - The transformer. See also your set text Dubrin, (2013): Ch 12 Communication and Conflict Resolution Skills (for leaders).

Another Evaluation Focus: The Leader-Managers Role in Communication Strategies

How well leader-manager sets direction for new communication processes. Leads by example: walks the talk. Provides resources for operational implementation at all levels: technical, human, intra-organisational, extra-organisational; and increasingly globally. Leaders actual focus on team-building. Leaders reliance on soft-skills (our earlier look at the The Karpin Report (1995; 2010). Other useful points in your Study Guide; also Simply the Best Workplaces

Evaluating Technologies Diffusion: LeaderManagers Need to Get the Media Mix Right
Availability of Advanced Information Technologies, new media

Coordination mechanisms

Outcomes

Communication patterns/preferences

Source: Adapted from Belanger (1997)

Summary: Important Leadership/Management Goal for Evaluating Communication.

LEADERS/MANAGE RS

Downward Communication

Mutual Influence

Power Sharing

EMPLOYEES

Upward Communicati on

Source: OHair (1992) Strategic Communication p. 177

Communication: Basic Evaluation Strategy

Methodologies.....could consider for your AMN435 Group Proposal or Later BEN910 Project

Four Main Stages: Possible Adaptation for Group Proposal.

1.

2.

3. 4.

5.

Identify the clients need and define a develop a proposed methodology. Preview expected communication outcomes to justify cost (can expand in Benefits section). Mention your team can collect, analyse and interpret the data. Potentially, could show your awareness of qualitative/quantitative approaches (no actual data collection needed). Include in group proposal section.

Above strategies are not mandatory they depend on your later group proposal topic!

Case Study: Previous AMN435 Group Proposal : QUT Student Housing Needs.

Focus Focus Groups Groups

SemiStructured interviews

Questionnaire

Issues Identification

Expand, Clarify Issues

Explicit Statistical Data

So How Do We Evaluate Communication?

Understand the organisations (or if for proposal) clients needs, then decide the method mix: Qualitative? Quantitative? Gap analysis? Mixed methods?

Preview: So How Do We Measure Leadership-Employee Communication Effectiveness?

Qualitative research. - interviews; focus groups; observation; critical incident reporting Quantitative research. - surveys, questionnaires Communication audits. Gap analysis via employee surveys - communication importance vs. communication satisfaction. Mixed methods?

The Gap Analysis

What exists now? What do we want in future? Useful rankings are - importance - satisfaction? http://www.mindtools.com/pages/article /gap-analysis.htm My own example with Mondial ECommerce : next weeks tutorial.

Qualitative Research : Continued Emphasis on Real World Subjects

QR meanings reside in people. Not all things are measurable.

Not everything that counts can be counted.not everything that can be counted counts Albert Einstein. May be simpler just to talk with employees/clients on their communication issues.

Qualitative Research: Interviews

In-depth interviews one of most commonlyused research tools. Initial clarification of communication issues. Structured or formal interview (prepared list of questions not deviated from). Semi-structured interview (balance between closed and open-ended questions to ensure maximum input from subject). The unstructured interview (totally openended). Similarly, Meyers (Telstra Consultancy) Semistructured interviews with 21 Australian managers on their individual organisations telework futures to assist Telstra with marketing campaign.

Interviews: Advantages

Elicit peoples perceptions, clarify assumptions, motivations, underlying issues . Useful for small samples (i.e., initial fact finding). Allows actors or subjects to speak for themselves. Often rich and deep knowledge. May be computer-coded using NVivo to elicit themes, their frequencies, in written text Ground later quantitative study.

Interviews: Disadvantages

Time-consuming for actual interview Training of interviewers (sensitivity, interpersonal skills) Need for transcripts May lack of precision (devil in the detail) Often need for thematic analysis (manually or via Nudist, NVivo and like software packages)

The Communications Audit: Another Take

Communications Audit/Quantitative Survey Defined:

Who says what to whom in what medium (i.e. E-mail, virtual meetings, face time, Skype, other media) with what frequency and with what effect communication satisfaction, work productivity?

See Hargie & Tourish (2000) but use more current research sources.

Other Methodologies: Useful Internet Sources

What Makes A Communications Audit?

http://www.communicationideas.co m/communicationaudit.html?gclid=CNfvxvzl6KICFdID 5QodSkJczA

Various other specific questionnaires auditing Leaders effectiveness in communication contexts.

Use of Focus Groups in Communication Evaluation

Advantages:

Tapping into peoples attitudes, opinions on communication problems. Eliciting feedback on organisational changes. Developing new communication practices, services. Re-vamping existing products, services Inviting may be low in cost.

Focus Groups: Communication Evaluation.

Disadvantages:

Less control (for researcher) than interview/s. Data difficult to capture, analyse? Group dynamics big talkers win? Need for observer, moderator? Not always representative of wider viewpoints.

Quantitative Methods: Questionnaires

The questionnaire generally seen as the most attractive method of quantitative data collection. Representative sampling (non-users+ users if appropriate) . Flexibility of the questionnaire (vs. interviews) Range of responses (N = 20, or N = 2,000) N=90 and beyond preferable for statistical clout. Objective data collection instrument. Isolation of predictive factors (where possible).

Questionnaires: Problems in Design

Research design Researcher bias Types of questions Decisions re types of scales (as per tutorial examples) Adequate sample (both users and non-users of the service N=90 for statistical clout User-respondents apathy (Survey Fatigue) Data collection Data analysis (appropriateness of statistical techniques)

Questionnaires: Popularity

Surveys frequently used to evaluate communication services (plenty of examples in Dalglish & Miller, 2010). Representative sampling (non-users+ users if appropriate) . Flexibility of the questionnaire (vs. interviews) Range of responses (N = 20, or N = 2,000). N=90 and beyond preferable for statistical clout. Objective data collection instrument (but need to avoid leader-manager bias in questions selection to get honest data).

Watch survey fatigue......

Review of IT Support in An Organisation: Employees Attitudes May Be the Key: Sample Questionnaire Items

1.

IT training contributes to my work effectiveness 1

Strongly Disagree

2
Moderately Disagree

3
Mildly Disagree

4
Mildly Agree

5
Moderately Agree

6
Strongly Agree

2.

IT Help desk staff can be counted on to assist me 1

Strongly Disagree

2
Moderately Disagree

3
Mildly Disagree

4
Mildly Agree

5
Moderately Agree

6
Strongly Agree

Internet Surveys

Rapid growth: usually questionnaires. Need careful designing. Assistance with Web design. Allow down-load of data straight into a computer file. Convenience is the buzz word!

Global Communication: Surveys

Global team norms and communication at Ernst & Young - future class handout of EYs practices being prepared for next week. Leadership philosophy-practices in communication at Mitsubishi. Virtual teams survey instrument as per later tutorial handout.

Example of Mixed Methods

Research Study

The QUT-Griffith Universities Wireless Productivity Study: DEST $1.36 million research grant. Qualitative approaches (interviews) . Quantitative approaches (our questionnaire = 1,374 respondents).

Our final research report to Federal Government.

Other Advantages of Mixed Methods Research

Advantages/applications of qualitative and quantitative research methods.

Applications extend beyond evaluation of just communication (our approach in Lecture 10 has necessarily been limited).

Can be applied to wide variety of practical work applications (we encourage this in BEN910).

Conclusions/1

Evaluation of leader-managers role is pivotal in ensuring communication effectiveness. Communication practices and outcomes remain questionable in many organisations.

Hence, evaluation of employees communication effectiveness essential leader-manager role. Leader-managers own communication competencies need frequent reviewingevaluation.

Conclusions/2

Need for both qualitative and quantitative communication analyses in todays organisations. Leader-manager awareness of value of communication audits and other methodologies. Leader-manager focus on need for training in team-building. New skills we/you can develop in your chosen IT profession? BEN910 a good training ground!