You are on page 1of 61

1666 K Street, NW

Washington, D.C. 20006


Telephone: (202) 207-9100
Facsimile: (202) 862-8430

)
CONCEPT RELEASE ON AUDIT QUALITY
INDICATORS

NOTICE OF ROUNDTABLE

Summary:

Public
Comment:

)
)
)
)
)
)
)
)

PCAOB Release No. 2015-005


July 1, 2015
PCAOB Rulemaking
Docket Matter No. 041

The Public Company Accounting Oversight Board is issuing this concept


release to seek public comment on the content and possible uses of a
group of potential "audit quality indicators." The indicators are a potential
portfolio of quantitative measures that may provide new insights about
how to evaluate the quality of audits and how high quality audits are
achieved. Taken together with qualitative context, the indicators may
inform discussions among those concerned with the financial reporting
and auditing process, for example among audit committees and audit
firms. Enhanced discussions, in turn, may strengthen audit planning,
execution, and communication. Use of the indicators may also stimulate
competition among audit firms focused on the quality of the firms' work
and, thereby, increase audit quality overall. Issues raised by the release
include: (i) the nature of the potential indicators, (ii) the usefulness of
particular indicators described in the release, (iii) suggestions for other
indicators, (iv) potential users of the indicators, and (v) an approach to
implementation over time of an audit quality indicator project. The Board
seeks advice on these subjects through the comment process and will
also convene a public roundtable about the concept release, on a date to
be determined during the fourth quarter of 2015. Additional details about
the roundtable will be forthcoming.
Interested persons may submit written comments to the Board at the
following address: Office of the Secretary, Public Company Accounting
Oversight Board, 1666 K Street, N.W., Washington, D.C. 20006-2803.
Comments may also be submitted by e-mail to comments@pcaobus.org
or through the Board's website at www.pcaobus.org. All comments should
refer to PCAOB Rulemaking Docket Matter No. 041 in the subject or
reference line. Comments should be received no later than 5:00 p.m.,
Eastern Standard Time, on September 29, 2015.

PCAOB Release No. 2015-005


July 1, 2015
Page 2
Board
Contacts:

I.

Gregory J. Jonas, Director, Office of Research and Analysis (202/5914320, jonasg@pcaobus.org), Stephen Kroll, Senior Advisor, Office of
Research and Analysis (202/591-4369, krolls@pcaobus.org), and George
Wilfert, Deputy Director, Office of Research and Analysis (949/435-7734,
wilfertg@pcaobus.org).

Introduction

The responsibilities of the Public Company Accounting Oversight Board (the


"Board" or "PCAOB"), under the Sarbanes-Oxley Act of 2002, as amended ("SarbanesOxley") 1 are all ultimately directed at improving audit quality and thereby benefiting
investors. The Board's audit quality indicator ("AQI") project, identified as a priority
beginning in 2013, 2 can be an important part of that effort. Its objectives are simply
stated: to identify a portfolio of quantitative measures of public company auditing
(called "indicators"), whose consistent use may enhance dialogue about and
understanding of audits and ways to evaluate their quality; and to explore how and by
whom the portfolio of indicators can best be used. 3 Ultimately, this effort may produce
higher quality audits.
This concept release describes and seeks comment on 28 potential indicators;
suggestions for other indicators are specifically sought. The Board hopes ultimately to
refine the list to a smaller number that is manageable and effective.
The 28 potential indicators fall into three groups.
The first is "audit
professionals," and includes measures dealing with the "availability," "competence," and
"focus" of those performing the audit. The second is "audit process," and includes
measures about an audit firm's "tone at the top and leadership," "incentives,"
"independence," attention to "infrastructure," and record of "monitoring and
1

Pub. L. No. 107-204, 116 Stat. 745 (2002). In addition to the direct
responsibilities given to the Board in Sarbanes-Oxley, the statute specifically authorizes
the Board to act "to promote high professional standards among, and improve the
quality of audit services offered by," accounting firms registered with the Board "in order
to protect investors, or to further the public interest." Sarbanes-Oxley Section 101(c)(5)
of the Sarbanes-Oxley Act, 15 U.S.C. 7211(c)(5).
2

See Public Company Accounting Oversight Board, Strategic Plan:


Improving the Relevance and Quality of the Audit for the Protection and Benefit of
Investors, 2012 2016 5 (Nov. 2012).
3

The possible application of the indicators to brokers and dealers in


securities that are not public companies but whose audits are within the jurisdiction of
the Board is discussed below, at pages 29 and 31 (Question 39).

PCAOB Release No. 2015-005


July 1, 2015
Page 3
remediation." The third is "audit results," and includes "financial statements," "internal
control," "going concern," "communications between auditors and audit committees,"
and "enforcement and litigation."
The individual indicators are intended to address different variables, but they
grow out of a core insight: without a tool such as a portfolio of AQIs the nature of the
audit, in which certain drivers of audit quality are less than transparent, makes
assessing quality difficult. The indicators discussed in this release are quantitative and
operate in an integrated manner along with contextual information for each audit or
related comparison. They are not formulas and may have their greatest use as
generators of questions for the auditor. Context provided by each audit to which the
indicators are applied and by the application of the indicators to the audit firm that
conducts the audit (and perhaps other firms) is essential to understanding their
meaning and implications.
Identifying relevant indicators is only the first step in using them. The release
discusses their potential availability and value to audit committees, audit firms,
investors, and audit and other regulators. A number of issues are involved, including
how to shape the indicators to maximize their value to users, how use of the indicators
should be phased in, who should provide the information involved and who should
receive it, whether and to what extent the information should be public, and whether any
continuing AQI project should focus initially on only certain audits or audit firms.
The five parts of the concept release following this Introduction each focus on
one or more of these issues. The release first explains the purpose of AQIs and the
AQI project. Next, the discussion lists, and describes the design of, the 28 potential
indicators. The release then turns to the best ways to move the AQI project forward and
asks which potential users of AQIs could find the indicators most immediately useful.
That discussion also explores how variations in the size of audit firms, or the size or
nature of audited companies, might be handled. The following section briefly describes
AQI outreach efforts in which Board staff has engaged, as well as contemporary efforts,
here and abroad, to promote an understanding of the nature and use of AQIs and
related data and the elements of audit quality. The release's final part is the detailed
description, in Appendix A, of the potential indicators.
The promise of AQIs, in generating insights into the foundations of audit quality,
both within and among audit firms, and in creating incentives for competition in quality,
is considerable. Still, the Board recognizes that at this stage the AQI project poses
many questions, from the appropriateness or operation of particular proposed indicators
to the way the information they generate might be obtained and used. This concept
release is intended to stimulate informed discussion and elicit detailed comments and
suggestions from all interested parties about the answers to these questions. Particular
questions of interest to the Board are noted in the release and in Appendix A.

PCAOB Release No. 2015-005


July 1, 2015
Page 4
II.

The Purpose of Audit Quality Indicators and the AQI Project

Under Sarbanes-Oxley, the Board oversees the audits of both public companies
and non-public brokers and dealers in securities. 4 The AQI project focuses on the
audits of public companies. An independent audit committee of the board of directors of
each public company 5 listed on a U.S. national securities exchange or association is
responsible for the appointment, compensation, and oversight of the work of the auditor
of that company's financial statements. 6 The shareholders of most listed and non-listed
public companies are asked to ratify the selection of their companies' respective
auditors. And investors, in considering their companies' financial condition, look to the
"preparation of informative, accurate, and independent audit reports." 7 The goal of the
AQI project is to improve the ability of persons to evaluate the quality of audits in which
they are involved or on which they rely and to enhance discussions among interested
parties; use of the indicators may also stimulate competition by audit firms based on
quality. The indicators proposed in this release are not put forward to modify, or to
suggest modification of, the objective of the audit of financial statements, or to impose
new audit performance requirements.
The 2008 Final Report of the Advisory Committee on the Auditing Profession to
the U.S. Department of the Treasury (the "Treasury Advisory Committee," sometimes
also called "ACAP") recommended that the Board, in consultation with other parties,
"determine the feasibility of developing key indicators of audit quality and effectiveness
and requiring auditing firms to publicly disclose these indicators." 8 The discussion of
4

See Section 101 (a) of Sarbanes-Oxley, 15 U.S.C. 7211(a). See supra

note 3.
5

In this release, the term "public company" is used in place of the more
technical term "issuer," in section 2(a)(7) of Sarbanes-Oxley, 15 U.S.C. 7201(a)(7).
6

Audit committees of some registered investment companies have similar


duties, and audit committees of both listed and non-listed companies have other
responsibilities under Sarbanes-Oxley. See the discussion infra notes 11, 23, and 24.
7
8

Section 101(a) of Sarbanes-Oxley, supra note 4.

United States Department of the Treasury, Final Report of the Advisory


Committee on the Auditing Profession, Chapter VIII, Recommendation 3 of Advisory
Committee on the Auditing Profession, Final Report to the Department of the Treasury,
(October 6, 2008), at VIII:14, available at http://www.treasury.gov/about/organizationalstructure/offices/documents/final-report.pdf. The "other" parties with whom the
Committee suggested the Board work included "auditors, investors, public companies,
audit committees, boards of directors, and academics." Id. See the discussion at Part V
of the concept release (Outreach; Other AQI Projects), infra at pages 31-34. The

PCAOB Release No. 2015-005


July 1, 2015
Page 5
that recommendation emphasized the gap such indicators could fill and the effect they
could have:
A key issue in the public company audit market is what drives competition
for audit clients and whether audit quality is the most significant driver.
Currently, there is minimal publicly available information regarding
indicators of audit quality at individual accounting firms. Consequently, it
is difficult to determine whether audit committees, who ultimately select
the auditor, and management are focused and have the tools that are
useful in assessing audit quality that would contribute to making the initial
auditor selection and subsequent auditor retention evaluation processes
more informed and meaningful. (Emphasis supplied.)
The Committee believes that requiring firms to disclose indicators of audit
quality may enhance not only the quality of audits provided by such firms,
but also the ability of smaller auditing firms to compete with larger auditing
firms, auditor choice, shareholder decision-making related to ratification of
auditor selection, and PCAOB oversight of registered auditing firms. 9
(Emphasis supplied.)
The Co-Chairs' Statement at the beginning of the 2008 Final Report noted that
the Treasury Advisory Committee's "Subcommittee on Concentration and Competition
[which studied the AQI question] discussed enhancing audit quality as a key element in
improving the viability and resilience of the auditing profession." 10
The lack of information identified by the Treasury Advisory Committee arises
from the inherent circumstances of auditing. Although a public company's audited
financial statements and related auditor's report are themselves made public, and
certain disclosures are now required by law to be made to the company's audit

Committee also recommended that the Board should monitor the indicators that it found
to be feasible. Id. at 16-17.
9

Id., at VIII: 14-15. The Board held its first discussion of the feasibility of
developing AQIs with its Standing Advisory Group in late October 2008, several weeks
after release of the Treasury Advisory Committee Report. The project was deferred
thereafter, and it was reopened by the Board in late 2012. See supra note 2.
10

Id. at II: 5. The Co-Chairs of the Treasury Advisory Committee were


Arthur Levitt, Jr., Chairman of the United States Securities and Exchange Commission
(the "Commission" or "SEC") from 1993-2001, and Donald T. Nicolaisen, the
Commission's Chief Accountant from 2003-05 and a former senior partner of
PricewaterhouseCoopers.

PCAOB Release No. 2015-005


July 1, 2015
Page 6
committee, 11 those items provide limited information on which to evaluate the quality of
an audit and explain how auditing actually takes place. The manner in which the audit is
conducted lies primarily under the surface, and the strengths and weaknesses of the
process are opaque. The sole observable output of the audit is a standardized report
that cannot be used to distinguish auditors from one another. Since this is so, it is
difficult to evaluate the elements of audit quality (although Board members have learned
that some audit committees are experimenting with various kinds of AQIs to inform
discussions with their companies' auditors), or to understand how it might be
strengthened; it is also hard to recognize and reward audit quality when it is present.
For these reasons, audits currently may fall within the class of what economists
call "credence goods." 12 That is, the quality and impact of the auditor's services may be
difficult or in some cases impossible to ascertain accurately. The auditor is usually in
the best position to determine the scope of the service required; the client has limited
ability to make a similar judgment, and the outcome of the service the quality of the
audit is either unobservable or can only be observed at significant cost to the audited
company or others.
As indicated above, the Board hopes through the AQI project to contribute to a
clearer view of auditing by identifying key variables, in the form of indicators, that may
reflect, and by doing so inform discussions about, audit quality, and then exploring how
the data generated by the indicators can be used prudently and effectively. The
indicators have been developed around three principles. The first is that, they should
11

See, e.g. Section 10A(k) of the Securities Exchange Act of 1934 (the
"Exchange Act"), 15 U.S.C. 78j-1k, and Rule 210.2-07 of Regulation S-X, 17 CFR.
210.2-07 which require the auditor to report to the audit committee (i) all critical
accounting policies and practices to be used in the audit, (ii) all alternative treatments of
financial information within General Accepted Accounting Principles ("GAAP")
discussed with management, their ramifications, and the treatment preferred by the
auditor, and (iii) other material written communications between the auditor and
management, such as any management letter or schedule of unadjusted differences.
See also Auditing Standard No. 16, Communications with Audit Committees, which
requires the auditor to communicate to the audit committee, among other things, the
following matters: (i) the overall audit strategy and timing of the audit, (ii) significant risks
identified by the auditor, (iii) timely observations arising from the audit that are
significant to the financial reporting process, (iv) critical accounting practices and
policies, (v) critical accounting estimates, and (vi) qualitative aspects of the company's
significant accounting policies and practices.
12

Monika Causholli and W. Robert Knechel, An Examination of the


Credence Attributes of an Audit, 26 Accounting Horizons 631(2012), available at
http://aaajournals.org/doi/abs/10.2308/acch-50265.

PCAOB Release No. 2015-005


July 1, 2015
Page 7
be quantitative wherever possible, to add consistency of approach and objectivity to
what would otherwise in most situations be only subjective judgments. The second is
that they should generate data that enables users to pose critical questions. The third is
that they should be used and function together as a "balanced portfolio" of audit quality.
No single indicator is likely to be determinative; in application to actual audits (or to
comparative analyses of, e.g., audit firm offices, industry audit practices, or even firms
themselves), the indicators should work collectively, so that certain indicators can
balance others to inform discussion.
The indicators are meant to be a tool. As such, they have inherent limitations
that have to be recognized if they are to be effective. They are not algorithms,
benchmarks, or safe harbors against enforcement or other claims, and they do not lead
directly to formulas for determining the quality of a particular audit or whether an auditor
has met its obligations. The reason AQIs cannot be used in any of these ways is that
analysis of AQI data almost always requires a context. Their meaning in evaluating
audit quality depends on the situation in which they arise. Reasonable explanations can
exist for divergent numbers, and a variety of other factors may affect a particular audit;
for example, the regulatory environment in which an audited company operates, or the
way accounting standards affect a particular company, may become an important prism
through which to view AQI data relating to a firm's audit. The quantitative character of
the indicators does not exclude in fact it can and often will require qualitative
analysis, and hence the judgment of the persons using the indicators, about context and
the interaction of the indicators as a group. Moreover, not all of the factors that drive
audit quality can easily be measured directly. For example, professional skepticism and
due care are critical characteristics for auditors.
Placing AQI data in context can involve what might be called "comparability." An
audit committee informed of the partner-manager-staff ratio for its upcoming audit might
want to know, to obtain context, how the ratio compares with that for other audits by its
auditor, and other auditors, and why there are similarities or differences. All audits,
even of companies of ostensibly the same size, do not raise the same issues. The
same need for context could arise in the case of engagement or retention decisions; an
audit committee considering whether to retain an audit firm might want to know how its
firm-wide indicators compare with those of firms of similar size and capacity. Without
the answers to questions of this sort, with data from both engagement and firm levels
based on consistent definitions of the indicators, the audit committee's ability to evaluate
what it has been told may be limited. Again, however, the question is not simply
whether numbers are comparable, but why (or why not), and at that point qualitative
analysis can be helpful. And since AQIs are not benchmarks, even comparability of
AQIs in two audits may not imply comparable audit quality.
Comparing AQIs from different audits or audit firms, for example in the context of
discussions between an audit committee and an engagement team, depends on

PCAOB Release No. 2015-005


July 1, 2015
Page 8
understanding what makes two engagements comparable and which differences among
firms or engagements matter. Can audits of companies in different industries be
comparable? What about different methods of measurement? Firms may measure
various audit activities in their own ways. They may even measure items differently in
different segments of their practices. Understanding whether and when these
differences are significant is one of the challenges of the AQI project.
The heart of the project, then, is learning what indicators can signal quality in the
conduct of audits. If one can measure a number of elements of an audit, one may have
a more informed basis for asking about, and understanding, how the audit firm
approaches the audit, and similar public company audits, and the results of that
approach. 13 Even if one cannot tie indicators directly to specific results, it may still be
possible to generate questions and trigger discussions that strengthen audit planning
and execution, and, equally important, create an environment conducive to sustained
audit quality and its recognition.
Finding the best ways to use firm-level data illustrates the sorts of inquiries the
Board hopes the AQI project may stimulate. When looking to that data for context, is
office, regional, industry, or firm-wide data the most likely to be useful? It is hard to
answer that question in the abstract. Firm-wide data may be crucial for tone at the top
or quality incentive questions, but analyses based on industry audits, or office or
regional experience, may be critical to other discussions, especially if there have been
quality questions about audits in the industry or geographic area in which a public
company operates.
The indicators can also have two broader effects. First, comparative information
about audit firms may over time help to drive a more vibrant "market in quality" for audit
services and stimulate competition among audit firms on that basis and may counter
pressures on audit firms to reduce audit effort or resources inappropriately; both
consequences may enhance audit quality across the profession. Second, as the project
evolves, the potential availability of AQI information may help investors become better
able to evaluate the audit quality associated with particular financial statements, with a
consequent effect on investment decisions.
The AQI project is closely related to two other Board projects studying the details
of audit quality. 14 These are the Division of Registration and Inspection's analysis of the
13

The Board's inspection program indicates that the quality of public


company audits inspected remains uneven. Some engagement teams do an excellent
job, while others are deficient in important respects.
14

As noted above, the efforts of the Board are all aimed, directly or
indirectly, at audit quality. Audit regulators in other parts of the world are also part of a

PCAOB Release No. 2015-005


July 1, 2015
Page 9
"root causes" of audit successes and deficiencies, and the Office of the Chief Auditor's
standard setting project updating the Board's quality-control standards. 15 The root
cause project is working closely with audit firms to identify factors that differentiate high
quality from deficient audits and audit practices. It has informed the AQI project to date
(so that there is a significant overlap in the areas on which the two projects focus), and
it is likely that the root cause project will continue to surface prime candidates for AQIs
in the future. The quality control project focuses, among other things, on the types of
information, including AQIs, that an audit firm should incorporate in its audit practice,
and then monitor, to allow it to flag and remediate problems before they result in audit
deficiencies. 16
Questions Overview
Readers of the release are encouraged to answer any and all questions posed
here and in later parts of the release in which they have an interest and to comment on
any aspect of the release not covered by specific questions. They are especially
encouraged to provide suggestions for alternative approaches in any relevant area.
Question 1. Is increasing knowledge about, and use of, the audit quality
indicators discussed in this release likely to provide insights about how to
evaluate, and ultimately improve, audit quality? If so, why? If not, why
not?
Question 2. Are the AQI project, and some number of the 28 specific
indicators described below, likely to build a strong knowledge base to
enhance discussions of audits among those involved in the financial
reporting process or other users of AQIs?
Question 3. Can the development of audit quality indicators, as
described in this release, have unintended consequences, either positive
similar effort to enhance the strength and usefulness of the auditing of public
companies.
15

All three of the initiatives are discussed in the Board's Strategic Plan for
2014-2018.
See Public Company Accounting Oversight Board, Strategic Plan:
Improving the Quality of the Audit for the Protection and Benefit of Investors, 2014
2018 at 10-12, 15, 18 (Nov. 2014), http://pcaobus.org/About/Ops/Documents
/Strategic%20Plans/2014-2018.pdf. See also, paper prepared for the Meeting of the
Standing Advisory Group of the Public Company Accounting Oversight Board (June 2425, 2014), http://pcaobus.org/News/Events/Documents/0624252014_SAG_Meeting/
06242014_AQI.pdf.
16

See also the discussion infra at page 14.

PCAOB Release No. 2015-005


July 1, 2015
Page 10
or negative, for audit committees, audit firms, investors, or audit or other
regulators? What are they? Can any negative consequences be
alleviated? How?
Question 4. What is the nature of the context that those using AQIs as a
basis for analysis and discussion will generally require to be able to
benefit from that use? Is the information required to build that context
available? Is access to the necessary contextual information feasible?
III.

Potential Audit Quality Indicators

Audit quality can be viewed from several perspectives. 17 One is an auditor's


operating in full compliance with professional auditing standards and applicable law. A
second is an auditor's meeting the needs of a public company's investors, and the
17

A classic academic definition speaks of "audit quality" as "the marketassessed joint probability that a given auditor will both (a) discover a breach in the
client's accounting system, and (b) report the breach." Linda E. DeAngelo, Auditor Size
and Auditor Quality, 3 J. Acct. & Econ 183 (1981).
The U.S. Government Accountability Office ("GAO") (then named the General
Accounting Office) provided a more detailed definition in a study of auditor rotation
required by Sarbanes-Oxley, one that incorporated more directly the standards for the
conduct of an audit in footnote 14:
[A quality audit is an audit conducted] in accordance with generally accepted
auditing standards (GAAS) to provide reasonable assurance that the audited
financial statements and related disclosures are (1) presented in conformity with
GAAP and (2) are not materially misstated whether due to errors or fraud. This
definition assumes that reasonable third parties with knowledge of the relevant
facts and circumstances would have concluded that the audit was conducted in
accordance with GAAS and that, within the requirements of GAAS, the auditor
appropriately detected and then dealt with known material misstatements by (1)
ensuring that appropriate adjustments, related disclosures, and other changes
were made to the financial statements to prevent them from being materially
misstated, (2) modifying the auditor's opinion on the financial statements if
appropriate adjustments and other changes were not made, or (3) if warranted,
resigning as the public company's auditor of record and reporting the reasons for
the resignation to the SEC.
GAO, Required Study on the Potential Effects of Mandatory Audit Firm Rotation, GAO04-216, note 14, at page 13 (November 2003),
http://www.gpo.gov/fdsys/pkg
/GAOREPORTS-GAO-04-216/content-detail.html.

PCAOB Release No. 2015-005


July 1, 2015
Page 11
marketplace, for independent, effective, and reliable audits of the company's financial
statements, conducted by auditors who exercise due professional care, including
professional skepticism; such audits, among other things, reduce the risk of material
errors or accounting fraud and provide timely reporting of material weaknesses in the
company's internal control over financial reporting ("internal control"), and of going
concern issues. A third is facilitating, as part of the process, the timely and effective
supply of information, most importantly to the company's audit committee and public
investors. The end result should be robust audits that provide "reasonable assurance
[that] the financial statements are free of material misstatement," and "present fairly, in
all material respects, the financial position of the Company . . . in conformity with
generally accepted accounting principles [or another applicable reporting framework]." 18
(In one sense, the higher the level of audit quality, the more certain users are that
financial statements are free of material misstatements.)
These perspectives reflect the dual nature of the audit, as a process and a result,
while confirming that the ultimate test is the strength of the result described in the
preceding paragraph. More important, they suggest the need for a broad approach to
identifying audit quality indicators, to reflect the different factors that can affect such
quality.
The framework for analyzing audit quality, developed by the AQI project and
discussed in this concept release, has three parts: (i) audit professionals, (ii) audit
process, and (iii) audit results. As indicated above, the audit professionals portion of the
framework includes indicators relating to auditors' availability, competence, and focus,
with emphasis on the details of staffing and experience; audit process includes
indicators relating to tone at the top and leadership, incentives, independence,
infrastructure, and monitoring and remediation; and audit results includes indicators
concerning financial statements, internal control, timely reporting of going concern
issues, communications between auditors and audit committees, and enforcement and
litigation. The framework focuses as much, if not more, on factors that have not
heretofore been observable generally those relating to audit professionals and
process as on those that have, relating to audit results.

18

See paragraphs .08 e. and h. of AU Section 508. The basic element of


the report in paragraph .08 h is: "[a]n opinion as to whether the financial statements
present fairly, in all material respects, the financial position of the Company as of the
balance sheet date and the results of its operations and its cash flows for the period
then ended in conformity with generally accepted accounting principles." Of course,
audits of public companies must be performed in accordance with the all applicable
reporting and related professional practice standards, including the rules relating to
evaluation of internal control on financial reporting.

PCAOB Release No. 2015-005


July 1, 2015
Page 12
Using this three part framework, the Board has identified 28 indicators as
candidates to provide insight into audit quality, out of a total of about 70 possible
indicators the project team reviewed. 19 The framework and proposed indicators reflect
the experience of the Board's Divisions (as noted above, the work of the Division of
Registration and Inspections' root cause initiative has been especially important), but
they are to some extent intuitive at this stage of the project. Thus, the "audit
professionals" factors are generally consistent with those discussed by the Treasury
Advisory Committee and others noted in section V, and the "audit process" factors
reflect elements in both the Board's existing quality control standards and widely-used
internal control frameworks. Criteria used to evaluate potential indicators (by a team of
experienced auditors from the Board's Office of Research and Analysis, Division of
Registration and Inspections, and Office of the Chief Auditor) included anticipated
correlation to audit quality, potential usefulness, potential for unintended consequences,
potential for becoming a "manage to measure" benchmark, scalability, quantifiability,
availability of data, redundancy, potential to become a lagging indicator, anticipated
precision of the signal the indicator would generate, and ability of the indicator to
generate insight into possible root causes of audit strengths or weaknesses.
The AQI project is focusing on ways to evaluate the potential indicators in the
future and compilation and evaluation of available data is ongoing. One goal of the
concept release is to generate discussion and use of the indicators that can aid in that
effort.

19

Of those, about 40 were presented to the Board's Standing Advisory


Group in its May 2013 meeting, which included break-out sessions to discuss possible
indicators; the Discussion Paper which included the 40 potential indicators can be found
at http://pcaobus.org/News/Events/Pages/05152013_SAG.aspx.
Many additional
potential indicators came from the ensuing discussion.

PCAOB Release No. 2015-005


July 1, 2015
Page 13

AUDIT PROCESS

AUDIT PROFESSIONALS

The 28 potential indicators are: 20


Availability

Competence

Focus
Tone at the Top and
Leadership

Staffing Leverage
Partner Workload
Manager and Staff Workload
Technical Accounting and Auditing Resources
Persons with Specialized Skill and Knowledge
Experience of Audit Personnel
Industry Expertise of Audit Personnel
Turnover of Audit Personnel
Amount of Audit Work Centralized at Service Centers
Training Hours per Audit Professional
Audit Hours and Risk Areas
Allocation of Audit Hours to Phases of the Audit
Results of Independent Survey of Firm Personnel

Incentives

14. Quality Ratings and Compensation


15. Audit Fees, Effort, and Client Risk

Independence
Infrastructure
Monitoring and
Remediation

16.
17.
18.
19.
20.
21.

Financial Statements
AUDIT RESULTS

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.

22.
23.
Internal Control
Going Concern
Communications
between Auditors and
Audit Committee
Enforcement and
Litigation

24.
25.
26.

Compliance with Independence Requirements


Investment in Infrastructure Supporting Quality Auditing
Audit Firms' Internal Quality Review Results
PCAOB Inspection Results
Technical Competency Testing
Frequency and
Impact
of
Financial
Statement
Restatements for Errors
Fraud and other Financial Reporting Misconduct
Inferring Audit Quality from Measures of Financial
Reporting Quality
Timely Reporting of Internal Control Weaknesses
Timely Reporting of Going Concern Issues
Results of Independent Surveys of Audit Committee
Members

27. Trends in PCAOB and SEC Enforcement Proceedings


28. Trends in Private Litigation

Appendix A (which should be read together with this section of the release),
describes the objective of each indicator, discusses briefly the reason for its inclusion,
and contains possible calculations for the indicator at both the engagement and firm 21
levels. In the case of one (indicator 7) calculation is suggested only at the engagement
level because the scale of an audit firm practice could make generation of meaningful
20

More detailed discussion of Audit Professionals, Process, and Results


indicators, appears at pages A-2, A-12, and A-19, respectively, of Appendix A.
21

As discussed at several points in this release, the definition of "firm" level


raises a number of its own issues.

PCAOB Release No. 2015-005


July 1, 2015
Page 14
data at the firm level difficult. In the case of three indicators (13, 14, and 26) calculation
is suggested only at the firm level, to assure sufficient sample size and anonymity.
Comment is sought below about ways to test the strength of the indicators as ways to
evaluate, and their usefulness for informing discussions about, audit quality.
In presenting the indicators for comment, the Board's goal is to identify a
manageable number of indicators that can help provide an effective practical picture of
audit quality to inform discussions that may produce improvements in quality. Certain
indicators on the current list may ultimately prove to be duplicative, and others may not
generate sufficiently useful information; comments may suggest new indicators that
should be added to the list. Again, the ability of the indicators identified to function as a
balanced group is thought to be critical to their successful use.
The choice of indicators is not fixed. The effort is new, and by their very nature
audit quality indicators must be capable of change over time to reflect advances in
learning and changes in the way audits are conducted. In addition, subsequent events,
for example restatements for errors affecting the public company using the indicators,
can cause particular indicators to be re-evaluated. One of the purposes of the project is
to assemble a sufficiently large collection of information about the indicators and
subsequent events to refine the ideas the release puts forward. Additional experience,
including that made possible by use of the first generation of AQIs, may suggest
valuable new or substitute indicators, or broader approaches to measuring audit quality.
In particular, analysis by the Board's Division of Registration and Inspections of data
about individual audits, audit firms, and audit quality, especially the work of the
Division's root cause analysis project, should become increasingly closely coordinated
with the AQI project and may prove critical in evaluating particular AQIs.
Questions - Selection of Indicators (as listed above and described in Appendix A)
Question 5. Should any indicators be omitted from the list proposed in
this release? Which indicators? Why?
Question 6. Should any indicators be added to the list? What are they?
Why? How would they be quantified?
Question 7. Which indicators are likely to be the most useful in
evaluating audit quality and informing discussions of audit quality? Why?
The least useful? Why?
Question 8. Which indicators, including any mentioned in response to
Questions 6 and 7, are in use today? How are they being used? Which
ones are relatively more effective? Less effective?

PCAOB Release No. 2015-005


July 1, 2015
Page 15
Question 9. Definition of the Indicators.
a)

Are the indicators clearly defined?

b)

Which indicators would benefit from a clearer definition?

c)

Are the suggested methods for measuring each indicator


clear?

d)

Are there other ways to measure particular indicators that


would be more effective?

e)

While most indicators depend only on data from a firm's


public company practice, some include information
concerning workload from both public company and nonpublic company practice. Are there other indicators that
would be more useful if they were extended to the breadth of
an audit firm's practice?

Question 10. Do particular indicators risk becoming too complex in


operation to reflect the reality of particular audit situations?
Question 11. Does the time lag between an audit year and the availability
of information for many of the results indicators (e.g., whether a
restatement has occurred) affect their value? How?
Question 12. Are there one or more indicators among the 28 that are
superior to other indicators on the list and cover the same subject or
subjects, so that one or more indicators are unnecessary for that reason?
Please identify the redundant indicators and explain.
Question 13. Are data available for each of the indicators? To what
extent, specifically, is the data already broken out in audit firms' operating
systems?
Question 14. The indicators operate at the engagement level, the firm
level, or in most cases both.

PCAOB Release No. 2015-005


July 1, 2015
Page 16
a)

How should "engagement level" be defined in the case of a


global audit in which work is referred to one or more "other
auditors" (whether or not the firm or firms involved are part of
the engagement firm's global network)? Who should make
that determination?

b)

Would one or more of the indicators be more useful if it also


operated at an audit firm's "office" or "regional" levels, not
merely at "engagement" and "firm-wide" levels (so that, e.g.,
the percentage of an office's work devoted to a single large
client would be known)? Which indicator or indicators?

c)

Would one or more of the indicators be more useful if it also


operated at the level of the audited company's industry or
economic sector (so that, e.g., indicators for the audit of a
particular bank could be compared with the average of
indicators for all of an audit firm's banking clients)?

Question 15. What are the elements of "context" required for successful
analysis of the 28 potential AQIs? Are those elements ordinarily available
to AQI users? If not, is it feasible to make the elements of context
available?
Question 16. Comparability.
a)

How important is comparability to the value of AQIs?

b)

What are the most important elements of comparability in the


analysis of AQIs?

c)

Is comparability more likely to be fostered by firm-wide data


(either within or among firms) or data focused on industry,
regional, or office practices?

d)

Does the existence of differences among firms in the way


certain matters
(e.g., classification of personnel) are
measured affect the value of AQIs if those differences are
disclosed? If they are not disclosed?

PCAOB Release No. 2015-005


July 1, 2015
Page 17
Question 17. How should audits of different size and complexity be
weighted in the calculation, analysis, and discussion of firm-level data?
Question 18. What are the costs and obstacles to audit firms of compiling
the relevant data? Can data be created at reasonable cost for any
indicator for which they are not now available? If not, is there another
indicator of comparable scope, either among the 28 or otherwise, for
which it would be less costly to obtain the necessary data?
Question 19. In what situations could generation of AQI data impose
collection or evaluation costs (whether involving specific indicators or the
use of AQIs generally) without commensurate benefits? Could those
costs be reduced? How?
Question 20. Could the collection and evaluation costs of AQIs be a
greater economic burden for smaller audit firms than larger audit firms?
Could this burden disadvantage smaller firms in competing for audit
business if perceptions of quality are driven by the indicators?
Question 21. In what ways should the various indicators be evaluated or
field-tested?
IV.

Use and Availability of Audit Quality Indicators

Identifying promising indicators is only the first step. For AQIs to help turn a
clearer view of audits into increases in audit quality, people who influence audit quality
must use them in their decision-making.
An AQI project can take different shapes for the future, depending on choices
among a number of variables. These include (i) the potential users involved, (ii) the
options for obtaining and distributing the relevant data, (iii) potential approaches to
phasing in AQIs (so that, for example, not all data would be available immediately), and
(iv) exclusion of certain audit firms or types of audits from the effort, at least initially, if
relevant indicators are not adequately scalable. Again, choices in any of these areas
could naturally change with experience.

PCAOB Release No. 2015-005


July 1, 2015
Page 18
A.

Users of AQIs: The Board believes that the potential primary users and
range of uses of AQIs can be summarized as follows:

Potential AQI User


Audit Committees

Potential Use (Decisions AQIs Can Influence)

Audit Firms

Investors 22

PCAOB (and other Regulators)

Assess reporting risk and audit quality


Retain and compensate auditors
Oversee auditors
Assess and manage risk
Improve quality control efforts and, ultimately,
audit quality
Identify root causes of audit deficiencies and
remediate weaknesses
Assess reporting risk
Vote shares
Inform policy-making
Assist root cause and quality control projects
Stimulate public discussion of, and market
demand for, quality

Other users of AQIs could include company management, the business press,
academics, and the general public. As noted above, the indicators are intended to
function as a balanced portfolio, and the initial assumption is that the portfolio would be
the same for all users. (Experience and research, however, may indicate the benefits of
modifying the group for situations where scalability or the nature of particular audits is a
consideration.) Different classes of users could receive different levels of disclosure of
AQI data.
Audit Committees. Independent audit committees of the boards of directors of
listed companies are directly responsible by statute or regulation for the appointment,
level of compensation, and oversight of their companies' auditors (including resolution of
disputes between the auditors and management concerning financial reporting), and
those auditors report directly to the audit committee. 23 Audit committees for all (listed or
22

Investors could only use AQI information if, when, and to the extent that
information is made publicly available. See the discussion infra at page 27. But of
course use of AQI data by audit committees to produce higher quality audits can benefit
the investors in the companies involved.
23

See Section 10A(m) of the Exchange Act and Exchange Act Rule 10A3(b). Foreign private issuers are exempt from these requirements if their home country
law requires their having an independent "board of auditors," with roughly similar
authority, and some other listed companies primarily passive investment vehicles, are
also exempt. See 17 CFR 240.10A-3(c)(6). Rule 32a-4 under the Investment Company

PCAOB Release No. 2015-005


July 1, 2015
Page 19
not) public companies must be given reports of critical matters related to the audit and
are responsible for preapproving provision of auditing and permissible non-audit
services by the auditor to the company. 24 AQIs may give the audit committees of both
listed and other public companies additional relevant data to explore these matters and
enhance dialogue with their auditors.
Based on discussion with a number of their members, it appears that at least
some audit committees are beginning to consider the idea of AQIs and have started to
question and track their company's auditors using ideas like those outlined in this
release. A more systematic approach to the definition and aggregation of AQI data
could help audit committees take these steps, especially producing enhanced dialogue
with their companies' auditors. But even if audit committees obtain information from
their audit firms about their own company's audits, they may lack comparative data
(derived from other audit operations of the firm, let alone from those of other firms) to
place that information in the context necessary to give it the greatest clarity. For
example, an audit committee might inquire about the staffing ratio (the ratio of
experienced senior personnel to all members of the engagement team) for its audit. The
average staffing ratio for all audits the firm performs and all audits the firm performs in
the company's industry, as well as comparable figures for audits by other firms (when
and if such data is available), might all provide necessary context to indicate what the
data means when an audit committee evaluates its audit engagement team or audit
firm. Finally, without a more systematic approach, audit committees may have no
assurance about the data's significance or the extent to which data from different audits
are derived from standard definitions and calculations.
An audit committee would likely focus first on AQI data at the engagement level
for the audit the committee oversees. The indicators are intended to provide information
to help frame the oversight and evaluation of a current or pending audit. 25 Even without
a broader set of data, information about an engagement team or its deployment may
highlight issues that, if addressed, could increase audit quality.
Act of 1940, 17 CFR 270.32a-4, requires oversight of the audit (but not selection of the
auditor) of registered management investment companies by an independent audit
committee of the board, but only if the company wishes to eliminate the need for a
shareholder vote on ratification of auditor selection. Governance is largely the province
of state or non-United States law for the remaining groups of public companies.
24

See Section 10A(h)-(k) of the Exchange Act, Regulation S-X Rule 201(c)(7) and Auditing Standard No. 16, supra note 11.
25

An exception would be a committee's review of tenders for new audit


engagements, when the audit committee would likely focus on firm-level data from
tendering audit firms (and potential competitors), to compare with firm- and
engagement-level data relating to prior audits.

PCAOB Release No. 2015-005


July 1, 2015
Page 20
Questions arise, however, about how to assemble data for an audit involving
more than one auditor. 26 Audits of global companies, for example, require participation
of auditors from multiple firms that may or may not be part of an affiliated network.
Providing AQI data for each firm could provide the most insight if audit quality differed
widely from firm to firm. But AQIs for the portion of an audit performed by separate
firms might be difficult to compile and might distort the picture of the audit as a whole.
Understanding and resolving these issues are challenges of the project going forward.
An audit committee would more than likely then turn to information about the
audit firm that performs (or is proposing to perform) the audit in question. The audit
committee retains a firm, and it seeks to associate its company with a reputable
organization that can provide the market with a basis for confidence in the quality of a
company's financial reporting. Moreover, the audit firm's overall commitment to quality
can shape the engagement team, which does not operate in isolation; tone at the top,
the incentive system, recruiting, retention, training, technology, tools, and knowledge
bases all affect the strength of the engagement team's work. The culture of the audit
firm, and not just the engagement team, is likely to influence audit performance. Given
these realities, the audit committee may have questions about how its engagement
compares with engagements conducted by its audit firm for similar companies. AQIs
related to the firm are likely to provide context for audit committees to evaluate
engagement level AQIs, and ask questions about them, far more effectively than would
otherwise be possible.
For the same reason, audit committees may find it helpful to compare AQI data
across firms (if the latter data are available). Quality is a relative concept, and
differences among firms may be instructive, when context and the details of AQI data
are taken into account. The availability of firm-level data, in context, may also help
answer the question posed by the Treasury Advisory Committee about the drivers of
competition among, and levels of quality at, different audit firms, so that AQI data,
properly used and understood, may help spark, and sustain, competition based on audit
quality and its improvement generally.
Finally, it may be useful to compare engagement AQIs to AQIs for other audits in
the same industry, especially if the audit requires specialized skills. For example, audits
of companies in the financial services or telecommunications sectors may require
auditors with advanced training and an engagement team with a disproportionate
number of experienced personnel. An audit committee may also be especially
interested in other audits by engagement teams from the same office or region as the
engagement team the audit committee is supervising.

26

See supra Question 14 a).

PCAOB Release No. 2015-005


July 1, 2015
Page 21
An audit firm's candid discussion of its AQIs with an audit committee, like its
discussion of its external and internal inspection results if that already occurs, 27 may
add value not only in relation to the committee's oversight and evaluation of a particular
audit, but also in relation to the committee's oversight of the financial reporting process
more generally. For audit committees that work with auditors who have a strong record
of performance, AQIs may offer comforting and confirming evidence of quality. In other
cases, the information may present insights (perhaps even troubling ones) that allow
audit committees to encourage improvement or seek stronger auditing from others.
Audit firms that provide information about the results of a PCAOB inspection coupled
with results of the firm's own internal quality reviews may help an audit committee
understand how the firm performed on specific audits and in high-risk areas across
audits; that information may in turn help the committee to ask more detailed questions
than would otherwise be possible about specific substantive audit issues, quality
control, and remediation. While AQIs are no substitute for the communications about
audit strategy, risks, and other matters required by Auditing Standard No. 16,
"Communications with Audit Committees," 28 the information the former contain may
complement those communications.
The Board recognizes that ultimately each audit committee will have to judge for
itself whether and how it wishes to use AQIs in its decision-making. The majority of
audit committee members to whom members of the AQI project team have spoken in
developing the project have expressed support. The Center for Audit Quality (the
"CAQ"), whose own AQI proposals are discussed below, 29 sees audit committees as
the logical recipients of the audit quality data they propose, and the Council of
Institutional Investors' list of factors audit committees should consider in determining
whether to continue the retention of an outside auditor contains several factors that
resemble AQIs. 30 Other audit committee members, however, question whether AQIs
27

See Information for Audit Committees about the PCAOB Inspection


Process, PCAOB Release 2012-003 (August 1, 2012).
28

Auditing Standard No. 16, supra note 11. The Standard is effective for
audits of fiscal years beginning on or after Dec. 15, 2012. The release that
accompanied approval of AS 16 by the Board states that "Auditing Standard No. 16
does not preclude the auditor from providing additional information to the audit
committee. Nor does the standard preclude the auditor from responding to audit
committee requests for additional information from the auditor." See PCAOB Release
No. 2012-004.
29
30

See the discussion infra at page 34.

See Council of Institutional Investors, Policies on Corporate Governance


section 2.13, available at http://www.cii.org/corp_gov_policies#2.13a.

PCAOB Release No. 2015-005


July 1, 2015
Page 22
can provide useful insights in light of the information already available from existing or
prospective auditors, the difficulty of tailoring AQIs to a company's unique
circumstances, and the possibility that AQIs could disrupt the standards and
expectations that audit committees must meet. Some have also noted that audit
committees' time is already taken up with a wide range of duties and that devoting the
time to use AQIs well could have the unintended consequence of spreading audit
committees' attention too thin. The Board specifically seeks comment on whether and
how audit committees can most directly benefit from audit quality indicators.
Audit Firms. Firms have long used some AQI-like measures to manage their
audit practices. They may use them to measure efficiency and profitability, to flag
audits or offices with a higher risk of audit failure, 31 or to make compensation decisions.
In addition, some firms are re-thinking how such measures may help them to
understand and respond to internal and PCAOB inspection findings, identify the root
causes of the deficiencies involved, and remediate those deficiencies, upgrading
management processes as a consequence. AQIs may provide useful perspectives in
all of these areas.
Audit firms may be in a unique position to make use of AQI data because of their
ability to correlate the information across many audit engagements to strengthen quality
control and risk management. Discussion with audit committees could broaden the
base of comparison for firms and encourage them to invest in AQI measurement.
Possible dissemination of AQIs from other firms might surface insights that a single firm
was not in a position to identify itself.
Thus, AQIs, applied carefully and thoughtfully, may become an important tool for
audit firms, not only in improving the strength of their audits but in allowing them to
distinguish themselves for audit quality and compete on that basis in the marketplace.
The Board hopes that firms' comments on the release will discuss their experiences in
analyzing audit quality, and that will continue to share those experiences with the AQI
project team as the AQI effort continues to take shape.
Investors. Investors are the primary beneficiaries of the financial reporting
process and the group at which audit quality is ultimately aimed. They elect the
directors of the company in which they invest. In addition, a large majority of public
companies request a shareholder vote to ratify the choice of auditor. 32

31

Firms may subject those audits to special procedures or to internal

inspection.
32

Data obtained from Audit Analytics indicates that 90 percent of the public
companies on the Russell 3,000 list as of April 2014 submitted at least one auditor
ratification proposal to shareholders between 2011-13. In its 2008 Report, the Treasury

PCAOB Release No. 2015-005


July 1, 2015
Page 23
At present, the visibility of the sources of audit quality to investors is even more
limited than it is to audit committees. Investors have no direct channel to the auditor;
communication is typically restricted to the standard auditor's report about a company's
financial statements and internal control over financial reporting. 33
The result is that investor impressions of audit quality come largely from the
frequency and magnitude of negative events, such as restatements for errors or fraud,
related to public company financial reporting, material failures of internal control over
financial reporting, or, on the positive side, the absence of these events. Such events
can certainly be significant and indicate lapses in audit quality or worse. But they
generally occur in a vacuum, without context. So they do not often help investors focus
on the components of the audit process and, more important, provide little information
about how quality audits are planned and executed. Over the past decade, the PCAOB
public inspection reports have been available to help inform investors' impressions of
audit quality, but by design these reports discuss particular audit deficiencies, rather
than the broader elements that can make for quality, and then on an anonymous basis.
Moreover, the reports are not necessarily representative of a firm's overall practice,
because inspections are risk-based and, in the case of the largest audit firms, involve
only a small portion of the firm's public company practice.

Advisory Committee noted that "[a]lthough not statutorily required, the majority of public
companies in the United Statesnearly 95% of S&P 500 and 70%-80% of smaller
companiesput auditor ratification to an annual shareholder vote." Report of the
Treasury Advisory Committee, supra note 8, at VIII: 20.
33

To address this circumstance, the Board has proposed changes to its


auditing standards and rules that can provide new information to shareholders about the
most critical issues addressed by the auditor, the auditor's evaluation of "other
information" in the document containing the financial statements, and the identity of the
engagement partner and certain other participants in the audit. That information would
be largely explanatory but can also help investors and others better evaluate audit
quality across engagements. See (i) Proposed Auditing Standards the Auditor's
Report of an Audit of Financial Statements When the Auditor Expresses an Unqualified
Opinion; the Auditor's Responsibilities Regarding other Information in Certain
Documents Containing Audited Financial Statements and the Related Auditor's Report;
and Related Amendments to PCAOB Standards, PCAOB Release 2013-005 (August
13, 2013), and (ii) Improving the Transparency of Audits: Proposed Amendments to
PCAOB Auditing Standards to Provide Disclosure in the Auditor's Report of Certain
Participants in the Audit, PCAOB Release No. 2013-009 (December 4, 2013);
Supplemental Request for Comment: Rules to Require Disclosure of Certain Audit
Participants on a New PCAOB Form, PCAOB Release No. 2015-004 (June 30, 2015).

PCAOB Release No. 2015-005


July 1, 2015
Page 24
Data from a portfolio of properly chosen AQIs might refocus investors' attention
on the broader elements of audit strength or weakness and the characteristics of quality
audits. It could alert investors to audit quality at a particular engagement, or even a
particular firm, provide them with context to understand what constitutes high-quality
audits, and enable researchers or analysts to provide investors with further insights into
audit quality at the engagement or firm levels. It could also inform investors voting to
ratify an audit committee's choice of auditor. By having these effects, AQIs could, again,
ultimately increase pressure for differentiation in the audit market in terms of quality and
hence drive overall enhancements in quality.
If and when AQI data were to be made publicly available (an issue discussed
below), 34 investors would likely focus first on engagement level AQI information about
particular companies in which they invest or might invest. They might also benefit from
firm-level data (whether reflecting industry, office, regional, or firm-wide data) for the
reasons discussed above in connection with usage of AQIs by audit committees,
namely comparison and context. Investors might receive AQI data from several sources
over time, including audit committee reports or documents published by audit firms or
the Board.
The ability to better distinguish variations in measures that relate to quality may
produce greater market differentiation among audits and stimulate competition in quality
that may also have an effect on securities prices. This in turn could generally help
investors given the public goods nature of securities prices.
A presentation by the Board's Investor Advisory Group Working Group 35 on Audit
Quality Indicators at the Advisory Group's October 16, 2013 meeting emphasized that
AQIs should "measure the quality of the actual audit," "help establish accountability for
audit quality," operate in a "forward-looking" way, and contain "information or predictive
content." The Working Group sought measures "that provide investors with timely
information with respect to the credibility of audits." Seventeen of the Working Group's
27 proposed indicators are reflected in the 28 potential indicators in this release, and
several others are the subject of questions on the indicators. The Board encourages
comments that discuss these issues in detail.
The Board. The Board's inspections staff already receives certain data that
resemble AQI data, and it has used AQI-like measurements in its work, primarily to flag
firms, offices, and audit engagements that are at a relatively higher risk of audit
deficiencies. Expanded AQI data, within the context discussed in this concept release,
34
35

See the discussion infra at page 27.

See infra note 41. The Investor Advisory Group is a forum for the investor
community to provide its views and advice on matters affecting investors and the work
of the Board.

PCAOB Release No. 2015-005


July 1, 2015
Page 25
can assist the Board in several ways. First, the data can inform the development of
inspection strategies. Second, the data can provide the Board with insights into trends
in overall audit quality at the firm, network, or profession levels. Third, it can broaden
the foundation for the Board's consideration of policy issues. Finally, as noted above,
expanded use and analysis of AQIs could inform analysis of the root causes of
inspection findings, evaluation of firms' remediation efforts, and the effort to update
auditing standards, including those related to audit firm quality control.
B.

Obtaining and Distributing AQI Data

The AQI information envisioned by this release comes most importantly from
audit firms. The data for 19 of the 28 potential indicators can be obtained only from the
firms (or, in one case, from an independent survey of firm personnel); the data for eight
of the nine remaining indicators (the "audit results" indicators, indicators 21-28) can be
derived from public sources, while the remaining potential indicator is a possible survey
of audit committee members. Information about background and context must also
generally come from the firms, who are in the best position to provide it for firm and
client industry indicators and for indicators relating to particular engagements. (As
experience with AQIs grows, third parties, including academic researchers, may
assume a role of assembling, analyzing and providing perspective for AQI information
that is public, as is already done for, e.g., financial statement restatements for errors.
The Board and other regulators could also analyze the indicators and provide context,
subject to applicable disclosure limitations.)
Others could be involved in distributing AQI data, even if firms are the source of
that data and much of the accompanying context. Data aggregators might collect and
publish AQIs derived from public information. Companies or audit committees could
include engagement-level AQIs (and perhaps firm-level AQIs, for context) in their
financial filings or audit committee reports to shareholders, as some do now to a limited
extent.
The Board could take one or more of several approaches to assisting in the
distribution of the data. For example, as and when appropriate as the AQI project
matures, it could (i) encourage firms and engagement teams voluntarily to discuss AQI
engagement- or firm-level data with audit committees, or to do so publicly, (ii) require
audit teams to provide that data to audit committees, (iii) collect and make "combined"
AQI data public over time, as a single set of weighted figures for comparable firms, (iv)
collate and make public on a firm-by-firm basis AQIs derived from public sources, and
(v) consider requiring reporting of the necessary data to the Board so that the Board
could make it public, or even require firms to do so directly. (Of course, the strength of
any approach, at the stage of the project involved, as well as any legal or other issues,
would have to be examined at the time.)

PCAOB Release No. 2015-005


July 1, 2015
Page 26
The timing of distribution of AQI data likely depends on, among other things, the
user involved, the company's complexity, and the timing of the audit cycle. It is possible
that the data could come to be included as a supplement to required communications at
the beginning of the audit cycle. 36 Audit committees may, however, also benefit from an
update following completion of planning and preliminary audit work and before the start
of final audit procedures. If engagement-level information were made available to
shareholders, directly or indirectly, disclosures might be made most effectively before
each year's requested shareholder vote to ratify the appointment of the auditor. Audit
firm-level AQIs might be made available to shareholders annually, for example, through
an annual report on audit quality released by firms for this purpose. Firms themselves
would also receive data about other firms when the latter made the data publicly
available, again, assuming that such a level of disclosure becomes part of the project.
C.

Implementation over Time

Of course, there is no need to adopt all of these approaches, and certainly no


need to adopt them simultaneously. Many projects are implemented in a series of
planned steps or phases rather than all at once. The advantages of a phase-in include
flexibility to alter the project as initial experience is evaluated and ability to build support
among the parties affected, if they perceive tangible benefits from each step along the
way. (On the other hand, a phased approach may delay the ultimate benefits of the
project and may make it more difficult to implement the project's later stages.)
The manner of phasing in an AQI project would depend on the project's details.
For example, the project could initially involve the Board's support of voluntary use of
AQIs, by engagement teams, audit committees, and audit firms. A voluntary approach
could give audit committees and auditors time to become comfortable with the
indicators and gain experience in using them to aid in decision-making. It would also
provide time for study of the indicators and their effectiveness. After a period of learning
and demonstrated benefits, the project could expand to include required disclosure of
AQIs to audit committees, public disclosure of AQI data, or both.
A phased approach might ultimately produce a solidly grounded result. Again,
variations are possible. To maximize voluntary use, the Board could specify the most
promising AQIs and provide standardized definitions and guidelines that would promote
comparable data. The project could even begin with a relatively small number of
indicators and expand the list for discussion over time, or consider pilot projects to
permit testing and evaluation of the indicators involved.
36

The CAQ suggests this approach in its paper on audit quality indicators.
See Center for Audit Quality, CAQ Approach to Audit Quality Indicators (2014) at 6,
available at http://www.thecaq.org/docs/reports-and-publications/caq-approach-to-auditquality-indicators-april-2014.pdf?sfvrsn=2.

PCAOB Release No. 2015-005


July 1, 2015
Page 27
One benefit of a phase-in would be to provide a basis for understanding more
fully the implications of making AQI information public. Firms are of course free to do so
now, and some are beginning to make some similar information public, as discussed
below. Other firms may do the same, providing context for audit committees and
beginning to do so for investors. But a general requirement that results in AQI data
becoming public, because of its very nature, would have to follow testing, and
evaluating, the use of AQIs and weighing the benefits and costs of public dissemination.
No decision has been made about this subject, but phasing in the use of AQIs may
produce the experience and information that can produce a basis for such a decision in
the future.
However the issues involved in moving an AQI project forward are ultimately
resolved, the Board will have the responsibility, as the Treasury Advisory Committee
intended, to monitor the terms and performance of any AQI project.
D.

Possible Exclusion of Certain Audit Firms or Certain Audits

The optimal scope for an AQI project is also an important issue; its consideration
raises three broad questions. The first is whether the project should extend to all, or
only some, registered accounting firms. The Treasury Advisory Committee suggested
that one of the uses of AQIs could be to "enhance . . . the ability of smaller auditing
firms to compete with larger auditing firms . . . ." 37 But evaluations of AQI data likely
need to consider differences in results in relatively larger and smaller firms, respectively,
and the ramifications of those differences.
Issues of scalability such as these are complex. The Board may need to
consider whether the management of a smaller firm is sufficiently different than the
management of a large firm that some AQI's are not as relevant for one type of firm as
the other, and how to create meaningful measures of quality for firms of significantly
different sizes. The Board could decide, for example, to focus the AQI project initially
on the largest audit firms, who audit the financial statements and internal control over
financial reporting of most of the nation's public companies (in terms of market
capitalization). This approach will provide more time to study scaling of the indicators,
but it may also limit the benefits for the excluded firms and their public company clients
and encourage the market to differentiate between firms solely by size (because AQIs
begin to apply to larger firms first) rather than audit quality.
The second question of scope facing the Board is which public company audits
should be reflected in indicator data. As discussed above, only listed companies are
required to vest auditor selection, compensation, and oversight in independent audit
37

United States Department of the Treasury, Advisory Committee on the


Auditing Profession Final Report, supra note 8.

PCAOB Release No. 2015-005


July 1, 2015
Page 28
committees; many registered management investment companies operate in a similar
fashion under the Investment Company Act of 1940. 38 Equally important, at all public
companies the audit committee (or if there is none the board of directors) must
preapprove any auditing or non-audit service performed by the company's auditor. The
fact that AQIs may potentially be of use to any public company directors with
governance responsibilities, their shareholders, and their auditors, is a consideration in
answering this question.
A third, related, question is whether to exclude from the AQI project audits in
certain industries where the nature, timing, and extent of audit work can differ from the
norm and differ equally widely within the particular group of audits involved. For
example, audits of employee benefit plans and registered investment companies are
likely to prove very different from the audits of global manufacturing companies. Are the
differences enough to render application of AQIs to the former audits irrelevant, or make
the comparison of AQIs in these situations misleading, or can problems be addressed
by comparing AQIs for one engagement against AQIs for other engagements in similar
industries or entities? Conversely, would excluding certain types of audits distort the
results of firm-wide public company audit comparisons, or suggest that only industrybased comparisons are valid? (Brokers and dealers in securities pose their own issues,
because few, if any, are free-standing public companies; many are small, and audit
requirements for those companies focus both on financial reporting to the Commission
and compliance with critical customer protection rules.)
Again, the Board could decide to exclude audits in certain industries from the AQI
project. But an exclusion may limit the benefits of AQIs and perhaps signal, incorrectly,
that audits in certain industries are less important than audits in other industries. As an
alternative, the project could initially apply to certain firms or to certain types of audits,
with the Board making clear its intention to expand the number of firms or audits to
which AQIs apply, over time. Finally, the Board could undertake special AQI efforts
aimed at entities with special characteristics, including brokers and dealers in
securities. 39
38

See discussion supra at note 23. Registered management investment


companies are required to select their auditor by a vote, cast in person, of a majority of
the disinterested members of the board of directors. See section 32(a)(1) of the
Investment Company Act of 1940.
39

As of May 24, 2014 (according to Commission data), 377 of the 4,007


brokers and dealers that filed audited financial statements with the Commission were
subsidiaries of public companies of various sizes, but the remaining 3,630 were
privately-owned companies, many of which were small. The Board is not aware of
information indicating that any of the brokers or dealers were free-standing public
companies.

PCAOB Release No. 2015-005


July 1, 2015
Page 29
Questions Use of Audit Quality Indicators
Question 22. For what class or classes of users would AQIs be most
valuable? Would some AQIs be more valuable than others to various
classes of users?
Question 23. Are there one or more groups, in addition to audit
committees, investors, audit firms, and the Board and other regulators,
that the Board should consider to be primary users of audit quality
indicators? If so, what are they? Does their need for the indicators, in
each case, differ from those of other primary users?
Question 24. Does the discussion of the uses of the indicators identify all
likely uses? If not, what other uses should be considered?
Question 25. How important to the usefulness of the indicators by audit
committees and other users is AQI engagement-level data? AQI firm-level
data for the audit engagement firm?
Question 26. To what extent do audit committees already receive AQIlike information from their audit firms? What are the most significant gaps
in the information they receive compared to the information that could be
contained in the potential AQIs?
Question 27. To what extent would engagement-level AQIs be useful to
investors? AQI firm-level data for the engagement firm? What AQIs
would be most useful? Why?
Question 28. Should engagement level AQI data be made public in
whole or part? Should firm level AQI data be made public in whole or
part?
Question 29. How important to the usefulness of the indicators by, audit
committees, audit firms, investors the Board and other regulators, and
others is the public availability of firm-wide AQI data for the audit firm that
performs a particular engagement?
How important is the public
availability of AQI data for other audit firms of comparable size?
Question 30. To what extent would firm-level data be more useful, for all
or some indicators, if it were broken out in industry categories?
Question 31. Would it be useful to phase in any ongoing AQI project?
For example, should the project be voluntary for at least some period? If
phasing is a good idea, what steps should the phasing involve? How
should any phasing of the project be monitored?

PCAOB Release No. 2015-005


July 1, 2015
Page 30
Question 32. How should AQI data be made available, either during a
phase-in or ultimately? Which of these approaches is preferable?
a)

By audit firms voluntarily to audit committees, at the


engagement level, the firm level, or both?

b)

By audit firms voluntarily to the public, at the engagement


level, the firm level, or both?

c)

By audit firms on a required basis to audit committees, at the


engagement level, the firm level, or both?

d)

By audit firms on a required basis to the public (whether


directly or through the Board), at the engagement level, the
firm level, or both? Would disclosure by audit firms directly
or by the Board be preferable?

Question 33. Should the Board consider steps to require audit firms to
make engagement- and firm-level AQI data available to audit committees?
To investors?
Question 34. Should distinctions be made, in the timing or nature of
AQIs, among the audit firms that audit more than 100 public
companies? 40 What potential distinctions would be most useful?
Question 35. Should smaller audit firms be treated differently than large
ones in designing an AQI project? What would small mean for this
purpose? Having less than a certain number of auditors? Auditing 100 or
fewer public companies per year and not being part of a global network of
firms?
Question 36. Should the size of the audited company set a limit on initial
application of an AQI project? What would an appropriate size be?
Should the fact that a public company is not a listed company affect the
way AQIs apply to it?
40

For 2014, nine U.S.-based firms audited more than 100 public companies:
BDO USA, LLP, Crowe Horwath LLP, Deloitte & Touche LLP, Ernst & Young LLP,
Grant Thornton LLP, KPMG LLP, MaloneBailey, LLP, McGladrey LLP, and
PricewaterhouseCoopers LLP See http://pcaobus.org/Inspections/Pages/default.aspx.
Sarbanes-Oxley requires the Board to inspect annually "each registered public
accounting firm that regularly provides audit reports for more than 100 issuers [i.e.,
public companies]. See Section 104(b)(1)(A) of Sarbanes-Oxley.

PCAOB Release No. 2015-005


July 1, 2015
Page 31
Question 37. How should the nature of the industry affect the design of
an AQI project? For example, is the nature of audits of investment
companies or employee benefit plans sufficiently different than that of
other public companies that the former require their own set of AQIs?
Question 38. Would excluding certain types of audits from an AQI project
distort the results of firm-wide public company audit comparisons, or
suggest that only industry-based comparisons are valid?
Question 39.
securities?

V.

Should an AQI project apply to brokers and dealers in

a)

Should the project apply to carrying brokers, introducing


brokers, or both? Should it apply differently to brokerdealers that are subsidiaries of public companies than to
broker-dealers that are privately-owned?

b)

What indicators would be most appropriate?


Would
indicators aimed at the special regulatory requirements for
broker-dealer reporting be advisable?

c)

Who would the users of the information be?

d)

Do the variations within the audited population make


comparability of information difficult?

Outreach; Other AQI Projects

In formulating the ideas reflected in this release, the Board and staff have
engaged in a broad project of outreach, speaking with representatives of a number of
interested parties, as the Treasury Advisory Committee suggested. They have met
several times with the Board's Standing Advisory and Investor Advisory Groups; 41 the
first of those meetings involved an extensive review of potential AQIs that helped guide
41

The staff made formal presentations to meetings of the Standing Advisory


Group on May 15-16 2013, that included break-out sessions to discuss possible
indicators (available at http://pcaobus.org/News/Events/Pages/05152013_SAG.aspx),
November
14,
2013
(available
at
http://pcaobus.org/News/Events/Pages
/11132013_SAG.aspx), and June 24-25, 2014 that included a presentation on
"Initiatives to Improve Audit Quality Root Cause Analysis, AQIs, and Quality Control
Standards" (available at http://pcaobus.org/News/Events/Pages/05282014_SAG.aspx).
The staff attended and participated in a discussion that included a presentation by the
Investor Advisory Group's Working Group on Audit Quality Indicators on October 16,
2013 (available at http://pcaobus.org/News/Events/Pages/10162013_IAGMeeting.aspx).

PCAOB Release No. 2015-005


July 1, 2015
Page 32
development of this release. Equally important, the project has been discussed with a
wide range of parties who reflect the classes of users outlined above. These include
the Commission staff and federal banking regulators, the National Association of
Corporate Directors (and smaller groups of audit committee members), the CAQ and
individual audit firms, and Financial Executives International. Presentations have been
made at various professional conferences, including ones sponsored by the American
Institute of Certified Public Accountants and by the American Accounting Association,
as well as at the PCAOB's own Academic Conference. Board staff have met as well
with representatives of the U.K.'s Financial Reporting Council (the "FRC"), and the
Canadian Public Accountability Board. Finally, the Board and staff coordinated and led
discussions on audit quality indicators at the April 2014 plenary meeting of the
International Forum of Independent Audit Regulators ("IFIAR"). Discussion of AQIs
during that meeting indicated that a number of audit regulators around the world were
using AQI-like ideas to plan and evaluate their inspections. Attention was again
devoted to the subject at the 2015 plenary meeting of IFIAR, which indicated that
interest in AQIs had continued to grow; a month earlier there had been extensive
discussion of AQIs at the IFIAR 9th Inspection Workshop in London.
The Board has also reviewed contemporaneous efforts to raise audit quality by
increasing audit transparency. In 2008 and 2009, the U.K.'s FRC, and the International
Auditing and Assurance Standards Board (the "IAASB"), began their own efforts to
understand and describe audit quality indicators. The FRC issued a February 2008
"audit quality framework (emphasizing firm culture, auditor skills and personal qualities,
and the strength of the audit process)," in the hope of "support[ing] effective
communication between auditors, audit committees, preparers, investors, and other
stakeholders on audit quality." 42 The IAASB issued "A Framework for Audit Quality
Key Elements that Create an Environment for Audit Quality," on February 18, 2014.
That document lists factors that can contribute to audit quality at the engagement, firm,
and national levels for financial statement audits; a number of the factors are reflected
in the AQIs discussed in this release, and the IAASB framework is based on an "input,
process, and output" model similar to the potential AQI framework discussed in this
release. But the IAASB framework does not attempt to create a system for quantifying
particular elements of audit quality. 43
42

Financial Reporting Council, The Audit Quality Framework (2008) 1,


available at https://www.frc.org.uk/Our-Work/Publications/FRC-Board/The-Audit-QualityFramework-(1).aspx.
43

International Auditing and Assurance Standards Board, A Framework for


Audit Quality, Key Elements That Create An Environment For Audit Quality (2014),
available at http://www.ifac.org/publications-resources/framework-audit-quality-keyelements-create-environment-audit-quality.

PCAOB Release No. 2015-005


July 1, 2015
Page 33
The European Union (2006), the Technical Committee of the International
Organization of Securities Commissions (2009), and the Basel Committee on Banking
Supervision (2008 and 2014), have also addressed aspects of the need for audit firm
transparency and the link between transparency and audit quality. 44 In addition, as
indicated above, independent audit regulators around the world are interested in the
possible use of AQIs and in making information about audit quality available to
interested parties. For example, the Swiss Federal Audit Oversight Authority publishes
certain aggregated AQIs for the country's five largest firms. 45 In the United Kingdom, the
FRC discusses its inspection reports, which include a numerical score, with the relevant
audit committees; the FRC is now considering the extent to which the content of
inspection findings should be made public by audit committees and is allowing audit
committees of FTSE 350 companies that wish to do so to make certain inspection
information public in the period before a final decision about the terms for publicity is
made. 46 The Australian Securities & Investments Commission has issued rules to
govern voluntary disclosure of AQIs, in part to assure comparability. 47

44

See Council Directive 2006/43/EC, Article 40, now implemented in


national regulations, Technical Committee of the International Organization of Securities
Commissions, Transparency of Firms that Audit Public Companies, Consultation Report
(2009), available at http://www.iosco.org/library/pubdocs/pdf/IOSCOPD302.pdf, Basel
Committee on Banking Supervision, External audit quality and banking supervision,
supra note 24, and External audits of banks (Bank for International Settlements, 2014),
available at http://www.bis.org/publ/bcbs280.htm.
45

Federal Audit Oversight Authority FAOA, Activity Report 2013 (2014), 2425, available at http://www.revisionsaufsichtsbehoerde.ch/bausteine.net/file/showfile.
aspx?downdaid=7814.
46

U.K. Financial Reporting Council, FRC Statement PN 69/14:


Transparency of AQR Findings (2014), available at https://www.frc.org.uk/News-andEvents/FRC-Press/Press/2014/November/Transparency-of-AQR-Findings.aspx.
The
Council's statement raises many of the issues discussed above, for example in
discussing the limits on grading of inspections.
47

Australian Securities & Investments Commission, Information Sheet 184


Audit transparency reports, Table 2, "Internal Indicators of Audit Quality," (2013) 3,
available at http://www.asic.gov.au/regulatory-resources/financial-reporting-and-audit/
auditors/audit-transparency-reports/.

PCAOB Release No. 2015-005


July 1, 2015
Page 34
Finally, over the last few years there has been increased focus on AQIs among
audit firms themselves, for internal management and public outreach purposes. One
firm published a 2013 Audit Quality Report that contains data on nine "transparency
data points" that resemble or in some cases are the same as the AQIs discussed in
Appendix A; the number grew to 12 in the firm's 2014 Report. A second firm is
beginning on the same path. The CAQ published its own discussion of audit quality
indicators in April 2014; the discussion, entitled "CAQ Approach to Audit Quality
Indicators," 48 focused on possible disclosures, for internal discussions, in ten subject
areas involving: audit firm leadership, characteristics of engagement teams, monitoring
by firms of audit engagement performance and quality control, and reliability of audit
reports; again the CAQ's proposals overlap with some of the AQIs suggested by the
Board. Finally, and ultimately more important, as noted above, some firms have turned
to data and concepts similar to those described in this release in working with the Board
on quality control and remediation.
VI.

Opportunity for Public Comment

The Board will seek comment for a 90-day period. Interested persons are
encouraged to submit their views to the Board. Written comments should be sent to the
Office of the Secretary, Public Company Accounting Oversight Board, 1666 K Street,
N.W., Washington, D.C. 20006-2803. Comments may also be submitted by e-mail to
comments@pcaobus.org or through the Board's website at www.pcaobus.org. All
comments should refer to PCAOB Rulemaking Docket Matter No. 041 in the subject or
reference line. Comments should be received no later than 5:00 p.m., Eastern
Standard Time, on September 29, 2015. The Board will consider all comments
received.
The Board will also convene a roundtable meeting in Washington, D.C., during
the fourth quarter of 2015, about this Concept Release. Additional details about the
roundtable will be announced at a later date.

48

Center for Audit Quality, CAQ Approach to Audit Quality Indicators (2014),
supra note 36.

PCAOB Release No. 2015-005


July 1, 2015
Page 35
On the 1st day of July, in the year 2015, the foregoing was, in accordance with
the bylaws of the Public Company Accounting Oversight Board,

ADOPTED BY THE BOARD

/s/ Phoebe W. Brown


Phoebe W. Brown
Secretary
July 1, 2015

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-1
APPENDIX A: POTENTIAL AUDIT QUALITY INDICATORS
The description of each potential audit quality indicator ("AQI") that appears
below has three parts. The first explains the nature of the indicator. The second
illustrates how the indicator could be calculated at the audit engagement and audit firm
levels. The third discusses briefly the reason the indicator has been suggested, and, in
some cases, issues the indicator may raise.
The illustrative calculations offered below for defining each indicator are only one
approach to doing so; the Board expects that other approaches may also be
appropriate. In addition, as discussed in the concept release itself, all of the indicators
are only as good as the context in which they operate; a measurement that is a valuable
indicator in one situation may, because of specific facts, be of little significance in
another. The description of the indicators has been kept as general as possible, in
keeping with the purpose of a concept release. Thus, for example, technical definitions
of various classes of firm personnel would likely be refined as part of a continuing AQI
project. The same is true of measures outside of firm-wide (for example measures by
industry, by region, or by office). The goal now is to outline the general concepts
around which each indicator operates.
The indicators focus on data for a 12-month period, unless the description
specifically indicates otherwise.
The concept release, of which this appendix is a part, contains general questions
about AQIs and their potential uses. This appendix contains a second set of questions
focused more specifically on the language of the 28 potential indicators themselves and
the manner in which they could operate. As before, readers of the release are
encouraged to comment on any matters not covered by specific questions. They are
especially encouraged to provide suggestions for other approaches to particular issues
relating to operation of the indicators.
Question 40. How might the description of each indicator and the
illustrative calculation be improved or replaced by other approaches that
would be more effective or easier to use?
Question 41. To what extent should the description of one or more
indicators and its illustrative calculation be revised to make clear that all
indicators are evaluated in context?
Question 42. To what extent could any suggested indicators produce
uninformative results either because of the context in which they operate
or because the variables they involve can be managed for results that
emphasized form over substance?

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-2
Question 43. How should the indicators be applied at the firm level? Are
different "firm" perspectives (firm-wide, region, office, industry practice)
appropriate for different indicators? Is firm-wide data always appropriate
for those indicators that call for firm-level data?
AUDIT PROFESSIONALS
Availability
1.
Staffing Leverage. The "staffing leverage" indicator measures the time of
experienced senior personnel relative to the volume of audit work they oversee.
Illustrative Calculations: 1
Engagement Level 2

Firm Level

a. Ratio of audit partners' chargeable a. Ratio of firm audit partners' chargeable


hours for the engagement to
hours to chargeable hours of all other
chargeable hours of all other
engagement personnel
engagement personnel
b. Ratio of firm audit partners' chargeable
b. Ratio of audit partners' chargeable
hours to firm audit managers'
hours for the engagement to
chargeable hours
chargeable hours of audit managers'
on the engagement
c. Ratio of firm audit managers'
chargeable hours to chargeable hours
c. Ratio of audit managers' chargeable
of all staff below manager
hours for the engagement to
chargeable hours of all staff below
manager on the engagement
Partners and managers are responsible for oversight of the audit and the audit
team, which will include less experienced staff. Sufficient time to oversee the work of
the audit staff is typically critical to quality. The lower the amount of partners' time per
audit managers and audit staff time, the wider the scope of partners' and managers'
supervision and review responsibilities, and the greater the risk that partners and
1

Hours spent by partners as engagement quality reviewers would not be


counted for purposes of these calculations.
2

The definition of "engagement level" in the case of global audits is an


open question on which comment has been specifically requested. See Concept
Release, Question 14 a), at page 16.

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-3
managers may not have sufficient time to supervise and review staff work and evaluate
audit judgments. Less extensive supervision raises the risk of less effective audit
procedures and a reduction in audit quality.
Question 44. Would addition of a calculation of staffing leverage
indicators that measures the ratio of partner and manager hours to total
audit hours be helpful?
2.
Partner Workload. The "partner workload" indicator generates data about the
level of work for which the audit engagement partner is responsible and the number of
claims on his or her attention.
Illustrative Calculations:
Engagement Level
a. Chargeable hours managed by audit a.
engagement partner for all public and
private clients for the current year
(planned) and prior year (actual)
b. Number of public clients, and number
of private clients, whose audits are b.
managed by the audit engagement
partner and audits for which that
partner is a quality control reviewer,
noting those with calendar year-ends,
for the current year (planned) and
prior year (actual)

Firm Level
Average chargeable hours managed
by public company audit engagement
partners for all public and private
clients for the current year (planned)
and prior year (actual)
Public company audit engagement
partners'
average
utilization
percentage for the current year
(planned) and prior year (actual)

c. Audit
engagement
partner's
utilization percentage 3 for the current
year (planned) and prior year (actual)
Heavy workloads could distract an engagement partner from giving adequate
and focused attention to an audit engagement. The figures generated by this indicator
can help bring that issue to light and aid understanding of the implications of division of
a partner's attention among several audit clients and competing client deadlines. (In
addition, the data can provide perspective on the leverage calculation if it shows, for
3

The term "utilization," used in this Appendix A, means a fraction whose


numerator is the number of all chargeable hours (i.e. for both public and private clients)
in a year and whose denominator is the number of working hours in a year.

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-4
example, that senior personnel in fact devote large amounts of time to an audit with high
staffing leverage or relatively low amounts of time to an audit with a low staffing
leverage.) However, workload figures may have different meanings. For example,
partners may supervise relatively few audit engagements because their time is devoted
to firm management or other firm leadership positions.
3.
Manager and Staff Workload. This indicator would provide information about
the workload of audit managers and audit staff.
Illustrative Calculation:
Engagement Level
For managers
respectively:

and

audit

Firm Level
staff, a.

Manager and audit staff average


utilization, respectively

a.

Utilization percentage for the b.


current year (planned) and prior
year (actual)

Manager and audit staff average


chargeable hours at different periods
within the year (e.g., during periods of
peak audit workload)

b.

Average hours worked per week


measured from day after clients'
year-end date through audit
opinion date, for all engagements,
by personnel level

The greater the workload, the greater the risk staff may have insufficient time to
perform appropriately the necessary audit procedures and take the other steps that
create a quality audit. Staff may become less effective when working long hours, and
such an environment may affect the level of due professional care they exercise. For
example, a heavy workload may create pressure on the staff to focus more on efficiency
in executing auditing procedures than on ensuring the effectiveness of those
procedures and of supervision of more junior engagement team members.
In applying this indicator, measurements may be made on a person-by-person
basis in the case of relatively small audit firms but on an average basis for larger firms.

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-5
4.
Technical Accounting and Auditing Resources. This indicator measures the
level of a firm's central personnel (or other resources engaged by the firm) available to
provide engagement teams with advice on complex, unusual, or unfamiliar issues and
the extent to which they are used in a particular engagement. 4
Illustrative Calculations:
Engagement Level

Firm Level

a. Technical resource chargeable


hours as a percentage of total
engagement hours

a. Size of a firm's "National Office" or other


technical audit resources as a percentage
of its total audit personnel, using a "fulltime -equivalent" measurement to account
for individuals who spend only part of their
time on technical resource matters

An audit firm's technical accounting and auditing (e.g., national office) resources
(or their equivalent) can enable it to deal with complex questions during an audit. The
measurement in this indicator provides a sense of a firm's capacity to resolve complex
accounting and auditing issues in an effective way. It may also provide a sense of
whether and how a firm promotes consultation and collaboration with others for the
benefit of audit quality.
Question 45. How should technical accounting and auditing resources be
measured in a situation in which those resources are retained from
outside the firm conducting the audit?
5.
Persons with Specialized Skill or Knowledge. This indicator measures the
use in an audit engagement of persons with "specialized skill and knowledge," other
than accounting and auditing personnel counted as technical accounting and auditing
resources under indicator 4. These individuals may be firm personnel or they may be
retained by the firm. 5

Governing professional standards require that a firms policies provide


reasonable assurance that engagement teams can consult with individuals outside the
engagement when appropriate. See Quality Control Standard 20.19. In most larger
firms the individuals are part of a "national office" staff, but they may be retained from
outside the firm especially in the case of smaller audit firms. Cf. Paragraph 15 of
Auditing Standard No. 16.
5

See Paragraphs 16 and 17 of Auditing Standard No. 9.

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-6
Illustrative Calculations:
Engagement Level

Firm Level

a. Chargeable hours by persons with a. Chargeable hours of persons with


specialized skill or knowledge (except
specialized skill or knowledge
as counted as technical resources
(except as counted as technical
under indicator 4), in total and by
resources under indicator 4) in total
functional specialty, as a percentage
and by functional specialty, as a
of an engagement's current year
percentage of a firm's actual
(planned) and prior year's (actual) total
chargeable hours
chargeable hours
An audit firm's capacity to provide valuation, actuarial, forensic, tax, technology,
financial instrument, legal, and other experts is increasingly necessary to plan and
perform complex audits. Recognition of the need to use experts, and their careful
deployment, can show a commitment to robust auditing in those industries in which their
use is beneficial.
In light of the different ways firms obtain the assistance of persons with
specialized skills or knowledge, the application of this indicator should measure effort,
not simply hours expended or revenue generated, to take account of the importance of
the role experts can play, but also, for example, the situation of experts who are
retained on a fixed-fee basis. Thus, appropriate measures could in some cases involve
hours, but in other cases reflect the percentage of total fees paid to experts.
Question 46. How should this indicator count participation by audit firm
personnel and managers who have dual skills (i.e., as accountants or
auditors but also as experts in another relevant discipline)?
Competence
6.
Experience of Audit Personnel.
This indicator measures the level of
experience of members of a particular engagement team and the weighted average
experience of firm personnel generally.

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-7
Illustrative Calculations:
Engagement Level

Firm Level

For partners, managers, staff auditors, a. Average experience for total


audit personnel
specialists, and engagement quality reviewers: 6
b. Weighted average years of
experience
for
partners,
managers,
staff
auditors,
and
b. Number of years in present assignment and
specialists respectively
personnel level

a. Number of years on the engagement

c. Number of years: (i) with the firm and (ii) in


the auditing profession
Auditors with relevant experience, both in general and with a particular client,
may be able to approach the audit in a more knowledgeable and effective manner. But
auditors who spend too much time on a particular team may begin to lose their capacity
for skepticism through simple familiarity. Thus, there may be a need to retain a balance
between preserving the benefit of an audit team's experience with a particular client and
adding new auditors who may provide a fresh look at audit issues. Evaluation of data
produced by this indicator should take that balance into account.
7.
Industry Expertise of Audit Personnel. This indicator addresses the
experience of senior members of the audit team, as well as specialists, in the industry in
which the audited company operates.
Illustrative Calculations:
Engagement Level

Firm Level

a. Number of years of cumulative experience of None


partners, audit managers, specialists, and
engagement quality reviewers, respectively, in the
audited public company's industry
Experience with a particular industry helps an auditor understand the industry's
operating practices, the critical audit and accounting issues confronting companies in
6

For purposes of the AQIs in Appendix A, the term "specialist" means


persons with specialized skill and knowledge, described in Indicator 5, who are part of
the engagement team. Thus, the term excludes technical accounting and auditing
resources discussed in Indicator 4.

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-8
that industry, and the best ways to resolve those issues to further audit quality. This
computation indicates the extent to which the firm has encouraged the development of
industry specialization and assigns partners, managers, specialists, and engagement
quality reviewers to audit engagements based on that specialization.
Question 47.
In measuring experience, would overall experience
(including auditing and accounting experience) in the relevant industry be
the best measure? Would such a measure disadvantage smaller firms?
Would a measure based on number of audits performed in a particular
industry be a better indicator for smaller firms?
Question 48. Are there ways to measure the industry expertise of a firm's
audit staff against its public company client base? How?
Question 49. Would adoption of the commonly-used Standard Industrial
Classification ("SIC") codes issued by the U.S. Department of Labor be
appropriate to define industries for purposes on the indicators? 7
8.
Turnover of Audit Personnel. This indicator measures turnover, that is,
transfers to other engagements or movement to other firms, at the engagement and,
more generally, at the firm, level.
Illustrative Calculations:
Engagement Level

Firm Level

a. Percentage of prior year's partners, a. Percentage of partners, managers,


managers, audit staff, specialists,
audit
staff,
and
specialists,
and engagement quality reviewers,
respectively, that have left the firm or
respectively, that have left the firm
left the firm's audit practice, in the
or been reassigned to another audit
preceding 12 months
engagement within the firm 8
The degree and nature of the changes in a company's audit team from year to
year help measure the readiness and ability of the team to perform a quality audit.
Some level of attrition is expected within audit firms. But a comparatively high rate of
turnover or auditor transfer within a firm may adversely affect audit quality.
7

The SIC code structure can be found at https://www.osha.gov/pls/imis/sic


_manual.html
8

The calculation should note separately the number of audit partners


whose rotation was required by Regulation S-X Rule 2-01(c)(6).

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-9
As noted in the discussion of indicator 6 (experience of audit personnel), the
benefit of retaining an audit team's experience with a particular client needs to be
carefully balanced with the benefit of adding new auditors who may provide a fresh look
at audit issues. Evaluation of data produced by this indicator in particular cases, and
issues of context and comparability, should take that balance into account.
Question 50. Should a distinction be made between partner retirements
and other turnover in applying this indicator?
9.
Amount of Audit Work Centralized at Service Centers. This indicator
measures the degree to which audit work is centralized by the audit firm at service
centers. 9
Illustrative Calculations:
Engagement Level

Firm Level

a. Percentage of total engagement audit


work (by chargeable hours) whose
performance is carried out on a
centralized basis at service centers

a. Percentage of audit work (by


chargeable
hours)
whose
performance is carried out on a
centralized basis at service centers

The degree to which work on the audit is carried out at service centers can be an
important element in audit quality. Centralizing audit work at service centers here and
abroad widens the geographic scope of the audit partner's supervision and review
responsibility.
Centralization of this sort can have a positive impact. It may concentrate
processing of audit work in the hands of people skilled in that processing and, by doing
so, may free experienced engagement personnel to focus on more complex or
judgmental areas of the audit.
On the other hand, centralization may introduce risks of insufficient or ineffective
communication among engagement team members and create obstacles to effective
supervision and staffing, for junior auditors especially. Centralization can also limit
junior auditors' exposure to basic audit processes and could reduce the ability of firms to
train their staffs about those processes, weakening the model under which staff auditor
training has historically taken place.

Centralizing audit work means assigning lower risk audit work to domestic
or foreign service centers established by the firm conducting the audit.

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-10
Despite the debate about centralization, however, the indicator can provide an
understanding of the extent to which audit tasks are or may be centralized in service
centers.
10.
Training Hours per Audit Professional. This indicator focuses on the hours of
relevant trainingincluding industry-specific trainingthat members of the engagement
team, and of the team's firm, have received.
Illustrative Calculations:
Engagement Level

Firm Level

a. Annual accounting and auditing a. Average annual accounting and


training hours, and industryauditing training hours, and industryspecific training hours for partners,
specific training hours, in total and for
audit managers, staff auditors,
partners, managers, staff auditors,
specialists,
and
engagement
specialists, and engagement quality
quality reviewers, respectively
reviewers, respectively
b. Total independence and ethics b. Average independence and ethics
training hours for personnel groups
training hours for personnel groups in
in "a"
"a"
The amount of continuing training auditors receive should increase their capacity
to perform effective audits. While the number of training hours is easily measured, the
quality of training is harder to gauge; the training must be relevant to raising audit
quality by, for example, focusing in some cases on back-to-basics audit issues that
have constituted inspection deficiencies and in other cases on more sophisticated
accounting and auditing issues specific to the industry in which audit clients of those
being trained operate. Independence and ethics issues are an important part of the
training curriculum as well, and the evolution and evaluation of training in that area
might also be followed.
The application of this indicator may depend on the size of the audit involved.
Thus per-person calculations may be appropriate for smaller audits, while averages
might be used, by class, for larger audit teams.
Question 51. Should training hours be computed on a per-person basis,
by personnel class, or as an average by class? Should the size of the firm
involved make a difference in this regard?
Question 52. How can the effectiveness of a firm's training program best
be measured?

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-11
Question 53. Should the effect of the way training is delivered (e.g., live,
web-based, or self-study) be factored into the evaluation of a firm's
training program? How?
Focus
11.
Audit Hours and Risk Areas. This indicator measures the time spent by
members of the audit team at all levels on risk areas identified by the firm during audit
planning.
Illustrative Calculations:
Engagement Level

Firm Level

a. Total
chargeable
hours,
and a. For audits by industry, computed
percentage of hours, by significant risk
separately, average chargeable hours
10
area for partners, managers, audit
overall and by significant risk area for
staff,
technical
accounting
and
partners, managers, audit staff,
auditing
resource
personnel,
technical accounting and auditing
specialists, and the engagement
resource personnel, specialists, and
quality reviewer, respectively, for the
the engagement quality reviewers,
current year (planned) and the prior
respectively, for the prior year (actual)
year (actual)
Measuring the hours that levels of an audit team devote to risk areas can
suggest whether audit managers have staffed the audit appropriately to reflect the risk
areas identified during the planning phase of the audit and the extent to which senior
members of the team have focused sufficiently on those areas. The data produced by
this and the following indicator should complement the picture of senior-level attention
to audit supervision created by indicators 1 through 3.
Measuring hours that other engagement teams from the same audit firm devote
to risk areas in audits of other public companies in the same industry as the
engagement client may provide grounds for a greater understanding of the nature of the
risks the firm identifies in audits of that industry, how it staffs to deal with them
generally, and the degree of focus its audit teams give them. This is another situation in
which context may be very important: different companies in the same industry may
present different risks due to, for example, systems, people, or process issues at one or
more of those companies.

10

See Paragraphs 70-71 of Auditing Standard No. 12 and Paragraph 9 of


Auditing Standard No. 16.

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-12
12.
Allocation of Audit Hours to Phases of the Audit. This indicator measures
the effort and staffing the audit devotes to audit planning, interim field work, and audit
completion.
Illustrative Calculations
Engagement Level

Firm Level

a. Current year's (planned) and prior a. Percentage of hours of the firm devoted
year's (actual) total chargeable hours
respectively to planning, quarterly
for each related audit phase (i.e.,
reviews, interim field work, final field
planning, quarterly reviews, interim
work up until report release date, and
post-field
field work, final field work up until report
work
until
audit
release date, and post-field work until
documentation completion date for
audit documentation completion date)
partners, managers, the audit staff,
for partners, managers, the audit staff,
technical resources staff, specialists,
technical resources staff, specialists,
and engagement quality reviewers
and the engagement quality reviewer,
respectively
Audit quality depends in part on proper planning and execution and on the way
the overall audit hours are phased to construct a successful process. The amount of
time allocated to planning for the audit can be critical, and it can be equally important
who participates in the planning process. The same is true with the other stages of the
process.
Question 54. Does the "percentage of hours" metric at the firm level of
this indicator provide a meaningful basis for comparison with the
engagement level of the metric? Would it help to disaggregate the
numbers by audit client size?
Question 55. Is there any way to expand this indicator to quantify audit
personnel experience with audit clients, to provide additional context?
AUDIT PROCESS
Tone at the Top and Leadership
13.
Results of Independent Survey of Firm Personnel. This indicator measures
an audit firm's "tone at the top" through use of a survey tool.

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-13
Illustrative Calculations:
Engagement Level

Firm Level
a. Anonymous independent 11 surveys of current and
former firm personnel about "tone at the top," quality
of supervision and training, and the extent to which
the firm promotes an environment that favors
speaking up about potential issues, and promotes
and rewards professional skepticism

None

An appropriate "tone at the top" and the way the firm communicates and stands
behind that tone is generally essential to foster professional skepticism, objectivity, and
independence in the firm's personnel. Data from anonymous independent surveys of
audit firm personnel could provide unique insights about staff perceptions of their firm's
commitment to critical elements of quality. 12
Question 56. Who should administer the survey described in this
indicator? What steps would be necessary to assure that the results of
anonymous surveys were comparable? Would the same set of questions
be necessary? Would the same individual or organization have to
administer each of the surveys?
Question 57. How often would a survey of this type have to be
administered to retain its validity?
Question 58. What other logistical issues might arise from a survey of
this sort?
Incentives
14.
Quality Ratings and Compensation. This indicator measures the potential
correlation between high quality ratings and compensation increases and the
comparative relationship between low quality ratings and compensation increases or
decreases.

11

An independent survey would require independently-determined


methodology, independently-drafted questions, and anonymity.
12

requested.

Comments on other ways to measure "tone at the top" are also specifically

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-14
Illustrative Calculations:
Engagement Level
None

Firm Level
a. Percentage of partners and managers, respectively,
with exceptional performance ratings on audit quality
b. Percentage of partners and managers, respectively,
with exceptional quality ratings who receive aboveaverage increases in compensation
c. Percentage of partners and managers, respectively
with low quality ratings
d. Average percentage compensation increase or
decrease for partners and managers, respectively, with
low quality ratings

Comparing internal firm quality ratings and compensation levels can provide an
important signal of the value a firm places on quality. This indicator would capture the
extent to which a firm's personnel evaluation process distinguishes among personnel at
each level of the firm based on audit quality and awards compensation accordingly.
Linking compensation to quality may provide strong evidence of the firm's commitment
to that goal and may create equally strong incentives for audit personnel.
Question 59. Can this indicator be applied to produce comparability
among firms, e.g. in terms of definitions of "exceptional performance
ratings" and "low quality ratings"? How?
15.
Audit Fees, Effort, and Client Risk. This indicator provides insight into the
relationship between engagement or firm audit fees and hours, on the one hand, and
levels of client risk, on the other.
Illustrative Calculations:
Engagement Level

Firm Level

a. Percentage change from prior year a. Percentage change from prior year in
in each of: (i) audit fees and (ii)
each of: (i) total audit revenues charged
chargeable hours for partners and
to public company clients and (ii)
managers, respectively, together
chargeable hours for partners and
with whether client was identified
managers, respectively, together with
by firm as high risk.
percentage of firm's public company
clients assessed as high risk.

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-15
The combination of falling audit fees and decreased audit effort can lead to
reduced audit quality. That combination can be especially serious for a high risk public
company audit client. This indicator attempts to highlight situations in which economic
pressures can create such a relationship. It may provide perspectives on: (i) how audit
fees vary with audit risk (e.g., to reflect a risk premium or higher audit effort) and (ii)
whether work is reallocated from senior to more junior auditors as fees change.
Question 60. One issue that this indicator raises is how to fashion a
workable definition of "high risk" that allows comparability among firms or
even among engagements within a firm. Comment is specifically
requested on that subject.
Independence
16.
Compliance with Independence Requirements. This indicator measures
several elements of a firm's independence training and monitoring program and the
importance it assigns to that program.
Illustrative Calculations:
Engagement Level

Firm Level

a. Percentage
of a. Percentage of firm personnel subject to firm's
engagement personnel
personal independence compliance reviews
subject
to
firm's
annually.
personal independence
compliance reviews.
b. Average of mandatory independence training
hours per audit employee and other firm
b. Average of mandatory
professional employees covered by Commission
independence training
independence rules (whether or not involved in the
hours per engagement
firm's audit practice), respectively
team member
c. Percentage of issuer audit engagements subject to
firm internal quality control reviews over
independence compliance annually.
d. Level of investment in centralized support for, and
monitoring of compliance with, independence
requirements per 100 public company audit clients
(for firms with 500 such clients)
e. Percentage of public company audit clients lost
due to independence violations.

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-16
Auditor objectivity is a critical precondition for audit quality, and several indicators
indirectly address that condition. The survey of firm personnelindicator 13 (Process)
is an example.
Other independence issues are equally important, including
arrangements involving mutuality of interest between the auditor and client, an auditor's
performing as a member of a client company's management, and an auditor's providing
prohibited nonaudit services.
Investment in training, a firm's independence function generally, and internal
compliance reviews can all illustrate a firm's commitment to objectivity and to producing
quality audits. The level of investment called for by this indicator may need to take
account of both personnel costs and the cost of technology.
Question 61. What other measures of independence, or independence
issues, would be appropriate? Would information generated by this
indicator be more meaningful if measurements were stratified by
personnel level?
Infrastructure
17.
Investment in Infrastructure Supporting Quality Auditing. This indicator
measures the amounts audit firms invest, in people, process, and technology, to support
the base on which quality auditing depends.
Illustrative Calculations:
Engagement Level

Firm Level

a. Investment in engagement team as a a. Investment in audit practice as a


percentage of revenue generated on
percentage of firm revenue
engagement
A firm's investment in auditing practice infrastructure (that is, in its people,
processes, and technology) can demonstrate its commitment to audit quality. Defining
the expenditures that represent such investment, however, is difficult. Firm-sponsored
advanced training in auditing and accounting certainly qualifies, but not all training is
consistent with that focus. Investment in audit technology can produce better audits,
but it is sometimes designed to streamline procedures to improve efficiency in a way
that does not improve audit effectiveness.
Question 62. In what ways can investments in infrastructure that are
relevant to improving audit quality best be defined?

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-17
Question 63.
How should such investments be measured?
Is
measurement in dollar terms (or dollars per auditor) appropriate? Can
such investments be measured at the engagement team level?
Monitoring and Remediation
18.
Audit Firms' Internal Quality Review Results. 13 This Indicator contains
information about the internal quality reviews conducted by each audit firm.
Illustrative Calculations:
Engagement Level

Firm Level

a. Results of any internal a. Percentage of public company audits subjected to


quality inspections of
internal quality review inspections by audit firm
audits of the engagement
client, including number b. Percentage of such inspections with one audit
(if
any)
of
audit
deficiency of a magnitude similar to a PCAOB Part
deficiencies
of
a
I finding
magnitude similar to a
PCAOB Part I finding
c. Percentage of such inspections with more than
one such audit deficiency
An audit firm's internal quality review program can show the level of attention the
firm pays to monitoring and improving quality in its audit practice. Analysis of audit
quality review findings requires care, however. Although a higher volume of findings
may raise questions about aspects of a firm's operations, a record of timely identification
and appropriate remediation could indicate a robust approach to internal quality review
that signals a firm's commitment to improved quality. In addition, a comparison of firm
and PCAOB findings may cast light on strengths and weaknesses of quality control
efforts more generally.
Question 64.
How should internal quality inspection findings be
compared to or analyzed alongside PCAOB inspection results in applying
indicators 18 and 19?

13

PCAOB inspection results and audit firm internal quality review results are
counted as audit process indicators, but they possess elements of both audit process
and audit results indicators.

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-18
19.
PCAOB Inspection Results. 14 This indicator contains information about
PCAOB inspection results relating to the engagement or the audit firm involved.
Illustrative Calculations:
Engagement Level

Firm Level

a. Results
of
any
PCAOB a. Number and percentage of PCAOBinspections of audits of the
inspected audits that result in a "Part I
engagement issuer as well as
finding"
the number and nature of any
Part I findings identified
b. Number and percentage of PCAOB
inspected audits that result in more than
one "Part I finding"
c. Number and percentage of PCAOB
inspected audits that led to a restatement
d. Number, nature, and dates of quality
control defects dealt with in released Part
II PCAOB inspection reports (and dates of
such releases), if any, combined with
information about firm's subsequent
remediation efforts
The Board's own inspections focus on whether audits are conducted in
accordance with the Board's rules and standards. They can provide insight, in their Part
I findings (and any quality control defects, described in Part II of an inspection report,
that becomes available if adequate remediation by firms with quality control defects
does not occur), about breakdowns that may cause audit deficiencies. Public inspection
findings may provide a baseline for evaluating other indicators (e.g., comparing staff
utilization rates, or use of persons with specialized skill and knowledge, with inspection
findings) and testing the efficacy of firms' internal quality control systems.
20.
Technical Competency Testing. This indicator seeks to measure the level of
technical competence of a firm's audit personnel, and the success of efforts to keep up
that level of competence.

14

See Information for Audit Committees about the PCAOB Inspection


Process, PCAOB Release 2012-003 (August 1, 2012).

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-19
Illustrative Calculations:
Engagement Level
a. Content requires study

Firm Level
a. Content requires study

Audit firms recognize that technical competence is critical for maintaining quality
in a rapidly changing business and financial environment. But there is at present no
recertification examination for auditors, as there is, for example, for medical specialties
in some states and for most members of the securities industry. State boards of public
accountancy typically impose continuing education requirements but do not require
retesting.
Comment is specifically requested on ways audit firms might measure technical
competence, encourage its development and maintenance, and report on the result.
AUDIT RESULTS
Financial Statements
21.
Frequency and Impact of Financial Statement Restatements for Errors. This
indicator measures the restatements for error of financial statements whose audit the
audit firm has performed.
Illustrative Calculations:
Engagement Level

Firm Level

a. Number and magnitude of audit a. Number and percentage (of audited


practice's restatements for errors
financial statements) of an audit practice's
at engagement level, computed
restatements
for
errors,
computed
annually.
annually, and magnitude of those
restatements.
b. The audit firm's top five annual
restatements measured by the magnitude
of those restatements.
The number and impact of restatements for errors (i.e., not for changes in
accounting principles) are generally considered a signal criterion of potential difficulties
in at least parts of an auditor's practice and approach to auditing. This indicator tries
to place restatements in context by focusing on their magnitude. Magnitude of
restatements could be measured in a number of ways, including impact of the
restatement on income, on cash flows and balance sheet, and on market capitalization.

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-20
Question 65. What are the best methods for measuring magnitude of
restatements for errors? Is one method superior to the others? Why?
22.
Fraud and other Financial Reporting Misconduct. This indicator is concerned
with reporting of fraud and other financial misconduct, at both the engagement and audit
firm levels.
Illustrative Calculations:
Engagement Level
a. Content requires study

Firm Level
a. Content requires study

Given the historical harm to investors from fraudulent financial reporting and
auditors' responsibilities to help prevent or detect fraud that materially affects financial
statements, 15 one or more AQIs on the auditor's work in the fraud area may be useful.
Although the content of specific AQIs requires further study, ideas include:
Positive indicators:

15

1.

Number of significant deficiencies or material weaknesses in controls


designed to address the risk of material misstatement due to fraud,
raised by the audit firm in the absence of an error or fraud that has
already occurred

2.

Number and severity of material or immaterial errors in financial


reporting from fraud or other financial reporting misconduct discovered
by the audit firm early enough to avoid errors in published financial
statements

AU Section 110.02 states, "The auditor has a responsibility to plan and


perform the audit to obtain reasonable assurance about whether the financial
statements are free of material misstatement, whether caused by error or fraud.
[footnote omitted]." AU Section 316 establishes requirements and provides direction
relevant to fulfilling that responsibility, as it relates to fraud, in an audit of financial
statements. Auditing Standard No. 5 states when planning and performing the audit of
internal control over financial reporting, the auditor should take into account the results
of his or her fraud risk assessment. "As part of identifying and testing entity-level
controls and selecting other controls to test the auditor should evaluate whether the
company's controls sufficiently address identified risks of material misstatement due to
fraud and controls intended to address the risk of management override of other
controls." See Paragraph .14 of Auditing Standard No. 5.

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-21
Negative indicators:
1.

Number of restatements for errors resulting from fraud or other


financial reporting misconduct with no previously reported material
weakness in internal control

2.

Number and severity of material errors in financial reporting from fraud


or other financial reporting misconduct that the audit firm did not detect
prior to restatements of financial statements

Developing AQIs related to these ideas is challenging. Data needed for the AQIs
may be difficult to obtain as it would require determining whether an internal control
deficiency related to fraud prevention, or determining whether errors in financial
reporting, resulted from fraud or other financial reporting misconduct. Also, the AQIs
listed above could reflect the riskiness of an audit firm's client base rather than the
quality of the audit firm's work.
Question 66. Would one or more AQIs related to fraud and other
financial reporting misconduct be helpful to discussions of audit quality? If
so, what AQIs would best inform those discussions? How could the
challenges listed above be overcome?
23.

Inferring Audit Quality from Measures of Financial Reporting Quality.

This potential indicator focuses on whether (and which) measures of financial


reporting quality used by investment analysts, academics, and regulators can also be
used as measures of audit quality.
Illustrative Calculations:
Engagement Level
a. Content requires study

Firm Level
a. Content requires study

Academics, analysts, and regulators have developed measures of financial


reporting quality. The measures are used by academics in their research, analysts, and
investors for investment decisions and by regulators to identify risky situations to
scrutinize. Examples include measures of the nature and size of accounting accruals,
the extent to which companies consistently and narrowly beat earnings targets, and the
complexity or relevance of financial disclosures.
Using reporting quality measures as AQIs requires answers to two questions: (i)
do the financial reporting measures reliably gauge reporting quality and, if so, (ii) do
they also provide useful inferences about audit quality? If the answers to those

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-22
questions are positive, a related issue is whether the measures provide insight into audit
quality at the audit firm level, at the specific engagement level, or both. For example, if
measures suggest the reporting quality for an audit firm's portfolio of clients differs from
the average for all public companies, does that suggest that the audit firm is performing
stronger or weaker audits? Or, at an engagement level, if measures suggest that the
specific company's financial reporting quality differs from the average, does that suggest
that the engagement team's performance is strong or should improve?
If the measures are reliable, and if users can infer audit quality from them, then
there are at least two ways forward for related AQIs:
1.

Identify specific AQIs related to financial reporting quality (i.e. specific AQI
measures)

2.

Ask auditors to report whether their firm uses measures of financial


reporting quality to measure risk and, if so, what those measures suggest
about the firm's and engagement team's audit quality (i.e. allow the firm
process to determine the AQI)

Question 67. Comment is requested on each of the issues raised about


this indicator. Would it be preferable to identify specific indicators related
to financial reporting quality or to focus on audit firms' measures of
reporting quality to measure risk? How would the latter approach control
for differences among firms?
Internal Control
24
Timely Reporting of Internal Control Weaknesses. This measure captures
the extent to which an audit firm identifies material weaknesses in an issuer's internal
controls over financial reporting on a timely basis.
Illustrative Calculations:
Engagement Level

Firm Level

a. Same as firm level but


concerning
audit
reports
for
the
engagement client

a. Percentage of findings of material weakness in


internal control over financial reporting with no
corresponding: (i) restatements for errors or (ii)
known errors
b. Percentage of: (i) restatements for errors, or (ii)
known errors, with no corresponding material
weakness in internal controls over financial
reporting identified in the prior year

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-23
In the words of Auditing Standard No. 5, and relevant SEC guidance, "effective
internal control over financial reporting provides reasonable assurance regarding the
reliability of financial reporting and the preparation of financial statements for external
purposes." 16 A firm's failures to identify material internal control weaknesses may raise
issues about staffing, training, or audit focus for these important issues.
It is unclear whether such material weaknesses serve as leading indicators (i.e.,
evidence of risks of future reporting flaws), lagging indicators (i.e., evidence that
reporting flaws may already have occurred), or both. Data assembled by the AQI
project can help illuminate that issue.
Going Concern
25.
Timely Reporting of Going Concern Issues. This indicator focuses on the
timeliness of the auditor's use of a going concern paragraph in its opinions.
Illustrative Calculations:
Engagement Level

Firm Level

a. Same as firm level "a," but a. The number and percentage of audit reports
concerning audit reports for the
with no going concern reference in the year
engagement client
preceding an engagement client's financial
distress, e.g., bankruptcy, troubled debt
restructuring, troubled buyout, or bailout
b. The five largest issuers by market
capitalization from the above indicator
Failure to include a going concern paragraph within an audit report in the face of
an issuer's reasonably foreseeable business distress (whether the distress results in
bankruptcy, a technical default, or a troubled buyout or bailout) can indicate issues
about the effectiveness of the auditing involved. This measure captures the extent to
which an audit firm identifies on a timely basis companies whose ability to continue as a
going concern is subject to substantial doubt.
All the same, any indicator focused on going concern issues can raise issues of
context and unintended consequences. Business difficulties are not always reasonably
16

Paragraph 2 of Auditing Standard No. 5 and Exchange Act Rules 13a15(f) and 15d-15(f). Paragraph 2 of Auditing Standard No. 5 provides that "[i]f one or
more material weaknesses exist, the company's internal control over financial reporting
cannot be considered effective," citing to Item 308 of Regulation S-K.

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-24
foreseeable even by the most experienced audit team. And some going concern
warnings are given for companies that do not experience financial distress; an indicator
should not provide an incentive for an auditor to consider giving a going concern
warning where one is not truly called for.
Question 68. How should factors such as difficulties in foreseeing
business difficulties, or the risk of providing an incentive for unnecessary
going concern warnings be reflected in an indicator of this kind?
Communications Between Auditors and Audit Committees
26.
Results of Independent Surveys of Audit Committee Members. This
indicator measures the effectiveness of the communication between auditors and audit
committees through use of a survey tool.
Illustrative Calculations:
Engagement Level

Firm Level
a. Anonymous independent 17 survey of audit
committee members overseeing one or more of a
firm's audit engagements, to evaluate level and
quality of communication between auditors and
clients

None

Communication between auditors and audit committees is at the center of the


audit process. Auditing Standard No. 16, states the "objectives of the auditor" to
communicate with the audit committee about the engagement and the auditor's role,
audit strategy and timing, and "timely observations arising from the audit that are
significant to the financial reporting process." 18
Data from anonymous independent surveys of audit committee members could
provide uniquely valuable information about the way auditors actually interact with audit
committees. Such surveys pose logistical issues about which comments are sought.

17
18

See discussion of independent surveys, infra at note 11.

Paragraph 3.a.-d. of Auditing Standard No. 16. A note following the


quoted language emphasizes the purpose of Auditing Standard No. 16 is "to encourage
effective two-way communication between the auditor and the audit committee
throughout the audit to assist in understanding matters relevant to the audit."

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-25
Question 69. Who should administer the survey described in this
indicator? What steps would be necessary to assure that the results of
anonymous surveys were comparable? Would the same set of questions
be necessary? Would the same individual or organization have to
administer each of the surveys?
Question 70. How often would a survey of this type have to be
administered to retain its validity?
Question 71. What other logistical issues might arise from a survey of
this sort?
Enforcement and Litigation
27.
Trends in PCAOB and SEC Enforcement Proceedings. This indicator
measures Board or SEC proceedings in audit and audit-related matters against an audit
firm.
Illustrative Calculations:
Engagement Level

Firm Level

a. Public, SEC or Board enforcement a. Public, SEC, or Board


enforcement
proceedings, measured over the
proceedings, measured over the preceding
preceding five years, against the
five years, against the firm or its partners,
firm or its partners, with respect to
with respect to audit matters
the engagement client
The frequency, nature, magnitude, and results of Board and SEC litigation in
audit and audit-related matters might help reveal weaknesses or strengths of a firm's
practice. Information generated by this indicator may include quality issues affecting
either particular firms or auditing in general. The length of the period required for
litigation, however, may create problems of timeliness of information.
Question 72. Should tabulation of cases for purposes of this indicator
include all cases filed or only cases that result in findings against an
accountant or accounting firm? What about settlements entered into
without an admission of wrongdoing?
28.
Trends in Private Litigation.
involving the audit firm.

This indicator focuses on private litigation

PCAOB Release No. 2015-005


July 1, 2015
Appendix A
Page A-26
Illustrative Calculations:
Engagement Level

Firm Level

a. Frequency, nature, and results of


private litigation relating to firm's audit
work for the engagement client

a. Frequency, nature, and results of


private litigation relating to firm's
public company audit practice

The frequency, nature, magnitude, and results of private litigation against audit
firms might reveal either weaknesses or strengths of a firm's public company practice.
Again, the information generated by this indicator may include quality issues affecting
either particular firms or auditing in general. But the quality of the information is
uncertain, given the fact that a particular litigation may or may not result in findings of
liability, and the amount of information derived from settled litigation is ambiguous.
Question 73. Should tabulation of cases for purposes of this indicator
include all cases filed or only cases that result in findings against an
accountant or accounting firm? What about settlements?

You might also like