The Cleft PalateCraniofacial Journal 53(2) pp.

187196 March 2016

Copyright 2016 American Cleft PalateCraniofacial Association

ORIGINAL ARTICLE
Evaluation of Facial Appearance in Patients With Cleft Lip and Palate by
Laypeople and Professionals: A Systematic Literature Review
Shiwen Zhu, B.D.S., M.D.S., Jayakumar Jayaraman, B.D.S., Ph.D., M.Paed. R.C.S. (Edin), Balvinder Khambay,
F.D.S. R.C.S. (Eng), M.Orth. R.C.S. (Edin), Ph.D.
Objective: To identify whether laypeople and professionals rate the facial appearance of
patients with cleft lip and palate (CLP) similarly based on viewing full facial images.
Design: Several electronic databases were searched. A hand search was performed among
reference lists and relevant journals. Studies that assessed the full facial appearance of patients
with CLP based on two-dimensional (2D) photographs, 3D images, or clinical examination by
laypeople and professionals using a visual analog scale (VAS) or a categorical rating scale were
included. Two authors independently assessed articles using methodologic-quality scoring
protocol.
Results: Eleven articles were included in qualitative synthesis, including four high-level and
seven moderate-level papers. Three studies found that laypeople were more critical than
professionals, three found there was no significant difference between laypeople and
professionals, and five reported that professionals were more critical than laypeople when
assessing facial appearance of patients with CLP.
Conclusions: It still remains unknown whether laypeople are more or less critical than
professionals when rating facial appearance of patients with repaired CLP. Professionals are
more familiar with the esthetic outcomes and difficulties of treating patients. The opposite maybe
true for laypeople; this disparity between what is achievable by professionals and what is
expected by laypeople may be a source of dissatisfaction in facial appearance outcome. Further
well-designed studies should be carried out to address this question and the clinical significance
of the difference in rating scores for patients with CLP.
KEY WORDS:

cleft lip and palate, facial appearance, laypeople, professionals, systematic review

The rst of these is a primary lip repair, which is carried out

to improve function, speech, and esthetics. Inadequate
correction of facial esthetics, in particular nasolabial
symmetry is thought to be a source of emotional distress
and dissatisfaction among this group of patients (Gkantidis
et al., 2013). This may be the reason for further surgical
revision procedures (Foo et al., 2013). In the present era of
patient-centered interdisciplinary management, in addition
to the patient, professionals and laypeople may also
indirectly affect the decision for further surgery. Even
though the patients opinion is paramount in requesting
further surgery to improve his or her perceived quality of
life (Chung et al., 2013), the opinion and support of
laypeople such as friends and peers have been shown to
have a positive effect on the psychological well-being of the
individual (Moss, 1997; Pedersen et al., 2007).
Facial attractiveness is of critical importance in various
social interactions (Little et al., 2007; Coetzee et al., 2014).
Social interactions of patients primarily depend on
perceptions of the wider community, which mainly consists
of laypeople or peers (Williamson, 1999; Rankin and
Borah, 2003). It has been shown that patients with cleft lip
and palate often experience negative social interactions as a
result of an unattractive facial appearance (Chung et al.,

Cleft lip and palate is the most common congenital

dysmorphology affecting the oral and maxillofacial region
(Stec et al., 2007). The prevalence of cleft lip and palate
varies between countries, socioeconomic status, and
ethnicities (Shaw, 2004). The average worldwide prevalence
of patients with cleft lip with or without cleft palate has
been reported as 7.94 per 10,000 births (Tanaka et al.,
2012). Among the ethnicities, the Chinese have the highest
prevalence of cleft lip and palate, affecting 16.63 per 10,000
births during 1996 to 2005, with an upward trend over time
(Dai et al., 2010).
In order to address the facial dysmorphology of patients
with cleft lip and palate, a series of surgical interventions are
undertaken at specic times during the childs development.

S. Zhu and Dr. Jayaraman are research postgraduate students

and Dr. Khambay is Clinical Associate Professor, Discipline of
Orthodontics and Paediatric Dentistry, Faculty of Dentistry, The
University of Hong Kong, Hong Kong, Hong Kong SAR.
Submitted June 2014; Revised September 2014, November 2014;
Accepted December 2014.
Address correspondence to: Dr. Balvinder Khambay, 2A, Prince
Philip Dental Hospital, 34 Hospital Road, Hong Kong. E-mail
bkhambay@hku.hk.
DOI: 10.1597/14-177
187

188

Cleft PalateCraniofacial Journal, March 2016, Vol. 53 No. 2

2013). Individuals with congenital craniofacial conditions

often experience discrimination in an employment and
social setting related to lower satisfaction with facial
appearance, quality of life, and self-esteem (Sarwer et al.,
1999). This highlights the need to involve laypeople as
potential peers in assessing facial appearance outcome.
Numerous studies have investigated the outcome of
postsurgical facial appearance in patients with cleft lip and
palate based on the views of laypeople and professionals.
Some studies have focused on cropped photographs of the
isolated nasolabial region (Bongaarts et al., 2008; Kim et
al., 2009, 2011; Mani et al., 2010), while others evaluated
the full frontal facial image of patients with cleft lip and
palate (Lo et al., 2002; Papamanou et al., 2012; Chung et
al., 2013; Gkantidis et al., 2013; Offert et al., 2013). The use
of cropped photographs may not be appropriate as they do
not indicate total facial harmony and may therefore be
misleading. The esthetic outcome of the repaired cleft lip
should not be viewed in isolation but should be based on
overall facial appearance as it is in other facial dysmorphologies (i.e., orthognathic patients).
There is agreement that laypeople and professionals rate
the facial appearance of patients with cleft lip and palate
consistently lower than noncleft individuals (Eichenberger
et al., 2014). However, when assessing the facial appearance
of individuals with clefts there are conicting opinions
between laypeople and professionals. Some studies have
reported professionals are more critical (Eliason et al., 1991;
Chung et al., 2013; Foo et al., 2013; Offert et al., 2013),
while others reported that laypeople were more critical
(Papamanou et al., 2012; Gkantidis et al., 2013; Eichenberger et al., 2014).
The aim of this systematic review was, therefore, to
determine whether laypeople and professionals rate the
facial appearance of patients with cleft lip and palate
similarly based on viewing full facial images.
METHODS
Eligibility Criteria
The selected studies compared full facial appearance
of cleft repair as judged by laypeople and professionals.
Studies that met the following criteria were included: (1)
patients with cleft lip with or without cleft palate; (2)
outcome of facial appearance assessed by laypeople and
professionals; (3) outcome measure based on a visual
analog scale (VAS) or a categorical rating scale; (4) use
of two-dimensional (2D) photographs, 3D images, or
real patients; and (5) assessment of full facial images.
Exclusion criteria were as follows: (1) patients without
cleft lip; (2) only one group of assessors; (3) use of video
images; (4) reviews, letter to editors, expert opinions,
and case reports; and (5) partial facial images, i.e.,
cropped or isolated surgical cleft region.

Information Resources
PubMed (all available articles until week 2 of June
2014), The Cochrane Library (1999 to week 2 of June
2014), Web of Science (1956 to week 2 of June 2014),
Scopus (1999 to week 2 of June 2014), and EMBASE
(1980 to week 2 of June 2014) were searched to identify
relevant articles. The reference lists in these relevant
articles were retrieved manually. The articles in Cleft
Palate-Craniofacial Journal (from January 2004 to May
2014) and American Journal of Orthodontics and
Dentofacial Orthopedics (from January 2004 to June
2014) were also manually searched based on their
relevance to facial appearance and the patients with
cleft lip and palate. No language limit was set.
Review Question
The patients, intervention, comparator, and outcome
(PICO) structure was applied to form the research
question. In this systematic review, patients are those
with cleft lip and palate. The comparison will be carried
out between the professionals and laypeople. The
outcome will be the difference in evaluation of facial
appearance by professionals and laypeople. Therefore,
the review question is whether professionals and
laypeople rate the facial appearance of patients with
cleft lip and palate similarly based on viewing full facial
images.
Search Strategy
Search terms were identied to t the research
question, including cleft, laypeople, professionals, assess, face, and appearance. Based on these search terms,
the search strategy was as shown in Table 1. PRISMA
guidelines were used to conduct the present systematic
review (Liberati et al., 2009).
Initially, two authors (S.Z. and B.K.) independently
searched the databases on the basis of title and abstract.
Papers that met the eligibility criteria were included. If
the eligibility of papers could not be identied by title
and abstract, the full-text articles were read by the
reviewers. An additional search was performed manually utilizing the reference lists of included papers and
relevant journals for identifying eligible articles.
Quality Assessment
To evaluate the methodologic quality of each article,
an eight criteria-based scoring protocol was used,
modied from Lagravere et al. (2005), as shown in Table
2. Two authors independently completed the information
extraction sheet for the included papers, which was
designed on the basis of the review question and PICO
structure, as shown in Table 3. Disagreement between

Zhu et al., EVALUATION OF FACIAL APPEARANCE BY LAYPEOPLE AND PROFESSIONALS

TABLE 1

Search Strategy for Electronic Databases

PubMed
((((((cleft) OR harelip)AND (((((laypeople) OR layperson*) OR
layrater*) OR non-profession*) OR general public) OR lay public)
AND (((((profession*) OR clinician*) OR orthodonti*) OR
surgeon*) OR maxillofacial surgeon*) OR oral surgeon* ) AND
((((((assess*) OR evaluat*) OR prefer*) OR perceive) OR
perception) OR view) OR psycholog* ) AND (((((face) OR facial)
OR dentofacial) OR profile) OR lateral) OR cephalometric*) AND
((((attract*) OR aesthetic*) OR esthetic*) OR appearance) OR
beauty
Cochrane Library
(cleft OR harelip) AND (laypeople OR layperson* OR layrater* or
non-profession* OR general public OR lay public) AND
(profession* or clinician* or orthodonti* or surgeon* or
maxillofacial surgeon* or oral surgeon*) AND (assess* OR evaluat*
OR prefer* OR perceive OR perception OR view OR psycholog*)
AND (face OR facial OR dentofacial OR profile OR lateral OR
cephalometric*) AND ( attract* OR aesthetic* OR esthetic* OR
appearance OR beauty)
Web of Science
(cleft OR harelip) AND (laypeople OR layperson* OR layrater* or
non-profession* OR general public OR lay public) AND
(profession* or clinician* or orthodonti* or surgeon* or
maxillofacial surgeon* or oral surgeon*) AND (assess* OR evaluat*
OR prefer* OR perceive OR perception OR view OR psycholog*)
AND (face OR facial OR dentofacial OR profile OR lateral OR
cephalometric*) AND ( attract* OR aesthetic* OR esthetic* OR
appearance OR beauty)
Scopus
(cleft OR harelip) AND (laypeople OR layperson* OR layrater* or
non-profession* OR general public OR lay public) AND
(profession* or clinician* or orthodonti* or surgeon* or
maxillofacial surgeon* or oral surgeon*) AND (assess* OR evaluat*
OR prefer* OR perceive OR perception OR view OR psycholog*)
AND (face OR facial OR dentofacial OR profile OR lateral OR
cephalometric*) AND ( attract* OR aesthetic* OR esthetic* OR
appearance OR beauty)
EMBASE
(cleft OR harelip) AND (laypeople OR layperson* OR layrater* or
non-profession* OR general public OR lay public) AND
(profession* or clinician* or orthodonti* or surgeon* or
maxillofacial surgeon* or oral surgeon*) AND (assess* OR evaluat*
OR prefer* OR perceive OR perception OR view OR psycholog*)
AND (face OR facial OR dentofacial OR profile OR lateral OR
cephalometric*) AND ( attract* OR aesthetic* OR esthetic* OR
appearance OR beauty)

two authors was resolved by discussion and consensus.

When two authors could not reach an agreement, a third
reviewer (J.J.) was asked and consensus reached.
According to the total score, studies were classied into
low (score ,6), moderate (score .6 and ,10), and high
(score .10) level of quality, as shown in Table 4
(Lagravere et al., 2005; Baratieri et al., 2011).
RESULTS
In total, 112 studies were identied following the
database searches. Eleven articles were nally included in
qualitative synthesis. The retrieval of PubMed, The
Cochrane Library, Web of Science, Scopus, and EMBASE
provided 19, 5, 18, 58, and 12 records, respectively. After
removing duplicates and ineligible articles, 11 publications

189

TABLE 2 Methodologic-Quality Scoring Protocol (Maximum

Score, 12 Points) Based on Lagravere et al. (2005)
I. Study design (8)
A. Full face photograph (2D or 3D), including frontal side (1)
B. Cleft patients: homogeneous (2) or heterogeneous (1)
C. Viewed by professionals (2)
D. Viewed by laypeople: nonmedical background (2) or medical
background (1)
E. Viewed on two separate occasions (1)
II. Study measurements (4)
A. Rating system: VAS scale 0 to 100 mm (2) or categorical scale (3-,
4-, 5-, 6-, 9-, and 10-point scale) (1)
B. Intrapanel agreement (1)
C. Interpanel agreement (1)

remained. Seven additional records were identied through

reference linkage by hand searching. After reading the fulltext articles, 11 of them were nally included in qualitative
synthesis. The PRISMA ow diagram is shown in Figure 1.
Number and Type of Patients With Clefts
The majority of studies evaluated the facial appearance of unilateral cleft lip and palate; only one study
evaluated patients with bilateral cleft lip and palate (Lo
et al., 2002). However, three of the studies used a
heterogeneous group for assessing patients with cleft lip
and palate (CLP) and, as well as including individuals
with unilateral cleft lip and palate (UCLP), also
included individuals with unilateral cleft lip (UCL)
(Meyer-Marcotty and Stellzig-Eisenhauer, 2009), bilateral cleft lip and palate (BCLP) (Chung et al., 2013; Foo
et al., 2013), and cleft palate (CP), cleft lip (CL), and
submucous cleft palate (Foo et al., 2013). The remaining
seven studies were homogeneous and included only
patients with UCLP (Eliason et al., 1991; Al-Omari et
al., 2003; Prahl et al., 2006; Papamanou et al., 2012;
Gkantidis et al., 2013; Offert et al., 2013; Eichenberger
et al., 2014). The number of patients varied from 12
(Papamanou et al., 2012; Gkantidis et al., 2013) to 80
(Foo et al., 2013), and ages ranged from infant to adult.
Raters
Laypeople
The composition of the lay panel was highly
variable in both number and demographic prole
ranging from four individuals (Foo et al., 2013) to 121
individuals (Chung et al., 2013). Most studies reported
using laypeople, but some studies also included
patients with clefts (Foo et al., 2013; Gkantidis et al.,
2013), parents of patients with clefts (Gkantidis et al.,
2013), and raters from a medical background (Eliason
et al., 1991; Chung et al., 2013) in the laypeople
group.

190

Cleft PalateCraniofacial Journal, March 2016, Vol. 53 No. 2

TABLE 3

Extracted Information From the Included Papers

Patients

Author (Year)

Number and Type of Cleft

Comparator
Laypeople Demographics and Number

Professional Demographics and Number

42 Professionals (orthodontists,
pediatric dentists, general dentists,
oral maxillofacial surgeons,
craniofacial geneticists,
craniofacial plastic surgeons, and
otolaryngologists)
24 Professionals (14 orthodontists
and 10 maxillofacial surgeons)

Chung et al. (2013)

17 CLP adults corrected by

orthognathic surgery;
17 CLP adults corrected by late
maxillary protection

121 Laypeople (registered nurses,

anesthesia nurses, technicians,
front and back office staff, and
administrative staff)

Eichenberger et al.
(2014)

20 UCLP adults,
10 control adult patients with a
Class I occlusion
29 UCLP early-grafted children,
30 UCLP nongrafted children

15 Laypeople

14 Laypeople

5 Professionals (members of the cleft

team)

Gkantidis et al.
(2013)

12 UCLP young adults

12 Professionals (6 orthodontists, 6
maxillofacial surgeons)

Foo et al. (2013)

80 CLP adults

48 Laypeople (12 adult laypeople

matched for patients, 12 adult
laypeople matched for parents, 12
patients themselves, 12 patients
parents)
4 Laypeople (2 laypeople with cleft,
2 laypeople without cleft)

Papamanou et al.
(2012)

12 UCLP adults

12 Adult laypeople

Meyer-Marcotty and
Stellzig-Eisenhauer
(2009)

30 UCL and UCLP adults

15 Laypeople

20 Professionals (10 Orthodontists,

10 oral and maxillofacial
surgeons)

Prahl et al. (2006)

41 UCLP infants

24 Laypeople

21 Professionals (members of the

cleft team)

Al-Omari et al.
(2003)

31 UCLP adolescents and adults

5 Laypeople (4 postgraduate
students, 1 schoolteacher)

5 Professionals (1 plastic surgeon, 2

orthodontists, 1 oral maxillofacial
surgeon, 1 speech pathologist)

Lo et al. (2002)

64 BCLP children, adolescents, and

adults

5 Laypeople

5 Professionals (cleft surgeons)

Eliason et al. (1991)

24 UCLP adolescents and young

adults

40 Laypeople (unfamiliar raters

included faculty and secretarial staff
in other medical department,
psychology graduate students, nursing
and medical students)

40 Professionals (9 speech
pathologists, 6 psychologists, 3
pediatricians, 3 nurses, 3
geneticists, 2 orthodontists, 1
social worker, 13 medical and
dental residents)

Offert et al. (2013)

Professionals
Studies could be divided into those that grouped
raters according to specialty (Lo et al., 2002; MeyerMarcotty and Stellzig-Eisenhauer, 2009; Papamanou
et al., 2012; Gkantidis et al., 2013; Eichenberger et al.,
2014) or those that grouped raters into interdisciplinary care teams (Eliason et al., 1991; Al-Omari et al.,
2003; Prahl et al., 2006; Chung et al., 2013; Foo et al.,
2013; Offert et al., 2013). Four of the studies based on
specialty subdivided the group into orthodontists and
surgeons, while one study used a group of professionals comprised of surgeons only (Lo et al., 2002).

5 Professionals:
3 nonsurgical (orthodontist,
dentist, psychologist),
2 surgical (plastic surgeons)
24 Professionals (12 orthodontists,
12 maxillofacial surgeons)

However, only three studies went on to compare the

rater scores between the two professional groups
(Meyer-Marcotty and Stellzig-Eisenhauer, 2009; Papamanou et al., 2012; Eichenberger et al., 2014), while
one combined them into a single professional group
(Gkantidis et al., 2013). The number of professional
raters ranged from ve (Lo et al., 2002) to 42 (Chung
et al., 2013).
Type of View
In addition to 2D frontal photographs, lateral prole
photographs, three-quarter views, worms eye, birds

Zhu et al., EVALUATION OF FACIAL APPEARANCE BY LAYPEOPLE AND PROFESSIONALS

TABLE 3

191

Extended
Assessment

Type of Image Viewed

Rating Method

Intraoperator

Outcomes
Interoperator

Results

Full frontal, full frontal

smiling, profile

10-Point scale:
1 - very unattractive,
10 - very attractive

Cronbach a . 0.95

ANOVA mean score

Professionals more critical

than laypeople

Frontal, left lateral

100 mm VAS:
0 - very unattractive,
10 - very attractive
5-Point scale (AsherMcDade):
1 - very good appearance,
5 - very poor appearance
100 mm VAS:
0 - not satisfied,
100 - totally satisfied

Linear regression model

Linear regression model;

mean VAS score

Laypeople more critical than

professionals

Cronbach a . 0.9

Spearman correlation
coefficient

Professionals more critical

than laypeople

Cronbach a . 0.8

Spearman correlation
coefficient

Laypeople more critical than

professionals

Frontal left/right profile

100 mm VAS with higher

scores reflecting very
attractive

None

Student t test, mean VAS

score

Professionals more critical

than laypeople

Frontal
right/left lateral
3/4 right/lateral
3D

100 mm VAS:
0 - not satisfied,
100 - totally satisfied
9-Point scale:
1 - very asymmetrical or
ugly,
9 - very symmetrical or
attractive
100 mm VAS, more right
position more attractive

Cronbach a . 0.8

Spearman correlation
coefficient

Laypeople more critical than

professionals

None

Spearman correlation
coefficient

Laypeople and professionals

rated similarly

Cronbach a

Z-score and ANOVA, mean

VAS score

Laypeople and professionals

rated similarly

2D, 3D, and clinical

assessment, frontal, right
and left profile, close-up
view of nasolabial and
submental area

5-Point scale:
1 - very good appearance,
5 - very poor appearance

Kappa score

Kappa score

Full-face, frontal, lateral,

worms eye views

3-Point scale:
3 - Good,
2 - Fair,
1 - Poor
6-Point scale:
1 - excellent facial
appearance,
6 - much worse than
average appearance

Kappa score

Student t test, mean score

Professionals were more

critical in rating full face
from clinical assessment;
laypeople rated full face
more critically when using
3D images
Laypeople and professionals
rated similarly

None

ANOVA and Student t test,

mean score

Full face, cropped images of

frontal and profile,
inferior view of nose
Frontal, right and left
lateral, 3/4 right lateral

Full-face cropped photos

Frontal and lateral

eye, and cropped images were viewed. Two studies used

3D images (Al-Omari et al., 2003; Meyer-Marcotty and
Stellzig-Eisenhauer, 2009), and one study rated the
actual patient (Al-Omari et al., 2003).
Rating Method
Only one study provided instructions to the panel on
how to rate the images (Eichenberger et al., 2014). A
second study provided a reference image that the
raters used as a benchmark for comparison (Prahl et al.,
2006). Two studies used a preassessment practice or
calibration session prior to the main study to standard-

Professionals more critical

than laypeople

ize the raters responses (Al-Omari et al., 2003; Offert et

al., 2013).
Five studies used a VAS (Prahl et al., 2006;
Papamanou et al., 2012; Foo et al., 2013; Gkantidis et
al., 2013; Eichenberger et al., 2014), and the remaining
studies used a categorical scale (Eliason et al., 1991; Lo
et al., 2002; Al-Omari et al., 2003; Meyer-Marcotty and
Stellzig-Eisenhauer, 2009; Chung et al., 2013; Offert et
al., 2013). All VAS scales ranged from 0 to 100 mm,
while categorical scores ranged from ve points (AlOmari et al., 2003; Offert et al., 2013), based on the
Asher-McDade et al. (1991) scale, to 10 points (Chung
et al., 2013).

Moderate
High
High
Moderate
Moderate
8
11
11
10
8
1
1
1
1
1
0
1
1
1
0
* Intraoperator assessment refers to the assessment for level of agreement among individual raters within one rater panel.
Interoperator assessment refers to the assessment for level of agreement between rater panels.

0
0
1
0
0
1
2
1
1
1
2
2
2
2
2
1
2
2
2
2
1
1
1
1
1

2
2
2
2
1

Moderate
Moderate
Moderate
High
Moderate
High
8
10
10
11
9
11
1
1
1
1
1
1
1
0
1
1
0
1
0
0
0
0
0
0
1
2
1
2
2
2
2
2
2
2
2
2
1
2
2
2
2
2
1
2
2
2
1
2

Quality
Author (Year)

1
1
1
1
1
1

Total
Score
Assessors,
Professional

Assessment of Study Quality

Interoperator and Intraoperator Assessment

Chung et al. (2013)

Eichenberger et al. (2014)
Offert et al. (2013)
Gkantidis et al. (2013)
Foo et al. (2013)
Papamanou et al. (2012)
Meyer-Marcotty and
Stellzig-Eisenhauer (2009)
Prahl et al. (2006)
Al-Omar et al. (2003)
Lo et al. (2002)
Eliason et al. (1991)

Full Face Photos

0 No, 1 Yes

Cleft Patients
1 Heterogeneous,
2 Homogeneous

Assessors, Laypeople
1 Medical,
2 Nonmedical

Rating System
1 Categorical Scale,
2 VAS Score, 0100 mm

Images Viewed
Twice
0 No, 1 Yes

Intraoperator
Assessment*
0 No, 1 Yes

Interoperator
Assessment
0 No, 1 Yes

Cleft PalateCraniofacial Journal, March 2016, Vol. 53 No. 2

TABLE 4

192

In these studies, the intraoperator assessment referred

to the level of agreement of individuals in each of the
two groups, i.e., professionals and laypeople. Five
studies used Cronbach alpha (Prahl et al., 2006;
Papamanou et al., 2012; Chung et al., 2013; Gkantidis
et al., 2013; Offert et al., 2013), one used a linear
regression model (Eichenberger et al., 2014), two used
kappa statistics (Lo et al., 2002; Al-Omari et al., 2003),
and three did not report any intraoperator assessment
(Eliason et al., 1991; Meyer-Marcotty and StellzigEisenhauer, 2009; Foo et al., 2013). Interestingly, one
study assessed agreement by rating duplicate images on
the same occasion (Eichenberger et al., 2014), and one
study repeated the entire rating process (Al-Omari et al.,
2003).
Interoperator assessment is referred to as the level of
agreement between rater panels, i.e., professionals and
laypeople. Four studies reported using the Spearman
correlation coefcient (Meyer-Marcotty and StellzigEisenhauer, 2009; Papamanou et al., 2012; Gkantidis et
al., 2013; Offert et al., 2013), three studies used analysis
of variance (ANOVA) (Eliason et al., 1991; Prahl et al.,
2006; Chung et al., 2013), two used t tests (Lo et al.,
2002; Foo et al., 2013), one used a linear regression
model (Eichenberger et al., 2014), and one used kappa
statistics (Al-Omari et al., 2003).
The interoperator and intraoperator reliability was
high for all studies except two in which the intraoperator agreement was fair (Lo et al., 2002) or not
stated (Eliason et al., 1991).
The individual rater reliability was not assessed in the
majority of studies. Only one study reported on the
reliability of each rater assessing the facial appearance
of patients with cleft lip and palate on two separate
occasions (Al-Omari et al., 2003).
Three of the 11 publications found that laypeople
were more critical than professionals, including two
high-level and one moderate-level studies; three studies
proposed that there was no signicant difference
between laypeople and professionals based on one
high-level and two moderate-level studies, and ve
reported that professionals were more critical than
laypeople when assessing the facial appearance of
patients with cleft lip and palate based on one highlevel and four moderate-level studies.
DISCUSSION
Numerous studies have reported on the outcome of facial
appearance of patients with repaired cleft lip and palate
based on the opinion of laypeople and professionals.
However, there is no consensus whether laypeople and
professionals rate the facial appearance of patients with
cleft lip and palate similarly based on viewing full facial

Zhu et al., EVALUATION OF FACIAL APPEARANCE BY LAYPEOPLE AND PROFESSIONALS

FIGURE 1

193

PRISMA ow diagram of study selection process.

images. This may be attributed to the various methodologies and outcome measures used in each of these studies.
The present systematic review was performed to determine
whether there is a difference between laypeople and
professionals in evaluating facial appearance.
Viewing Media
Four methods have been used to evaluate full facial
appearance following cleft lip and palate repair (Sharma
et al., 2012): direct clinical assessment (Al-Omari et al.,
2003), 2D photography (Eichenberger et al., 2014), 3D
imaging (Meyer-Marcotty and Stellzig-Eisenhauer,
2009), and video-graphic assessment (Morrant and
Shaw, 1996). Only one study reported on the equivalency of rating facial appearance outcome by direct
clinical assessment, 2D photography, and 3D imaging
by professionals and laypeople (Al-Omari et al., 2003).
The study concluded that there was a statistical
difference in the outcome score (modied AsherMcDade) (Asher-McDade et al., 1991) between professionals and laypeople following direct clinical assessment and rating of the full facial 2D images;
professionals consistently scored higher indicating a
poorer outcome. However, when 3D images were
viewed, laypeople gave a poorer outcome score but this
was not statistically signicant. This single high-quality

study indicates that the viewing media may have an

effect on the perceived outcome of the repair. One
possible explanation put forward was that laypeople are
not trained to critically assess the outcome of cleft lip
surgery clinically or in 2D; this together with a degree of
sympathy toward the patient may lead to a lower
outcome score. Interestingly, in 3D there was no
statistically signicant difference in outcome score
between laypeople and professionals; a possible explanation is that both groups of raters were unfamiliar with
viewing 3D images and hence gave similar scores.
Cleft Group and Rater Group Selection
The inclusion criteria of patients with clefts and
laypeople also varied between studies. The homogeneity
of cleft patients was not present in some studies (MeyerMarcotty and Stellzig-Eisenhauer, 2009; Chung et al.,
2013; Foo et al., 2013). It has been reported that facial
growth is different in patients with bilateral cleft lip and
palate than in those with unilateral cleft lip and palate
(David et al., 2011). This may indicate that different
cleft types could affect the result of the nal outcome,
and heterogeneity in case selection should be avoided
(Anderson et al., 2013). The age range of patients with
clefts in each study also varied, ranging from infants
(Prahl et al., 2006) to adults (Meyer-Marcotty and

194

Cleft PalateCraniofacial Journal, March 2016, Vol. 53 No. 2

Stellzig-Eisenhauer, 2009; Papamanou et al., 2012;

Chung et al., 2013; Foo et al., 2013; Eichenberger et
al., 2014). The difference between rating infants and
adults may result from the various responses to babyishness and cuteness among raters (Hildebrandt and
Fitzgerald, 1978).
Another potential source of bias is the inclusion of
individuals with a medical background in the panel of
laypeople (Eliason et al., 1991; Chung et al., 2013).
These individuals may have previous knowledge or
exposure to cleft patients and may not represent the
valid opinion of laypeople (Chung et al., 2013). Thus,
for future studies, the homogeneity of both cleft patients
and laypeople with a nonmedical background is
recommended.
The largest discrepancy between studies is the
selection of professionals. One opinion is that raters
from different professions could differ in the evaluation
of facial esthetics in patients with cleft lip and palate
(Eichenberger et al., 2014). The professionals should be
divided into separate groups according to their disciplines in order to reduce the professional bias. The other
opinion is that different specialists in the panel of
professionals could represent the professional opinion
(Gkantidis et al., 2013). Since the aim of present
systematic review is to identify the difference between
professionals and laypeople, different specialists were
regarded as a whole and included in the professional
panel.
Besides laypeople and professionals, two studies
included patients with cleft lip and palate and their
parents in the rater group (Foo et al., 2013; Gkantidis et
al., 2013). This is of importance because patientcentered healthcare requires health professionals, patients, their families, and friends to cooperate as a
treatment team (Lutz and Bowers, 2000). However,
patients with cleft lip and palate and their parents may
not be regarded as laypeople. It has been reported that
professionals, as well as cleft patients and their parents,
were less critical than laypeople in assessing facial
appearance (Gkantidis et al., 2013). Therefore, in future
studies it may be necessary to exclude patients and
parents of cleft patients from the laypeople group and
view them as a separate assessment panel.
Control Group
The use of a noncleft control group would determine
if professionals and laypeople rated facial appearance
similarly and provide a baseline measure. However, only
one study used a noncleft control group and found that
professionals rated facial appearance higher than
laypeople when assessing the full face of individuals
(Eichenberger et al., 2014). None of the remaining
studies recruited individuals without clefts as a control
group. This could be attributed to the aims of the

studies, which were to identify the difference between

laypeople and professionals when assessing patients
with cleft lip and palate, rather than assessing individuals without clefts.
Outcome Measure
For outcome measure, the VAS and categorical scales
were commonly used by researchers. It was found that
the categorical scale seemed to be more discriminative
than the VAS (Prahl et al., 2006). However, there was
no standard categorical scale of 3, 5, 6, 9, and 10 points
applied in the studies (Eliason et al., 1991; Lo et al.,
2002; Al-Omari et al., 2003; Meyer-Marcotty and
Stellzig-Eisenhauer, 2009; Chung et al., 2013; Offert et
al., 2013). The VAS scores have been shown to be more
reliable, valid, and sensitive than the categorical scale,
such as the Likert scale, especially for scar rating (Grant
et al., 1999; Duncan et al., 2006). Even though VAS
scores and Likert scales are presented in metric
variables, which allow quantitative evaluations (Gkantidis et al., 2013), VAS is often preferred over
categorical scales.
Since both VAS and categorical scale are subjective
rating systems, reliability of these ratings should be
identied. Viewing facial appearance on two separate
occasions could determine the reliability of ratings given
by each individual rater. Similarly, intraoperator
assessment should be performed to assure the agreement
within a panel of raters. Interoperator assessment is
carried out to investigate the agreement among different
groups of raters. Only one study investigated the
reliability of viewing the images on two separate
occasions (Al-Omari et al., 2003).
Following this systematic review, 11 articles were
classied according to different levels of evidence
quality. Laypeople were found to be more critical than
professionals in two high-level and one moderate-level
study (Papamanou et al., 2012; Gkantidis et al., 2013). It
was inferred that professionals were more familiar with
the esthetic outcomes and difculties of treating patients
with cleft lip and palate, and rated less critically than
laypeople (Eichenberger et al., 2014). In addition, one
high-quality paper and two moderate-quality studies
reported no difference between laypeople and professionals (Al-Omari et al., 2003; Prahl et al., 2006). The
remaining studies, one high-quality paper and four
moderate-quality studies, found that professionals rated
more critically than laypeople. Direct comparison of the
different studies is difcult due to the difference in
viewing media, the composition of the rater group, and
the outcome measure.
The detailed recommendations for the study design
and outcome measure are listed in Table 5. These
recommendations will be helpful in investigating rating
differences of facial appearance between laypeople and

Zhu et al., EVALUATION OF FACIAL APPEARANCE BY LAYPEOPLE AND PROFESSIONALS

TABLE 5

195

Recommendations for Study Design and Measurements

Recommendations

I. Study design
A. Selection of images
B. Diagnosis of patients with
clefts
C. Laypeople
D. Professionals
E. View the images
II. Study measurements
A. Subjective rating system
B. Intraoperator agreement
C. Interoperator agreement

Clinical assessment, 2D photographs, or 3D imaging which include full frontal assessment/views of the face
Only one type of patients with clefts could be included to guarantee homogeneity
Those from a medical background, previous knowledge of patients with clefts, parents of patients with clefts, and
individuals with cleft should not be included
Different specialists could be regarded as a whole to represent the professional opinion
Raters should view the images on two separate occasions after calibration
VAS is preferred, and mean and median of VAS scores of full frontal face should be recorded
The level of agreement between rater panels should be performed
The level of agreement among individual raters within one rater panel should be performed

professionals in future studies. For example, full facial

imaging of a homogeneous group of cleft patients of a
similar age should be the target subjects for assessment.
One rater panel should be laypeople with no prior
knowledge of patients with clefts and no medical
background. The other panel should be professionals,
even from different disciplines, but who are part of an
interdisciplinary care team. Both professionals and
laypeople should assess the overall facial appearance
of patients with cleft lip and palate using VAS scores on
two separate occasions after detailed viewing instructions and calibration. An intraoperator and interoperator reliability should also be performed.
The clinical impact of a rating difference between
laypeople and professionals could have an inuence on
the decision for secondary surgical procedures for
patients with cleft lip and palate. One possibility is that
laypeople and professionals assess the facial appearance
similarly, and both agree further surgical intervention
would be benecial. The other possibility is that
laypeople and professionals rate facial appearance
differently. If laypeople are more critical, then the
surgical team may need to manage expectations or
discuss the possibility of further surgery. If, however,
professionals are more critical, then the facial appearance outcome should be accepted even though the
professional may feel the result is suboptimal and can be
improved.
In summary, this review highlights the need for
standardized viewing media that is valid, i.e., records
the true clinical facial appearance of the individual,
composition of a standardized rater panel, and a
standardized validated outcome measure. There are
apparently many denitions of patient-centered care,
but the Institute of Medicine denition is thought to be
the most acceptable (Mills et al., 2014): providing care
that is respectful of and responsive to individual patient
preferences, needs, and values and ensuring that patient
values guide all clinical decisions (Institute of Medicine, 2001). This highlights the importance of including
laypeople or the cleft patients peers as judges in the

assessment of soft tissue facial outcome following

surgical repair.
CONCLUSIONS
It still remains unknown whether laypeople are more or
less critical than professionals when rating the facial
outcome of patients with repaired cleft lip and palate.
Professionals are more familiar with the possible esthetic
outcomes and difculties of treating patients with cleft lip
and palate. The opposite may be true for laypeople; this
disparity between what is achievable by the profession and
what is expected by the laypeople may be a source of
dissatisfaction in the outcome of facial appearance. Further
well-designed studies should be carried out to address this
question.
REFERENCES
Al-Omari I, Millett DT, Ayoub A, Bock M, Ray A, Dunaway D,
Crampin L. An appraisal of three methods of rating facial
deformity in patients with repaired complete unilateral cleft lip
and palate. Cleft Palate Craniofac J. 2003;40:530537.
Anderson NK, Jayaratne YS, Zwahlen RA. Facial aesthetics and
perceived need for further treatment among adults with repaired
cleft as assessed by cleft team professionals and laypersons. Eur J
Orthod. 2013;35:849.
Asher-McDade C, Roberts C, Shaw WC, Gallager C. Development of
a method for rating nasolabial appearance in patients with clefts of
the lip and palate. Cleft Palate Craniofac J. 1991;28:385390;
discussion 390391.
MT, Maia
Baratieri C, Alves M Jr, de Souza MM, de Souza Araujo
LC. Does rapid maxillary expansion have long-term effects on
airway dimensions and breathing? Am J Orthod Dentofacial Orthop.
2011;140:146156.
Bongaarts CA, Prahl-Andersen B, Bronkhorst EM, Spauwen PH,
Mulder JW, Vaandrager JM, Kuijpers-Jagtman AM. Effect of
infant orthopedics on facial appearance of toddlers with complete
unilateral cleft lip and palate (Dutchcleft). Cleft Palate Craniofac J.
2008;45:407413.
Chung EH, Borzabadi-Farahani A, Yen SL. Clinicians and laypeople
assessment of facial attractiveness in patients with cleft lip and
palate treated with LeFort I surgery or late maxillary protraction.
Int J Pediatr Otorhinolaryngol. 2013;77:14461450.

196

Cleft PalateCraniofacial Journal, March 2016, Vol. 53 No. 2

Coetzee V, Greeff JM, Stephen ID, Perrett DI. Cross-cultural

agreement in facial attractiveness preferences: the role of ethnicity
and gender. PLoS One. 2014;9:e99629.
Dai L, Zhu J, Mao M, Li Y, Deng Y, Wang Y, Liang J, Tang L, Wang
H, Kilfoy BA, et al. Time trends in oral clefts in Chinese Newborns:
data from the Chinese National Birth Defects Monitoring Network.
Birth Defects Res A Clin Mol Teratol. 2010;88:4147.
David DJ, Smith I, Nugent M, Richards C, Anderson PJ. From birth
to maturity: a group of patients who have completed their protocol
management. Part III. Bilateral cleft lip-cleft palate. Plast Reconstr
Surg. 2011;128:475484.
Duncan JA, Bond JS, Mason T, Ludlow A, Cridland P, OKane S,
Ferguson MW. Visual analogue scale scoring and ranking: a
suitable and sensitive method for assessing scar quality? Plast
Reconstr Surg. 2006;118:909918.
Eichenberger M, Staudt CB, Pandis N, Gnoinski W, Eliades T. Facial
attractiveness of patients with unilateral cleft lip and palate and of
controls assessed by laypersons and professionals. Eur J Orthod.
2014;36:284289.
Eliason MJ, Hardin MA, Olin WH. Factors that inuence ratings of
facial appearance for children with cleft lip and palate. Cleft Palate
Craniofac J. 1991;28:190193; discussion 193194.
Foo P, Sampson W, Roberts R, Jamieson L, David D. Facial
aesthetics and perceived need for further treatment among adults
with repaired cleft as assessed by cleft team professionals and
laypersons. Eur J Orthod. 2013;35:341346.
Gkantidis N, Papamanou DA, Christou P, Topouzelis N. Aesthetic
outcome of cleft lip and palate treatment. Perceptions of patients,
families, and health professionals compared to the general public. J
Craniomaxillofac Surg. 2013;41:e105e110.
Grant S, Aitchison T, Henderson E, Christie J, Zare S, McMurray J,
Dargie H. A comparison of the reproducibility and the sensitivity to
change of visual analogue scales, Borg scales, and Likert scales in
normal subjects during submaximal exercise. Chest. 1999;116:1208
1217.
Hildebrandt KA, Fitzgerald HE. Adults responses to infants varying
in perceived cuteness. Behav Processes. 1978;3:159172.
Institute of Medicine. Crossing the Quality Chasm: A New Health
System for the 21st Century / Committee on Quality Health Care in
America, Institute of Medicine. Washington, DC: National Academy Press; 2001:40.
Kim JB, Strike P, Cadier MC. A simple assessment method for
auditing multi-centre unilateral cleft lip repairs. J Plast Reconstr
Aesthet Surg. 2011;64:195200.
Kim SC, Nam KC, Rah DK, Cha EJ, Kim DW. Assessment of the
cleft nasal deformity using a regression equation. Cleft Palate
Craniofac J. 2009;46:197203.
Lagravere MO, Major PW, Flores-Mir C. Long-term dental arch
changes after rapid maxillary expansion treatment: a systematic
review. Angle Orthod. 2005;75:155161.
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gtzsche PC, Ioannidis
JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA
statement for reporting systematic reviews and meta-analyses of
studies that evaluate healthcare interventions: explanation and
elaboration. BMJ. 2009;339:b2700.
Little AC, Burriss RP, Jones BC, Roberts SC. Facial appearance
affects voting decisions. Evol Hum Behav. 2007;28:1827.
Lo LJ, Wong FH, Mardini S, Chen YR, Noordhoff MS. Assessment
of bilateral cleft lip nose deformity- a comparison of results as

judged by cleft surgeons and laypersons. Plast Reconstr Surg.

2002;110:733738.
Lutz BJ, Bowers BJ. Patient-centered care: understanding its
interpretation and implementation in health care. Sch Inq Nurs
Pract. 2000;14:165183.
Mani MR, Semb G, Andlin-Sobocki A. Nasolabial appearance in
adults with repaired unilateral cleft lip and palate: relation between
professional and lay rating and patients satisfaction. J Plast Surg
Hand Surg. 2010;44:191198.
Meyer-Marcotty P, Stellzig-Eisenhauer A. Dentofacial self-perception
and social perception of adults with unilateral cleft lip and palate. J
Orofac Orthop. 2009;70:224236.
Mills I, Frost J, Cooper C, Moles DR, Kay E. Patient-centered care in
general dental practice a systematic review of the literature. BMC
Oral Health. 2014;14:64.
Morrant DG, Shaw WC. Use of standardized video recordings to
assess cleft surgery outcome. Cleft Palate Craniofac J. 1996;33:134
142.
Moss T. Individual variation in adjusting to visible differences. In:
Lansdown R, Rumsey N, Bradbury E, Carr T, Partridge J, eds.
Visibly Different. Coping With Disgurement. Oxford: ButterworthHeinemann; 1997:121130.
Offert B, Janiszewska-Olszowska J, Dudkiewicz Z, Brudnicki A,
Katsaros C, Fudalej PS. Facial esthetics in children with unilateral
cleft lip and palate 3 years after alveolar bonegrafting combined
with rhinoplasty between 2 and 4 years of age. Orthod Craniofac
Res. 2013;16:3643.
Papamanou DA, Gkantidis N, Topouzelis N, Christou P. Appreciation of cleft lip and palate treatment outcome by professionals and
laypeople. Eur J Orthod. 2012;34:553560.
Pedersen S, Vitaro F, Barker ED, Borge AI. The timing of middlechildhood peer rejection and friendship: linking early behavior to
early-adolescent adjustment. Child Dev. 2007;78:10371051.
Prahl C, Prahl-Andersen B, vant Hof MA, Kuijpers-Jagtman AM.
Infant orthopedics and facial appearance: a randomized clinical
trial (Dutchcleft). Cleft Palate Craniofac J. 2006;43:659664.
Rankin M, Borah GL. Perceived functional impact of abnormal facial
appearance. Plast Reconstr Surg. 2003;111:21402146; discussion
21472148.
Sarwer DB, Bartlett SP, Whitaker LA, Paige KT, Pertschuk MJ,
Wadden TA. Adult psychological functioning of individuals born
with craniofacial anomalies. Plast Reconstr Surg. 1999;2:412418.
Sharma VP, Bella H, Cadier MM, Pigott RW, Goodacre TE, Richard
BM. Outcomes in facial aesthetics in cleft lip and palate surgery: a
systematic review. J Plast Reconstr Aesthet Surg. 2012;65:1233
1245.
Shaw W. Global strategies to reduce the health care burden of
craniofacial anomalies: report of WHO meetings on International
Collaborative Research on craniofacial anomalies. Cleft Palate
Craniofac J. 2004;41:238243.

Stec M, Szczepanska
J, Pypec J, Hirschfelder U. Periodontal status
and oral hygiene in two populations of cleft patients. Cleft Palate
Craniofac J. 2007;44:7378.
Tanaka SA, Mahabir RC, Jupiter DC, Menezes JM. Updating the
epidemiology of cleft lip with or without cleft palate. Plast Reconstr
Surg. 2012;129:511e518e.
Williamson C. The challenge of lay partnership. It provides a different
view of the world. BMJ. 1999;319:721722.