Schapiro – Georgia Paper Critique

Dear Mr. Bechard,

Dr. Feingold forwarded your message regarding the Schapiro study along to me for comment. I
am the statistics consultant for Dr. Feingold's HIV/AIDS and Trafficking Project at UNESCO
and a PhD candidate in the Department of Development Sociology (specialties in demography
and research methods) at Cornell University.

I am sorry I do not have more time to devote to a full-length critique of the paper, but it in short,
I find it extremely problematic on a number of levels. Claims of the researchers to the contrary, I
find the study to be very light in 'scientific' rigor and incommensurately heavy in findings and
policy recommendations. Since they referenced a total of only 2 research papers (and 1 ABC
news article), I would advise reading the reports cited first. The several claims to scientific
validity stated in the Schapiro paper hinge on flimsy assumptions derived from what appear to be
fairly biased readings of the Chicago study and the Georgia CSEC paper. I haven't followed up
on the Chicago study, but I would definitely contact those authors and get their opinion on the
Schapiro study. The Georgia CSEC paper appears to be a policy paper and is probably not peer
reviewed. Note that they cite this paper far more often than the peer-reviewed Chicago study.

After a quick reading of the paper, I have a lot of comments about its various shortcomings in
terms of its near complete disregard for substantiation of research in a body of literature, its
insufficient elaboration of the methodology, its full lack of attention to ethics, and the various
extrapolations / conclusions they draw out of the "statistics." I use quotations to draw your
attention to the fact that the statistics presented as findings are only descriptive statistics of an
extremely biased sample. The researchers, however, present these descriptive statistics as
findings, without demonstrating that they have rigorously analyzed the data either for validity or
for statistical accuracy. As a general statistics comment, it should be noted that descriptive data
are insufficient for making public policy, especially when the research sample from which these
data are generated is likely to be extremely biased. The researchers make a cursory nod to the
clear need for attention to sampling bias, and very creatively proffer a + 5% sampling error in
footnote 6 with absolutely zero theoretical or methodological rationale.

The researchers seem to cite the dearth of research on the "demand-side" for purchasing sex as
sufficient reason to not ground their methods in any literature, and yet their rather inadequately
articulated conceptual framework operates as a basic supply-demand model of micro-
economics. Before I make any comment as to their particular methods, I should just point out
that there is a tremendously rich literature (albeit varying considerably in quality) on a range of
black-market activities including sex trafficking and is grounded in a variety of frameworks from
the socio-cultural to micro-economic. An article making multiple claims to scientific validity
should begin with a critical review of this literature and clearly demonstrate the basis for
embarking into methodologically uncharted territory.

Another useful literature that probably should have been examined and discussed in relation to
constructing this methodology discusses the propensity of people to be deceptive about
themselves in non-illegal activities over the internet. They note that men may not be willing to
be honest about their identities for fear of being caught, but they present the descriptive findings
as conclusive evidence that a giant population of men are engaging in sex with underage girls.
Perhaps with the exception of the 6% of respondents that specifically request an underage
female, I am not convinced by these methods that advertising "young" females is sufficient proof
that a man is specifically seeking to exploit a child. Of course this finding is of ethical and legal
concern, but it is not analyzed in relation to any other research.

It may very well be that the methods employed to "capture" respondents are commensurate to the
goals of the research. Very serious considerations of ethical issues aside, the authors do very
little to convince me that their methods are designed sufficiently well to base their claims as
anything other than circumstantial. What was the survey instrument that they used? Did they
contact a range of adult services listed to understand what kinds of questions were typically
asked of potential clients when they devised this survey instrument? Some methods questions to
think about in relation to survey and research design might be: What percentage of the adult
services ads listed specifically use the word "young"? What other descriptors are used (e.g.,
beautiful, sexy, brunette, older), and what numbers of hits do these ads receive? Are the clients
(assuming they tell the truth about themselves) statistically significantly different than those who
answer ads for "young" females? Did the interviewers ask the respondents what they wanted in a
young looking female? Can Craiglist provide information on the number of hits that these
service sites receive? What information do police and FBI taskforces have on these websites?
Did the ad or the interviewers specifically discuss sexual intercourse or specific sex acts? Such
basic information about background research done to devise a reliable methodology, if any
related background research was done at all, would make this project considerably more reliable.

Finally, I might note that the only description provided about the men interviewed is age.
Assuming that the ages provided by these men are in fact their real ages (already recognized as
suspect by the authors), the authors do not mention whether they acquire any other type of
information about the profiles of men seeking sex with underage girls. They, rather, seem to
imply through their extrapolations that all men are equally capable and therefore at risk of
engaging in this pathological behavior. As I understand it, they then extrapolate their findings to
the entire population of Atlanta males and produce numbers of "sexual predators" that are either
massively inflated or truly terrifying. They provide no information on the profiles of the men
that could be of real use. What are the incomes of these men (assuming a supply-demand model
we would want to know how much they are charging for sex with an underage girl, how they are
expected to pay, and whether men could afford this)? What are their criminal histories? What
are their marital profiles? Do they have children? What other possible and KNOWN factors are
linked to engaging in sexual exploitation of children? I do not claim to know these factors, but
research that extrapolates from an admittedly biased sample must ground the work in some
kind of knowledge about the assumed population.

If I were to follow this up more critically, I might investigate the political position of the group
that commissioned the Schapiro study and the various political campaigns the Schapiro group
typically contracts with. It's interesting to me that one of the only 2 lead researchers at the the
Schapiro group is trained in religious studies at Baylor U. I am not necessarily calling into
question their ability to conduct research, but the background of the research and the researchers
might suggest that, rhetoric in the study to the contrary, conservative opinions about sex work in
general are being privileged over scientific rationality.

It's truly disappointing that most research on sex trafficking is so poorly executed. Poor research
will not inform strong policy to effectively deal with such a horrific practice. If you are looking
for people to provide a more thorough critique of the research methods, I can also point you to
research institutes that specifically advise highly trained researchers on survey sampling
methodologies.

Good luck with your work, and please let me know if I can be of more help.

Best regards,
Amanda Flaim

On Wed, Sep 15, 2010 at 11:27 PM, David Feingold <ophidianfilms@hotmail.com> wrote:
Amanda -- Back sick (but getting better) from a great trip to the Wa.
I find the methodology, in particular the extrapolations, of this study pretty suspect.
Interested in your view as a stats ace.
Best,
David

David A. Feingold, Ph.D.

Director,
Ophidian Research Institute
Home (Bangkok):66(0)2 - 258-8796
Mobile: 66-(0)87-930-2511

Subject: FW: Georgia Research Study

Date: Wed, 15 Sep 2010 04:39:36 +0200
From: trafficking.bgk@unesco.org
To: ophidianfilms@hotmail.com

From the trafficking mailbox.

From: Raymond Bechard [mailto:raymond@raymondbechard.com]

Sent: Tue 8/24/2010 3:34 PM
To: d.feingold@unescobkk.org; trafficking@unescobkk.org
Subject: Georgia Research Study
Dear Dr. Feingold,

I know you are terribly busy. However, I am currently finishing a book on commercial sexual
exploitation of females in the United States focusing on the Federal trial of US v. Dennis Paris in 2007.
The book criticizes the use of inaccurate numbers and statistics by many in the government and 'anti-
trafficking community.'

Your essay in the book, 'Sex, Drugs, and Body Counts' has been remarkably helpful in my argument.
Thank you for that and all your fine work.

I wonder if you have heard of the attached report out of Georgia. Not being an expert on analytics,
methodology, statistics or their interpretations, I am interested to see what you think of the attached
report. If you could take a look and give me your comments, it would be most helpful.

If you do not have the time, please refer me to someone who may be able to give me an expert
opinion on the report.

All the best

Raymond Bechard
Connecticut
USA