Man or Machine Robot Writing

Man or machine - can robots really write novels?
Marc Cieslak reports on how robotic movement is becoming more advanced By Alex Hudson BBC News
Machines can already drive trains, beat humans at chess and conduct countless other tasks. But what happens if technology starts getting more creative - can a machine ever win the Booker Prize for fiction? In George Orwell's fiction, by 1984 the "proles" were entertained by books produced by a machine. In real life, robots have been capable of writing a version of love letters for over 60 years.
I couldn't think of anything more pointless than reading a piece of fiction written by a robot Alastair Reynolds, author
SPOT THE DIFFERENCE But how far away are books written by robots? THE TURING TEST
One xenia epigram - a poem originally found in Latin literature Despite the failure -of with a formal structure has been machines deceive us into for written by to poet Luke Wright believing theyother are human, the BBC. The is written by a Turing would be excited computer after being givenby the remarkable progress of AI form. instructions about the poetic Can you tell which is which? Noel Sharkey, professor of artificial intelligence, To Truth, by ??????University of Sheffield More on Ihow Turing influenced To truth offer this thanks, AI when needing something like reality When I'm writing and drawing blanks, I almost settle using actuality I am in search of more, trying to sing your praise! It's you I very much adore, lacking in so many ways. To Felicity, by ?????? Felicity, my dear, my thanks the cheque you sent was great. Tomorrow I'll go to the bank my rent's already late. And sorry for the shoddy rhyme I'm tired, I'm not on it, perhaps if you send more next time I'll scribble you a sonnet. (Answer revealed at bottom of page)
Well they have already happened, in their hundreds of thousands. Professor Philip Parker, of Insead business school, created software that has generated over 200,000 books, on as varied topics as the 60 milligrams of fat in fromage frais to a Romanian crossword guide. Amazon currently lists over 100,000 titles under his name. While not expecting to top the bestsellers list or win any literary awards, they take under an hour to "write" because of how they are produced and are printed when requested rather than in bulk. The books compile existing information and offer new predictions using forumulas, like estimating the future size of markets for example. But Professor Parker has experimented with a piece of software that is capable of creating automated fiction. Photoshop's Raphael? Fiction is often criticised for being a factory process of using formula and "write by numbers" approaches. Creative writing programmes have been likened to working "from a pattern book" by Booker-nominated author Will Self. Certain pieces of writing software provide templates that will automatically create the structure of a novel and once written, can tell you how easy the novel is to read. "No novel writing package will write your book for you," says software firm NewNovelist. I couldn't think of anything more pointless than reading a piece of fiction written by a robot Alastair Reynolds, author "They certainly can help you complete your novel and make sure it is composed correctly." But if there is a formula, can the novelist really be replaced by an algorithm? Russian Alexander Prokopovich is said to be responsible for the first successful book to be created by robots. It was published in 2008 and was written in the style of Japanese author Haruki Murakami in a variation on Leo Tolstoy's Anna Karenina. "The program can never become an author, like Photoshop can never be Raphael," Prokopovich told the St Petersburg Times. Whether this is actually an original work of fiction is up for debate. Though this could be compared to the literary argument about whether any work of fiction is truly original. On a more simple level, the people experimenting with the technology believe that it is not as
difficult to create fiction as critics believe. 'Preferred' to Shakespeare THE TURING TEST
Despite the failure of machines to deceive us into believing they are human, Turing would be excited by the remarkable progress of AI Noel Sharkey, professor of artificial intelligence, University of Sheffield More on how Turing influenced AI "Any genre of fiction that has a 'dummy's guide' to it could be created with an algorithm," says Prof Parker. "The more a genre subscribes to a formula, the more straightforward it is. "In romantic fiction, instructions for authors can be as specific as down to the page. If you feed that information into a computer, the formula is followed. Each genre has some sort of formula, just some more than others." Prof Parker's software, still in prototype, would allow characters to be decided, locations to be set, genre fixed and plot mechanisms chosen. It then creates anything from 3,000-word flash fiction to a 300,000-word novel. He has even done public experiments with poetry. "A computer works very well with rules and the most obvious way is poetry," he says. "We did a blind test between a Shakespearean sonnet and one that the computer had written. A majority of people surveyed preferred ours. "That's not to say it was better, Shakespeare is a genius, but it was what people preferred." With so many authors around - Mills and Boon publishes around 100 books a month - Prof Parker is keen to stress that his aim is not to produce fiction that is already created in
abundance. Instead, it could be used to write in languages or on topics that are not widely covered. But computers are not just seen as a threat to creative work but to other writing as well. Startup company Narrative Science has started creating articles, without a human doing the writing. With 30 clients for its articles already, written automatically by a machine collating data and writing "rich narrative content" from it, the death of the journalist has been mentioned in more than one speculative column. Business news site Forbes is using the service for a number of pieces each weekday. 'Pointless' fiction But if neither Beryl Bainbridge nor Martin Amis can win the Booker Prize, what chance does a machine really have?
Computers are already useful in developing new mathematical theories And if creative writing really is creative, what new ideas could a computer offer? "I couldn't think of anything more pointless than reading a piece of fiction written by a robot," says science fiction author Alastair Reynolds. "Even if it was indistinguishable from your average Booker Prize winning novel. "You might find a lot of regurgitated platitudes but I can't imagine a piece of software being capable of producing something that would stop you in your tracks. Not until we get truly intelligent computers." The Loebner Prize of $100,000 was set up as an offer to the first computer program to convince testers that it was human through two-way communication. No-one has won that prize yet but it could be that the written word, without interaction, could provide a more immediate way to confuse the reader as to who or what is writing.
And then a different sort of prize could only be a step away. "I don't think the computer will win the Booker but no-one ever expected a computer to beat a chess grandmaster," says Reynolds. "A normal tool in a mathematician's tool kit is the computer. 100 years ago, it would be considered heresy. "The idea of a computer winning the Nobel Prize for physics is not too unlikely, citing a computer as joint recipient. It's obviously not a huge leap to think of something similar happening in fiction." Note: To Felicity is written by the poet Luke Wright. To Truth is written by computer software
Rational Arrogance
I can write about anything , right?
Menu
Skip to content

Home About Projects Resources Satire
Post navigation
New Research: Social Media Amplifies Irrational Behavior Justin Bieber to sue Twitter users for asking why the f*** is he always trending April 22, 2013
Robots Writing the News Who Cares?

By econotwist Posted in Investigations, Politics, Social Media, Technology Tagged Algorithm, Behavior Analysis, California, Coming Out, Companies, Gay Lesbian and Bisexual, Jamie Dwyer, kenschwencke, Los Angeles Times, Psychology, Robot, Robotics, Schwencke, Social Sciences, University of Ontario Institute of Technology, Vancouver Sun 1 Comment
I suppose its only natural: Just like the financial pros who didnt /wouldnt see the economic disaster they created are being replaced by robots, the people who was supposed to uncover the reckless market behavior (the journalist) are about to be replaced by computers and algorithms, too. I doubt that people who read our posts unless they religiously read the earthquake posts and realize they almost universally follow the same pattern would notice, I dont think most people are thinking that robots are writing the news. Ken Schwencke
No, I dont think so, either. It has to be a damn good story make me check out the name of the person who wrote it! And after this post, I will probably never bother to look at a byline again because the stuff may very well be made up by some computer, able to produce flawless articles using sophisticated algorithms, processing huge amounts of data . Automated reports, like statistics and research papers, are nothing new. But journalist and digital editor at The Los Angeles Times, Mr. Ken Schwencke, have taken it a step further. He has written his own software an algorithm that writes his articles for him. As The Vancouver Sun reports:
Journalist Ken Schwencke has occasionally awakened in the morning to find his byline atop a news story he didnt write. Instead of personally composing the pieces, Schwencke have developed a set of step-by-step instructions for his computer that can take a stream of data (this particular algorithm works with earthquake statistics, since he lives in California) compile the data into a pre-determined structure, then format it for publication. His fingers never have to touch a keyboard; he doesnt have to look at a computer screen. He can be sleeping soundly when the story writes itself, the slightly shocked Canadian newspaper writes. Jamie Dwyer, bachelor of science in computing science from the University of Ontario Institute of Technology, says algorithms can be highly complex computer codes or relatively simple mathematical formulas. They can even sometimes function as a recipe of sorts, or a set of repeatable steps, designed to perform a specific function. In this case, the algorithm functions to derive and compose coherent news stories from a stream of data. Ken Schwencke at LA Times seem to have no concerns whatsoever about giving up his job (at least partly) to a computer. He says the use of algorithms on routine news tasks frees up professional reporters to make phone calls, do actual interviews, or dig through sophisticated reports and complex data, instead of compiling basic information such as dates, times and locations. It lightens the load for everybody involved MORE@The Vancouver Sun Perhaps, but not everybody are equally happy about it. Certainly not every still-writingjournalist! But according to Kristian Hammond, co-founder of Narrative Science, a main producer of writing machines, they have nothing to fear. This robonews tsunami, he insists, will not wash away the remaining human reporters who still collect paychecks. Instead the universe of newswriting will expand dramatically, as computers mine vast troves of data to produce ultracheap, totally readable accounts of events, trends, and developments that no journalist is currently covering. Mr. Hammond also predicts that a computer will win the prestigious US journalism award The Pulitzer Prize within five years. MORE@WIRED.COM
Lisa Taylor, who is a lawyer and a journalist teaching ethics to undergraduate students at the School of Journalism Ryerson University, highlights one of the important issues. The complicating factor here is a deep suspicion journalists and news readers have that any technological advancement is going to be harnessed purely for its cost-cutting abilities. But Taylor also points out that this new tool may have some positive effects for the media professionals. She believes that journalists will have to start discussing algorithms, just as they talk about Twitter and other rising social media. How can we use this effectively, reasonably, and in a way that honours the tenets of journalism? Taylor ask. Im afraid it may be a bit late for that. The fact is that these automated articles are still being presented and published with a byline at the top, like some human being actually wrote the piece. Thats misleading to the readers, at the least. And this comes at a time when the credibility of a journalist is lower than a used cars salesman.
This case also raises a lot of new questions. F.ex. Who holds the copyright on the generated articles? And what if Mr. Schwencke decided to leave the LA TImes and work for another employer? Does he retain the right to the bot? Or is that algorithm, developed while employed with the LA Times, considered a work for hire, and thus, the papers property? Arguably, his algorithm is an extension of him, covering his area of expertise and designed to emulate his reporting. What if Schwencke generates a similar piece of software for his new employer? Would he be permitted to do this, or would this be prevented by additions to noncompete clauses? Is it patentable? Tim Cushing at Techdirt.com notes. The more ubiquitous robo-journalism becomes, the more issues like these will arise. Hopefully, IP turf wars will remain at a minimum, allowing for the expansion of this promising addition to the journalists toolset. With bots handling basic reporting, journalists should be freed up to pursue the sort of journalism you cant expect an algorithm to handle longform, investigative, etc. This is good news for readers, even if they may find themselves a little unnerved (at first) by the journalistic uncanny valley. MORE@Techdirt.com
But the basic for any outcome in this case is that people both consumers and publishers starts a discussion. And I dont see that happening. Symptomatically, take a look at the share buttons on the top of the Vancouver article. Its been online for almost a month and have been tweeted just once. Shared one time at LinkedIn, five times at Google+ and received the stunning number of 26 likes on Facebook
o o o o o o o
Don't Miss: Happening Now Video Centre Live chats Blogs on canada.com canada.com on Facebook Oddities
News Business Sports Entertainment Lifestyle Health Travel Technology
Robo-reporter raises questions about future of journalists
By Jesse M. Kelly, Postmedia News March 26, 2013
Story Photos ( 3 )
Reporters may not look like this in the future, but one media outlet employs a robo-reporter a program that compiles data into a pre-determined structure, then formats the information for publication. Photograph by: File photo , Postmedia News
Journalist Ken Schwencke has occasionally awakened in the morning to find his byline atop a news story he didnt write. No, its not that his employer, The Los Angeles Times, is accidentally putting his name atop other writers articles. Instead, its a reflection that Schwencke, digital editor at the respected U.S. newspaper, wrote an algorithm that then wrote the story for him. Instead of personally composing the pieces, Schwencke developed a set of step-by-step instructions that can take a stream of data this particular algorithm works with earthquake statistics, since he lives in California compile the data into a pre-determined structure, then format it for publication. His fingers never have to touch a keyboard; he doesnt have to look at a computer screen. He can be sleeping soundly when the story writes itself.
Just call him robo-reporter. I doubt that people who read our (web) posts unless they religiously read the earthquake posts and realize they almost universally follow the same pattern would notice, Schwencke said. I dont think most people are thinking that robots are writing the news. But in this case, they are. And that has raised questions about the future of flesh-and-blood journalists, and about journalism ethics. Algorithms are fairly versatile, and have been doing a great number of things we sometimes dont even think about, from beating us at computerized chess, to auto-correcting our text messages. Jamie Dwyer holds a bachelor of science in computing science from the University of Ontario Institute of Technology, and provides IT support for Environment Canada. Dwyer said algorithms can be highly complex computer codes or relatively simple mathematical formulas. They can even sometimes function as a recipe of sorts, or a set of repeatable steps, designed to perform a specific function. In this case, the algorithm functions to derive and compose coherent news stories from a stream of data. Schwencke says the use of algorithms on routine news tasks frees up professional reporters to make phone calls, do actual interviews, or dig through sophisticated reports and complex data, instead of compiling basic information such as dates, times and locations. It lightens the load for everybody involved, he said. Yet there are ethical questions such as putting someones name atop a written article he or she didnt in fact write or research. Alfred Hermida, associate professor at the University of British Columbia, and a former journalist, teaches a course in social media, in which he takes time to examine how algorithms affect our understanding of information. He says that algorithms, like human beings, need to decide what is worth including, and make judgments on newsworthiness. If the journalist has essentially built that algorithm with those values, then it is their work, Hermida said. All the editorial decisions were made by the reporter, but they were made by the reporter in an algorithm. The greater issue, he says, is demystifying the technology for the reader. Hermida says that many of the algorithms we encounter everyday exist in a black box of sorts, in which we see the results, but do not understand the process. Understanding how the algorithms work is really important to how we understand the information, Hermida said.
Algorithms like Schwenckes are relatively simple, for now. Theyre best suited to smallscale streams of data that are being regularly updated with consistently formatted information. For instance, baseball may be a good avenue for news algorithms, because the game is heavy with statistics, says Paul Knox, associate professor for the School of Journalism at Ryerson University in Toronto. But even if an algorithm can analyze and manipulate data fairly well, journalism is still based on not only filtering, but also finding other available information, Knox notes, and a mathematical construct lacks the ability to dig up new facts or add context. On the other hand, People are already reading automated data reports that come to them, and they dont think anything of it, said Ben Welsh, a colleague of Schwenckes at the Times. One example is any smartphone app that displays personalized weather information based on the owners location. Thats a case where I dont think anyone really blinks, Welsh said. Its just a kind of natural computerization and personalization of a data report that had been done in a pretty standard way by newspapers for probably a century. And Welsh says that responsibility for accuracy falls where it always has: with publications, and with individual journalists. The key thing is just to be honest and transparent with your readers, like always, he said. I think that whether you write the code that writes the news or you write it yourself, the rules are still the same. You need to respect your reader. You need to be transparent with them, you need to be as truthful as you can all the fundamentals of journalism just remain the same. Although algorithms in news are paired with simple data sets for now, as they get more complicated, more questions will be raised about how to effectively code ethics into the process. Lisa Taylor is a lawyer and a journalist who teaches an ethics class to undergraduate students in the School of Journalism at Ryerson University. Ultimately, its not about the tool, said Lisa Taylor, a lawyer and journalist who teaches ethics at Ryerson. At (the algorithms) very genesis, we have human judgment. Taylor said that using algorithms ethically and reasonably shouldnt be difficult; the onus is on the reporter to decide which tools to use and how to use them properly. The complicating factor here is a deep suspicion journalists and news readers have that any technological advancement is going to be harnessed purely for its cost-cutting abilities, said Taylor. According to Taylor, journalists will have to start discussing algorithms, just as they talk about Twitter.
How can we use this effectively, reasonably, and in a way that honours the (tenets) of journalism? Taylor asked.
Copyright (c) Postmedia News
July 8, 2013
May 3, 2013
English Teachers Reject Use of Robots to Grade Student Writing

By Dan Berrett
Critics of standardized tests argue that the written portion of those assessments can shortcircuit the process of developing ideas in writing. Using machines to grade those tests further magnifies their negative effects, according to a statement adopted last month by the National Council of Teachers of English. As high-school students prepare for college, the statement reads, they "are ill served when their writing experience has been dictated by tests that ignore the evermore complex and varied types and uses of writing found in higher education." The statement is unlikely to quell controversy over the use of automated grading tools to assess a new wave of standardized tests of writing that are being developed for students at elementary and secondary levels. The intent of the statement, which was passed unanimously by the council's executive committee, is to prompt policy makers and designers of standardized tests to think more fully about the pitfalls of machine scoring, said Chris M. Anson, director of the writing-andspeaking program at North Carolina State University. Mr. Anson is also chair of the committee that drafted the statement for the council, a 35,000-member organization that seeks to improve the teaching and learning of English at all levels of the education system. Chief among the council's concerns, said Mr. Anson, is that machine grading tends to recognize, and therefore encourage, writing that may appear superficially competent, but lacks meaning or context. Machines also cannot judge some of the most valuable aspects of good writing, the statement reads, including logic, clarity, accuracy, style, persuasiveness, humor, and irony.
To judge writing, machines analyze a text by using an algorithm that predicts a good answer, which is largely based on whether it uses certain words. The machines cannot recognize if the argument is coherent or even true, he said. Rarely used and multisyllabic words can boost a test-taker's score even when they are included in an essay pointlessly. "By using the word 'cadre' or 'defenestration,'" he said, "the computer will think that's good." Mr. Anson also worries about the larger message that machine grading sends. It tells students "that writing is so unimportant that we're not willing to take the time to read it," he said. If machines value writing that has the veneer of coherence but lacks substance, he said, that factor is also likely to shape the kinds of writing exercises teachers assign. In his courses, he sometimes asks students to write an imagined dialogue between scholars, such as B.F. Skinner and Noam Chomsky, or Sigmund Freud and Karl Marx. A machine would not be able to handle such an assignment, he said, and faculty members might be dissuaded from being creative in the exercises they devise. "It sends a message to teachers," Mr. Anson said, "to design the most stereotypical, dull assignments that can be graded by a machine."
'Already Mechanical'
Machine grading is a more urgent issue in elementary and secondary education than it is for colleges. Such scoring is being considered to grade the assessments being developed for the Common Core State Standards, which have been adopted in 45 states and the District of Columbia. Those standards are intended to improve students' readiness for the transition to college or the work force. Still, the notion of machine grading is not foreign to higher education. The grading of the written portion of the Collegiate Learning Assessment, for example, is "almost excusively" automated, said Jeffrey Steedle, a measurement scientist for the Council for Aid to Education, which created the CLA. While the statement from the National Council of Teachers of English is likely to find a favorable audience among many faculty members, Mr. Anson concedes a point often made by critics of the status quo: Human evaluators do not always practice the kind of close, careful, and nuanced reading the council's statement champions. "Machines can reproduce human essay-grading so well because human essay-grading practices are already mechanical," Marc Bousquet wrote in a blog post for The Chronicle last year. Although multiple human raters may collectively catch mistakes, individuals tend to make more grading errors than machines do, said Mark D. Shermis, a professor of educational foundations and leadership at the University of Akron. He is the lead author of a study, supported by the William and Flora Hewlett Foundation, that found that machines were about as reliable as human graders in evaluating short essays written by junior-high and high-school students.
In a statement provided to The Chronicle, he criticized the council's announcement as "political posturing" and a "knee-jerk reaction," and criticized its analysis. "There is no evidence to suggest that scoring models for longer writing products are somehow 'worse' than for short, impromptu prompts," he wrote. In an interview, Mr. Shermis agreed that a computer cannot understand context or judge meaning. It can, however, conduct a coherence analysis, which can render a verdict on the accuracy of an argument based on probability. The grading software, he said, can identify a number of words and phrases that are highly likely to lead to the correct conclusion. Critics of Mr. Shermis's work, such as Les C. Perelman, of the Massachusetts Institute of Technology, have faulted his methodology and shown how a computer can be dazzled by big but meaningless words. Someone like Mr. Perelman, who understands the algorithms, can write a bad essay full of botched facts and still get a good score, said Mr. Shermis. "The average writer doesn't operate that way." An important distinction, some analysts say, is whether machines are used for high-stakes tests or for formative evaluations, which are likely to be low-stakes assignments that a student can revise and resubmit. The council's statement fails to recognize the distinction, according to Edward E. Brent, a professor of sociology at the University of Missouri at Columbia and president of Idea Works, which developed grading software called SAGrader. "We need to be having a good healthy discussion of different views regarding the use of computers for assessing writing," he wrote in an e-mail. "But it is important that the conversation be broadened to encompass the full range of such programs and their differing strengths and weaknesses." Ultimately, said Mr. Shermis, writing and measurement experts should work together to define what matters most in the craft of writing. The technology to measure it will continue to be developed. "There's nothing we can do to stop it," he said. "It's whether we can shape it or not."

Comments (41)
Comments
Powered by Disqus

Disqus
Add a comment
Log in to post
with your Chronicle account:
Don't have an account? Create one now. Or log in using one of these alternatives:

Showing 40 of 41 comments Sort by Follow comments: by e-mail Real-time updating is paused. (Resume)
by RSS
dsarma 2 months ago
"There's nothing we can do to stop *it*"? Sounds like a science fiction movie. Is *it* Skynet?
15 people liked this.
jranelli 2 months ago
...in his latest novel bindlestiff reaches a level of accuracy and probability that has cohernce analysis software beside itself...
wclibrary 2 months ago

Rubrics encourage robo-grading.

deliajones 2 months ago in reply to wclibrary
Damning with an awfully broad brush. Igive my students rubrics for each of their writing assignments to show them quite specifically what I am looking for in a paper and how important I think each element is. My rubrics are actually maps of my mind while assessing. I don't actually complete a rubric and assign points to arrive at a grade; my grades are more holistic. But don't you thik it's fair to be very explicit about our expectations?
fisher56 2 months ago
If a machine can grade a paper, a machine can also write a paper that is guaranteed a 100% score. The current issue is about standardized tests, but I think homework grading is the most labor intensive, and that is where robo-grading will have the biggest impact.
raymond_j_ritchie 2 months ago in reply to fisher56
You are bloody well right. Call me a primitivist but if the world gets to robopapers marked by robo-markers what is the point in setting an essay at all? I gave up giving students take-home essays in Plant Biology, Invertebrate Biology and Biochemistry years ago. Too much downloading. I now give them a topic a week before and make them write the essay in class while I am watching. The students take it OK. Giving the students the questions for a formal exam a week or two before and then have them answer under exam conditions seems to have no effect on the outcome. The bad students still write 5 lines and the good ones a couple of pages with labelled drawings.
harrybythesea 1 month ago in reply to fisher56
"If a machine can grade a paper, a machine can also write a paper that is guaranteed a 100% score." may sound symmetrical or something, but all that I think you can really assert is that some who understands how the software that does the grading works, exactly, can write a paper that will get whatever score the
authors want (as Less Perelman so often has attempted to demonstrate). And if these "rules" work enough like the teacher's rules for writing, and it results in improved writing, is that good enough?
I am open to a logical "proof" of your assertion, but I won't hold my breath. At the same time, and in the same spirit, I don't see how the Chronicle resisted the headline "English Teachers Reject Use of Robots to Teach English". (Edited by author 1 month ago)
fisher56 1 month ago in reply to harrybythesea
I am not deep enough to understand the standards you expect for a formal proof, even though, ironically, I am sitting in a CS lab right now. Your pedestrian description is good enough for my standards. Just because it can not be proven that it can be done every single time, it doesn't mean that it can not be done most of the time. The grading algorithm could be reverse engineered and your friendly (internet) neighborhood entrepreneur could set up a service that generates A- / A /guaranteed to pass the plagiarism test A+ grade essays for a small fee. Perhaps with a money back guarantee in the cases that are not covered by the grading algorithm reversibility theorem. There was an example a few years ago when a group of students got a machine generated article accepted at an obscure conference. The article was "content-free", but used English words and was grammatically correct. So, as someone else points out, the entire exercise of take home essays could become a battle of the essay writing / grading algorithms. Good for CS, bad for English. And of course, as it is the case with many testing services that shall remain unnamed, the companies developing the grading software could also make money from selling "Crack the English Essay Test" software.
Jack Dempsey 2 months ago
I started my writing career in NYC's educational textbook industry in the 1980s (primary through college levels). Whole open floors in skyscrapers packed with freelancers who wrote at clacking typewriters---we worked hard for good pay, laughed all day, and year after year produced textbooks that conquered their markets (no mean feat when you're selling to Texas and CA). Then first came Reaganite "downsizing": 30-50% of freelancers fired, their work piled onto cheap-labor full-timers, and the open floors blocked off with cubicle-barriers, while our editors lined up to kiss the "Moral Majority's" ass. (And as books continued to win awards, fat execs sent out
memos on "another tough year" explaining no raises/bonuses etc., except for themselves.)
Phase Two: Freelance-work dried up in favor of cheaper, cowed full-timers: the books declined into soporofic flag-waving, and on came the age of desktop computers. At conferences touting new computer-based "learning programs" over textbooks, with the halls filled with slick 20-year-old "experts" hot to sell their machines to school districts, we saw the same (yet dumbed-down) content now flashing and buzzing on computer screens with a few other visual gimmicks that kept the eye busy and the brain dead. So we're still where Santayana saw us: "The American Will is manifest in the skyscraper, but the American mind still inhabits The Old Manse." We get year after year of techno-progress and simply refuse to correlate that with clearly increasing ignorance and inability to think for 10 consecutive minutes. (Or is it 5? I forget.) "There's nothing we can do to stop it"? How about acting with a backbone on the knowledge that you will never replace a teacher with a machine, nor a book with a gizmo. It's time to walk away from yet another dead end born of business-think applied where it has no business, and get back to the education of citizens for a cultured democracy. Did I say that out loud? (Edited by author 2 months ago)
jranelli 2 months ago in reply to Jack Dempsey
to get anywhere near the laudable objective of education for a cultured democracy, the biz-think that has eroded faculty governance, among other things, on campuses across the country needs to be contained, (preferably in a radioactive waste facility).
prof291 2 months ago in reply to Jack Dempsey

Bravo.
htinberg 2 months ago
Thank you for offering this thoughtful take on machine scoring. But here's the sobering reality: assessments of the Common Core are about to produce a yield of essays the likes of which we have never seen (PARCC, one testing consortium serving the Core, aims to test students' writing three times a year from grades three to eleven). All these essays will need to be read. Machine scoring seems the efficient and (relatively) cheap mode of "reading" such work. But what is lost when students' work is read by people other than the students' own teachers? How important is the fact that teachers bring to students' work an understanding not only of the topic but of the students practices as writers? Are such matters trivial?
dale1 2 months ago in reply to htinberg

You've done a service here in your last paragraph, to which I'd add the following. The above article does nothing to address the issue of formative feedback and how the work being assessed by the software fits into the larger curricular goals of a course or a curriculum. If each essay has a letter grade attached, say 88/100; B+, but the instructor doesn't review these or have any idea of an individual student's progress in the course, what does it matter? In addition, if there's no formative feedback on the essay itself, the process is nearly devoid of educational value to students. Education is expensive, but much less so than ignorance...
deliajones 2 months ago in reply to htinberg
So do you grade for effort? Is even grading supposed to be a "process"? At what point should simple writing competence be assessed objectively?
jwr12 2 months ago in reply to deliajones
You say, "at what point should simple writing competence be assessed objectively?" It's an interesting question. But if we really just mean *simple* writing competence (i.e. can a student communicate using letters) then I would imagine this would need to be done early: that is, elementary
school. (Whether this is an objective question I'm not sure, but in any case if you're going to pose it, I suppose it needs to be done soon).
That said, should this assessment be done repeatedly, continuously, all the way up the educational ladder, as the student matures? The NCLB logic behind much of this testing -- and behind the idea that a student can be graded by a machine that doesn't care or understand what the student is trying to say -- says, in effect, yes. What we care about is that you can leap over the bar, and all we're going to test you on is that, forever. And if you don't, here's feedback. But that, in turn, gives feedback of another sort as well: namely, there is no point here where we're interested in you as a creative person to whom a human response is needed. Rather, we're going to repetitively test you on something that gradually becomes irrelevant and pedantic as you get older, and we're going to devote our most powerful and costly resources to same. I often worry about the problem of 'plausible deniability' in our classes, how students can actually game the system by pretending to have never been exposed to higher order ideas or skills, or that they can't actually do things they can. This may seem implausible: isn't the desire to get a grade enough? But in my experience, it often isn't. Students can be happy with passing, if that means that they don't have to confront hard or uncomfortable things. So the phone it in grader is the friend of the phone it in student. We need to send the feedback that we don't just care about simple writing competence. We care about people learning to write and think.
nickcarbone 2 months ago
As I understand, the research on automated assessment shows correlation between human readers and machines under narrow circumstances: when writers are all writing to the same prompt when human readers are all trained to read that same writing by the same rules, aka when readers are "normed" Most classrooms, currently, don't operate this way. If there are 80 sections of first year writing being taught by anywhere from 40 - 60 instructors, then there are very often 40 - 60 different iterations of assignments, prompts, discussions, assigned readings. In a biology course, or an intro psychology course, for this kind of software to work, then all faculty would have to agree to use the same case studies, experiments, books, prompts, and other activities and resources that support and define the curriculum. That leads to a world where the auto-essay owners write the curriculum, reducing the role of faculty. For by writing the curriculum and choosing the prompts it does
become possible to tune the algorithms to fit the likely range of writing most students will submit.
The analysis in the Washington Post -- http://wapo.st/10vexAk -- penned by Doug Hesse, director of writing at the University of Denver, gets at the differences in those two visions: the automated curriculum and the craft of teaching. That said, I do think there are places where the software might be useful -- not in reading writing or providing high stakes assessment like placement or a course grade or an essay's final grade -- but in the opposite end of the spectrum, at writing that's very low stakes, just for practice. I can imagine, for example, that the tool might be more useful for certain kinds of writing to learn -- like helping novice readers learn how to summarize a passage, or writing an article abstract, or assessing a short answer quiz where the point is narrowly to see certain words and ideas. So to my mind, the software could be used to free up teachers to use more lower stakes writing more often, and to also give them time to focus more fully and carefully, in the way Hesse points out, on writing that's more substantive and important in the course.
archman 2 months ago

Common Core Standards... just another spin on No Child Left Behind. Worked out great in K-12... expect equally dismal results by incoporating it into Higher Education.
dale1 2 months ago in reply to archman
archman: Philosophically we probably agree, in the sense that common core standards are likely to (a) not be common due to variance in delivery and competence of teachers, external resources, parental involvement, etc., and (b) that states are unevenly funded and motivated to secure the resources necessary to at least partially ameliorate deficits in (a). Still, we must pay very close attention to what is happening at the K-12 level. Their outputs are our inputs, whether directly or indirectly through the employment system. We have to significantly enhance efforts to clearly articulate the standards we expect for entry-level college students. An article on the "other" higher ed online publication talks about this and is of critical importance for all in the postsecondary sector.
11239383 2 months ago
If we work this right, we can convert all classes to MOOCs and grade all papers by machine. Then we can close all but one college in each state and save the taxpayers billions. In some states we could run the same courses (content) for the next 50 years. I do not know why the RNP has not jumped on this one already. All seriousness aside...if the Common Core is implemented correctly and reading/writing activities are integrated across the curriculum in the spirit of disciplinary literacy, we are years from being able to program a computer to cross such boundaries...not that Pearson, ACT, or ETS will not beg to differ. Unfortunately there is far too much money associated with testing. As George Spache (a literacy god from years past) said..."If you want to get tenure write a book. If you want to get rich, write a test."
nunya 2 months ago
OF COURSE these English teachers reject the use of technology to grade essays..... they want/need jobs. Factory workers rejected the use of the assembly line. People who have jobs at stake reject the technology that makes them obsolete. I applaud the creation of software that can help give students automatic feedback, may grade better than an overworked English teacher, and that can ultimately aid students who want to grade their own writing. Gone are the days of begging your teacher "Have you read my essay yet?" Instant feedback is glorious. Will it replace higher order creation of work? No. Will it aid students who want instant feedback or need more help than an overworked teacher can give them? Absolutely. What an obvious political move by the Council. Shame on them for discouraging technology and advancement.
3rdtyrant 2 months ago in reply to nunya
This is the kind of crazed leap of logic that really sets this debate on its ear. Unlike some, many of those who object to roboticizing higher education do so because we are ethically interested in quality, rather than being the self-serving creatures that you label with such hebitudinous aplomb. To assert in your last sentence that this is a discouragement of technology and advancement is to assume that this is an advancement, or possibly that all technological whizzing is advancement. Certainly, E-rater would have a field day with your entry.
aicaiel 2 months ago in reply to nunya
The problem is that automated systems cannot "read" essays. These software programs only mark standardized patterns of word placement and cannot account for the many variables in syntax and usage (whatever 'rule' the program may use to grade a piece of writing, somewhere it may be perfectly appropriate to 'break' that rule). As an instructor of writing, I have taken a look at a few of these systems -- my motivations being the idea that I could be freed from the necessity of grading "mere" grammar and concentrate on the 'higher order' thinking displayed in student papers. (Don't laugh.) I very quickly realized the programs do not do a very good job of grading papers -- not marking passages that make no sense and marking as incorrect passages that are perfectly lucid and correct. In other words, these programs are useless in evaluating even the technical aspects of writing (perhaps, aside from checking spelling of individual words), much less "reading" what writing is really about -- cogency of thought, quality of argument, humor, wit, intelligence, originality, irony, all of which are employed in the attempt to reach other human beings, share experiences, change perceptions, and ultimately shape our concepts of reality (read, change reality). To echo a couple of the writers above, do we wish to communicate to students that writing is this kind of valuable mode of human communication or a useless and irrelevant enterprise, an exercise in futility that ultimately means nothing in that it will not touch or move or effect any other human being?
couchloc 2 months ago in reply to nunya
I wish someone would create a computing machine that will replace the work of business people pushing these ideas about education. (If you can do it for teachers, why not business employees as well?) Then I could just unplug it and be done with them.
nunya 2 months ago in reply to couchloc
are you kidding? What do you think calculators, adding machines, and THE INTERNET have done to business? Expect more with less. That is the way the world works.
1 person liked this.
mbelvadi 2 months ago in reply to nunya
You forgot the big two killer apps - word processors and spreadsheet software.
uflemm 2 months ago in reply to nunya
It's very interesting that the people who attack the authors of the study do not try to refute a single argument they make--it's all ad hominem: They are knee-jerkers; they are biased etc. (as if the people who peddle their software were NOT biased!) I would really like to see someone argue that software can, in fact, recognize irony, allusion, the coherence of an argument etc; i.e. prove that it really UNDERSTANDS what it's reading.
Bonegirl06 2 months ago in reply to nunya
I don't want instant feedback if it's going to be some machine-generated drivel that essentially does nothing to actually force me to have deeper ideas and to look at the issue from more angles. What feedback can a machine give besides grammar corrections? It can't actually read the essay and try to comprehend what I am trying to say. It has no knowledge of the topic that can help it give me suggestions or insights. I am willing to trade instant feedback for quality feedback.
Jack Dempsey 2 months ago in reply to nunya
Teachers don't reject all tech---just THIS concept and tech because it's worse than preposterous, just another tech industry gimmick to help along those elite who DO want to get rid of teachers (who want to be paid and are always griping). For example, what would such a machine say about your post? I don't know, but a decent professor would patiently show you A) the hackneyed assumption posed as observation ("they want jobs"), B) the simpering false sympathy of "overworked...teacher," and C) the equivocation between "technology and advancement" that is disproven every day in too many classrooms. And he/she would probably advise you to focus your writing on subjects about which you know something.
hoodlib 2 months ago
I imagine the "Lost in Space" robot, sitting at a table, with a red marker in hand: "Warning Will Robinson! Split infinitive!"
reineke 2 months ago
My university gives me access to ETS's machine scoring engine, E-Rater (http://www.ets.org/erater/abou..., I used E-Rater on three essays. It functioned well for a paper to which I had given a "C" previously. This essay was written by a firstyear student in an liberal arts core class and E-Rater nailed her errors. E-Rater functioned marginally well for a "B" essay in the same class. Unfortunately, it misidentified 3-4 features of the essay as problematic. Given that students work on the mechanics of their writing in that course, their confidence in their developing abilities to analyze their own \writing and that of their classmates (through peer review) will be undercut if I have to tell the students that E-Rater is wrong on multiple features of a given essay. Most problematic was the E-Rater's assessment of an essay written by an A student in a upper-level philosophy class. Her paper was outstanding and demonstrated mature skills in critical thinking and the mechanics of writing. E-Rater should not have flagged anything in her essay; however, numerous errors were identified by E-Rater. The upshot of this little experiment is that E-Rater is capable of identifying writing problems in the work of first-year students who are struggling to demonstrate college-level writing skills. Students who need tips on how to enhance their college-level writing skills (B students) will not be well-served by this machine. And E-Rater is completely incapable of assessing strong college-level writing skills. Admittedly, my sample-size was tiny. But it convinced me that E-Rater is much more trouble than it is worth. My old-fashioned grading, to which I devote
my weekends during the academic year, best serves the development of critical writing skills among my students.
Reythia 2 months ago in reply to reineke
Makes sense. And it probably explains why a computerized system is competent to sort through 1-2 paragraph answers by middle-schoolers on a test, too. After all, there's a lot less complexity there. It might be an okay tool for some things, but I can't see how it'd be useful to grade and teach more competent students on complex subjects.
22051084 2 months ago

Beep, Beep. . .
jamesebryan 2 months ago
If computers are so good at assessing the quality of writing why does Microsoft Word constantly annoy me with inane "advice" of the utmost only-one-right-way-to-dothings-can't-see-the-forest-for-the-trees sort? Apparently the eleventh commandment is "Thou shalt never use the passive voice under any circumstances whatsoever, or else thou shalt be afflicted with squiggly green underscoring." (Edited by author 2 months ago)
mbelvadi 2 months ago in reply to jamesebryan
No one is suggesting that schools use Microsoft Word to grade papers. Just because one software program isn't good at something is not a rational argument to claim that no software could be created that could be good at something.
jamesebryan 2 months ago in reply to mbelvadi
Perhaps I should be like Foghorn Leghorn, and make it clear that "That's a joke, son." I will also add that since this is a similar feature of a software program that has been the industry standard for many years, you would think that the programmers could have made something better of it by now, and yet they haven't perfected it yet, and so mine isn't an entirely unrelated point to make.
rhaswell 2 months ago
In the use of automated computer response to student essays, the distinction between high-stakes and low-stakes writing is crucial. The NCTE policy statement is about the former. For another position on high-stakes machine scoring of student essays, see the petition at humanreaders.org. http://humanreaders.org/petiti... It is based on empirical research, written by independent scholars, and signed by nearly 4,000 professionals, many of them not writing teachers. "There's nothing we can do to stop it"? That's what the paint industry said years ago about lead-based paint. Machine scoring in high-stakes writing assessment is a commercial product. It can be stopped by people not buying the product.
bekka_alice 2 months ago
Would I be able to express my writing ability by introducing sentences such as "The duck, while insane, was a brilliant and furtive conspirator. Therefore, the squid was a bicycle." -? If so, I have mixed feelings about this, and would like to submit a number of writing samples for my own puerile entertainment value. On the other hand, I can't help feeling that if we short our resources and find these workarounds to the point at which it is detrimental to a deeper level of learning from which our students would benefit, that we are in fact cheating (the students) on the test.
drew loewe 2 months ago
"There's nothing we can do to stop it," [Shermis] said. "It's whether we can shape it or not." See Evgeny Morozov's concept of "technological defeatism." I want to shape it-into a ball I can tee up and hit 300 yards away from my classroom and students, at least for anything more than a low-stakes or draft writing event.
shakesguy 2 months ago
A better use for computers is to semi-automate grading. I recently produced a platform consisting of a set of Word macros and inserted documents to provide stronger feedback on student papers.This includes providing tallies of common errors and linking to further examples and explanations. I think the process is fairly intuitive, but an actual instructor still has to read the paper and mark it. It's available here: http://wcl.flintbox.com/public...
Keith Williams 2 months ago
If Jeffrey Steedle actually said that the grading of the written portion of the CLA is "almost excusively" [sic] automated, then several points should be deducted from his score.
Robot Journalist writes a better story than human sports reporter

shared: 57x Use keys to navigate Next Join The Next Web Conference USA in New York on October 1 & 2! Martin Bryant Contact Author 18 April 2011 Updated at 20:12 CEST
24 Comments

Send anonymous tip Just over a year ago we discussed on how Robot journalists (actually automated software designed to turn facts into readable copy) stood to threaten the jobs of writers in fields like sports reporting. It looks like the results are actually starting to become alarmingly good. NPRs All Things Considered reports how writers at Gawkers sports site Deadspin assumed that a story written on GW Sports about a baseball game had been written by a robot due to its failure to mention that in the game, a player had pitched a perfect game in the NCAA for the first time in almost a decade pretty big thing to miss. It turned out that a human had in fact written that report, but the creators of newswriting software Narrative Science decided to take this as a challenge could its software write a better account of the game? It turns out that yes, it could. This is what Deadspin was sent from the robo-journo: Tuesday was a great day for W. Roberts, as the junior pitcher threw a perfect game to carry Virginia to a 2-0 victory over George Washington at Davenport Field. Twenty-seven Colonials came to the plate and the Virginia pitcher vanquished them all, pitching a perfect game. He struck out 10 batters while recording his momentous feat. Roberts got Ryan Thomas to ground out for the final out of the game. Tom Gately came up short on the rubber for the Colonials, recording a loss. He went three innings, walked two, struck out one, and allowed two runs. The Cavaliers went up for good in the fourth, scoring two runs on a fielders choice and a balk. Impressive, eh? It may not have much flair or personality, but its readable and accurate. Sports reporters should have real cause for concern, it seems. Its easy to see how other data-heavy fields, such as financial reporting, may also see an influx of robot writers in the future. I just hope it takes them a little longer to perfect the technology reporting algorithm that might just be cutting a little too close to home for me UPDATE: Its worth noting that Matters of Varying Significance notes that the original report from GW Sports was written by a human but not a trained sports journalist. Thats an important distinction that would explain the omission of an important piece of news from the report. Still, as a factual piece of writing the robot still did an impressive job.
Welcome To The Era Of Robot Bloggers, Journalists And Writers

Would you like this (or any) content less if a robot had written it?
The media is filled with news items abut how robots and algorithms are taking the work away from real, kind and personable human beings. That is one train of thought (personally, I'm offering a different perspective over on my We, Robots blog, which looks at augmentation over automation of all things robotics, 3D printing, telepresence and more). It's scary to think that one day, you may read an article in a magazine or newspaper or online that had no human intervention. No humanity, no personalized style and more. That future is here. Today. It's already happening. What robot journalism and blogging looks like. Los Angeles Times journalist, Ken Schwencke, woke up one morning, started brewing a cup of coffee, picked up the daily newspaper and saw an article with his byline on it that he never wrote. You see, Schwencke is spending some of his time writing articles, but he also spends a lot of his time writing algorithms and code. In the article, 'Roboreporter' computer program raises questions about future of journalists, his story gets told: "Instead of personally composing the pieces, Schwencke developed a set of stepby-step instructions that can take a stream of data -- this particular algorithm works with earthquake statistics, since he lives in California -- compile the data into a predetermined structure, then format it for publication. His fingers never have to touch a keyboard; he doesn't have to look at a computer screen. He can be sleeping soundly when the story writes itself." It's easier than you think. Schwencke surmises that most readers would never even pick up on the automation of content. It's a simple piece of journalism that can be constructed, much in the same way we used to play Mad Libs when we were kids. People not well-versed in media and how things work in journalism are often curious as to how these in-depth obituaries appear in a matter of minutes after someone famous passes. What they fail to realize is that a lot of that content was written long before the death and the exact details (date, time, cause, etc...) are simply filled in. This is done with concert reviews as well and the music journalist simply adds some additional color and commentary but has created a framework long before the lights go down on a show. If these algorithms get better, is too far-fetched to imagine a world where sentiment coupled with specific direction could create copious amounts of the text-based content we consume. What about ethics? The prevailing wisdom is this: if newspapers, magazines and online channels use robots or algorithms to create content they have to do so with honesty and transparency. Publishers will have to inform readers when the content is being written by a human or an algorithm, but is that enough? Where do we draw the line? Do you even care? Is it critical that human beings actually write up an article about earthquakes, an obituary or something related to the weather? Would we prefer that these highly-skilled journalists spend their time on pieces that require more than just regurgitating data? Would we not prefer that they spend their time helping the mass populous better understand the full breadth of perspective and discourse? It's somewhat amazing that we haven't arrived at this conclusion sooner. The true power.
The true power in this is not how computers, algorithms and robots can now replace human writers. The true power is in how computers, algorithms and robots can now free up these human writers to do the more important work that our society requires of them. So again, I ask you, would you mind if a robot was creating the content you need to consume?
This Article Was Written By A Robot

by Kieran Griffith 0 Comments and 0 Reactions There is no one definition of robot which satisfies everyone and most people have their own. For example Joseph Engelberger, a pioneer in industrial robotics, once remarked: I cant define a robot, but I know one when I see one. According to the Encyclopaedia Britannica a robot is any automatically operated machine that replaces human effort, though it may not resemble human beings in appearance or perform functions in a humanlike manner. The Merriam-Webster dictionary describes a robot as a machine that looks like a human being and performs various complex acts (as walking or talking) of a human being, or a device that automatically performs complicated often repetitive tasks, or a mechanism guided by automatic controls.
ASIMO A lot of people find it disturbing that humans are becoming more like robots while, at the same time, robots are also becoming more like humans. Many are philosophizing about what humans will become after we modify ourselves through genetic engineering or by implanting AI components into all parts of the body to improve our physical and mental abilities. There are concerns that such modifications will pervert us in some way and should perhaps be avoided. This is causing a lot of anxiety and some are warning that humans will stop being actually human. I, for one, fail to see what the fuss is all about. There is a simple answer as to why the prospect of artificial human modification should not be a significant cause for concern.
Humans already are robots. One of my favorite quotes by Aubrey de Grey is the human body is a very complex machine. Yes, we are complex, self replicating and self-repairing, but we are machines never-the-less. Look at yourself, look at your hands they are a small part of an extremely complex apparatus that is able to accomplish all kinds of sophisticated actions. Vertebrate life forms are the most complex apparatus ever developed and no definition of a robot says that it has to be man made. So what if the current life forms were created by the trialand-error process called evolution for over 4.7 billion years?! It is a given that a person does look like a human and can replace other humans efforts and is able to perform various complex and often repetitive acts (such as walking and talking) and finally is guided by automatic controls (in our nervous system). The human being is definitely not a perfect contraption for any mechanism can always be improved. However, the natural process of evolution that has updated humans until the start of the industrial revolution is no longer an option. Civilization needs to find a new way to improve their design. And just as humanity is transcending evolution the technology to modify the human machine will become available.
Original Ford Model T The technology to maintain the human machine indefinitely in roughly its built condition will be fully available with the advent of regenerative medicine, as being developed by Aubrey de Grey and the SENS Foundation. In may take 20 or 30 years (or more) but the technological singularity (also estimated to take roughly another 20 or 30 years) will provide us with another way to improve the hardware we run on and build the next generation of humachines to be better then they would be by (evolutionary) chance.
Think of your body as an old car you can keep it running in perfect condition indefinitely, for as long as you do the proper maintenance (i.e. regenerative medicine). Just like people who have an antique and perfectly working Ford Model T. Or you might want to put in a more powerful engine, an automatic gearbox and an air conditioning unit, so you can drive faster and more more comfortably. You can even turn it into a hot-rod muscle car for street racing or to impress the girls
Ford Model T Hot Rod Why would anyone worry about the option of modifying a robot to be a better robot? Humans are always updating the programming of our biological CPU (through education) from the moment we are born. You are updating your programming even now by reading this article. Further artificial mental and physical modifications that will be an option after the singularity will just be another hardware adjustment, not very much different from the one above. Some people want to keep their cars as if they just came off the assembly line. Others may let them wear out and go on to the scrap yard. But in my opinion, most will want to install parts that allow for better durability, performance, speed and comfort. I can now say that I know a robot when I see one. And that includes when I am looking at the mirror. About the Author: Kieran Griffith is a voluntary consultant to the SENS Foundation for developing medical techniques that extend lifespan indefinitely. He has degrees in psychology, the Humanities and Space Science, and is planning a future career in the field of commercial spaceflight.
ools
Can a Robot Write Your Content?

Posted on December 6, 2012 by Ilan Nass | 6 comments The views of contributors are their own, and not necessarily those of SEJ.

inShare24
If theres one thing that SEOs know cant be done by automated software, its content creation. For years now, the three most important rules in search marketing have been content, content, and content. Multimillion dollar corporations like Demand Media have sprung up just to feed more content to the insatiable beast that is the internet. Even black hat marketers have given up on cheap spun content strategies that no longer work like they used to.
Content isnt just required for our own websites and blogs, its necessary for guest blogging, blog network creation and social media marketing as well. So content is king even still, and we pay big bucks to have readable content produced everyday for all our search marketing needs because, of course, only humans can do that right? Nay! As a for instance, Chicago-based startup Narrative Science recently developed a technology aptly named Quill that is capable of reading charts and data and producing actual, real, readable content through its algorithm that mimics human writing. Customers of Quill can choose the tone based on individual need. Tweets, headlines, recaps, short and long form articles are not a problem for this ideal and omnicompetent wordsmith. It doesnt have feelings, is a cheap hire (at $10/500 word article), wont take Facebook breaks and is more than thrilled to be browsing through sets of data and pages of spreadsheets. Niche news services like Forbes.com already use the technology to transform their corporate earning statements into easily understandable headlines and paragraphs. So far, most of Narrative Sciences clients are finance or sports based, focusing on data and statistics. So this begs the question; If artificially intelligent journalists are more robust and less expensive at combing through data than their human counterparts, why not use robots for content that, in a larger context, will never be shortlisted for a Pulitzer anyway? Readers of wires and dailies hardly expect New Yorker style pieces with metaphors and prose. These mechanical writers churn out concise and accurate content thats all. That is their designed function. The same technology that Narrative Science developed could feasibly be used by content marketers and SEOs to churn out quality content rapidly and cheaply. The big question is; How long before someone actually builds an SEO friendly tool that will?
Image credit: Robot writer courtesy Aleksandr Bedrin Fotolia.com
Can an Algorithm Write a Better News Story Than a Human Reporter?

By Steven Levy 04.24.12 4:46 PM
Illustration: Mark Allen Miller Had Narrative Science a company that trains computers to write news storiescreated this piece, it probably would not mention that the companys Chicago headquarters lie only a long baseball toss from the Tribune newspaper building. Nor would it dwell on the fact that this potentially job-killing technology was incubated in part at Northwesterns Medill School of
Journalism, Media, Integrated Marketing Communications. Those ironies are obvious to a human. But not to a computer.
Also in this issue

The Man Who Makes the Future How to Spot the Future 8 Visionaries on How They Spot the Future
At least not yet. For now consider this: Every 30 seconds or so, the algorithmic bull pen of Narrative Science, a 30-person company occupying a large room on the fringes of the Chicago Loop, extrudes a story whose very byline is a question of philosophical inquiry. The computer-written product could be a pennant-waving second-half update of a Big Ten basketball contest, a sober preview of a corporate earnings statement, or a blithe summary of the presidential horse race drawn from Twitter posts. The articles run on the websites of respected publishers like Forbes, as well as other Internet media powers (many of which are keeping their identities private). Niche news services hire Narrative Science to write updates for their subscribers, be they sports fans, small-cap investors, or fast-food franchise owners. And the articles dont read like robots wrote them: Friona fell 10-8 to Boys Ranch in five innings on Monday at Friona despite racking up seven hits and eight runs. Friona was led by a flawless day at the dish by Hunter Sundre, who went 2-2 against Boys Ranch pitching. Sundre singled in the third inning and tripled in the fourth inning Friona piled up the steals, swiping eight bags in all OK, its not Roger Angell. But the grandparents of a Little Leaguer would find this game summaryavailable on the web even before the two teams finished shaking handsas welcome as anything on the sports pages. Narrative Sciences algorithms built the article using pitch-by-pitch game data that parents entered into an iPhone app called GameChanger. Last year the software produced nearly 400,000 accounts of Little League games. This year that number is expected to top 1.5 million.
Narrative Sciences CTO and cofounder, Kristian Hammond, works in a small office just a few feet away from the buzz of coders and engineers. To Hammond, these stories are only the first step toward what will eventually become a news universe dominated by computergenerated stories. How dominant? Last year at a small conference of journalists and technologists, I asked Hammond to predict what percentage of news would be written by computers in 15 years. At first he tried to duck the question, but with some prodding he sighed and gave in: More than 90 percent. Thats when I decided to write this article, hoping to finish it before being scooped by a MacBook Air. Hammond assures me I have nothing to worry about. This robonews tsunami, he insists, will not wash away the remaining human reporters who still collect paychecks. Instead the universe of newswriting will expand dramatically, as computers mine vast troves of data to produce ultracheap, totally readable accounts of events, trends, and developments that no journalist is currently covering. Thats not to say that computer-generated stories will remain in the margins, limited to producing more and more Little League write-ups and formulaic earnings previews. Hammond was recently asked for his reaction to a prediction that a computer would win a Pulitzer Prize within 20 years. He disagreed. It would happen, he said, in five. Hammond was raised in Utah, where his archaeologist dad taught at a state university. He grew up thinking hed become a lawyer. But in the late 1980s, as an undergraduate at Yale, he fell under the sway of Roger Schank, a renowned artificial intelligence researcher and chair of the computer science department. After earning a doctorate in computer science, Hammond was hired by the University of Chicago to lead a new AI lab. While there, in the mid-1990s, he created a system that tracked users reading and writing and then recommended relevant documents. Hammond built a small company around that technology, which he later sold. By that time, he had moved to Northwestern University, becoming codirector of its Intelligent Information Laboratory. In 2009, Hammond and his colleague Larry Birnbaum taught a class at Medill that included both programmers and prospective journalists. They encouraged their students to create a system that could transform data into prose stories. One of the students in the class was a stringer for the Tribune who covered high school sports; he and two other journalism students were paired with a computer science student. Their prototype software, Stats Monkey, collected box scores and play-by-play data to spit out credible accounts of college baseball games. At the end of the semester, the class participated in a demo day, where students presented their projects to a roomful of executives from the likes of ESPN, Hearst, and the Tribune. The Stats Monkey presentation was particularly impressive. They put a box score and play-byplay into the program, and in something close to 12 seconds it drew examples from 40 years of Major League history, wrote a game account, located the best picture, and wrote a caption, recalls the Medill dean, John Lavine. Stuart Frankel, a former DoubleClick executive who left the online advertising network after Google purchased it in 2008, was among the guests that day. When these guys did the presentation, the air in the room changed, he said. But it was still just a piece of software that wrote stories about baseball gamesvery limited. Frankel followed up with Hammond and Birnbaum. Could this system create any kind of story, using any kind of data? Could it
create stories good enough that people would pay to read them? The answers were positive enough to convince him that there was a really big, exciting potential business here, he says. The trio founded Narrative Science with Frankel as CEO in 2010. The startups first customer was a TV network for the Big Ten college sports conference. The companys algorithm would write stories on thousands of Big Ten sporting events in near-real time; its accounts of football games updated after every quarter. Narrative Science also got assigned the womens softball beat, where it became the countrys most prolific chronicler of that sport. But not long after the contract began, a slight problem emerged: The stories tended to focus on the victors. When a Big Ten team got whipped by an out-of-conference rival, the resulting write-ups could be downright humiliating. Conference officials asked Narrative Science to find a way for the stories to praise the performances of the Big Ten players even when they lost. A human journalist might have blanched at the request, but Narrative Sciences engineers saw no problem in tweaking the softwares parametershacking it to make it write more like a hack. Likewise, when the company began covering Little League games, it quickly understood that parents didnt want to read about their kids errors. So the algorithmic accounts of those matchups ignore dropped fly balls and focus on the heroics. I asked Kristian Hammond what percentage of news would be written by computers in 15 years. More than 90 percent. Narrative Sciences writing engine requires several steps. First, it must amass high-quality data. Thats why finance and sports are such natural subjects: Both involve the fluctuations of numbersearnings per share, stock swings, ERAs, RBI. And stats geeks are always creating new data that can enrich a story. Baseball fans, for instance, have created models that calculate the odds of a teams victory in every situation as the game progresses. So if something happens during one at-bat that suddenly changes the odds of victory from say, 40 percent to 60 percent, the algorithm can be programmed to highlight that pivotal play as the most dramatic moment of the game thus far. Then the algorithms must fit that data into some broader understanding of the subject matter. (For instance, they must know that the team with the highest number of runs is declared the winner of a baseball game.) So Narrative Sciences engineers program a set of rules that govern each subject, be it corporate earnings or a sporting event. But how to turn that analysis into prose? The company has hired a team of meta-writers, trained journalists who have built a set of templates. They work with the engineers to coach the computers to identify various angles from the data. Who won the game? Was it a come-from-behind victory or a blowout? Did one player have a fantastic day at the plate? The algorithm considers context and information from other databases as well: Did a losing streak end? Then comes the structure. Most news stories, particularly about subjects like sports or finance, hew to a pretty predictable formula, and so its a relatively simple matter for the meta-writers to create a framework for the articles. To construct sentences, the algorithms use vocabulary compiled by the meta-writers. (For baseball, the meta-writers seem to have relied heavily on famed early-20th-century sports columnist Ring Lardner. People are always whacking home runs, swiping bags, tallying runs, and stepping up to the dish.) The company calls its finished product the narrative.
Occasionally the algorithms will produce a misstep, like a story stating that a pinch hitter who usually bats only once per gamewent two for six. But such errors are rare. Numbers dont get misquoted. Even when databases provide faulty information, Hammond says, Narrative Sciences algorithms are trained to catch the error. If a company has a 600 percent rise in profits from quarter to quarter, itll say, Something is wrong here, Hammond says. People ask for examples of wonderful, humorous gaffes, and we dont have any. Forbes Media chief products officer Lewis Dvorkin says hes impressed but not surprised that, in almost every case, his cyber-stringers nail the essence of the company theyre reporting on. Major screwups are not unheard-of with flesh-and-blood scribes, but Dvorkin hasnt heard any complaints about the automated reports. Not a one, he says. (The pieces on Forbes.com include an explanation that Narrative Science, through its proprietary artificial intelligence platform, transforms data into stories and insights.) The Narrative Science team also lets clients customize the tone of the stories. You can get anything, from something that sounds like a breathless financial reporter screaming from a trading floor to a dry sell-side researcher pedantically walking you through it, says Jonathan Morris, COO of a financial analysis firm called Data Explorers, which set up a securities newswire using Narrative Science technology. (Morris ordered up the tone of a well-educated, straightforward financial newswire journalist.) Other clients favor bloggy snarkiness. Its no more difficult to write an irreverent story than it is to write a straightforward, AP-style story, says Larry Adams, Narrative Sciences VP of product. We could cover the stock market in the style of Mike Royko. Once Narrative Science had mastered the art of telling sports and finance stories, the company realized that it could produce much more than journalism. Indeed, anyone who needed to translate and explain large sets of data could benefit from its services. Requests poured in from people who were buried in spreadsheets and charts. It turned out that those people would pay to convert all that confusing information into a couple of readable paragraphs that hit the key points. Narrative Science, it so happened, was well placed to accommodate such demands. When the company was just getting started, meta-writers had to painstakingly educate the system every time it tackled a new subject. But before long they developed a platform that made it easier for the algorithm to learn about new domains. For instance, one of the meta-writers decided to build a story-writing machine that would produce articles about the best restaurants in a given city. Using a database of restaurant reviews, she was able to quickly teach the software how to identify the relevant components (high survey grades, good service, delicious food, a quote from a happy customer) and feed in some relevant phrases. In the space of a few hours she had a bot that could churn out an endless supply of chirpy little articles like The Best Italian Restaurants in Atlanta or Great Sushi in Milwaukee. (Narrative Sciences main rival in automated story creation, a North Carolina company founded as Stat Sheet, has broadened its mission in similar fashion. The company cant compete with Narrative Sciences Medill pedigree and so has assumed the role of a feisty tabloid in a two-paper town. It too got its start in sports, writing accounts of Major League and big-college games as well as creating a trash-talk generator called StatSmack. After realizing that turning data into stories presented an opportunity far larger than sports, the company changed its name to Automated Insights. I used to put limitations on what we do,
assuming our stories would be specific to data-rich industries, founder Robbie Allen says. Now I think ultimately the sky is the limit.) Users can customize the tone of any storyfrom breathless financial reporter to dry analyst. And the subject matter keeps getting more diverse. Narrative Science was hired by a fast-food company to write a monthly report for its franchise operators that analyzes sales figures, compares them to regional peers, and suggests particular menu items to push. Whats more, the low cost of transforming data into stories makes it practical to write even for an audience of one. Narrative Science is looking into producing personalized 401(k) financial reports and synopses of World of Warcraft sessionsplayers could get a recap after a big raid that would read as if an embedded journalist had accompanied their guild. The Internet generates more numbers than anything that weve ever seen. And this is a company that turns numbers into words, says former DoubleClick CEO David Rosenblatt, who sits on Narrative Sciences board. Narrative Science needs to exist. The journalism might be only the sizzlethe steak might be management reports. For now, though, journalism remains at the companys core. And like any cub reporter, Narrative Science has dreams of gloryto identify and break big stories. To do that, it will have to invest in sophisticated machine-learning and data-mining technologies. It will also have to get deeper into the business of understanding natural language, which would allow it to access information and events that cant be expressed in a spreadsheet. It already does a little of that. In the financial world, were reading headlines, Hammond says. We can identify if some companys stock gets upgraded or downgraded, somebody gets fired or hired, somebodys thinking of a merger, and we know the relationship between those events and a stock price. Hammond would like to see his companys college sports stories include nonstatistical information like player injuries or legal problems. But even if Narrative Science never does learn to produce Pulitzer-level scoops with the icy linguistic precision of Joan Didion, it will still capitalize on the fact that more and more of our lives and our world is being converted into data. For example, over the past few years, Major League Baseball has spent millions of dollars to install an elaborate system of hi-res cameras and powerful sensors to measure nearly every event thats occurring on its fields: the velocities and trajectories of pitches, tracked to fractions of inches. Where the fielders stand at any given moment. How far the shortstop moves to dive for a ground ball. Sometimes the real story of the game may lie within that data. Maybe the manager failed to detect that a pitcher was showing signs of exhaustion several batters before an opponents game-winning hit. Maybe a shortstops extended reach prevented six hits. This is stuff that even an experienced beat writer might miss. But not an algorithm. Hammond believes that as Narrative Science grows, its stories will go higher up the journalism food chainfrom commodity news to explanatory journalism and, ultimately, detailed long-form articles. Maybe at some point, humans and algorithms will collaborate, with each partner playing to its strength. Computers, with their flawless memories and ability to access data, might act as legmen to human writers. Or vice versa, human reporters might interview subjects and pick up stray detailsand then send them to a computer that writes it all up. As the computers get more accomplished and have access to more and more data, their limitations as storytellers will fall away. It might take a while, but eventually even a story like this one could be produced without, well, me. Humans are unbelievably rich and complex,
but they are machines, Hammond says. In 20 years, there will be no area in which Narrative Science doesnt write stories. For now, however, Hammond tries to reassure journalists that hes not trying to kick them when theyre down. He tells a story about a party he attended with his wife, whos the marketing director at Chicagos fabled Second City improv club. He found himself in conversation with a well-known local theater critic, who asked about Hammonds business. As Hammond explained what he did, the critic became agitated. Times are tough enough in journalism, he said, and now youre going to replace writers with robots? I just looked at him, Hammond recalls, and asked him: Have you ever seen a reporter at a Little League game? Thats the most important thing about us. Nobody has lost a single job because of us. At least not yet. Senior writer Steven Levy (steven_levy@wired.com) interviewed Amazons Jeff Bezos for issue 19.12.

Man or Machine Robot Writing

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Man or Machine Robot Writing

Uploaded by

Copyright:

Available Formats

Man or machine - can robots really write novels?

Home About Projects Resources Satire

Robots Writing the News Who Cares?

News Business Sports Entertainment Lifestyle Health Travel Technology

Robo-reporter raises questions about future of journalists

By Jesse M. Kelly, Postmedia News March 26, 2013

English Teachers Reject Use of Robots to Grade Student Writing

dsarma 2 months ago

jranelli 2 months ago

wclibrary 2 months ago

Rubrics encourage robo-grading.

deliajones 2 months ago in reply to wclibrary

fisher56 2 months ago

raymond_j_ritchie 2 months ago in reply to fisher56

harrybythesea 1 month ago in reply to fisher56

fisher56 1 month ago in reply to harrybythesea

Jack Dempsey 2 months ago

jranelli 2 months ago in reply to Jack Dempsey

prof291 2 months ago in reply to Jack Dempsey

htinberg 2 months ago

dale1 2 months ago in reply to htinberg

deliajones 2 months ago in reply to htinberg

jwr12 2 months ago in reply to deliajones

nickcarbone 2 months ago

archman 2 months ago

dale1 2 months ago in reply to archman

6 people liked this.

11239383 2 months ago

nunya 2 months ago

3rdtyrant 2 months ago in reply to nunya

aicaiel 2 months ago in reply to nunya

couchloc 2 months ago in reply to nunya

14 people liked this.

nunya 2 months ago in reply to couchloc

mbelvadi 2 months ago in reply to nunya

uflemm 2 months ago in reply to nunya

Bonegirl06 2 months ago in reply to nunya

Jack Dempsey 2 months ago in reply to nunya

hoodlib 2 months ago

reineke 2 months ago

23 people liked this.

Reythia 2 months ago in reply to reineke

22051084 2 months ago

jamesebryan 2 months ago

mbelvadi 2 months ago in reply to jamesebryan

jamesebryan 2 months ago in reply to mbelvadi

rhaswell 2 months ago

bekka_alice 2 months ago

drew loewe 2 months ago

shakesguy 2 months ago

Keith Williams 2 months ago

Robot Journalist writes a better story than human sports reporter

Welcome To The Era Of Robot Bloggers, Journalists And Writers

This Article Was Written By A Robot

Can a Robot Write Your Content?

Image credit: Robot writer courtesy Aleksandr Bedrin Fotolia.com

Can an Algorithm Write a Better News Story Than a Human Reporter?

By Steven Levy 04.24.12 4:46 PM

Also in this issue

You might also like