And It Don't Stop « PREV | NEXT »: Hello, Police? I Want to Report a Theory

April 12, 2010

Needed: A New Role For Criticism

The entertainment industry is lining up against plans to create a futures market on movie box office grosses. The scheme would establish a market where investors could bet on how much money movies take in at the box office.

A week after expressing its opposition against the launch of two new movie futures exchanges, the Motion Picture Assn. of America has assembled a coalition of entertainment industry representatives to urge the Commodity Futures Trading Commission to delay its approval of the ventures.
Meanwhile, researchers at HP Labs report that they have been monitoring Twitter comments about movies and can predict box office sales with 90+ percent accuracy. They...

started by monitoring movie mentions in 2.9 million tweets from 1.2 million users over three months. These included 24 movies in all, ranging from Avatar to Twilight: New Moon.

Then they took two different approaches, dealing with two very different performance metrics: the first weekend performance, which is largely built on buzz and the second weekend performance, which is largely built whether people actually like the movie.

To predict first weekend performance, they built a computer model, which factored in two variables: the rate of tweets around the release date and the number of theaters its released in. Lo and behold, that model was 97.3% accurate in predicting opening weekend box office. By contrast, the Hollywood Stock Exchange, which has been the gold standard for opening box-office predictions, had a 96.5% accuracy.

Meanwhile, to predict second-weekend performance, the authors created a ratio of positive tweets to negative ones. Then they blended that with the Tweet rate metric in another prediction algorithm. This time, the method was 94% accurate.

Do box office grosses equal popularity? Not quality, surely. What, then? America is obsessed with movie box office grosses, even though those numbers are considered bogus. By Saturday night, stories about weekend box office numbers are already washing over the web.
How do they know how many people bought tickets for Sunday when it's still Saturday? They don't. But the BO horse race has become so important that if a movie doesn't hit big its first few days (as "confirmed" by BO numbers) it doesn't make it much past the second weekend. So this report card is crucial to the bottom line, even if the projections are fantasy.

And reviews? Critics panned Clash of the Titans but it still grossed $60+ million on its opening weekend. More and more movies seem to be critic-proof. And movies showered in acclaim, such as Hurt Locker, can win the industry's highest honors and critical acclaim and no one sees them (after nine months in release and a starburst of Oscar glory, the film still hasn't sold more than $16 million in tickets).

Many online editors report that reviews don't pull traffic on their sites. A couple have told me that reader surveys rank reviews well below other content. Variety editors said in dropping their staff movie critics a few weeks ago that they doubted readers would notice. Sure some will. But in the current economic model, for Variety's purposes, its editors are right.

Then there's this chart from Forrester Research:

People are saying online that they would rather hear from their "friends" than critics when they're making choices about the culture and information with which they want to engage. And Washington Post media critic Howard Kurtz weighs in this morning by wondering if the days of the newspaper critic are over.

So where does all this leave reviews? And professional critics? Perhaps we need to refine the definition of the purpose/place/standing of reviews. Kurtz suggests that:

The notion that the blog culture has rendered critics redundant is ultimately an argument against specialists. And that debate stretches from the world of culture to the nature of journalism itself -- and whether its practitioners, who poke and probe on the public's behalf, have outlived their usefulness.
He's wrong. It isn't an argument against specialists, it's an argument against expertise. Americans glorify the voice of the everyman, even when/if the everyman has nothing to say, or hasn't spent more than a second thinking about something. We distrust experts because they presumably "know something" the rest of us don't. Experts have lower approval ratings than Congress.

So how is it that critics and experts got such a bad rep? Partly I think it's because people think every time an expert is proven wrong on something, it's proof they should be dismissed. But infallibility is different than expertise. Is it that people aren't sophisticated enough to see the difference? I don't have an answer, but it seems a question worth pondering...
April 12, 2010 10:47 AM | | Comments (2)


Experts inherently cut against the democratic impulse of our society. Democracy liberates, but it also levels. Experts, who by definition create a kind of hierarchy and authority, don't resonate in that kind of context.

I wouldn't bet my reputation on it, since I read the HP authors' paper for about 5 minutes, but I think the "97.3% accurate" figure is nonsense. In the paper itself, the authors *never* claim the regression model can make predictions that are "97.3% accurate" -- they claim an adjusted R-squared of 0.973, which is a different thing. That number tells you how accurate the regression model was when applied to the data the model was developed on. Using this number as a measure of accuracy is a bit like saying "I have a model that's 100% accurate because I inputted all the correct answers after the fact."

Leave a comment


Creative Commons License
This weblog is licensed under a Creative Commons License.


    ARTicles ARTicles is a project of 
    the National Arts Journalism Program, an association of some 500 journalists in the United States. Our group blog is a place for arts and cultural journalists to share ideas and information, to celebrate what we do, and to make the case for its continuing value. ARTicles is edited by Laura Collins-Hughes. To contact her, click here.

    ARTicles Bloggers Meet our bloggers: Sasha Anawalt, MJ Andersen, Alicia Anstead, Laura Bleiberg, Larry Blumenfeld, Jeanne Carstensen, Robert Christgau, Laura Collins-Hughes, Thomas Conner, Lily Tung Crystal, Richard Goldstein, Patti Hartigan, Glenn Kenny, Wendy Lesser, Ruth Lopez, Nancy Malitz, Douglas McLennan, Tom Moon, Abe Peck, Peter Plagens, John Rockwell, Werner Trieschmann, Lesley Valdes and Douglas Wolk. more

    NAJP NAJP is America's largest organization dedicated to the advancement of arts and cultural journalism. The NAJP has produced research, publications and discussions and works to bring together journalists, artists, news executives, cultural organization administrators, funders and others concerned with arts and culture in America today. more

    Join NAJP Join America's largest organization of arts journalists. Here's how more

see all archives


Recent Comments