Monday, October 01, 2007

The bigger the better, for movie critics

Using the same 2005 movie roster as was utilized in previous posts on the superiority of amateur movie mavens to professional critics in predicting box office success, I accounted for a movie's budget. The thought is that since critics are likely to be more forgiving of low-budget productions than plebians are, if we take away this self-imposed handicap, movie critics will fare better.

Critics are not meaningfully biased toward higher budget movies (the correlation is a statistically insignificant .07). But mavens are hardly anymore so (with a correlation of .16, again outside statistical significance at 95% confidence). Given this negligible relationship, it is not surprising that taking budget into account only increases the relationship between critical reviews and box office performance from .30 to .33.

Interestingly, though, performance varies between critics and mavens depending on production costs. Contrary to what I thought would be the case, critics actually do much in the high-budget range than anywhere else. They even perform better than mavens. But they do so horribly in the middle- and especially low-end range, that overall mavens are better.

Breaking the roster into thirds based on budget, the review-box office relationship is as follows, for critics and mavens:

High-budget: .60, .55
Medium-budget: .24, .38
Low-budget: .08, .30

Both get much better as budgets increase, undercutting my earlier supposition that critical reviews are of little value from the perspective of determining what movies will become blockbusters by introducing a huge caveat. When it comes to big dollar productions, they share the public's tastes to a substantial degree.

I'm definitely not a knowledgeable movie connoisseur by any stretch, but might this be because expensive productions tend to be less 'intellectual' and more about pleasing the audiences' senses than more targeted, lower-end films are? Thus, they provide more objective criteria for evaluation. It is also worth noting that the high-budget list is dominated by action and science fiction titles, the two genres where critics do quite well.

Or might my original premise be unfair to critics as a whole because the way reviews are amalgamated isn't optimally helpful? That is, the rating services give an average score for films. But the number of reviewers (both critics and mavens) varies widely by movie. Professional critics do not review every movie that comes down the pike, and movie enthusiasts even fewer. Movies that illicit a response in the reviewer, be he professional or otherwise, are the ones he'll most likely rate and write about. More likely than not, even if he isn't particularly impressed, he probably tends to get more from the movie than the next guy who decided not to write about the film. If this is the case, movies rated by the greatest number of people should be the top performers, whereas the ones receiving sporadic comment languish in the face of uninterest.


Steve Sailer said...

One factor is that critics, unlike mavens, seldom see the preview trailers that paying fans see. At critics' screenings, they show just the film and nothing else. So, I seldom have any idea how the trailer looks like or, especially, what the public's reaction to the trailer is.

To take an ancient example, if twenty years ago I was attending critics' previews instead of buying a ticket, I would have had no idea that the preview trailers for "Robocop" were driving paying audiences wild with enthusiasm. At several movies before "Robocop" came out, I heard the audience spontaneously break into chants of "Ro-bo-cop! Ro-bo-cop!" during the trailer.

John S. Bolton said...

Slightly off the focus point here, Could someone explain how it happens that in possibly hundreds of movies where one would expect that someone would yell 'stop thief' that never happens. The last time I saw someone running off with something stolen, there were cries of stop thief, and they were stopped. In movies though, no such solidarity and common customs seems ever to be allowed to appear. I suspect that this is partly from the liberal wish for freedom-for-aggression, since the plots allow for the thieves to be caught, and it could be in this way which reinforces continuity of civilization.

Audacious Epigone said...


Good point. That's a golden marketing opportunity being missed, if people going to see a movie aren't able to see advertisements for another movie. They're all accessible online, but being presented with something and having to hunt it down are two different things, especially if it's a genre that doesn't, generically, appeal to you, but when you happen to see the trailer you're pumped.


Heh, instead they just watch with tongue-gaping stares. Unless there's a hero around to do the stopping.