Using the same 2005 movie roster as was utilized in previous posts on the superiority of amateur movie mavens to professional critics in predicting box office success, I accounted for a movie's budget. The thought is that since critics are likely to be more forgiving of low-budget productions than plebians are, if we take away this self-imposed handicap, movie critics will fare better.
Critics are not meaningfully biased toward higher budget movies (the correlation is a statistically insignificant .07). But mavens are hardly anymore so (with a correlation of .16, again outside statistical significance at 95% confidence). Given this negligible relationship, it is not surprising that taking budget into account only increases the relationship between critical reviews and box office performance from .30 to .33.
Interestingly, though, performance varies between critics and mavens depending on production costs. Contrary to what I thought would be the case, critics actually do much in the high-budget range than anywhere else. They even perform better than mavens. But they do so horribly in the middle- and especially low-end range, that overall mavens are better.
Breaking the roster into thirds based on budget, the review-box office relationship is as follows, for critics and mavens:
High-budget: .60, .55
Medium-budget: .24, .38
Low-budget: .08, .30
Both get much better as budgets increase, undercutting my earlier supposition that critical reviews are of little value from the perspective of determining what movies will become blockbusters by introducing a huge caveat. When it comes to big dollar productions, they share the public's tastes to a substantial degree.
I'm definitely not a knowledgeable movie connoisseur by any stretch, but might this be because expensive productions tend to be less 'intellectual' and more about pleasing the audiences' senses than more targeted, lower-end films are? Thus, they provide more objective criteria for evaluation. It is also worth noting that the high-budget list is dominated by action and science fiction titles, the two genres where critics do quite well.
Or might my original premise be unfair to critics as a whole because the way reviews are amalgamated isn't optimally helpful? That is, the rating services give an average score for films. But the number of reviewers (both critics and mavens) varies widely by movie. Professional critics do not review every movie that comes down the pike, and movie enthusiasts even fewer. Movies that illicit a response in the reviewer, be he professional or otherwise, are the ones he'll most likely rate and write about. More likely than not, even if he isn't particularly impressed, he probably tends to get more from the movie than the next guy who decided not to write about the film. If this is the case, movies rated by the greatest number of people should be the top performers, whereas the ones receiving sporadic comment languish in the face of uninterest.