Why The Wall Street Journal Rankings Are Bogus [A Takedown]

By alcalde in Blog on September 16, 2010 at 4:09 pm | 8 Comments

This week, the venerable Wall Street Journal tried its hand at ranking undergraduate schools, a slippery and notoriously contentious endeavor, and returned some wild results.

It may come as a surprise that perennial powerhouses Harvard, Duke, Columbia, NYU, Stanford, Wharton, the University of Chicago, and The University of Texas at Austin didn’t make the list of 25 schools that “top recruiters” prefer for filling entry-level jobs. What the?

UT marketing researcher Matt Turner was surprised, as were observers from Boston to L.A. who divined from the results that if you wanted to get a good job in business go to a state school (just not UT). Even Time was seduced.

Schools like Penn State (No. 1) and Texas A&M (No. 2) and Florida (No. 9) and Texas Tech (No. 18) loved the survey, of course. The question is whether it’s worth a darn.

Turner says no, and he took to the McCombs School blog yesterday to, how shall we say, raise some concerns with the methodology. It would be safe to summarize that Turner wonders what planet the Wall Street Journal is reporting on.

“The WSJ’s Top 25 Recruiter Picks flies in the face of everything we observe in our recruiting operations and anything that has been reported in other major publication rankings,” Turner wrote. “Only two rankings exist specifically for undergrad business programs: U.S. News & World Report and Bloomberg Businessweek. For the record, McCombs ranks no. 7 in the former and no. 10 in the latter.”

Even more curious, part of the Bloomberg Businessweek rankings is a survey of recruiters — just what the WSJ says it has done. How has Texas done on that survey recently? McCombs ranked 2 in 2010, 3 in 2009, 2 in 2008, 5 in 2007, and 5 in 2006.

The WSJ’s Top 25 Recruiter Picks flies in the face of everything we observe in our recruiting operations and anything that has been reported in other major publication rankings.

This isn’t the first time the WSJ has veered far from the norm in its fitful attempts at rankings. They used to rank MBA programs until multiple failures to develop a working methodology led them to abandon the effort in 2007, Turner said. “Year to year, the WSJ MBA ranking was a roller coaster ride for schools, with dramatic upswings and crashing descents utterly unconnected with real-world performance,” Turner wrote.

How about starting salaries? The median for McCombs undergrads is $54,000 a year. That’s between $7,000 and $9,000 better than A&M or Tech graduates.

Now, to be fair, Turner is responding on behalf of McCombs to a study that judged the entire university. Still, the starting salary numbers for a generic UT grad stack up with those of the best schools in the nation — certainly as well as A&M and Tech graduates — despite a much heavier emphasis at UT with educating liberal arts majors, social workers, and students who pursue graduate degrees in the humanities and sciences. 

Turner has more to bark about, and you should read his whole post here, but just by way of passing shot, consider that the WSJ did not collect any data from the schools, it published no data points, and provided nothing that would allow for any quantitative comparison.

The WSJ said that, where possible, it pulled school-related data on number of students, tuition, application deadline, undergraduate enrollment, and admissions contact information directly from each institution’s website. In some cases, data was collected via school profiles on collegeboard.com.

But here’s a kicker: in the press release announcing the rankings, the Journal included a URL to its methodology that does not work. Huh, figures.

Photo by Val Cook