Mythbusters: Some Ranking/Guide Examples

If you haven’t read my most recent Mythbusters post, College Rankings and Guides, you may want to take a look at my general thoughts and conclusions before you look at the specific examples I cite here. The long and short is that I feel rankings and guides can be used as a tool, if they’re used carefully and you make sure you’re familiar with their methodology. They’re best used early in the college process, as you’re searching, and should probably not be factors in the decisions you make about where to apply and enroll.

Here are some thoughts and illustrations using a few common examples that I often encounter..

Example 1: U.S. News and World Report

The classic example of ranking controversy is U.S. News and World Report‘s Best Colleges list, and I think it’s a perfect demonstration of the rankings-are-only-as-good-as-their-methodology point. I don’t see any problem with using USNWR as one way of getting to know some of the nation’s best colleges. But I would hesitate to use it too heavily in forming opinions or making decisions — in other words, in actually ranking colleges — unless you’re absolutely familiar and comfortable with how it ranks schools.

Did you know that 25% of a college’s ranking in USNWR is based on a peer survey filled out by the presidents, provosts, and deans of other colleges? That is to say, one quarter of a college’s score comes from the opinions of about 2,000 higher education administrators across the country. There are merits to this methodology, of course — as USNWR points out, it helps take the “intangibles” into account (all their other variables are purely mathematical statistics) — but there are some significant shortfalls as well. It tends to favor more established colleges and disadvantage up-and-coming or rapidly-changing colleges; it disadvantages colleges that don’t fit the typical mold or don’t fall neatly into established categories; and, like it or not, it means that relationships and politics play a role in the ranking (the popularity and visibility of Richmond’s current president across the national academic scene sure hasn’t hurt Richmond’s ranking, I’m sure — despite the fact that Dr. Ayers couldn’t care less about rankings). Some of the best colleges in the country spend exorbitant amounts of money on informational/promotional pieces directed at the administrators of other colleges, for the express purpose of boosting their reputation survey score. This is, in my opinion, an unfortunate waste of resources, and I’m proud to say that Richmond is not among these schools.

So how valid are the USNWR rankings? Well, completely valid — as a combined statistical measure of a college’s “reputation,” retention, faculty resources, student selectivity, financial resources, graduation rate performance, and alumni giving rate, in that order of importance. But if those are the exact statistics and order of importance on which you want to evaluate and select a college, go right ahead. (Just be careful not to get too nitpicky… the difference between #10 and #200 may be significant, but the difference between #10 and #20 probably isn’t all that great.)

Example 2: Forbes

Okay, I confess in advance that I’m a little more heated up about this one — primarily because Richmond has sat on Forbes’ “Most Expensive U.S. Colleges” list for several years now. The Forbes list is, as with every other ranking, a perfectly accurate reflection of exactly what it measures. In Forbes’ case, this is tuition. Just tuition.

Now I’m not sure what sane college student or parent would tell you that tuition is the only cost of college, unless the student is living at home and commuting. We all know that room and board are a huge factor as well — whether a student is living on campus or off, there is always going to be a cost of living associated with college.

When you factor in room and board, Richmond drops from #11 to #77 for the 2009-2010 year (see the complete list here). This is a perfect example of how methodology can have a very significant impact on ranking. The city of Richmond has a very reasonable cost of living, and the University has an extremely low room and board compared to many of our peer schools, especially given the quality of residential life we offer (have you eaten in our dining hall or seen our new rec center??) But that low cost of living is not factored into the ranking at all. (Nor, for that matter, is financial aid, but that’s a completely different blog post.)

Now, don’t get me wrong — I’m not saying a Richmond education is not an investment, or even that it isn’t among the most expensive colleges in the U.S. (it’s still in the top 100 by total cost, after all). But it frustrates me that Richmond is known for being “at the top of that expensive college list,” just because Forbes chooses to use tuition as its only measure of college cost. If they included room and board, we wouldn’t hear about Richmond at all. It’s all about the methodology.

Example 3: The Princeton Review

There are a lot of different college guides out there, and some are better than others. I think it’s fine to use guides as an introduction to colleges, perhaps when you’re putting together your initial list of schools to visit — but I would not rely on them to develop your knowledge of a school or help you make decisions. Here’s why.

Even in my three short years, I’ve seen the role that politics, impressions, and stereotypes can play in these things. Prior to last year, Richmond’s page in the Princeton Review was, in a word, atrocious — horribly out-of-date, at least a decade old in the image it presented of the University, with quotes that came from the mid-90s. (In case you weren’t aware, Richmond has changed drastically since then.) We tried, year after year, to suggest changes and updates, but the response was generally that “this is what the students are telling us.” Then, last year, one of the movers and shakers at Princeton Review came to visit our campus; he was here for other reasons, but he was kind enough to spend some time with us in admission as well as with students, faculty, and administrators. And guess what? Poof! Our page in this year’s edition was an utter transformation, with fresh language that reflected the present reality. Obviously we’re happy for the change and grateful for the Princeton Review‘s attention; but the staff at every guidebook can’t visit every college campus every year. Like it or not, the impressions and prejudices of the editorial staff at any guidebook are going to seriously affect how a college is presented.

So, please, feel free to use guides — I personally consider Princeton Review and Fiske to be among the best — but rely on your personal experiences, campus visits, and interactions with people currently affiliated with the university (students, faculty, admission people) to form your opinions.

7 Comments

  1. Param
    Posted March 26, 2010 at 3:40 am | Permalink

    Well, this puts international students in a big dilemma. If rankings are based on skewed, unofficial and often inconsequential data, guide books can be out of date and visiting campuses is an option that many (most?) of us can’t avail; how should choose which college we want to attend?

    At the end of the day, I feel that the final choice will be made on the basis of financial aid; atleast for international students seeking some financial aid.

    PS- I often joke with my friends that I’ll end up at the college which has the best viewbook 😀

    • Tom
      Posted March 26, 2010 at 10:47 am | Permalink

      Param,

      You raise some excellent points! International students don’t often have the opportunity to visit, which can put them in a very different situation. As I said in the post, I do think you *can* use rankings and guidebooks, as long as you’re careful and make sure you understand the potential drawbacks of any particular ranking or guidebook.

      Still, I would encourage international students to make as much use as possible of (a) colleges’ websites, which tend to be more up-to-date than the guidebooks; (b) as you rightly point out, college viewbooks and other informational materials that we send (hope our viewbook stacks up well in your list!); and (c) contact with colleges through e-mail, blogs like this one, and social media (like Facebook and online chats), especially if there are opportunities to contact current students or professors. (Contact with current international students is also a big plus, if it’s available.) I think international students can still get a good sense of fit for U.S. colleges, even if they don’t have the opportunity to visit.

      And, of course, you’re absolutely right to say that financial aid will ultimately play a big role for many international students. Funds at U.S. colleges are often limited (Richmond is lucky to have a modest pool of international aid available, though we’d still like to have a lot more than we presently do).

      Tom

  2. Griffin Meads
    Posted April 3, 2010 at 1:54 pm | Permalink

    I have visited the University of Richmond campus twice – once in my freshman year and this past Spring. What a transformation the campus has made within such a short period and the future is even brighter. By all means, use the guide book but nothing compares to visiting the campus and listening and talking to the individuals at the University of Richmond.

    • Tom
      Posted April 5, 2010 at 11:52 am | Permalink

      I completely agree, Griffin! Thanks for your thoughts.

  3. Burns
    Posted April 6, 2010 at 12:26 pm | Permalink

    Yeah, UR seems really “hot” right now.

  4. Jennifer Helfer
    Posted September 16, 2010 at 3:39 pm | Permalink

    I am so excited about applying to University of Richmond. I have emailed the admissions counselor, looked at your brochures sent to me, and plan to talk with an admissions rep when she comes to my HS in Buffalo. Will it hurt my chances of admission if I can’t visit the campus in person though?

    • Tom
      Posted November 4, 2010 at 6:31 pm | Permalink

      Jennifer,

      We don’t take interest into account in our admission process — with the huge exception of students who commit under our early decision plans — so visiting campus will not improve your chances of admission, and not visiting will not hurt your chances. We have such a geographically diverse applicant pool, it wouldn’t be reasonable for us to expect all interested students to visit our campus. Plus, the internet and social media mean that students can find out quite a bit without ever setting foot on campus.

      Tom


One Trackback/Pingback

  1. […] Admissions Blog « Some tips for juniors Mythbusters: Some Ranking/Guide Examples […]

Post a Comment

Required fields are marked *
*
*

%d bloggers like this: