Read the back-and-forth over law school ranking

I wrote today about a heated exchange over U.S. News and World Report's decision to "unrank" the University of St. Thomas School of Law from its "Best Law Schools" list. Here are the letters, in full.

Dean Thomas Mengler's open letter to U.S. News:

Dear Mr. Morse:

I should now be a strong believer in the adage no good deed goes unpunished. We learned last Friday through your blog post that your publication placed the University of St. Thomas School of Law in the "unranked" category after it immediately self-reported that two submitted data points related to "at graduation employment" were conflicting - one accurate, one inaccurate. We have some questions about your decision to automatically "unrank" a school. I write to you today to do two things: first, to make two inquiries about your decision, and second, to express my concerns about the impact of your decision on schools' incentives to correct errors promptly.

As you now know, the form you rely on for calculating rankings asks for "at graduation employment" in two places - on lines 164 and 169. As you also know from our self-reporting, in response to one question we reported correct data (line 169), but in response to the other question we inadvertently reported incorrect data (line 164). Ironically, we were not even required to answer the question that produced an error, but we did so in an effort to be as transparent as possible. Other schools that elected not to answer the question were imputed a percentage.

First, I would like to inquire about your methodology. What we do not know is whether you relied entirely on the incorrect data in your initial ranking of our law school. Did you consider line 169 - the correct "at graduation" employment percentage in your initial ranking of our law school? Did anyone at U.S. News discuss - or notice - the fact that the two data points are in conflict?

Second, is the decision to place a law school in the unranked category following a self-reported inadvertent error pertaining to conflicting data a new policy for U.S. News? Historically, and as recently as last fall, when other schools were discovered to have lied - not just, as in our case, to have made a mistake - you decided not to "change [your] long-standing policy of not revising previously published rankings." In those cases, you were hopeful that a "public outcry" would serve as a deterrent. Is your decision to "unrank" a change in policy?

We remain very sorry that we mistakenly submitted an inaccurate number in one of the two places for reporting the data. We will continue to strive to be as transparent as possible in providing consumer information. Our mission calls us to focus on professional formation of our students and graduates with an emphasis on integrity and accountability. There was no question in our minds that we had to correct the mistake immediately, and we will live with the consequences of the mistake. However, we also hope that we can continue to dialogue with you about these important issues and how we can work together to create a better system.

Warm regards.

Sincerely,

Thomas M. Mengler

Dean and Ryan Chair in Law

Dear Dean Mengler:

I am responding to your "Open Letter to Bob Morse." As the editor and chief content officer of U.S. News & World Report, I have the ultimate responsibility for everything we publish. While Bob is indeed one of the most experienced and respected journalists in the field of editorial data, any decisions we make are institutional and not personal.

In the rare cases where we must make corrections or clarifications, the circumstances are discussed by a team of people and the ultimate decision is made by me. My responsibility is to our readers and our policy is to make them aware of new developments or changes in information. Because these corrections are so unusual, given the hundreds of thousands of data points we publish each year, we evaluate them on a case-by-case basis. A key factor is the confidence level we have in the integrity of the data.

The incorrect data submitted by St. Thomas was troubling because the school was given multiple opportunities to correct it after submission. As part of our normal interaction with schools, we sent an assessment report showing the submitted data including St. Thomas's report that it had an "at graduation" employment rate of approximately 80 percent, which was highly inflated. St. Thomas did not request to change its "at graduation" employment rate as part of this assessment process. As part of the final data verification, we next asked the school for confirmation that all submitted data was correct, including the "at graduation" employment rate data used in the assessment calculation. St. Thomas returned a signed verification form stating that all data was correct.

When the error was pointed out to us by St. Thomas, we concluded that the correct number would have had a material impact on St. Thomas's ranking in the 2013 law school rankings. Whether intentional or unintentional, St. Thomas received a rank it should not have received. We moved St. Thomas to the unranked category for the 2013 rankings both to inform our readers and to reflect the change in status as a ranked institution.

I'm reassured to hear that St. Thomas is committed to the integrity of its data and we will continue to work with you to make sure we can share it with our readers. As to the notion that any perceived penalty will dissuade other law schools from admitting mistakes, we will continue to have faith in the stated ethics policies of each of these institutions. We also will continue to believe that those who educate the nation's lawyers will not be deterred from correcting the record – as St. Thomas rightly did -- when they make a mistake.

Sincerely,

Brian Kelly

Editor