Shame on you Poets & Writers Magazine by Stacey Harwood

Once again, Poets and Writers Magazine has published creative writing MFA/Phd program rankings that are based on a poll of would-be applicants to such programs who visit a blog.  Clearly P&W doesn't let its ethics get in the way of perpetuating a scam to boost circulation. After three years of seeing these rankings gain traction, nearly 200 MFA/Phd-program teachers and administrators, led by Robert Pinsky, Leslie Epstein, and Erin Belieu, have lodged a protest, in the form of an open letter (reproduced here by Dan Nester) to P&W  that cites many of the same reasons for objecting to the rankings that we have noted here, here, and here since they were first published in 2009.

Here are a few of my thoughts on this mess: 

Poets & Writers has entrusted the task of the rankings to a blogger/poet who has no credentials as a research scientist and who is not disinterested in the outcome:  For each year of the rankings he has been enrolled in one of the programs that is being ranked.  This fact alone should disqualify him for the job. 

In attempting to shore up its phony methodology, P&W wastes a lot of ink on why other rankings fall short. This is like saying my poem is good because yours is bad. 

P&W is wrong to assert that a scientific ranking of MFA programs is not possible. A ranking is possible, though to construct it properly would require time, money, and commitment, resources that might be better spent elsewhere. The underlying data, if collected by a reputable research firm, under the auspices of the Association of Writers & Writing Programs and with the cooperation of its members, could establish a ranking that would give MFA/Phd applicants valuable information to guide their decisions about graduate programs. 

Many of the signatories to the open letter are affiliated with our most prestigious universities.  To lend more depth and credibility to their criticism, they should ask a member of a social science research department to review the ranking methodology and comment on its soundness.  A first-year graduate student could do this in an afternoon; the outcome would underscore the extent to which P& W is flouting its responsibility to its readers.  

The MFA programs should immediately pull their advertisements from P&W.  Why continue to support a publication that is so lacking in journalistic integrity? 

--sdh

Comments

[John Findura]

You're right, Stacey, there are a number of errors with the rankings. I think that the most overlooked factor is the faculty. When I chose the MFA program I attended, the most overwhelming factor was not class size or whether they were CGSR (?!) compliant. I went where I went because that's where the people I wanted to study with were located.

[A.S.]

As someone who applied to MFA programs in 2010 and will apply again this year, I must say that I find them immensely helpful. I've seen rankings set up this way before, ordered by the votes received, and I always take that into consideration--ultimately, the responsibility to fully research each program falls on my shoulders. The rankings are not what make my decision about where to apply. They're a starting point, and they're a great reference as well for information that would take days to compile if I had to search each program's website myself (funding, cohort size, length, etc). Several schools to which I will apply this year are represented on the list of protestors, but they are schools I may never have considered if not for the rankings.

[Trisha H]

You are absolutely incorrect, Stacey. Despite the fact that there may be a few errors on the list, the people who are providing these lists are providing valuable information to those of us who are planning on applying to apply to MFA programs. It is YOUR journalistic integrity that should come under fire with this.

One has to wonder why you are so opposed to prospective students having a general understanding of what they are getting themselves into when they are applying. Perhaps there is something missing from your side of the story that we should be aware of? If not, I can't help but believe that you have some ulterior motive for wanting the rankings pulled. What makes you an authority on the P&W rankings? Where are YOUR credentials.

Shame on YOU, Stacey for trashing a man's reputation for no reason. It is obvious that the man posting this effort has put incredible amounts of time and energy into getting accurate information - not "an afternoon" like you idiotically suggest. Believe me, a lot more effort goes into his research than one of your little slanderous blogs.

[Clarissa]

But what else is out there for us (potential applicants) to use? You can argue whatever you want about the ranking methodology, but at the end of the day Seth Abramson has compiled an absolute wealth of valuable information for an MFA applicant that you can't find anywhere else. You can dispute the rankings, but you can't dispute the information on funding, teaching load, cost-of-living, etc. Most MFA program websites don't provide such clear information about their program; if anything, Seth's rankings have pushed MFA programs to be more transparent about their programs, particularly where funding is concerned.

I, for one, am grateful to Seth and P&W for publishing this information. P&W publishing the rankings doesn't equal them telling anyone where to apply - it's just information for an applicant use during the difficult and stressful MFA application process. We can take it or leave it as we see fit.

[M]

At the end of the day, I don't care about the rankings to be honest. I care about the funding, teaching load, cost-of-living information, those things cited above by Clarrisa which are indisputable. These are things MFA applicants have a right to know, and Seth has done a hell of a job putting it all in one place for us. For that alone, he should be commended. In my experience, there are some MFA programs and websites that can be pretty nebulous and downright shady with their information. Seth has worked to provide MFA applicants with a service that forces programs to be more transparent. I can think of about 300 fellow MFA applicants (just off the top of my head, at least) who would share my support for Seth.

[Stacey]

Clarissa and A. S. Thank you for your comments. I'm relieved A.S. that you aren't basing your graduate school decisions on the P&W rankings.
Trisha, There are more than a few errors in the rankings and if you follow the links to my earlier posts about them and read the AWP letter I link to and the recent open letter to P&W you should be able to understand what they are, beginning with the sampling methodology. You will also learn more about my credentials.
I am not alone in objecting to the rankings: 190 poets, writers and MFA/Phd program administrators have objected to the rankings on grounds similar to mine.
There is nothing in my post that says that I object to prospective students having information.
I’m astonished Trisha that you’ve chosen to comment here without first reading the material to which you object so strongly.

[Anna]

I have found the information provided by P&W MFA rankings immensely helpful as well. Clear, concise, available in one place and from a source that actually has viable experience dealing with the process of applying to MFA programs has more value and integrity, in my honest opinion, than any research scientist, unfamiliar with the subjectivity of the process could present.

It is wrong to assert the opinion that it should/could be done better elsewhere. The point of the matter that it hadn't been done and there was a need filled by someone taking the initiative to compile this information as a benefit to the community. More likely, the blogger and poet you mention found himself wanting a resource that was unavailable and took the initiative to create it. What is the harm in that again?

[Veronica Popp]

I don't understand why the writer is being so negative. The MFA Listings are very helpful in choosing a program.

[Jonathan]

I also am an applicant who has found the rankings (and even more so, the data that comes with them) extremely helpful. I don't think they are perfect, nor should they be the final or only basis for an applicant's decision, but no one (including Seth Abramson or P&W) makes the claim that they should be. Without the rankings I shudder to think how incredibly uninformed my decision making process would be, especially as I've been shocked to discover how uncommunicative many programs are about basic data regarding funding and acceptance rates (that's a generalization, sure, but I believe a true one).

All in all then, I completely DISAGREE with any call to boycott P&W or the MFA rankings.

[Tory]

I think that these rankings are misleading in as much as popularity is the main factor for how they are ordered. They ensure programs like Iowa and Michigan, wonderful programs though they may be, perennially remain at the top of the rankings while up and coming programs and lesser known programs remain just that. This wouldn't be an issue worth talking about if these rankings were still published on Seth's blog--but the fact that they appear in Poets & Writers, a well-respected, well circulated, and influential magazine, makes them seem as though they are word of god. I think a listing of programs alphabetically, or by indisputable factors like funding, faculty/alumni accomplishment, would be more than adequate and useful without creating a false hierarchy based exclusively on how well known a program as. I alo wonder if Seth, as the progenitor of this ranking, fails to see that his opinions (which he has made known explicitly, albeit in the past) influence which programs are "popular."

[Trisha H]

No need to be so "astonished." I have read the information you've provided and also the letter signed by all the professors who have disagreements with the rankings.

Bottom line, Seth has done a hell of a job putting the rankings together and they provide an immense amount of valuable information. You objections are ridiculous and I'm astonished that you would so blatantly bash on someone's hard work - work that IS useful and IS credible - just for the sake of doing so.

I can understand disagreeing with someone, but using your meager points as a reason to call for people to immediately pull their funding? Like I said, I think you've got something personal going on here.

Seth is doing an incredible job. You, on the other hand, are just sitting around and pointing fingers. When you do something useful to help all of us prospective students, you can get back to me. Until then, your opinion on anything MFA is pointless.

[Emily S.]

I applied in 2006 without access to any of Seth Abramson's excellent and very relevant research, and I entered the worst program for my needs and interests. Studying Seth's rankings would have changed everything, including the amount of debt I am still paying off from ideas I had as a naive 21-year-old about what was important in a program.

Unfortunately, there is very little information available to applicants about the programs to which they apply. The programs are not as forthcoming with this information as they ought to be, and Seth is the person who has compiled it for those without access. Until there is complete transparency from each school about every quantifiable aspect of each program, Seth's research is absolutely the authority on the subject.

You don't have to agree with his methodology, Ms. Harwood, but you don't need to offensively trivialize his research by saying someone could do this "in an afternoon." Really? I don't know why you feel compelled to participate in the MFA applicant community conversation if you know so little about it.

Like you, I would love for someone to fund the necessary MFA research, but please remember that Seth is committed to this project on a volunteer basis. We have ONE person working tirelessly to organize this research-- uncompensated and beleaguered though he is-- and I fail to see how putting forward this information "flouts" journalistic integrity, or whatever other alarmist language you have decided is an appropriate description for his work. This "shame on you" rhetoric is, frankly, ridiculous.

Applicants deserve to know what the data is, and deserve to make well-informed decisions based on the truth. Seth Abramson has provided that resource.

[Jake Winston]

The rankings are totally misleading and anyone relying on them is nuts. Hey, you're going to study creative writing -- you can't judge that or the teaching of that by statistics combined in an arbitrary way by some loudmouth on the net.

[SP]

Tory, your last point is a good one. Seth doesn't just make the rankings and let them stand for themselves; he participates in the conversation. For example, he has spoken ill of faculty members and programs on the Facebook group (from which most of these commenters have come, by the way; it's not a coincidence that people keep posting the same opinion within minutes of each other), and because his devoted followers consider him an "authority" or "expert," many of them listen to his advice. This negatively affects those programs and their reputations, even if those effects are small-reaching, and that's a shame. I think Stacey makes good points about the flawed methodology used, but I also think she may be right in her estimation that Seth, as an aggressively opinionated person with personal stakes in the game, for all his admirable passion and spirit, may not be the best person for the "official" job (the publication in P&W makes the ranking seem more legitimate than it is; the material may be useful, but it is best suited to a personal blog unless some changes are implemented). That he is ready and willing to do the work is, frankly, irrelevant to the argument Stacey is making. Also, other commenters: that the rankings have aided you does not magically make the methodology better, or mean that nothing needs to change here.

[Laura Orem]

This isn't just Stacey's beef with P&W methodology. The signatories of the open letter are some of the most respected writers and teachers of writing working today. Even if there was something personal going on, plainly the methodology of P&Ws annual rankings are problematic to many people. Regarding several commentors' assertions that they found P&W's rankings helpful, I don't doubt that you did: if you have no other information, any information at all is valuable.

David Fenza at AWP has also criticized P&Ws means and methods. He says:

"Good advice on choosing a writing program should help you discover your own literary affinities, but no magazine’s centerfold of academic rankings is up to that task. Although AWP values Poets & Writers as a peer, Poets & Writers’ annual rankings of writing programs cheapen their usual standards. Rankings of writing programs simulate literary affinities; one should never confuse that simulation with finding one’s own authentic literary ties. Such rankings do for creative writing what pornography does for love. The rankings are provocative and good for commerce, but the heart of the matter resides elsewhere, in your own preferences as a reader, in your own sensibility as a writer, in your own love for certain books—and not in the dubious statistics of Poets & Writers’ polling."

P&W should pay attention to what the Writing Program community is saying. This isn't about Seth Abramson, and it isn't about Stacey. It's about P&W presenting information in a disingenuous and inaccurate way. I disagree with Fenza in that I believe there is a need for reliable, accurate, and thorough information for potential applicants of MFA programs. It would be nice if P&W, with its wide readership and long and respected history, considered what its critics are saying and filled the need for its audience, instead of stubbornly sticking to this problematic, inaccurate, and limited method.

As for the issue about finding jobs with an MFA, SNORT.
In this economy? Really? You go to an MFA program to study and learn your craft with writers who speak to you, not to get a job. If you are getting an MFA to get a job, go get an MBA instead, and good luck to you. These days, you may end up as a barista at Starbucks with the poets and novelists anyway.

[Kelly]

The biggest problem with the rankings, as mentioned in the letter, is the fact the group being polled is a very small and insular group, namely MFA applicants who are visiting a specific site (formerly The MFA Blog, currently the "closed" Facebook Page "MFA Draft 2012") where Seth Abramson, the person conducting the poll, makes frequent comments and posts that likely influence these applicants' decisions and choices. There are other problems with the methodology, but this is the biggest one and the one that the writers who signed the letter seem to object to most.

Stacey is right that a more accurate and objective type of ranking is possible. I think that potential applicants should absolutely have a voice in the poll, but so should graduate creative writing professors, current MFA students and recent MFA grads, undergraduate creative writing professors, maybe even editors of literary magazines and presses. In other words, a wider spectrum of opinions would create a more accurate type of picture of how these programs are actually regarded.

Seth has argued that the applicants he's polling are much more knowledgeable about MFA Programs today than the people teaching in them or even the undergraduate professors advising them, but I find this claim a little weak. I've been advising undergraduate creative writers on MFA Programs for over twenty years, and over that time I feel I've gained a great deal of insight into many of these programs (mainly through former students reporting back to me about their experiences) and also know a good deal about the basic nuts and bolts: program size, financial aid offerings, duration, student to faculty ratio, etc.

Seth has further argued that if he polled people who had graduated from programs, were enrolled in programs, or currently taught in programs, their opinions or rankings would be biased, but this is an easy thing to fix. All you'd need to do (as Edward Delaney did when he conducted his poll for the Atlantic) is require that the person being polled didn't list any institution where he or she had an affiliation.

The point is there is a way to do a broader, more accurate ranking that would likely silence the many critics of the current poll. The problem is, as Stacey points out, this type of poll would require a lot of time and money, something that Seth, as a PhD. student, might not have. It would also require the willingness of a lot of the people mentioned above to participate, which might be hard to obtain--at least, at first.

Still, just because the resources to conduct such a poll don't exist right now, doesn't mean that we should automatically accept the current poll as accurate, simply because "it's the best that we have at the moment."

Seth pours an enormous amount of time and energy into compiling the data for his rankings, which is why I can understand why he defends them so vehemently. But at a certain point, when so many people reject your methodology--and there were a lot of people I respect on that list--you have to stop fighting with everyone who questions it and ask yourself whether there might in fact be a problem.

[Rebecca Brett]

Is the perpetrator of the P & W rankings qualified for the task? Is he truly disinterested?
My third question is this. Shouldn't we figure out a more creative approach to evaluate the pluses and minuses of any programs we're considering applying to?

[Tom]

The P&W list is helpful to a great extent in that it's one-stop-shopping for finding information about funding, size and teaching load. Many in the applicant pool know that this list is not the final word but a comprehensive and valuable resource. If professors and program administrators want to be a little more open and proactive about their program's style, the best place in this day and age would be to throw interviews, workshops or lectures on a medium like "youtube." Three programs that pique my interest, Wyoming, Iowa and North Carolina at Wilmington, have utilized this approach.

[Leslie McGrath]

My goodness, I read the original post here by Stacey just a couple of hours ago and suddenly Best American Poetry is Popularity Central. I'm so pleased! Stacey works very hard in terms of looking for and scheduling bloggers for this site-- and well as providing regular content herself. As one of the bloggers here, I've been encouraged to blog about my opinion on anything American poetry. Why shouldn't Stacey have the same ability as the dozens of us she features?

It's already been mentioned that Stacey didn't start this conversation. I believe Erin Belieu did. And hundreds of poets, including myself, joined her.

I'm in an unusual situation here, being a poet with four years of graduate training (including four courses in grad level experimental methodology and multivariate statistics) in psychology. I've published as a psychologist. The P&W methodology is, in my opinion, very elementary. Not only that, but I believe it gives too much weight to student funding and not enough to the pedagogical efforts of the faculty. Also, where is the followup data that anyone attending an MFA program might most want to see: evidence of publication, both in lit mags and published books by graduates? Wouldn't this be a relevant measure of an MFA program's "success"?

I hold no negative feelings toward Seth Abramson, who clearly works very hard on this ongoing project. What I'd like to see, however, is the addition of a research methodologist in the organization and statistical analysis. It wouldn't be terribly expensive and might go a long way in terms of legitimizing the results in the eyes of poets. It might also be more helpful to the people considering graduate training in literature.

I've heard from a number of sources at the 2011 edition of Best American Poetry is particularly good. I'm looking forward to using it in my classes....and hopefully attending the launch reading in NYC soon.

All thanks to Seth, Stacey, David, Erin, and anyone else who spends a significant amount of time working on behalf of other writers.

[MG]

As SP pointed out, it's not a coincidence that so many current applicants are suddenly showing up within minutes of each other to defend the value of Seth's rankings. I have no doubt they've all been sent by Seth himself from the Facebook page MFA Draft 2012, where he presents himself as the leading authority on all things MFA and rules over the current group of MFA hopefuls (and where he is also, coincidentally, conducting his poll for next year's P&W rankings!)

He used to do the exact same thing over at The MFA Blog, but at least that was an OPEN forum where graduates of the programs he attacked could defend their programs and administrators. Now he does it in a closed group that you have to be "allowed" to join and where he doesn't have to listen to any dissenting opinions.

If there's any other poll in the country that's conducted in such a heavily biased environment, I'd like to know where.

[Jim Valvis]

The problem, of course, isn't that P&W ranks MFAs-- anymore than this blog's anthology claims to know what are the "best" American poems. Nor is the problem that they are informing readers about their opinions about degrees and/or schools. The problem is the psuedo-science shine they buff on those opinions.

Stacey seems to imply a better methodology by a more capable statistician will yield better results, but here I agree with P&W that it will not. There are simply too many variables, too many unknowns, and too many opportunities for bias. A ranking of MFAs, like a ranking of poems, is hopelessly opinion. It is impossible to say, scientifically, what is the "best". For me, lower tuition is important, for you, big-name instructors, for someone else, location, for another, an emphesis on poetry. It is only possible to say what I think is best-- for "best" is not a scientific result.

A better question might be if a subscription to P&W is worth the money. I decided several years ago that it wasn't. But that's not science, just my opinion.

Another question is if *any* MFA is worth the money people spend on them. That's a question each must answer for himself.

[A.S.]

It should be said that some of those who have commented here were directed here from the Facebook group...but not only those who are defending the rankings.

I also want to point out that Seth puts out a list of underrated programs, providing the traits that make those programs worth considering.

And Iowa's reputation is certainly not reliant upon these rankings.

[Thomas]

Trisha,

If you think the rankings are credible I think that a statistics course would be of much greater use to you than any MFA course.

[Fletch]

The guy behind this scam ain't fooling nobody. Fletch

[Kat]

As a student newly enrolled in an MFA program, as of this fall, I have to speak in defense of the rankings. Although I agree with the open letter that it would be responsible of P&W to clearly acknowledge the limitations of the rankings on the same pages they're presented on on their website and in the magazine, I cannot overemphasize how valuable I find the information compiled within the rankings. When I applied for MFA programs in the fall of 2010, I found all of the raw data compiled within the rankings an invaluable resource. While I personally have no interest in the popularity of each program with upcoming applicants (one of the major criteria in the current ranking process), information about, among other things, selectivity, cost of living in the programs' areas, and -- above all -- funding were an indispensable springboard for my research into each program to which I considered applying.

Though I agree that speaking to current students and reading the work of faculty members is an excellent step in the research process, I value the information gathered in the rankings because it gave me a starting place to figure out the potential logistics of each program. No matter how much I admire the work of writers on faculty at various universities (hey there, NYU), the sad facts are that I am a twenty-something going into a field with zero job security and minimal prospects, and have another sixty years of my life to worry about student loans and my credit score. Because programs are, understandably, more willing to flaunt their sterling faculties than, for example, their often complicated financial aid situations (and with budget slashing at state universities, I can't say that I blame them), it is hard to find this information elsewhere. And, as an applicant, this is information I cannot live without.

I agree that no applicant should base his or her application decisions entirely off of the list, and that individually researching each program is a vital step of the application process. I suspect you're right, that the rankings might be more palatable if gathered by a professional organization (if someone would agree to fund it). That failing, P&W could consider more clearly stipulating the limitations of the rankings when they print them; it seems to me, even, that the rankings could possibly be repackaged into a less competitive form -- perhaps just a comprehensive list of programs, including all of the raw information gathered in the current rankings. But as a former MFA applicant, as a current MFA candidate, and as a student whose concerns range from the super-concrete (will I be living in my 2003 Corolla for thirty years as I pay off my student loans?) to the sublime (literature! in the end this is all about literature, right?), I applaud the current rankings for making as much information as possible available for applicants, so that they can apply to programs that will be the best match for them. And whether or not this information could conceivably be packaged in a different form or otherwise reshuffled, I commend and personally thank P&W for making it all available to the people who need it, in the best form in which it currently exists.

I think it is possible that the rankings could exist in a different form, but think it would be in the best interest of all if those who disagree with the current style of rankings pushed for reform, instead of scratching the fixture altogether, because offering raw information to applicants is a unique and, in my experience, incredibly valuable service.