K99/R00 Reviews

So, I finally got the reviews for the K99/R00 grant application I submitted to the NIH in June. Since the K99 application guide that I wrote back then has turned out to be one of the most-read articles on my site, I wanted to write a follow-up post about my reviews for those who are interested.

How the K99 is scored

First, let’s discuss how the applications are scored. Your application will be read by three reviewers on the study section, each of which will score your grant in five categories. In each category the scores can range from 1 to 9. 1 is the best and 9 is the worst. Here are the categories:

1) Candidate

  • This score reflects what they generally think of you and your background, publication record, etc. I think it’s mainly based on what you write in the “Candidate” section and your biosketch, as well as your reference letters.

2) Career Development Plan / Career Goals / Plan to Provide Mentoring

  • This score is based on your training plan, career goals and objectives, the mentor plan, letters from your co-mentors and collaborators, and possibly the description of institutional environment. The reviewers are evaluating whether the training you’ve proposed is appropriate given your background, career goals, and timeframe.

3) Research Plan

  • This score is based on your research proposal, ranging from its conceptual merits and potential impact on the field to its technical feasibility and experimental design.

4) Mentors and Co-mentors, Consultants, and Collaborator(s)

  • This score reflects not only the letters that your collaborators provide but also how well you convince the reviewers that these people are amazingly qualified and will greatly contribute to your project.

5) Environment and Institutional Commitment to the Candidate

  • Here’s where the reviewers look the statements regarding equipment, facilities and resources, and the letter of institutional commitment. They will judge whether your institution, department, and lab are a good place for doing the kind of science you’ve proposed and training you to be a PI.

In addition, the reviewers will look at your other statements regarding vertebrate animals, training in the responsible conduct of research, resource sharing plan, budget, etc. But for these things the reviewers will simply indicate whether the plan is “acceptable” or not. If something is not acceptable you need to fix it before they’ll give you the grant, but it may not impact your score (unless it pisses them off).

What’s a good score?

So you’ll get the 5 category scores from 3 reviewers, and you’re hoping for mostly 1s and 2s. But none of these numbers really matter. All that matters is the final impact score, and this isn’t simply an average of the individual scores.

Here’s how it works in the study section. First, the three reviewers who actually read your application will present their opinions to the entire section and lead a discussion about it. Then all the members get to vote. They each come up with a single score from 1 to 9 (again, 1 is the best), those numbers are averaged, and that average is multiplied by 10 to get your impact score, ranging between 10 and 90.

The successful K99 applications that I’ve seen received scores of 20 or lower. But the scores don’t always relate directly to funding. Which applications get funded will be decided at a separate council meeting at least a month after the initial review.

Most of the decisions are made based on the impact scores, but I’ve heard that sometimes there can be exceptions for extenuating circumstances. And the distribution of scores varies for each batch of applications—if there were a lot of great scores in your batch then you’ll need a better score to get funded, and vice versa.

So, how did I do?

I received my impact score about four months after I submitted the grant. You don’t get the reviews (“summary statement”) till a month later.

My score was 42. While 42 may be the answer to life, the universe, and everything, it is not a good K99 score. As mentioned above, anything in the 10s is great, 20s are good, above that not so much.

So for a month I wondered what the reviewers hated about my application and whether I had given everyone terrible advice in my K99 application guide and would have to crawl into a hole and hide.

Then I got the reviews. They were actually pretty good.

The reviewers generally liked my research proposal. Two of them mentioned some specific weaknesses, but nothing that seemed that major. The scores I got for the Research Plan category were 3, 3, and 2.

The reviewers loved my training plan and all the other stuff I wrote in the “Candidate” section. I also got high marks for the Mentors/Collaborators and Environment/Institutional Commitment sections, which I expected since my advisor, collaborators, department, and university are all very highly regarded in the field. For these three categories (#2, 4, and 5 listed above) my scores were mostly 1s with a couple of 2s. Yay.

So what didn’t they like?

The fact that I haven’t published any papers during my postdoc.

Seriously, that’s about it. [Update: see below for new information from my PO.]

(Aside: It’s not like I didn’t anticipate this. This was the very reason I started putting together a paper last year and gave my advisor a complete manuscript in February. Unfortunately my advisor is a busy guy, and long story short, he still hasn’t signed off on submitting my paper.)

My lack of publications was reflected in the scores for the Candidate category, which were 4, 4, and 2. But even those scores aren’t terrible. If you average all the individual scores I got for the five categories, it comes out to 1.9, which (multiplied by 10) would be an impact score of 19. That’s a far cry from 42.

So it seems like that one thing – a lack of publications – completely overshadowed everything else. It also seems like maybe the rest of the study section didn’t like my application nearly as much as the three main reviewers did, possibly because they had differing opinions on the importance of one’s publication record.

My summary statement mentioned this specifically: “While all reviewers did not have the same level of concern about this issue, some on the panel expressed strong reservations.” Looks like those people won.

I’ve heard that the Program Officer (PO) can give you additional feedback and guidance on your reviews. My PO is currently out of the country so I have to wait till December to speak with him, but I’ll update this post if his feedback changes anything.

Update on 12/6/2016: I finally got to speak with my PO and he told me that yes, the main issue was my lack of papers, but in addition, some members of the study section had concerns about my research plan. Some specific issues he mentioned (which were alluded to in the reviews) were that the models I proposed were too simple, the preliminary data wasn’t strong enough, and the data analysis methods were limited. So for those of you worried that my story means your application will be rejected purely based on a lack of papers, this information should give you hope that maybe a really really strong research plan can overcome that limitation.

Will it get funded?

I highly doubt my application will get funded. I’ve been told that you never know, even with a score that seems unfundable, because we don’t know what kind of scores they gave to the rest of the applications. There’s also the possibility of writing some kind of appeal if you felt you were unfairly reviewed or have an extenuating circumstance.

IF I can improve my publication record in the next 6 weeks before the council meeting by getting my paper submitted or even accepted (ha), then that might be grounds for an appeal—but it’s a longshot.

Update on 12/6/16: My PO told me that for this study section, there’s no way that a score of 42 is fundable. Since I’m not eligible to resubmit my application, he recommended that I instead apply for a K01 award, which has a longer eligibility window. I haven’t looked into the K01 very much and I’m not sure I have the energy to apply for any more NIH grants as a postdoc, but I thought I’d mention it for others in a similar situation.

Final takeaways

The good news for me is that most of what I wrote was received pretty well. The main thing that brought me down was something entirely out of my control. The comments I got on my research proposal were fairly constructive and could help me on future grant applications.

The good news for you is that this probably means my K99 application guide does contain some solid advice. In particular, I was praised for assembling a diverse team of collaborators and a “clear and excellent” training plan. I also think my statements addressing the institutional environment, facilities and resources, equipment, etc. were successful in convincing the reviewers that I’m working in a great environment.

In addition, all the work I put into the writing did not go unnoticed (though it may go unfunded). The reviewers said my application was “well-organized” and “beautifully written”. This didn’t magically happen on its own; it took lots of editing and rewriting. So don’t put yours together at the last minute—leave plenty of time to rework it! Even if they’re not explicitly scoring you on your writing, good writing and organization makes it much easier to understand the content.

That’s all for now. I hope hearing about my experience is helpful for those of you putting together your own applications or navigating the review process. And if anyone else wants to share their own experience, feel free to comment below!


Comments

K99/R00 Reviews — 18 Comments

  1. Hi!

    Great blog!

    I’m a post-docs from harvard and I’m writing my first K99 for Feb round. Very stressed and overwhelmed!

    Your blog has been a life saver in finding out what I need to do.

    Just to say thank you!

  2. Thank you!

    I wonder if you’d be able to share your K99 application? It’d be very helpful as I don’t know anyone in my department/hospital who has applied for it.

    Thank you again!

  3. Anita, this is a really amazing gift to anyone contemplating a K99. And your writing is wonderfully clear, with a nice note of humor! I enjoyed reading from start through updated finish–it was quite a cliff hanger. Sorry you didn’t land the award, but you made the most out of your efforts by sharing your experience. Just know that for every reader who posted a reply, there are probably 10 more readers frantically referring to it while struggling with their current applications this cycle. THANK YOU for your generosity!!!!

  4. Well, I found this kind of late, so it won’t be helpful to you now, but I was in the same boat as you. I submitted my first round K99 in the June cycle, and the revision would have been due in March, a month after my eligibility officially hit the wall. Because of that, when I got my score back in early october, which was a 21, I spoke with my PO about my options. She said that funding would be iffy, as i was outside the funding cut, but she pushed to have my reviews released to me early. I fixed the issues that were raised in the reviews as best as I could and resubmitted in November. This time I got a 10, so barring anything funky going on with funding, it will get funded. Fortunately, my issues were easy to fix (I have a decent number of publications from my postdoc so far so not one of the criticisms), but it is possible to get reviews back early for K99s if you are up against the time wall.

  5. Hi!
    Really helpful and fortunate to see your blog at this stressful moment.
    I’m also under preparation for a k99. I’m wondering where I could find more information/requirements for the reference letters?

    libra

  6. Omg!!!! You have no idea how relieved I am to read this post. I checked out your original K99 post as well. As a postdoctoral fellow, I am formulating my K99 grant application and I will surely go over your original K99 post as well. It is very easy to follow and you write so well. Thank you once again for the wonderful post. I wonder if you would be able to share your K99 application? I am not sure if anyone from my department has applied for it, but having yours as an example would be great.

    Thank you

  7. Very nice post and helpful comments. However, as a senior (in the cane and slippers sense of the word) study section member and frequent NIH reviewer let me say that your publication record is not “entirely out of your control”. It is the one thing you need to control and build vigorously. How else can we evaluate your potential for future contributions to the field? If your advisor is too busy to read your manuscript, that also does not bode well for the training experience. A weakness here will make most reviewers I know uncomfortable and likely sink an application.

    • You are making two separate points. One is that publication record can be an important indicator of future success; yes, this may be true (or not… a debate for another time). This is completely independent of your second point regarding whether or not one’s postdoc publication record is within their control. If a postdoc has been extremely productive, completed a research project, and written at least one manuscript, but their advisor refuses to allow them to submit it within the K99 timeframe, then this is not a situation they can control unless they can go back in time and choose a different lab. Whether or not it “bodes well” is a separate issue.

  8. Hi Anita,
    This is a great blog and I’m glad I stumbled upon it. I’m about to start writing my K99 application and was wondering if I could get a copy of your application as a sample.

  9. Hi – thanks so much for this post, very helpful! I have a question about salary, and maybe this varies by institution… But I can’t seem to find the answer! Does the training/post doc portion of the grant pay at the same rates as the NRSA fellowship stipends, or is this a way to secure a higher paying position while remaining a post doc? Thanks!

Leave a Reply

Your email address will not be published.