K99/R00 Reviews

So, I finally got the reviews for the K99/R00 grant application I submitted to the NIH in June. Since the K99 application guide that I wrote back then has turned out to be one of the most-read articles on my site, I wanted to write a follow-up post about my reviews for those who are interested.

How the K99 is scored

First, let’s discuss how the applications are scored. Your application will be read by three reviewers on the study section, each of which will score your grant in five categories. In each category the scores can range from 1 to 9. 1 is the best and 9 is the worst. Here are the categories:

1) Candidate

  • This score reflects what they generally think of you and your background, publication record, etc. I think it’s mainly based on what you write in the “Candidate” section and your biosketch, as well as your reference letters.

2) Career Development Plan / Career Goals / Plan to Provide Mentoring

  • This score is based on your training plan, career goals and objectives, the mentor plan, letters from your co-mentors and collaborators, and possibly the description of institutional environment. The reviewers are evaluating whether the training you’ve proposed is appropriate given your background, career goals, and timeframe.

3) Research Plan

  • This score is based on your research proposal, ranging from its conceptual merits and potential impact on the field to its technical feasibility and experimental design.

4) Mentors and Co-mentors, Consultants, and Collaborator(s)

  • This score reflects not only the letters that your collaborators provide but also how well you convince the reviewers that these people are amazingly qualified and will greatly contribute to your project.

5) Environment and Institutional Commitment to the Candidate

  • Here’s where the reviewers look the statements regarding equipment, facilities and resources, and the letter of institutional commitment. They will judge whether your institution, department, and lab are a good place for doing the kind of science you’ve proposed and training you to be a PI.

In addition, the reviewers will look at your other statements regarding vertebrate animals, training in the responsible conduct of research, resource sharing plan, budget, etc. But for these things the reviewers will simply indicate whether the plan is “acceptable” or not. If something is not acceptable you need to fix it before they’ll give you the grant, but it may not impact your score (unless it pisses them off).

What’s a good score?

So you’ll get the 5 category scores from 3 reviewers, and you’re hoping for mostly 1s and 2s. But none of these numbers really matter. All that matters is the final impact score, and this isn’t simply an average of the individual scores.

Here’s how it works in the study section. First, the three reviewers who actually read your application will present their opinions to the entire section and lead a discussion about it. Then all the members get to vote. They each come up with a single score from 1 to 9 (again, 1 is the best), those numbers are averaged, and that average is multiplied by 10 to get your impact score, ranging between 10 and 90.

The successful K99 applications that I’ve seen received scores of 20 or lower. But the scores don’t always relate directly to funding. Which applications get funded will be decided at a separate council meeting at least a month after the initial review.

Most of the decisions are made based on the impact scores, but I’ve heard that sometimes there can be exceptions for extenuating circumstances. And the distribution of scores varies for each batch of applications—if there were a lot of great scores in your batch then you’ll need a better score to get funded, and vice versa.

So, how did I do?

I received my impact score about four months after I submitted the grant. You don’t get the reviews (“summary statement”) till a month later.

My score was 42. While 42 may be the answer to life, the universe, and everything, it is not a good K99 score. As mentioned above, anything in the 10s is great, 20s are good, above that not so much.

So for a month I wondered what the reviewers hated about my application and whether I had given everyone terrible advice in my K99 application guide and would have to crawl into a hole and hide.

Then I got the reviews. They were actually pretty good.

The reviewers generally liked my research proposal. Two of them mentioned some specific weaknesses, but nothing that seemed that major. The scores I got for the Research Plan category were 3, 3, and 2.

The reviewers loved my training plan and all the other stuff I wrote in the “Candidate” section. I also got high marks for the Mentors/Collaborators and Environment/Institutional Commitment sections, which I expected since my advisor, collaborators, department, and university are all very highly regarded in the field. For these three categories (#2, 4, and 5 listed above) my scores were mostly 1s with a couple of 2s. Yay.

So what didn’t they like?

The fact that I haven’t published any papers during my postdoc.

Seriously, that’s about it. [Update: see below for new information from my PO.]

(Aside: It’s not like I didn’t anticipate this. This was the very reason I started putting together a paper last year and gave my advisor a complete manuscript in February. Unfortunately my advisor is a busy guy, and long story short, he still hasn’t signed off on submitting my paper.)

My lack of publications was reflected in the scores for the Candidate category, which were 4, 4, and 2. But even those scores aren’t terrible. If you average all the individual scores I got for the five categories, it comes out to 1.9, which (multiplied by 10) would be an impact score of 19. That’s a far cry from 42.

So it seems like that one thing – a lack of publications – completely overshadowed everything else. It also seems like maybe the rest of the study section didn’t like my application nearly as much as the three main reviewers did, possibly because they had differing opinions on the importance of one’s publication record.

My summary statement mentioned this specifically: “While all reviewers did not have the same level of concern about this issue, some on the panel expressed strong reservations.” Looks like those people won.

I’ve heard that the Program Officer (PO) can give you additional feedback and guidance on your reviews. My PO is currently out of the country so I have to wait till December to speak with him, but I’ll update this post if his feedback changes anything.

Update on 12/6/2016: I finally got to speak with my PO and he told me that yes, the main issue was my lack of papers, but in addition, some members of the study section had concerns about my research plan. Some specific issues he mentioned (which were alluded to in the reviews) were that the models I proposed were too simple, the preliminary data wasn’t strong enough, and the data analysis methods were limited. So for those of you worried that my story means your application will be rejected purely based on a lack of papers, this information should give you hope that maybe a really really strong research plan can overcome that limitation.

Will it get funded?

I highly doubt my application will get funded. I’ve been told that you never know, even with a score that seems unfundable, because we don’t know what kind of scores they gave to the rest of the applications. There’s also the possibility of writing some kind of appeal if you felt you were unfairly reviewed or have an extenuating circumstance.

IF I can improve my publication record in the next 6 weeks before the council meeting by getting my paper submitted or even accepted (ha), then that might be grounds for an appeal—but it’s a longshot.

Update on 12/6/16: My PO told me that for this study section, there’s no way that a score of 42 is fundable. Since I’m not eligible to resubmit my application, he recommended that I instead apply for a K01 award, which has a longer eligibility window. I haven’t looked into the K01 very much and I’m not sure I have the energy to apply for any more NIH grants as a postdoc, but I thought I’d mention it for others in a similar situation.

Final takeaways

The good news for me is that most of what I wrote was received pretty well. The main thing that brought me down was something entirely out of my control. The comments I got on my research proposal were fairly constructive and could help me on future grant applications.

The good news for you is that this probably means my K99 application guide does contain some solid advice. In particular, I was praised for assembling a diverse team of collaborators and a “clear and excellent” training plan. I also think my statements addressing the institutional environment, facilities and resources, equipment, etc. were successful in convincing the reviewers that I’m working in a great environment.

In addition, all the work I put into the writing did not go unnoticed (though it may go unfunded). The reviewers said my application was “well-organized” and “beautifully written”. This didn’t magically happen on its own; it took lots of editing and rewriting. So don’t put yours together at the last minute—leave plenty of time to rework it! Even if they’re not explicitly scoring you on your writing, good writing and organization makes it much easier to understand the content.

That’s all for now. I hope hearing about my experience is helpful for those of you putting together your own applications or navigating the review process. And if anyone else wants to share their own experience, feel free to comment below!


Comments

K99/R00 Reviews — 43 Comments

  1. Hi!

    Great blog!

    I’m a post-docs from harvard and I’m writing my first K99 for Feb round. Very stressed and overwhelmed!

    Your blog has been a life saver in finding out what I need to do.

    Just to say thank you!

  2. Thank you!

    I wonder if you’d be able to share your K99 application? It’d be very helpful as I don’t know anyone in my department/hospital who has applied for it.

    Thank you again!

  3. Anita, this is a really amazing gift to anyone contemplating a K99. And your writing is wonderfully clear, with a nice note of humor! I enjoyed reading from start through updated finish–it was quite a cliff hanger. Sorry you didn’t land the award, but you made the most out of your efforts by sharing your experience. Just know that for every reader who posted a reply, there are probably 10 more readers frantically referring to it while struggling with their current applications this cycle. THANK YOU for your generosity!!!!

  4. Well, I found this kind of late, so it won’t be helpful to you now, but I was in the same boat as you. I submitted my first round K99 in the June cycle, and the revision would have been due in March, a month after my eligibility officially hit the wall. Because of that, when I got my score back in early october, which was a 21, I spoke with my PO about my options. She said that funding would be iffy, as i was outside the funding cut, but she pushed to have my reviews released to me early. I fixed the issues that were raised in the reviews as best as I could and resubmitted in November. This time I got a 10, so barring anything funky going on with funding, it will get funded. Fortunately, my issues were easy to fix (I have a decent number of publications from my postdoc so far so not one of the criticisms), but it is possible to get reviews back early for K99s if you are up against the time wall.

  5. Hi!
    Really helpful and fortunate to see your blog at this stressful moment.
    I’m also under preparation for a k99. I’m wondering where I could find more information/requirements for the reference letters?

    libra

  6. Omg!!!! You have no idea how relieved I am to read this post. I checked out your original K99 post as well. As a postdoctoral fellow, I am formulating my K99 grant application and I will surely go over your original K99 post as well. It is very easy to follow and you write so well. Thank you once again for the wonderful post. I wonder if you would be able to share your K99 application? I am not sure if anyone from my department has applied for it, but having yours as an example would be great.

    Thank you

  7. Very nice post and helpful comments. However, as a senior (in the cane and slippers sense of the word) study section member and frequent NIH reviewer let me say that your publication record is not “entirely out of your control”. It is the one thing you need to control and build vigorously. How else can we evaluate your potential for future contributions to the field? If your advisor is too busy to read your manuscript, that also does not bode well for the training experience. A weakness here will make most reviewers I know uncomfortable and likely sink an application.

    • You are making two separate points. One is that publication record can be an important indicator of future success; yes, this may be true (or not… a debate for another time). This is completely independent of your second point regarding whether or not one’s postdoc publication record is within their control. If a postdoc has been extremely productive, completed a research project, and written at least one manuscript, but their advisor refuses to allow them to submit it within the K99 timeframe, then this is not a situation they can control unless they can go back in time and choose a different lab. Whether or not it “bodes well” is a separate issue.

  8. Hi Anita,
    This is a great blog and I’m glad I stumbled upon it. I’m about to start writing my K99 application and was wondering if I could get a copy of your application as a sample.

  9. Hi – thanks so much for this post, very helpful! I have a question about salary, and maybe this varies by institution… But I can’t seem to find the answer! Does the training/post doc portion of the grant pay at the same rates as the NRSA fellowship stipends, or is this a way to secure a higher paying position while remaining a post doc? Thanks!

  10. Hi Anita, Many many thanks for an insightful and detailed description of the K99 process. As with everything postdoc, we as usually left to our own devices to figure things out, and this came as a real life-saver.
    Thanks!
    Dan

  11. This is a beautiful blog and somehow made me less isolated in these “lost land of academic environment”…I am not surprised at all about the publication point but consider this: as you said now they limit to 4 years postdoctoral training time and as you clearly evaluate it means you need to think about it 2.5 half years before that 4 years “deadline”, so how could they expect anybody to have a successful publications record?do they know how long it takes to publish nowadays? And don’t forget that not all the PIs will allow you to put your discoveries in your k99 since it means they can not use it for their own NIH grant, this is what I was told…the academia needs a deep revolution if they want to keep Research growing. Thanks for this great detailed blog and all the best of luck in your future! Elena

    • Thanks Elena. Yes, I think some academics are living in a different world than (most of) the rest of us… for example I was told in a K99 seminar that an “adequate” publication record is 1 paper per year! I’ve noticed that even people who get the K99 are often criticized for their record: one friend who had a Nature paper in <4 years was criticized by all 3 reviewers for his "disappointing" record, and another friend was told that 6 papers in <9 years (grad school + postdoc) isn't that great.

      • In all honesty, 6 papers in 9 years is NOT that great. With the current competitive nature of science, and particularly the K99/R00, publishing multiple papers a year in your postdoc is almost required. Sure, one amazing Nature paper could put you over the top, but some reviewers will argue that you had 1 successful project that you probably didn’t even start!

        You make a good argument that publishing isn’t everything, and I agree with that, but you are selling yourself in these proposals almost like a job application. It’s hard to get the job when other people applying for the job have a more consistent successful record of performing that very job?

        • Anyone who thinks that counting up someone’s publications can indicate a “consistent successful record” of being a good scientist exemplifies exactly what’s wrong with the system.

  12. Le sigh. Postdoc here, googling around to begin the process of K99 applications but all I’m really finding is a lot of reasons to not even try, even though my boss is encouraging it. I’m in chemical biology, so I’m with you that a limit of 4 years and an expectation of publication productivity AND promising preliminary data is a lot.

    So, what ended up happening – did you dig deep and go for the K01? I wouldn’t blame you one bit if you didn’t. I’m really feeling discouraged and I haven’t even tried yet. (I mean, I haven’t had a lot of luck, applied but didn’t get an F32 and a few others.)

    • No, I haven’t applied for anything else since then because my paper still hasn’t been published, and I don’t really believe it’s possible to get any kind of postdoc grant 5 years in without a publication.

      Sorry that you’re feeling so discouraged, but know that the vast majority of us postdocs are in the same boat, and rejection is the norm in academia– even for those who are considered successful. The high chance of rejection doesn’t mean we can stop trying! If you have a decent CV and think you can put together a good proposal and make a case for why you’re almost ready to apply for jobs (but still need a bit more training), then it may be worth a try. Of course, depending on your circumstances your time may be better spent on other things. Despite the rejection, I did find the K99 to be good practice for writing grants and really helpful in thinking about job applications and my future research plans. Good luck either way!

    • Nope, just the impact score. And apparently the range of scores can vary widely for each study section, but your PO can help give you an idea of whether your score is close to fundable.

  13. This and the previous post are both excellent. I too am googling around to see if I should start the K99 process. I am quickly approaching my four year limit and didn’t really want to do it but recently people are saying K99 is almost a necessity for a job in academia. Unlike your situation I do have some papers: a first author paper in review as of now (with the so-called glamorous journal, so it’s very hard to say it can be accepted any time soon), and two second-author IF>10 papers published. However, it looks like there are tons of efforts involved and I am wondering if spending all these time to get another paper done would actually be more beneficial than doing a K99 submission, especially that I can only have one submission, so mostly not getting funded in the end.

    • Yeah it’s a hard call. The K99 is not a necessity for an academic job, but it certainly helps a lot. If you feel you’re a good candidate on paper (postdoc publications, good grad school CV, in a good lab at a good dept/university) and also feel like you have the ideas for a proposal, or that developing those ideas is something you want to do (it’s very helpful for job apps) then I’d go for it. Also might depend on the funding % at your institute.

      • Hi Anita, thanks. My situation does satisfy most of what you mentioned, however based on my colleague’s experiences, first time application is still almost impossible to go through, most of them do make it in the second attempt though. Since I only can try once, I have decided against using at least two months of my time on the application, especially since I expect to deal with tough reviewers pretty soon on my main postdoc paper..

  14. This and your previous blog (the comprehensive guide) are great and very helpful! Thanks Anita! Your blogs are written clearly and informative from which I have learned a lot about the K99/R00 application. Thanks again and wish you success in securing a faculty position.

  15. Hi Anita,

    In my mind, this is a wonderful blog. I learned a lot about how to prepare for K99. You are so generous. I only can try once since the 4-year limit. But, anyway, who knows what will happen if we don’t try anything. Furthermore, we can learn a lot from the failure and find what we should do in the future. Every circle, there are only about 20% applicants can get this grant. That means 80% of applicants got rejected. There is one successful chance when you apply five times. So it is common. What we can do is try our best to do it. Not only for K99, it is for every right thing. So I will have a try with the great help of your post.

  16. Hi Anita, I want to thank you for a great blog. I eventually submitted my K99/R00, what a rollercoster of emotions during the writing process but your sample was such a great help. I finally got my Impact score (34) and the payline is 30/31 for NCI. I still have 2 years left in the 4-yr limit so I may be resubmitting depending on our discussion with the PO. Thanks again.

    • does anyone have an idea what is the NCI impact score payline for K08, mine came back 29, still waiting for the summary statement

  17. Thank you so much for your story! I just submitted and am in the hopeful phase. Good luck with your future endeavors.

  18. Congratulations on your recent faculty job! I was wondering if you ended up securing any NIH (or other) grants before becoming successful with your faculty application. Good luck!

  19. Hi Anita, thank you so much for your helpful posts! I learned a lot about the K99 application from your posts! Congratulations on getting the R01 grant! I second that publication is not an indicator of future academic achievements. I sincerely wish you a great future!

Leave a Reply to Anita Devineni Cancel reply

Your email address will not be published. Required fields are marked *