Don’t Let Your Research Questions Go Out Without PICOTS

Doing Research / Grants & Funding

All the best aims are wearing PICOTS (pronounced “peacoats”). Specification of your PICOTS* is the minimum outerwear required to prevent your research question from being caught in a downpour of questions. Having these details tucked in gets you ready to have a meaningful conversation with colleagues, evaluate feasibility, brainstorm about how to get the best study done, and prepare to share your concept:

P = Population

I (E) = Intervention (Exposure)

C = Comparator

O = Outcome

T = Timing

S = Setting

Use PICOTS as a checklist for operationalizing research questions and probing how the research would shape up under different assumptions. Ask:

Population: 

  • What group of participants is ideal?
  • Whom does this imply we need to include or exclude?
  • How would we operationalize those criteria?
  • What influence will that have on ability to identify participants/recruit well?
  • Do we need to worry about proof of concept or generalizability more at this stage?

Intervention:

  • What will participants do/experience in the study that is being tested for its effects?
  • What dose, frequency, intensity will be tested?
  • Do we need to invoke a specific behavioral or causal model?

Or Exposure:

  • What is the behavior, biomarker, experience, metric for which we are interested in evaluating the effect?
  • How will it be measured?
  • How will we ensure quality of the measure?

Comparator:

  • What comparison provides the most relevant contrast (e.g. usual care, no intervention, placebo, etc)?
  • What analytic approach will best support the comparison?
  • Does this comparator help test our causal model or could it be stronger and more direct?

Outcome:

  • What is our measurable outcome?
  • How will measurement be operationalized?
  • Do we need primary and secondary outcomes?
  • Can we achieve adequate power to assess the outcome?
  • If there is loss to follow-up, do we have alternative ways of assessing outcomes?

Timing:

  • Over what time frame will participants be recruited?
  • What is the time period over which intervention will be conducted for an individual participant?
  • How long after completion of intervention will measures be collected?
  • When will outcomes measured? How wide is the tolerable window for measurement?

Setting:

  • Where will the research be conducted or participants be recruited (e.g. academic tertiary care center, network of health department clinics, community-based, etc.)?
  • What are the characteristics of that setting?
  • If extant data, what was the setting in which the data was developed?

Try it, you’ll like it. And it’s better than the alternative of getting soaked later by questions and requests for details needed to clarify your concept for the research.

Taking PICOTS for a Spin

For example, if you’re interested in asking: “Do community-based lifestyle interventions really work?” or “What determines who stays in community-based lifestyle interventions?” work the PICOTS:

Initial Question: “Do community-based lifestyle interventions really work?”

Goal: Pilot intervention study with a primary aim of determining if an intervention results in weight loss

In this case a pilot would be a typical approach for estimating the effect size, feasibility, participant satisfaction, loss to follow-up, and need for adjustments to inform design of a future definitive randomized trial. So we sketch a picture of what the study could look like:

P: Adult women with physician’s permission who are registered for the first session of the 12-week New Beginnings Program, and who speak English or Spanish.

I: Structured small group (n=5 to 8) coaching program with 1) specific weekly goal setting targets (eliminating sodas, understanding metabolic effects of exercise and tracking, counting carbohydrates, planning daily physical activity, enhancing sleep, writing an individual vision for one’s health, making a long term health contract with oneself, etc.) 2) three small group resistance and circuit training coached sessions each week, 3) social media peer connections, and 4) individualized exercise, diet and stress-reduction prescriptions.

C: Women who have applied for the program and are eligible but who are currently on the wait list with an anticipated wait time of 14 or more weeks.

O: Primary outcome will be weight loss, measured as difference between first measurement (in pounds to one decimal place on scale provided and calibrated by the study) at intake session and weight at the last group session. Outcomes will be grouped by completion status where completers attended ≥75% of schedule sessions and non-completers fewer sessions. Weight loss will also be described by group for each of the 12 weeks. For secular trend among those with an intention to lose weight the wait list comparison (secondary analysis) group weight will be collected from initial application (or as documented in physician’s permission letter in application) and weight at intake session, adjusted for elapsed time between application and start of program.

T: The intervention will last for 12 weeks of structured lifestyle and exercise coaching. Informal peer and social media networks established during the intervention will continue unsupervised after completion. Secondary outcome data will be collected at 3, 6, and 12 months after completion of the intervention.

S: Privately owned gym facility partnered with non-profit (501C3) to provide a comprehensive lifestyle intervention program to means-tested low income women, the majority of whom are age 40 and older, African American and weigh, on average, more than 200 pounds.

Initial Question: “What determines who stays in community-based lifestyle interventions?”

Goal: Observational study of whether baseline mental and physical health status, locus of control, and dispositional optimism are associated with completion of a community-based lifestyle intervention

P: Adult women with physician’s permission who are registered the 12-week New Beginnings Program which is a structured small group (n=5 to 8) coaching program with 1) specific weekly goal setting targets (eliminating sodas, understanding metabolic effects of exercise and tracking, counting carbohydrates, planning daily physical activity, enhancing sleep, writing an individual vision for one’s health, making a long term health contract with oneself, etc.) 2) three small group resistance and circuit training coached sessions each week, 3) social media peer connections, and 4) individualized exercise, diet and stress-reduction prescriptions.

E: Lower levels^ of physical and mental health as assessed by Short Form 36, lower self-efficacy (assessed by Generalized Self-efficacy Sale), and greater pessimism (assessed by the Revised Life Orientation Test) at baseline.

C: Higher levels^ of physical and mental health as assessed by Short Form 36, internal locus of control, and greater optimism at baseline incorporated into logistic regression models to assess association of characteristics with outcome.

^ Cut offs to be determined by distribution of traits in context of national normative reference data.

O: Program completers will be classified as those who attended ≥ 75% of scheduled sessions and non-completers fewer sessions. Will also capture week of attendance for secondary analysis in time-to-event analysis.

T: The assessment will be completed within 12 weeks.

S: Privately owned gym facility partnered with non-profit (501C3) to provide a comprehensive lifestyle intervention program to means-tested low income women, the majority of whom are age 40 and older, African American and weigh, on average, more than 200 pounds.

But I can’t possibly know these details when I first think the thought!?

True, but you can get much closer than you think. Start by daydreaming and then add parameters that are initially fantasy. The approach to shaping questions jumpstarts thinking that then leads to:

  • Productive generation and sifting of research ideas.
  • Greater focus for literature review.
  • Strategic thinking about multiple aspects of feasibility .
  • Weighing the best choices for measures of exposure, covariates, and outcomes.
  • Enhanced ability to rapidly gather input from others.

Related Posts:

Acing Your Observational Research Aims

All research proposals – grants, dissertations, internal funding – must ace the description of aims.  Many scientific questions are interesting.  Not all are useful.  You must persuade your readers that the proposed aims/hypotheses to be tested and the related analysis will fill gaps in scientific knowledge.

Don’t Crash on Approach

Getting the approach – the methods section of your grant –  fine-tuned is literally the heart of it all. You must land your science smoothly. Study section members know, and recent evidence confirms, your grant’s score is not an equal weighting of component scores. NIH criterion scores are for significance, innovation, approach, investigators, and environment.

* Gordon Guyatt initially described PICOTS in Guyatt G, Drummond R, Meade M, Cook D. The Evidence Based-Medicine Working Group Users’ Guides to the Medical Literature. 2nd edition. McGraw Hill; Chicago: 2008. Subsequently the framework became standard for formulating inclusion and exclusion criteria for conduct of systematic evidence reviews and meta-analyses of interventions.

Conveying Institutional Support

Grants & Funding

Grant reviewers want to invest in success.  If you’re applying for a career development award, you must convey the support of your institution.  If your chair doesn’t want to invest in you, why should the NIH or other funding agencies?

Dr. Nancy J. Brown, chair of the Department of Medicine at Vanderbilt University Medical Center and author of hundreds of letters of institutional support, recently shared her top suggestions for crafting a letter that shows your department chair is invested in your success.

(And yes, you will be drafting these letters.)

The Kiss of Death. Never, ever say that institutional support is contingent on receiving the award.  See the point above about grant reviewers wanting to invest in success.

The letter should promise what you need to succeed. This is typically at least 75% protected time for research and career development activities. Spell out teaching, clinical, and administrative duties you will put down upon receipt of the award, or duties from which you will continue to be protected if you don’t have them already.  The letter should also describe the appropriate office and lab space, equipment, and other resources (including access to research populations) the division or department will provide.

Success looks like: Your becoming an independent, tenured investigator.

Open strong. The opening recommendation should include superlatives, e.g., “I write to provide my strongest support…”

Provide some (superlative) biography. This person is a star, and his/her unique qualities are…  When drafting a letter, this is where you let your ego off the leash.  Statements that may seem hyperbolic to you may be appropriate.  If this is uncomfortable for you, ask your mentor to include more superlatives when he or she revises the draft.

Include a bit about your science. The letter should convey that you’re doing important work, and your chair knows about it.

Commit. As above, spell out how your time will be protected, space provided, and other items and investments you need to succeed.  Count everything.  For example, if you work somewhere that sells access to an electron microscope in small chunks for a small fee rather than you having to buy the whole $2 million thing yourself, count how much money that’s worth, especially if it explains why your research funds may seem low.

Close with a strong summary of why you are a promising investment.

Use good etiquette. Always provide a draft. Use the right hierarchy for your department—if it should go to your mentor, then division chief, then chair, send it that way.  Allow sufficient time for busy people to read, revise, and submit the letter.  (Think weeks, not days.  And follow our guide to pacing the components of your proposal.)

Common grammatical and style errors to avoid:

  • Comma rather than a colon after the greeting (a business letter always uses the colon, even if it’s to your best friend).
  • Use of passive voice.
  • Use of “their” as a singular pronoun (“The candidate will then select their…” vs. “The candidate will the select his or her…”)
  • “Data” are plural. “Datum” is singular.
  • Split infinitives: Any word in between “to [verb].” For example, “to boldly go where no man has gone before…”  Good for Star Trek.  Not for you.
  • Spelling errors of any kind.

Drafting a letter of collaboration? Check out advice from collaboration pros.

Not that Kind of Selection: Tales of Picking Which Grants to Write

Grants & Funding

In my last post, I blogged about the different types of grants that are available to early stage investigators (ESIs) and the benefits of these awards. If you are like me, you were overwhelmed when you saw the list the first time. There are too many grants to write as a new principal investigator (PI), so I have developed a series of questions that helps me decide which ones to pick. As always these days, n=me.

The caveats: I am a year and a half into a tenure-track position in a clinical department at an R1 medical school. I have a good start-up and a fairly expensive wet lab research program. I am not an expert in time management and I have a small lab, so the amount of data and publications are limited. I still work at the bench a bit, and my main focus is covering salaries rather than supplies. I was awarded a K99/R00 as a postdoctoral research fellow, so I have an established track record of funding and additional funds in the laboratory. My goal, therefore, is to establish stable funding in the lab before I fall off the K cliff. Your priorities may be different, and the answers to these questions may weigh differently in the final calculus of whether to write an application.

The questions: When I am notified about a new request for applications, either through my institution, an email from relevant foundations, or the NIH, I work through the following questions:

Is this grant a priority? A large part of being a PI is identifying research priorities. If you have been submitting grants and reviewers keep commenting on a lack of senior author publications or quantity of preliminary data, your time might be better spent completing papers and pilot experiments. The point of a start-up package is getting your research program launched, and that includes getting the first couple papers out and completing pilot experiments with ready-to-use funding. In my R00 transition documents, I included a timeline for submitting my first R01, and I met that goal this year. Whether the reviewers will share my enthusiasm for the project, or criticize my productivity on the K99/R00, remains to be determined.

Am I eligible and in research scope? Some of these grants have very specific eligibility requirements, so confirm you satisfy all of the eligibility criteria. These grants can also have a very defined research focus. As wonderful as the ESI MIRA (R35) through NIGMS is, my research is not of interest to the NIGMS.

How big is the award? I am looking for multi-year project funding. In general, I do not apply for smaller, yearlong grants. When my start-up is gone, I expect I will be much more interested in the smaller, pilot-level awards. Decide whether these smaller grants satisfy your needs, for example establishing a track record of funding, before applying.

How long is the application? The more involved the application, the longer the funding period needs to be and the larger the award. If the grant is very very competitive, but the application is short and easy, I will apply, regardless of how small the odds.

How many applications will be awarded? The smaller the number of grant awards, the tighter the applicant pool needs to be. For example, if there will be one award made for studying Terrible Disease, I probably will not apply. If, however, there is one award being made for studying Super Important Signaling Cascade by a Lab Mouse in Terrible Disease, I will probably apply. I use this criteria to justify applying for some of the very prestigious awards, like Searle or Pew, where they are funding a limited number of applicants, but the applicant pool is quite small too.

Is there are an early investigator pool? This should be a no-brainer. If you are competing with every Terrible Disease researcher as an ESI, you are unlikely to be given the award. I am saving these grants for when I am more established.

Is the application re-usable? Some applications have a very unique format (like the MIRA or DP2) or a very specific research focus. If I am unlikely to submit the application anywhere else, I think carefully about whether I want to invest the energy in a grant whose parts will only be used once, maybe twice.

Will I get feedback? I do not rule out grant applications that do not provide feedback, but I am likely to submit the same idea somewhere else that does provide feedback. It is impossible to gauge interest in your research ideas if reviewer comments are not provided. If these are your first grants, make sure you are getting some feedback on the grants you submit.

Is this a place I can get more money in the future? I am a big believer in building relationships with funding agencies. Even though some of these prestigious foundation awards give you money and make your CV stand out, not all of them have grants for mid-career faculty. I am more interested in building my relationship with the NIH, which will hopefully fund me for many years to come.

Are there predoctoral and postdoctoral grants? I am also thinking about the future of my trainees. If I establish a funding reputation at a foundation, I want to know that my trainees can benefit from my establishment there when it comes to mentored awards in the future.

When is it due? There are a large number of grants due in September and October. Be kind to yourself and do not submit all your grants for the year in a two-month period. Submitting many grants in the same time period is stressful, but more importantly, it raises the very real possibility of multiple grant rejections during a short time frame. As new PIs, we are still working on thicker skin, so spread out the potential rejections.

The one question I do not ask: I never ask myself if I am competitive for a grant. I apply and let the reviewers decide.  I have been in this game for too long to believe what anyone tells me when it comes to what a grant awardee “looks like.”

In an ideal world, I would write all the grants. In the real world, grant writing competes with laboratory work, training, mentoring, and paper writing, which are very important parts of successful grant applications. I am sure other new PIs have different strategies and more senior PIs will have their own opinions too, but this has been my attempt to keep sane. Feel free to share yours in the comments. Stay tuned for more tales!

 

Did I miss an important point? Do you have questions or concerns about the post? Or perhaps an anecdote to contribute! Feel free to send some electrons my way in the comments, via Twitter @PipetteProtag, or through traditional electronic mail pipette.protagonist@gmail.com

More Resources

Grant Funding Strategy: Which Grants to Apply For?

Designing Your Career

Which Study Section Should I Pick? Try the Assisted Referral Tool!

Not that Kind of Grant: Tales of Early Career Investigator Grants

Grants & Funding

As a graduate student and postdoctoral fellow, my mentors wrote National Institutes of Health (NIH) and Terrible Disease Foundation grants. That was it. Imagine my surprise when I started as a new principal investigator (PI), and I was inundated with grants of which I had never heard, some of which had money dedicated to early stage investigators (ESI). What are these grants? And where do they come from? Here is a semi-comprehensive list of the grants in the US that I have seen, mostly with a focus on the ESI/young investigator/new PI category. As always these days, n=me, and this is my experience with grants up to now.

The basics: ESI status as described by the NIH is a PI “who has completed their terminal research degree or end of post-graduate clinical training, whichever date is later, within the past 10 years and who has not previously competed successfully as a PI for a substantial NIH independent research award”. Career development awards, small pilot grants, and instrumentation grants are excluded from this list. Foundations and other government agencies, like the Department of Defense (DOD), can modify this definition. Some do not make exceptions for time spent in clinical training, others choose to limit the eligibility window to time in faculty position rather than time after PhD and/or MD, while others define their young investigators by how much money they have been granted for their independent research programs.

Grant types: Grants come from different places, including your institution, foundations, non-NIH federal agencies and of course, the NIH. Here are the different types of awards, with examples, that have arrived in my Inbox since I started.

Internal grants: Your institution will have pots of pilot money available for new projects. These tend to be smaller, one year awards, designed to generate preliminary data for R01-level grants. These are a great way to get some pilot funding for early studies and build on your track record for funding success. Some of these funds come from large center grants (Clinical and Translational Science Awards [CTSA] Program, American Cancer Society (ACS), Cancer Center Support Grants [CCSGs] for NCI-designated Cancer Centers [P30]) or program projects that have pilot programs built into the grant (Program Project Grants [P01], Specialized Grants [P50]). Applying for these pilot funds is a great way to build relationships with these centers and programs. Other funds can come from the institution, philanthropy, or the state.  These applications are shorter to write, do not require much preliminary data, and the outcomes are announced much faster than external awards. If you are fortunate to have a career development award, you may not be eligible for some of these internal grants as a ESI. I have applied for one pilot grant, and while unsuccessful, it was reviewed fairly, quickly, and the provided feedback was incorporated into a subsequent application.

Foundation grants: Foundation grants have smaller grants for pilot studies but also R01-level funding awards for ESIs. There are the disease specific foundations (American Cancer Society, American Heart Association, March of Dimes, American Diabetes Association, etc.) and more general biomedical research foundations (Searle, Pew, MallinckrodtBeckman, etc.). Some of these require nomination by your institution, so you first submit a letter of intent to your institution and write the grant if you are the institutional nominee. These general biomedical awards are highly competitive, and a member of my mentoring committee fondly refers to them as beauty pageant awards. Whether or not my application will be pretty enough will be determined in the spring. I am also preparing an application for Terrible Disease of Interest Foundation for the fall, for which I am much more optimistic.

Non-NIH government grants: The DOD, Department of Energy (DOE), and Veterans Affairs (VA) all have ESI grants for which you can apply, if you or your research satisfy the criteria. These are a great alternative to NIH funding and can be a better fit for your research. For example, the DOD CDMRP focuses on disease research that will drive improved outcomes for servicepeople, so their interest in basic science is more limited. Moreover, the DOD has very specific areas of interest, so if you focus on a specific disease in their research portfolio, these grants may be a good fit for your developing research program. I have written one of these grants already and am revising it for this year’s submission.

NIH: Of course, the biggest player in the room is the NIH, with various grant types. As an ESI, I have been contemplating the R01, R21, MIRA, and DP2 grants. You are probably most familiar with R01s, which serve as the standard funding mechanism for most laboratories. These are five-year grants and require preliminary data. They do, however, have an ESI bump, where good ESI grant applications outside of the funding range will be considered for funding. What this looks like in practice varies by institute, but a general summary can be found here. Conversely, the R21s are shorter, pilot style grants that require less preliminary data. However, these grants are competitive, are not offered in all institutes, and do not offer the ESI bump. MIRA and DP2 grants a very different beasts, having completely different writing styles and scopes work. These are more “big picture” grants, focused on the general research plan of the investigator rather than specific aims. The MIRA (Maximizing Investigators’ Research Award for Early Stage Investigators, R35) is offered through NIGMS and is focused on research in the scope of the institute. My research does not fall under the purview of NIGMS, but I think it is a phenomenal grant for new PIs working in pure basic science. The DP2, also known as the NIH Director’s New Innovator Award, is focused on highly innovative science and has no preliminary data requirements. The DP2 focuses only on the applicant and there are no co-investigators or collaborators to include. It is a really interesting grant mechanism. Whether they think I am an interesting applicant remains to be determined. Finally, R15 and R03 grants have restrictions based on institutional funding and institute of interest, respectively, but these are also great grants for ESIs.

How to learn about these grants: The internal grant applications requests will come from your institution. For foundations, your institution is likely to have a Foundation Relations office that will send out requests for applications for foundations and societies, often times from places of which you have never heard. Reach out to this Office, meet some of the staff, and get on the email list. You can also sign up for emails with funding announcements from non-NIH agencies like DOD’s CDMRP. Finally, the NIH has a weekly email of all funding notices for which you can register.

I hope you have enjoyed this primer on ESI grants and are gleefully planning your application schedule. In my next post, I will describe my selection process for deciding which grant applications to write. For now, I have to finish outlining this DP2. Stay tuned for more tales!

Did I miss an important point? Do you have questions or concerns about the post? Or perhaps an anecdote to contribute! Feel free to send some electrons my way in the comments, via Twitter @PipetteProtag, or through traditional electronic mail pipette.protagonist@gmail.com

Using NIH RePORTER to Find Your Guide

Grants & Funding

In much the same way the Assisted Referral Tool can help you pick a study section, the Program Official option for NIH’s Matchmaker tool provides insight into the Program Officer who works with the most projects that look like yours.

To use it, visit the Matchmaker portion of NIH RePORTER. (New to RePORTER? Here’s how to find out a wealth of information about grants on your campus and elsewhere.) In the text box, you can enter a paper or grant abstract, or any other text you want to search on up to 15,000 characters. Click the “Similar Program Officials” button.

In a few moments, the system will return a list of up to 175 program officers, starting with those whose grant portfolio most closely matches the text you put in. Click on a name to bring up a list of the active grants for whom that person is the PO. Are they in the same ballpark as your research? Then go back to your search results and grab the email address of that PO to start a conversation.

Why might you want to talk to your PO before submitting a grant? Several reasons:

  • To find out if the institute is enthusiastic about your research area or if there might be a better fit.
  • To get clarification on which study section is ideal.
  • To confirm the appropriate FOA.
  • To get suggestions for alternative programs, FOAs, or institutes.
  • If you have questions about your budget or scope of work.

Also of note: Matchmaker will give you graphs of how many POs at each institute work with grants that look like yours, as well as how many of which grant mechanisms (R01, R03, K08, etc.) they handle.

Additional Resources

Harness the Immense Power of Nosiness in NIH RePORTER

Which Study Section Should I Pick? Try the Assisted Referral Tool!

Video: How to Use NIH Matchmaker

What I Wish I’d Known Before I Wrote My K

Grants & Funding

Three K awardees (K01, K08, K23) share the advice they wish they’d received before preparing career development awards. Writing your K? Listen up.

Training Plan and Mentors

Have an endgame, and goals to get there. In your training plan, you should be able to articulate your research aspirations for the longterm (and make sure that fills a niche in your field). Work backwards to establish long-term, intermediate, and short-term (i.e., K) goals.

This is your chance to tell your story. How does your background set you up to achieve your endgame goals? What else do you need to get you there?

Mentors probably can’t teach you piano, but they can open doors to resources you need.

It’s common and encouraged to have interdisciplinary mentoring committees of 3 or 4 senior faculty. For your application you need to identify scientifically accomplished investigators. This assures reviewers you can access their resources, problem-solving advice, career development coaching, and a larger network of colleagues. While peers and near peers may still be the people you walk down the hall to see when you have a question, your primary mentor(s) should be able to open doors for you. Reviewers will look up your mentors in NIH RePORTER. If they see that your mentor has only one grant and it’s running out next year, they’ll suspect, “This person can’t keep their own career together. How are they going to help someone else?” (Here’s a great description of who should be on your committee.)

Be specific about how you’ll learn and how your mentors will help you. Make explicit the learning activities (e.g., courses, regular meetings, writing a review article, carrying out scientific project, mastering a specific technique) and scholarly products (some overlap with learning activities) related to each training goal. Identify each training need with a mentor or consultant. Reviewers will focus on specifics: Coursework and whether it ties into the proposed science; special training opportunities like workshops, frequency and content of mentorship meetings; justification for the research funding outside the PI salary support.

Summarize all activities and products in a comprehensive timeline. Show reviewers how the training and science will unfold and be sure to include targets for grant submission and resubmissions.

Have someone with K review section experience review this section (even if they’re outside your subject area).

Research Strategy

Your research plan:

  • has to be feasible (be careful not to be “too ambitious”)
  • have strong and sellable novelty/impact
  • should facilitate your training in a (set of) new method(s) or a higher level of skill

Propose research with just the right amount of complexity. If one of your figures looks like this, seek help. Maybe from a mentor!

Find the sweet spot for the complexity of the science you propose. A multisite clinical trial would be viewed as too difficult to pull off on a K, while complete reliance on an existing data set might be viewed as not ambitious enough.

Non-dependent aims are key. This applies to all grants, including K’s. If Aim 1 tests the effectiveness of Drug A vs. Drug B and Aims 2 and 3 propose to explore why Drug A works better, you’re dead in the water if it doesn’t.

Show that you know when to submit your R. A K is a five year grant to do 2.5 to 3 years of work preliminary to a more sweeping study. To ensure success, you should get enough data in the first two years of your K to write and submit your R01 or equivalent in your third year.

Tables and pithy summary-type figures are your friends. Reviewers who are not assigned readers may read only aims, abstract, tables, and figures before deciding if they will read more.

Study section choice matters. Try to direct your grant to a study section that you think will understand the grant and its merits. Your program officer can be enormously helpful here; ask if specific study sections have been responding well to similar types of research and they can usually tell you.

Funding decisions are unpredictable. Sometimes really good ideas aren’t funded.

Sometimes you have to “play the game.” Study funded K awards and give review panels what they’re looking for – don’t neglect expected content. All parts of the proposal matter.

Letters of Support

Drafting your own letters is standard. It will feel weird, but it makes the grant more of a unified package. Unleash your ego and talk yourself up. Mentors and collaborators will take your draft and modify it to give it their own voice.

Give your letter writers time. A month’s notice is standard. More is better. Two weeks is pushing it. Remember, you’ve taken the advice above and asked high-powered scientists to mentor you—this means they are busy traveling, going to meetings, and trying to fit their own science around institutional responsibilities.

Resubmissions

Resubmission in progress.

Plan to resubmit. Remember resubmission is expected. Most eventual K awardees have to resubmit their K. Think ahead about what you can accomplish between submissions.

Show productivity. Work to get a paper out or make progress on one of your aims. You might even ask for a shorter funding period in the resubmission, because you’ve made progress. This is impressive to reviewers.

Have someone new look at it. To ensure your response to reviewers strikes the right tone, ask someone uninvolved or only peripherally involved with your work or the grant to look at it and provide outside perspective.

If reviewers liked a section, keep it. Don’t court trouble by changing what worked. Some successful resubmitters have included a copy of the scores each reviewer gave on each section and noted that since Approach or RCR or whatever was criticized the most, that’s where they’ve focused their revisions.

And Last…

Start early and pace yourself.

It will take longer than you think. Start well before you think you need to. Our grant pacing guru recommends at least 16-20 weeks out.

Advice that applies to other grants (e.g., R01’s) may not be optimal for a K. R01s don’t have mentor letters or letters of recommendation, for example.

All of the sections matter, even components like RCR and Vertebrate Animals. We’ve seen grants rejected for review for inadequate RCR.

Don’t ever look at a grant score on vacation. You say you won’t, but…

Thanks to panelists Josh Fessel, PhD, MD (who also blogs at the Edge); Katherine Gotham, PhD; and Wesley Self, MD, for their great advice.

More Resources

More Things I Wish I’d Known Before I Wrote My K

Which Study Section Should I Pick? Try the Assisted Referral Tool!

Grants & Funding

One important aspect of submitting grants to the NIH is selecting an appropriate study section.  While the general descriptions on the CSR website can be helpful, they often require further investigation, which can (and should) include contacting SROs, talking to current or former study section members, and/or looking up similar types of grants on RePORTER to see where they were reviewed.

However, it can often be difficult determining just where to start or where your grant may really fit in.  CSR has provided a new tool called the Assisted Referral Tool (ART) to help you find the perfect home for your grant.

ART allows you to paste in text from your grant (typically your Specific Aims) and provides a list of recommended and ranked study sections based on matches with words found in your text.  You may be surprised to find study sections you had not considered in the past (it also recommends SBIR/STRR special emphasis panels). ART is another tool to add to your arsenal when applying for NIH funding.  Visit ART.

Additional Resources

Discover What’s Getting Funded with NIH Matchmaker

Harness the Immense Power of Nosiness in NIH RePORTER

Diversify Your Funding Portfolio

Three (Grant) Peeves in a Pod: Appearance Matters

Grants & Funding

Since the holy trinity important things have come in threes—listen up. Beauty is in the eye of the reviewer and we like simple elegance:

1) Avoid gratuitous attention grabbing.

Really? All caps, bold, italics, and underlining in the same paragraph? If your use of emphasis is unrestrained, it irritates rather than helps my focus. Likewise, organization of sections must be uniform. You don’t want me to waste mental energy trying to intuit the meaning of inconsistent headers and sub-headers.

2) Keep your figures sleek.

Extraneous details in cartoons of mechanisms, causal models, and other illustrations pet my peeves. Including pathways you are not studying or arrows pointing to features that don’t matter to your grant makes me wonder if you ran out of time to customize or don’t know how to tighten up your thinking.

3) Give me simplicity.

Ditch tangles of text for lists. Highlighting innovations or detailing inclusion/exclusion criteria often doesn’t require full sentences. Bullets and enumeration are beautiful with the added benefit of making content easier to absorb and find again.

Finally, bilateral justification is of the devil: It impairs reading speed/efficiency, decreases recall of content, and causes stilted writing.

Make your ideas simple and attractive because even if I am only subconsciously peeved it can hurt your grant score…and only nearly perfect scores clear the payline.

More Peeves, More Cranky:

Formatting

Check Yourself

Staying on My Good Side

Trials and Tribulations

Write Better

Not that Kind of Progress: Tales of Setting up a New Laboratory

Grants & Funding

If you are amongst the truly lucky job applicants, you are probably setting up a laboratory as we speak. Having recently concluded our set-up, I can tell you what worked for us, and what we could have done differently. As with many things in science, there are many ways to approach this process, but here are the things that needed addressing and how we did it. N= me and my lab.

Take pictures of your old lab before you leave: The smartest thing I ever did was document extensively the equipment, plastics, buffers, and so on that we used in my postdoc lab. This was particularly useful with my microscope purchase. I did not know the specifics of the microscope that we had used for our dissections, but I knew the model and that we needed all the zoom stops. Now, said microscope model had been out of production for years by the time I ordered it, but its new equivalent is equally great. I also spent a lot of time copying lab notebook pages I could not take, protocols in the lab, plasmid maps, primer lists, antibody logs, genotyping approaches, and chemicals. If applicable, I highly recommend copying your former lab’s Institutional Animal Care and Use Committee (IACUC) and Institutional Review Board (IRB) protocols. Yes, this all takes time, but not nearly as much time as a) calling back to your old lab and requesting copies or b) recreating these documents yourself.

Complete the administrative stuff: As a postdoc, I did not appreciate the amount of administrative work my mentor did. When you start your lab, you will be responsible for developing your own laboratory chemical hygiene plan, Institutional Biosafety Committee (IBC) plan, IACUC protocol, and IRB protocol. You will also need to be added to all the administrative systems so you can approve employee time, place orders, submit grants, and monitor your budget expenses. This will likely include online and/or in person training. While the least fun, most of these administrative tasks need to be completed first, before you can do the science.

Evaluate your needs: The first step of setting up a lab is identifying what types of experiments you will be performing. If you are starting a lab with a career development award, this is fairly straightforward. If you are (temporarily) unfunded, this is a more existential question. What kind of a scientists are you? What equipment do you need every single day? What equipment do you use once per month? What Cores can you use instead? Are you going to use the same tried and true methods for a technique? Or are you going to update the newest quantitative methods? I should have spent a little more time contemplating whether I would use the same old technique or implement the newest technology in the lab. Had I done so, I would have likely spent a little less money on some small equipment we will rarely use.

Decide on your initial purchasing strategy: There is no ideal strategy for purchasing. You will either pay with money or time. Shopping around for the best deals saves you money, but it costs time to collect quotes, identify less expensive alternatives, and figure out purchases outside of the procurement system. The savings, however, can be substantial. Home appliance stores, Amazon, EBay, smaller vendors, and used equipment resellers offer great savings, if you are inclined to spend the time searching. For my lab, we went with the path of least resistance by purchasing through our institution’s preferred vendors. By buying multiple pieces of capital equipment from one place, I received smaller equipment (fridges, freezers, hot plates, pipettes etc.) for free. Many vendors have new lab programs that provide considerable discounts and/or bundled items when you purchase capital equipment. I do highly recommend, and your institution may in fact require, getting multiple quotes for capital equipment. Again, there is no perfect approach—just get it done.

Order capital equipment: Every capital equipment vendor you talk to sells the best, newest, and most powerful Science Machine you have ever seen. Their competitors’ Science Machines are garbage and will break tomorrow. The good news is, if you can tolerate a bit of back and forth, you can receive really competitive pricing on said equipment through evaluating multiple quotes on similar Science Machines. This is well worth your time, as are demos of said equipment. Demos are incredibly useful if you are debating implementing new technology in the lab and/or do not have a preference between Science Machine from Science Stuff and Research Resource. In addition, expensive Science Machines come with service contracts. Talk to your network about whether you need to invest in them or not.

Staff the lab: Who will work in your lab is highly dependent on your environment and your research. Since I am in a clinical department, graduate students cost as much as postdocs, and I need additional grant support before I commit to training a graduate student. A postdoc, with a shorter training window and the ability to apply for career development awards, is a much more reasonable hire. If you are in a basic science department and work with less expensive model organisms, you may be able to afford more staff or trainees. Talk to your mentors and colleagues. See what they recommend. When to hire your first technician or bring on your first trainee (graduate student or postdoc) is also a highly personal and variable decision. I was told to hire someone very early so they could help set up the lab. I decided, instead, to hire a technician when the lab was about half set up, so said technician had something scientific to do and not just unpack boxes. This has worked out well so far and the technician has diligently generated data while completing the lab set-up process.

With your lab set-up well in hand, it is time to talk about navigating some of the other aspects of your new job, including forming a mentoring committee, navigating service, and teaching. Stay tuned for more tales!

 

Did I miss an important point? Do you have questions or concerns about the post? Or perhaps an anecdote to contribute! Feel free to send some electrons my way in the comments, via Twitter @PipetteProtag, or through traditional electronic mail pipette.protagonist@gmail.com

5 Ways to Improve the RCR Section of Your K

Grants & Funding

  1. Remove all the fluff. We don’t need for you to tell us that you are “committed to RCR at every level of your career” when you can demonstrate it through your experiences. What have you actually done that shows me that you’ve had conversations about research integrity or that you’ve grappled with difficult decisions?
  2. Don’t kill me with “RCR.” I know lots of scholars who specialize in research integrity who physically shudder when people use the term “RCR.” Some of them will only allow RCR if the writer puts a “the” in front of it. Let’s hope you don’t get that reviewer. Although I’m not like that, I do get tired of reading RCR over and over again. It loses its meaning. Try switching it up a bit. Use “research integrity” or “research ethics.” You can even use “bioethics.” Use what feels right to you. It is also helpful to think about the specific content areas and the subtopics within them. Don’t just call it “human subjects research” when you could write about informed consent or data safety monitoring.
  3. It’s not a grocery list. There is no need to try to squeeze in every content area in the RCR. No one is going to check to make sure you covered all the areas. It’s quality of exposure and experience, not quantity of topics. I’ve read some plans that were literally a bulleted list of all the content areas and confirmation that the author had learned about them in CITI training. Well, we at least know this scholar can copy and paste.
  4. Show some progression. Your past RCR instruction should include several examples of passive RCR learning, which I refer to as being a consumer of RCR. You need to move beyond the passive and talk about how you translate that knowledge through active experimentation with research integrity in your current RCR instruction, which includes experiences in the lab or the clinic, with your mentors or as part of a peer mentoring group. Finally, you should write about your planned or future RCR instruction, in which you become an ambassador for RCR throughout biomedical academia. You should show that you will move from being simply a consumer of RCR instruction to a creator of RCR instructional materials (for example). Instead of attending a seminar or grand rounds about RCR you should be leading one. Instead of simply being mentored in the RCR you should be mentoring others. Progression is everything.
  5. Be specific. Every discipline has its own hot topics in the big research ethics tent. What’s the issue for you? Maybe you’re working in liver transplant and a hot topic today is the ethical issue of paying organ donors or the ethical issues surrounding using an organ from a diseased person. Or you’re an orthopedic surgeon and you want to explore patient-centered alternatives to arthroscopic partial meniscectomy. What are the pertinent ethical issues here? Could you tease some of them into your future planned instruction? The more specific you can be in writing about your hot topic (without making it feel like a checklist, see above) the better it will read.

Author: Sam Gannon, EdD

More Resources

Counting What Counts in Responsible Conduct of Research Training 

Responsible Conduct of Research (RCR): The Importance of Ethics, Whistleblowing, and Mentoring

Designing Your Career