Category: Proposal Structure

George Heilmeier Proposal Metrics

In the IEEE Founders Award Lecture published in The Bridge, 1992, (National Academies press) on Some reflections on innovation and invention, luminary George Heilmeier put in eight simple questions what anyone proposing a project should be able to answer.

  1. “What are you trying to do? Articulate your objectives using absolutely no jargon.
  2. How is it done today, and what are the limits of current practice?
  3. What’s new in your approach and why do you think it will be successful?
  4. Who cares? If you’re successful, what difference will it make?
  5. What are the risks and the payoffs?
  6. How much will it cost?
  7. How long will it take?
  8. What are the midterm and final “exams” to check for success?”

https://en.wikipedia.org/wiki/George_H._Heilmeier

A proposal should be as jargon free as possible, and certainly its objective should be clear to all, experts and non experts. If you absolutely have to use jargon in the title, it must be defined in the abstract. Definitions cannot wait till the reviewer gets inside your grant. Why? Title and abstracts create the first impression. If the proposal presents the objective in a way that the reviewer feels his or her nascent interests are aligned with the proposal, a conformation bias is likely to appear – which of course, plays in your favor. It is difficult to feel good about something you do not understand!

The gap is what answers the second question. Remember that to innovate you extend, or modify what already exists (“done today”, “current practice”). Blue sky research grants exploring the pitch-dark side of research are not for everyone. You need an outstanding track record for those. The grey side is more accessible.

Novelty is expected, often in methods and approach, but with novelty comes risks, and risks are expressed in levels of confidence. That confidence is built over years of experience and a deep understanding of what your abilities are, but not only. The “You” is now plural. “You” are a team.

That you care about your project is a given, but do others care as much as you do? Who are these people? And if they do not know they should care, will you be able to show why they should, in a compelling way? Before you ask other people to get into your shoes, find out whose feet are blistered, feeling pinched, or in daily need of a shoe horn. For that, investigate, not the limitations that are there, but the consequence of these limitations.

Risk is a word rich in meaning. It spells potential trouble, but it also promises great rewards, not just payback, but payoffs – returns on the grant investment. Although it is not becoming to talk hard dollars when one is concerned with genes and quarks, it nevertheless will ultimately translate into hard currency for someone down the line. The economics of research matter to the grantor. If that payoff in dollars is further down that you would like, at least reveal the path that will get there. Build a dissemination plan to ensure long term impact.

Finally, just like students have midterm and final exams, build a solid evaluation plan to make sure you know if you are on track (or not) and meeting your specific aims, on time,  but also quantitatively and qualitatively. Good evaluation criteria are hard to come by. Learn from other PIs how they do their formative and summative evaluation plans. And remember that success is not what you say it is, but what others agree it is. So make sure you give them your success evaluation criteria before they create alternative ones you will disagree with. Deciding on these criteria is a team exercise. Involve your grant team. In the process of trying to establish these success criteria, you may discover that some of your specific aims are vague, not measurable, and require rewriting.

 

Give me the Specifics…. on the Specific Aims

When people want to know more than generalities, they say “Give me the specifics”, in other words: “Give me the precise details so that I can get a better idea of what’s going on here – don’t waffle!” A tragic mistake young PIs make is to leave the specifics for the inside of the grant and keep the the abstract and the one page specific aims at a high level. So the question is: how encompassing should  my aims be: the larger the aim, the less aims I need, right? The number of specific aims is limited to one or two for small grants, or maybe three or four for larger grants – but they can include sub-aims.  another question is “how specific should I be in my specific aims?”

Good question! To answer it you could look at the specific aims section of  a senior PI’s funded grants and learn by example. But learning by example is just like being thrown in at the deep end of a pool and asked to learn how to swim by watching others slice the water effortlessly in the pool. Chances are you will swallow a gulp or two, or maybe even drown before you can figure it out. Such technique is more likely to succeed if you have a minimum of grant knowledge, for example by reading our book (free serving unabashed plug), or if a mentor is holding the long bar in the water a few feet in front of you. Otherwise, you could look for discussions or podcasts on the topic. One such podcast I discovered came as a result of taking a number of MOOCs (Massive Open Online Courses) on Data Science organized by John Hopkins. In the effort report podcast, Roger Peng, prof in Biostatistics, my MOOC instructor and podcaster, with Elisabeth Matsui, Prof in Pediatrics, (both senior PIs at Johns Hopkins University) talk about specific aims . Skip the first 6 minutes to get to the topic. They discuss the one page specific aims commonly used for NIH grants.

They say that the specific aims page is a capsule (think concise one page) that is read by all review panel members – hence it’s importance (most time in a review panel, the full grant will only be read by two or three main reviewers). Roger and Elizabeth both concur with Professor Leo Chalupa, who extensively wrote on grants that half the time spent on writing the grant is spent on writing the specific aims page. So, besides its wide readership, why is that page so important?  It is the page that will “win over” the reviewer. It acts as a sort of ratchet. You do not move to the next cog unless you pass this one.

 The aims are the tasks, not the research questions answered, nor the hypotheses. The aims are related to the main objective, but they could be independent. Elizabeth uses the effective metaphor of “triangulation” where three different experiments (approaches) are tried to answer the same research question. These three tasks are specific aims. Most times, however, they are somewhat interdependent, and always logically connected.  How they all fit and all serve the main objective is clear. Logically connected does not mean deeply serially dependent, however. If aim 2 depends on the full accomplishment of aim 1 (think of a domino effect – the first one must fall before the next one can), your grant application may be rejected – unless of course, aim 1 is risk free, more like a project igniter. Just make sure the wick is not wet!

5177093939_cbae459a37_z

“scrolls” by psyberartist  CC BY 2.0

The podcast helps disambiguate three common adjectives: detailed and explicit and narrow. Narrow often bores and fails to raise sufficient interest. Explicit conveys idea that the aim is unfolded, unrolled, like a scroll. Unfortunately, the podcast only indirectly mentions what is on the scroll, so I will add to the dialogue. The task (aim) description is accompanied by one or several of the following: the purpose for the task, how it will be accomplished (approach or methodology), and, unless implicit, how the task relates to the clearly defined objective or the other aims. Being specific by stating What, what for, Why, and How shows the reviewer you “know what you are doing”, and you gave much thinking to what you will do. Nothing is superficial or lacks direction like a vaguely defined fishing expedition. “Laser sharp” focus will ensure your grant will be productive.

Elizabeth then presents her “secret aims-page weapon“: a crisp conceptual roadmap drawn in diagram form that shows the logical and relationship links (with directional arrows) between the objective, the aims and the various elements under each aim. It saves much reviewer time and acts as a booster to understand and value the grant.

Jargon is specific, and a method may be very specific to a problem. But this type of specification creates problems for the reviewers who are not totally domain experts. As Roger Peng correctly points out: some grants have multiple audiences which partially overlap in knowledge: biostatisticians, health epidemiologists, environmental exposure experts,… So he writes the aims for multiple audiences and justifies the importance of the choice made in each aims (methods, significance). Writing for multiple audiences and assuming a lower level of background knowledge makes it easier to “circulate” your specific aims page to people from different fields for feedback before submission.