The joy of specificity

 “The project will deliver significant impact, which will be ensured through effective dissemination.”

We all love some good impact, and we know that the funders love it too. And they’re terribly keen on dissemination, so hopefully there are some more brownie points to be had there. But when I see phrases like the one above in a research-funding proposal, unencumbered by any additional information – and believe me, I frequently do see them – my heart sinks. The total lack of detail suggests that the applicant either doesn’t really care about impact or dissemination, or hasn’t actually given them any thought. Quite possibly both. A good reviewer will always pick up on such a lack of specificity, and their response to it will not be positive. The same goes for any other areas in which the reviewers are expecting some substantive information, but are instead just given bland assurances with little or no detail to back them up.

When I review research proposals, two words that seem to crop up time and time again in the comments I write are ‘specific’ and ‘specificity’. Always in the context of encouraging more of them. The Oxford English Dictionary gives us the following helpful definition:

 

Specific

adjective

clearly defined or identified

 

If you’re serious about doing something, and committed to doing it properly, then what better way to convince someone of that than to identify and describe clearly the details of exactly what you will do – and perhaps how and when you’ll do it?

I know that space in the proposal document is always limited, and that there’s always so much to cover. But, assuming for a moment that you have thought through all of the various diverse proposal details, cutting out the specifics of stuff that the funder really cares about is really not a good strategy for success.

Below are some areas where I commonly find specifics to be lacking, but where at least some degree of specificity is pretty much essential.

Impact

‘Impact’ means things that change, usually (or at least hopefully) for the better, as a result of a research project or programme. It can happen over different timescales, and in different domains – in the academic world and in ‘real-world’ domains such as the economy, society, health and wellbeing. Whole books have been written about research impact, and it isn’t even the main focus of this blog post, so what follows will necessarily just be a brief overview. But when it comes to writing about the impact that you hope and anticipate your project will have, be sure to give some details about the following where they apply.

So, for academic impact:

• Who will benefit – researchers and scientists working in which particular disciplines and fields, and on what particular topics? If possible, name some of the nationally- and internationally-prominent institutions, centres, groups and perhaps even individuals in these disciplines and research areas – these will be among your key academic beneficiaries.

• How will they benefit? How, specifically, will the new knowledge, results, data, methods, models, approaches or whatever it is you hope to produce in your project help and advance them in their work? What will they be able to do (or do better), that they cannot already do now?

Perhaps you envisage that your project results will have specific benefits for other scientists working in areas where there will be downstream real-world impact in the future – if so, then spell this out.

For health, economic and societal impact:

• Once again, who will benefit – which specific groups of people (for example patients with a particular medical condition and their families; and/or professionals working in a particular field)? What about industry sectors, possibly even individual companies? And other organisations, such as the third sector – perhaps including some specific charities? Government is often a beneficiary of research, particularly where the research is likely to influence policy, regulatory matters and/or public-sector practice. Which specific sections of national or local government will benefit?

• Also once again, how will the benefits be felt? It’s not enough simply to list beneficiaries, and hope that it’ll be obvious to the reviewer how they will benefit from your research. Spell out exactly what you anticipate will change as a result of the research, and over what timescales, in the context of a problem that you have identified. No one expects a three-year project to completely solve a major problem, but reviewers will expect some specifics in terms of changes (or precursors to change) that are realistically achievable.

When it comes to the ‘who?’ of your beneficiaries, avoid using broad, unqualified terms like ‘stakeholders’ without explaining exactly what you mean by them. You should leave the reviewers in no doubt as to who your project stakeholders are and why they hold a stake in your research and its outputs. Words like ‘practitioners’ and even ‘policymakers’ can be similarly vague if you don’t make clear the particular area/s in which they’re practising or making policy.

Remember, a description of specific impact (whether academic or real-world) is a powerful answer to the question “what’s the point of doing this research?” If you’re unable or unwilling to address this question clearly by giving some specific details of your project’s anticipated impact, then you definitely can’t expect your reviewers to do it for you. They’ll be asking that question, not answering it.

Dissemination and impact-maximising activities

Dissemination is not impact. But making sure that the people, groups and organisations who will use, act on and benefit from your research and its findings actually get the information they need is normally a vital step towards achieving and maximising impact.

As with impact itself, specificity – or at least some specific examples – is vital for demonstrating that you’re serious about communication and dissemination, and have given them some proper thought. Having identified who, specifically, will benefit from your research, funders will expect you to have planned some targeted communication and dissemination activities, designed to ensure that the potential beneficiaries are given everything they need to realise those benefits. If your research is focused on informing policy in a particular area, for example, then you’ll need to make sure that the findings reach the relevant policymakers; if your vision for impact is to change practice, then likewise it’s vital to ensure that key decision makers and probably also practitioners in the area of focus are kept informed. When it comes to academic impact, you’ll want to ensure that all the right academics and scientists get to hear about your findings.

Conferences and publications are the bread and butter of dissemination to fellow academics and sometimes to professionals and practitioners in the area of focus. So which conferences will you be attending, and why have you selected them? When it comes to publications, which journals will you be targeting, and once again why? From a dissemination standpoint, the answer to the ‘why?’ question here will relate to each conference’s or journal’s potential to reach your target audience/s. So you’ll probably have picked the conferences and journals that are best attended, most widely read and most influential among the people and groups you want to reach.

Be specific about other aspects of your plans for dissemination and engagement. Don’t just allude vaguely to ‘impact activities’ – spell out what these will be. And don’t hide behind vague and generic terms like ‘stakeholder impact event’ – describe what form it will take, and which stakeholders you will be targeting. Similarly, when it comes to project outputs aimed at informing beneficiaries, try to give a flavour of what they will be. ‘Detailed summary of findings for practitioners in older-adult social care’ is helpfully more descriptive than just ‘Stakeholder report’.

Research objectives

It’s often said that research objectives should be SMART, where the ‘S’ stands for ‘specific’. (The remainder of the ‘SMART’ acronym is accounted for by ‘measurable’, ‘achievable’, ‘relevant/realistic’ and ‘time-bound’.) But what does this actually mean in practice?

A reviewer will scrutinise the list of objectives with the aim of determining whether they’re do-able, whether it will be clear and apparent when each objective has been completed, and whether completing each objective represents a sensible and appropriate step towards achieving the overarching project goal. They can only evaluate the research objectives on this basis if they are all properly specific.

For example, to borrow from my oft-used (if rather simplistic) housebuilding analogy, ‘to complete kitchen floor’ really doesn’t cut the mustard as a specific objective. How will people know when the kitchen floor has definitively been completed? Indeed, will everyone agree that it has been completed? What was the original specification, against which everyone can compare the supposedly-complete floor?

If on the other hand the objective had been ‘to tile entire kitchen floor area with large-format tiles as supplied, laid direct to cement screed and grouted, with all work completed to BS 5385’, then there’s little room for ambiguity. Everyone can agree whether or not the work is complete. The completeness or otherwise of the work is measureable, and completing this particular (and very specific) objective is demonstrably a relevant key step towards the project’s overall goal of refurbishing the kitchen.

A project aim is not the same as a research objective, but specificity matters here too. ‘Improve outcomes for leukaemia patients’ is a laudable goal, but it lacks specificity. On the other hand, ‘reduce diagnosis delays in leukaemia by developing a new streamlined rapid-diagnosis pathway’ is much more specific, and gives the reader a real essence of the project in just a handful of words. Sometimes a two-part aim statement can work well, with a more general part followed by a specific part. For example: ‘To improve outcomes for leukaemia patients presenting with non-specific symptoms, by reducing diagnosis delays through development of a new rapid-diagnosis pathway’.

Very similar principles apply to project titles, where it’s really important to give a clear and concise ‘nutshell’ indication of what the project is about.

Explaining budget costs

When it comes to justifying your requested resources, which many funders will require you to do, specificity is once again to the fore if you’re going to satisfy the inquisitive reviewer and convince them that the costs you’re claiming for are real and necessary. Avoid lumping various diverse costs under a single sub-heading, such as ‘consumables’, without any further breakdown or explanation. Instead, set out for the reviewer what the various different types of consumable will be, giving quantities and a proper cost breakdown for each category. If you’re claiming for travel, explain how many people will be travelling, and give details of where, how and why. The same goes for things like conference fees.

Much of your budget is likely to relate to staff time – your own, that of your co-investigators, and the cost of employing postdoctoral researchers and technicians – and here again the key to justifying these costs is to be specific. What will people be doing in the project, and how much of their time will it take? What salary scales and grades will researchers and technicians be on, and why are these justified? Specificity. Specificity. Specificity.

Project and risk management

With apologies for starting to repeat myself, this is another area where some specificity is needed but is often lacking. In terms of managing the project and ensuring that milestones and deliverables are accomplished in a timely manner, who, specifically, will do what – and (if applicable) how and when (or how often) will they do it? Give details of responsibilities and task ownership, and the management structures that will be in place to steer the project. When it comes to identifying project risks, specificity is vital. What are the particular research and project risks that you have identified, how likely and serious are they, who will ‘own’ them, and what can be put in place to mitigate them?

Data management

Few enjoy writing about managing the data used and produced by their proposed research project, but it’s not uncommon for funders to ask for a data management plan. Lack of interest in (and perhaps knowledge of) the subject of data management may result in the temptation to write something anodyne about ‘storage on secure servers’ and leave it more or less at that, but a proper data management plan will require a fair amount of specific detail. What types of data, for example, will the project handle and produce? What formats will the data be in? What will the volume of data be? How will you ensure that data-quality standards are met? How and where, specifically, will data be stored, backed up and curated? What metadata standards will be used for the data, and what documentation will be in place? How will data be archived, preserved and shared? And so on, addressing some very specific points that may need to be covered in some detail.

Specificity – everywhere!

I’ve covered here a handful of the areas in which at least a degree of specificity is required, but where grant applicants frequently resort to rather bland, generalised statements that are free from any specific details. But as a rule, it’s wise to include some specifics wherever possible, since doing so will always reduce ambiguity, demonstrate your commitment and attention to detail, indicate thoroughness of planning, and generally strengthen your research proposal. Increasing, of course, its chances of success.

The views and opinions expressed in this blog are mine alone and are in no way endorsed by my employer. Factual information and guidance are provided on a 'best-endeavour' basis and may become out of date over time. No responsibility can be taken for any action or inaction taken or not in respect of the content of this blog. 

Ten possible outcomes but just one of them positive: Navigating the evaluator's internal flowchart

Perhaps unsurprisingly given that I’m a bid writer, I spend quite a lot of time trying to get inside the mind of a ‘typical’ proposal evaluator (reviewer or panel member). How does she really think? What is the decision-making process that concludes in him liking or not liking a proposal? And how can we influence that process?

Is there even such a thing as a typical evaluator? In my more ideological moments, I like to think of them as super-humans with almost mythical levels of intelligence, able to scan through large piles of proposals with astonishing speed and assimilate their respective merits and demerits without favour or bias before ranking them perfectly according to a complex matrix of criteria. Such is their accuracy and consistency, any two of my super-human evaluators would rank that pile of proposals in exactly the same order.

Evaluators are human too...

I’ve never sat in on a grants panel, nor reviewed proposals for a funder, but rumour has it that the above characterisation may not be entirely representative. Certainly, we’ve all seen instances when two reviewers’ opinions on the same proposal appear to be diametrically opposed. And while the panel itself may subsequently impose a measure of sanity in such situations – the wisdom of crowds in action, perhaps – some of those individual panel members may not be quite as ‘on-it’ as we’d like to think they are. I’ve seen it suggested – admittedly with tongue at least partly in cheek – that there are often three identifiable types present on a grants panel:

1. The one who knows everything: Has a robust opinion on all the proposals, dominates proceedings and seems to know a great deal about many things
2. The perpetually-baffled one: Seems to struggle with many aspects of all the grants under review, leading to lots of questions and difficulty forming a definitive opinion
3. The slightly scatty one: Appears to be at least four grants behind the panel as a whole, and may not be as familiar with each of the proposals as perhaps they might be

Of course, if you’re a panel member yourself then I’m sure you don’t fall into any of these categories. But perhaps you’ve encountered them? How then to write a proposal that accommodates all the different types of evaluator who might review it?

Flowcharting the evaluation process

My suggestion would be to pitch the proposal at the ‘super-human evaluator’ who is rigorous, informed, methodical and objective in their appraisal. Make sure your proposal covers all of the important criteria that they’ll be looking out for and using as the basis for their evaluation. To this end, I have had a go at flowcharting their decision-making process, and I think it should probably look something like this:

You are, in essence, telling a story, and it’s vital that the story you tell in your proposal weaves in all of the essential points that will guide the evaluator to the decision you want them to make.

So what about catering for those less-than-perfect evaluators, like the three caricatures we met above? My advice would be to ensure that your proposal is presented as clearly and as nicely as possible. By which I mean well-ordered thoughts and a logical narrative flow; clear and straightforward language that is no more complex than it needs to be; and presentational niceties such as hierarchical headings, white space, figures, and so on. The sorts of things, in fact, that I consider in some of my previous blog posts.

The views and opinions expressed in this blog are mine alone and are in no way endorsed by my employer. Factual information and guidance are provided on a 'best-endeavour' basis and may become out of date over time. No responsibility can be taken for any action or inaction taken or not in respect of the content of this blog.

The power of a good story (and that all-important ‘However…’ clause)

“There’s nothing in the world more powerful than a good story. Nothing can stop it.”

So said a chap named Tyrion Lannister who, my internet tells me, is a character from a TV programme called Game of Thrones. And he’s right, I reckon, at least when it comes to grant applications. Sure, many funders state that the overriding factor they use when they make funding decisions is the excellence of the science, and indeed the science in your proposal (assuming it’s in a scientific area of research) will definitely need to be first-rate. But to grab the reviewers’ attention and sell them your science over and above the other competing proposals, you’ll need to embed your description of it within a very clear and compelling story. A story that explains very clearly what the point of doing that science is – why it really matters, and how we stand to benefit from it.

 

When I was in primary school, I was taught that stories are like fish. They have a beginning (the head), a middle (the body) and an ending (the tail). I’m not sure whether the extent of my English teacher’s knowledge of piscine anatomy would stand up to close scrutiny, but I like the idea of identifiable key components without which a story is incomplete.

In the case of a research proposal – certainly the types of life-science proposal with which I’m most familiar – I’d suggest that the story will often break down into the following components:

The ‘problem’ statement

An identifiable problem, which could be a gap in current scientific knowledge and/or a ‘real-world’ challenge relating to society, public health, wellbeing or whatever.

An explanation of why the problem is important – its size, nature, severity and impact on the people whom it affects.

The ‘current state of the art’ overview

A description of where we are now in this area of science and its application – what we already know and can do, and how that has had benefits to date for the scientific field and for the domain of real-world impact on which the proposal is focused (for example, using blood biomarkers for cancer diagnosis). Wherever possible and appropriate, it's very valuable to be able to point to some promising preliminary data that sits at the cutting edge of the science. 

The ‘however’ clause

This is critical – it’s fundamental to justifying why the proposed research needs to be done. Great and highly promising though the current science may be, this is the big ‘but’. A description of what we don’t yet know, what we can’t yet do, that prevents us from achieving so much more in this particular scientific area. If only we could move the field on…

The ‘leap forward’ description

Having prepared the ground by setting out the above points in clear terms, this component of the story is a vital part of the jigsaw – a description of a novel and compelling idea for moving forward, overcoming the current obstacles, plugging the knowledge gap and advancing the field. The aforementioned preliminary data often plays a critical role in giving credence to this novel idea. 

The ‘impact’ promise

Just outlining the problem is not enough – here we state specifically how and to what extent we will address the problem. Scientific leaps forward are great, and in an ideal world we’d fund them all just for the sake of curiosity and the notion that producing new knowledge is a worthwhile end in itself. But funding and resources are limited, so without wishing to sound too grand we need to select those proposals that promise most benefit for mankind. Specificity is the watchword here – what, exactly, do you intend will change as a result of the research?

In an applied proposal, you will be describing things like new practices, new processes, new guidelines, changed policies, perhaps new products (for example drugs or medical devices). They don’t have to change or come into being directly on conclusion of your project, but you’ll need to describe how the project will advance the status quo towards those changes ultimately being realised.

In a fundamental-science proposal, much or all of the immediate impact is likely to be academic. So in the life sciences at least, you’ll probably be describing how the new knowledge you deliver will support and advance the work of other scientists who are undertaking impactful research in specific areas.

And did I mention specificity?

The ‘timeliness’ reinforcement

Why do this research now? If it’s such a good and promising idea then why has no one done it before? And even if it couldn’t have been done until now, is it really so urgent? This component of the story explains why your idea’s time has come (perhaps the technology just didn’t exist a few years ago) and describes, without hyperbole, why the work must now be done without delay (perhaps the important problem you have identified is escalating rapidly).

The remit reminder

This is a focused explanation of why, specifically, the funder should care about all this. Every funder has a remit area and strategy for supporting research, and as much as something may be of general concern to the world as a whole, if it’s not within a funder’s remit then their interest in it will be limited. It may well seem abundantly obvious by this stage that the proposed research and its intended impact will fall within the funder’s remit, but spell it out clearly for the reviewers. This part of the story should refer to specific elements of the funder’s mission, remit and strategic focus, and explain explicitly and convincingly why and how the proposed research will support these.

Building the story

You won’t necessarily assemble these key components in the same order as above and they won’t all necessarily be self-contained chunks of narrative – some may serve as a thread that permeates the proposal as a whole. But in the life sciences at least, the majority of strong proposals are likely to be underpinned by all or at least most of these basic components.

There are of course other components to the story. You’ll note, for example, that I’ve barely touched above on the scientific detail and methodology, and of course without these you don’t have a research proposal. But, for our purposes here, this aspect of the proposal is all actually – believe it or not – largely secondary. It sits adjacent to the ‘why bother?’ story, and its purpose is to establish the credibility of your big idea. Anyone can say they’ll change the world, but if they propose to do so by witchcraft and magic alone then they may not be taken very seriously by those who hold the purse strings.

It’s no coincidence that the storyline building blocks I’ve outlined above would underpin a strong lay summary. I maintain that in the life sciences at least, it should probably be possible to boil down almost every research proposal to a handful of clear, strong statements from which my daughter (who is in Year 7) would grasp the essence of what the proposal is trying to achieve and understand why that matters in the real world.

This story-based narrative provides a central framework upon which to hang many of the essential parts of a research proposal. An overarching aim, for example, will be framed with clear reference to the problem you have identified and how you seek to address it. The main hypothesis and research questions will address the particular gaps in knowledge that you have identified, and the research objectives will outline the specific steps you will take to achieve the project’s aim. Drilling down further to the methodological detail, the description of this should be organised in such a way as to explain in practical terms how you will achieve each of your research objectives.

So: Get the work-plan done and you’ll achieve your objectives. Achieve the objectives and you should deliver against your aim. Accomplish the aim and you’ll have made an impact against the problem. Which we already know is important, matters to the funder, and needs to be tackled now.

There are usually other parts to the story that need telling, some of which may be peripheral but all of which are nevertheless important in their own right. How, for example, will you archive and share your data? What concrete steps will you take to maximise impact? If you’re applying for a fellowship then there’s a second story to tell that’s as important as the story behind the project – one about you and your motivation, your long-term career plans, and why fellowship funding is essential for realising your full potential as a researcher.

A real example – a great story in action

Below is an example of a grant that was funded a few years ago by the Medical Research Council (MRC). The project’s title, The ‘Medical Bypass’: a new treatment for obesity and diabetes’, gives a strong hint as to what the story is about. You can read a bit more about the research on UKRI’s Gateway to Research (GtR) website, but in a nutshell:

Obesity is an important problem (it affects one in four people in the UK, and it’s getting worse). It’s a major cause of diabetes and other serious diseases. (The ‘problem’ statement, with a ‘timeliness’ reinforcement.)

We have just one anti-obesity drug, which is not very effective. We do though have gastric bypass surgery, which works well for treating obesity and diabetes. (The ‘current state of the art’ overview.)

But gastric bypass is expensive, irreversible and not without risk, having a 1 in 300 mortality rate. (The ‘however’ clause.)

This research will set out to prove the concept of administering satiety hormones (part of our body’s signalling mechanism that tells us when we’re full) to achieve similar results to a surgical bypass. (The ‘leap forward’ description.)

Ultimately, the goal of the research is to develop an effective treatment for obesity and diabetes that is safe and cost-effective. Without such treatments, obesity levels in the UK are projected to reach 50% plus by 2050. (The ‘impact’ promise, plus more timeliness.)

The short information provided on the GtR website doesn’t explicitly include a remit reminder. It does though describe how the research, which was to be done using rats, was designed to justify a trial of the proposed ‘medical bypass’ approach in overweight patients. This, together with the focus area of the project (treatments for obesity and diabetes), leaves us in no doubt that the research is squarely within MRC’s remit. If the research had been of a more fundamental-science nature then a strong and explicit remit statement would probably have been essential.

The research itself involved measuring levels of gut hormones – peptide YY(PYY), glucagon-like peptide-1 (GLP-1) and oxyntomodulin (OXM), and a good description of the project's science would have been absolutely essential for enabling the reviewers to evaluate the credibility of the idea and assess the quality and feasibility of the project. But while the science might well have been the part of the proposal that interested the applicant most, it’s worth noting that it barely features within the central ‘good story’ that makes a compelling case for funding the project and getting the science done.

The views and opinions expressed in this blog are mine alone and are in no way endorsed by my employer. Factual information and guidance are provided on a 'best-endeavour' basis and may become out of date over time. No responsibility can be taken for any action or inaction taken or not in respect of the content of this blog.

Something boring that everyone hates (Justification of resources)

Justification. Of. Resources. Surely three of the most hated words in academia, right up there with ‘key performance indicators’ and ‘increased teaching load’. Such a dull thing to write about, and what on earth to write anyway? Is it not utterly obvious why you need an ELISA instrument and three litres of blocking-buffer solution? Anyway, there’s so little time left before the deadline, and you still haven’t finished editing the Case for support. And you’ve been working on this grant pretty-much solidly for the last two months. I feel your pain.

 

And yet. Funders want to get maximum bang for their research-funding buck, so value for money is often right up there with scientific excellence and impact potential when it comes to assessing a proposal and deciding whether or not to fund it. Indeed, it’s inextricably linked. In their guidance for reviewers, for example, the Medical Research Council sets out just three core assessment criteria – importance; scientific potential; and value for money, on which they ask the following question:

Resources requested: are the funds requested essential for the work, and do the importance and scientific potential justify funding on the scale requested? Does the proposal represent good value for money?

To take another example, the peer-review template that BBSRC reviewers are asked to complete includes the following question:

Value for money

Please comment on the value for money of the proposal.

Sure, some reviewers will only write a line or two here, particularly if they generally like the look of the proposal overall. But others will go into quite forensic detail, interrogating your budget almost on a line-by-line basis and maybe questioning the need for some of the things in it. Why risk exposing any vulnerabilities in your proposal, when just a little bit of extra effort would allay such concerns and head off the possibility of receiving potentially-damning criticism from reviewers?

Just to rewind slightly for a moment, I always emphasise the fact that putting together a sound research proposal consists of two distinct jobs: planning the project down to the last detail – the what, the when, the how, the who; and writing about the project in a manner that will sell it to a would-be funder. You should never try to do the second job before you’ve finished the first one. As well as making sure you actually have something to write about, this ‘plan it, then write about it’ approach has another clear advantage. When it comes to working out your budget, you should know exactly what you’re going to need in terms of resources (including people’s time, equipment, consumables and so on). You’ll have a very clear idea of the items of expenditure and the quantities involved, and of course you’ll have a sound understanding of exactly why you need them.

Justifying resources – the clue’s in the name…

As a bid writer I quite regularly see ‘Justification of resources’ documents that are little more than lists. In effect, they just replicate the budget – a list of items the applicant claims are needed, and their cost. But they don’t explain why they’re needed.

Imagine auditing the schedule of costs for your new-build house. One of them is for ‘Dancing girls: £10,000’. On the face of it, the first thing you’d do would be to fire the builder and get in someone else who’s less intent on spending your cash on hedonistic frivolity. But wait – it’s a high-end build, and the design brief calls for ornamental statuary to decorate the courtyard. Had the builder only itemised this as ‘Life-size “dancing girl” stone statues for external ornamentation (courtyard; one for each corner) x4 @ £2,500 each: £10,000’ then you could have saved yourself the unnecessarily-raised blood pressure and an angry telephone call. Sufficient information to explain and justify costs puts suspicious minds at rest and pre-empts unnecessary queries.

How then to justify your costs properly and keep your reviewers firmly on side?

  

Step 1: Don’t miss anything out

The first step in convincing the wary reviewer that you really do need everything you’re requesting in your budget is to make sure you actually mention it in your justification. Once the budget’s finalised and locked down, go through it line by line and list everything that needs justifying. Some funders will tell you which costs do and don’t need to be justified – UKRI, for example, provides cross-Council guidance on the Je-S website. Some of the individual Research Councils also give guidance, for example this from EPSRC. If there’s no specific guidance from the funder, then it’s wise to justify pretty much everything, and definitely all directly-incurred costs (project-specific stuff like travel, consumables, and postdocs who are dedicated to the project), any equipment, and any directly-allocated costs (permanent-staff time, specialist facilities charges, and perhaps pool technicians). Your list will form the basis of your justification of resources, and because you based it on your finalised budget you won’t miss anything out.

Step 2: Set out your justification narrative clearly and intuitively

Reviewers know they have to look at your justification of resources so they can comment on the ‘value for money’ criterion, but most of them will enjoy doing so about as much as you enjoyed writing it. So make it easy for them. Group items under clear and logical sub-headings, and set everything out neatly and consistently. Use bold to enable them easily to pinpoint particular items. This may not be storytelling but it’s still narrative, so write clearly and completely and don’t lapse into barely-intelligible shorthand, text-speak or similar. The basics of sound grammar continue to apply. Do be sure to check and adhere to the funder’s procedural guidelines with regard to things like document length, word-count limit and font. Which of course you always do!  

Step 3: Explain clearly why you need stuff

And so we get to the part that really seems to exercise minds. Steps one and two only require a bit of organisation and attention to detail, but this final step is the one that’s right up there with pulling teeth for many of us. Yet the truth is that if you can’t justify something in your budget convincingly to someone else then you simply don’t need it.

To return to my house-building analogy, consider the following costs: bricks; slates; copper finials; Ferrari.

 

On the face of it, the first two barely need justifying. Houses need walls and roofs, and bricks and slates are as good a material as any for these purposes. But actually, as a scrupulous sort I’d want to check that the quantities of these are appropriate, and that I’m not shelling out for unnecessarily-expensive or otherwise-inappropriate types of brick and slate. So I might prefer to see something like:

Standard house bricks (215mm x 65mm) to build 15m x 12m two-storey house @118 bricks per m² of wall (double-leaf) with 5% overage allowance – 12,500 bricks 

Spanish roof slates (600mm x 300mm) for 240m² roof @13 slates per m2 (390mm overlap) with 10% overage allowance – 3,430 slates

If you’re a quantity surveyor or a builder then please don’t examine my quantities here too carefully – no doubt they’re incorrect. But for our purposes, the above is clear and intelligible. It gives the sceptically-minded reader enough information to determine whether or not whoever wrote it is having a laugh, or making it all up on the hoof. And it does so in a way that’s understandable to the non-expert, without taking any sort of condescending tone that might irritate an expert reviewer.

The third item on our list above needs a bit more explanation. Copper what? But I commissioned a fairly ornate roof, and so the following extra detail would reassure me that it’s all legitimate:

Decorative copper roof finials (14 inch) for roof-ridge ends (x2) to ornament roof as specified

And what about the last item on the list? Well, no amount of explanation is likely to convince me that my house build requires the purchase of a Ferrari. It’s unjustifiable and so shouldn’t be in the budget. As such, it’s going to cause me to question whether I’ve picked the right builder to give my money to.

How all this translates to a research proposal

The safest approach is never to assume. Something that may seem so obvious to you as to need no explanation may not be at all clear to someone else, perhaps a reviewer from outside your immediate area of scientific specialism. Make clear why the item – and the quantity stated – is necessary for the project to go ahead as described in the proposal.

I won’t reproduce the UKRI guidance, but in short keep in mind that you’re answering the question why here. Why have you chosen that international conference? Why do four of you need to attend? Why will the project require 20% of your time (in other words, what specifically will you be doing in the project that would take that amount of time)? What particular skills, expertise and capacity will your co-investigators bring to the project, and how will they be deployed? Why do you need a postdoc for the whole of the project’s duration, and why do you need to recruit at that particular pay grade? Why will they need that high-end laptop? Why do you need those lab consumables – what specifically are they for and why is that necessary for the project?

The other key thing is to give a proper breakdown where appropriate. The sub-headings ‘Travel’ and ‘Lab consumables’ tell me very little on their own, for example, so simply giving a total figure for each of these isn’t going to be at all helpful to the critical reviewer. For travel, I’d want to know who’s travelling, where, how and for how long, so that I can judge whether the ‘why’ in the explanation seems reasonable and proportionate. Similarly, when it comes to lab consumables I’d want to know what exactly that sub-heading means – what specifically are they, how many of them are needed and how will they be used. Once again, this would enable me to judge whether the explanation of why they’re needed seems appropriate.

Some examples

Here are some examples, provided by the Natural Environment Research Council, of how to justify key items in a research-project budget:

Investigator effort: The PI will spend an average of 2.5 hours per week throughout the 36 months which will cover grant administration, guiding the PDRA in fluid inclusion analysis, and co-ordinating the research team. The Co-I effort varies: Co-I One will spend an average of 5 hours per week guiding the PDRA in microtextural techniques and data analysis, and contributing to project meetings. Co-I Two will spend an average of 7.5 hours per week training and guiding the PDRA in Ar/Ar dating, which encompasses sample preparation, the use of the laserprobe and noble gas mass spectrometry, data collection, data quality assessment, data reduction and presentation. Ar/Ar data interpretation is critically dependent on the measured sub-grains (from microtextural analysis) and microthermometry (from fluid inclusion analysis) and will involve modelling of the measured Ar/Ar data with respect to grain sizes and temperatures under the guidance of Co-I Two.

Conference attendance: We seek funding for conference attendance for the PDRA and investigators. This is essential for the international dissemination of results and to provide networking and career development opportunities for the PDRA. We have chosen AGU (San Francisco 2008) and Geofluids (Adelaide 2009), both of which will attract wide international audiences both in the specific fields of the proposal and more broadly.

Source: NERC; see the whole example Justification of resources here 

So to finish

Be aware that it can be as important to request sufficient resources, appropriate to your proposed approach and methodology, as it is to avoid requesting too much. An expert reviewer with relevant scientific experience will spot a project that’s under-resourced and will rightly be concerned that you’ll be unable to deliver.

A final point. We all know that, for various good reasons, putting together a research proposal is often a rush job. A good reviewer though will be looking out for projects that have been meticulously planned down to the last detail, leaving no stone un-turned. These tend to make for the best-written, clearest proposals, and a well-thought-through budget and accompanying justification are definitely hallmarks of a carefully-planned research project. By showing the reviewer that you’re on the case and have given due attention to detail you’ll be increasing your chances of success. Conversely, in a highly-competitive funding environment a poor justification of resources can be enough to kill an otherwise-decent proposal. So never let it be the ha’p’orth of tar for want of which the ship ends up being spoiled.


The views and opinions expressed in this blog are mine alone and are in no way endorsed by my employer. Factual information and guidance are provided on a 'best-endeavour' basis and may become out of date over time. No responsibility can be taken for any action or inaction taken or not in respect of the content of this blog. 

Deal or no deal? – The all-important PI response

For this month’s blog post I’ve resisted the temptation to write something about coronavirus. I think everyone’s hearing quite a bit about the ‘c-word’ right now, and there’s just a chance that some are getting a teensy bit fed up with it. So I decided instead to write about something that’s almost equally feared by those on the receiving end of it; grant reviewers’ feedback. As with certain other things, we know it’s coming sooner or later, and part of us just wants to get it over with. But at the same time, when it does finally arrive, we dread the moment when we’ll find out exactly what it’s like. Will it be mild, or severe?

Let’s pause for a moment and think about the purpose of the reviewers’ feedback, and of your response to it as the lead applicant on the proposal. (My focus here is on the UKRI peer-review system, but various other research-funding bodies operate in a similar way.) Peer reviewers will be asked to consider the importance, scientific potential and value for money of the proposed research. In doing so they’ll often raise some questions about points that aren’t clear to them, or where the detail seems insufficient, and they’ll make observations based on their perception of how the proposal stands up against the main evaluation criteria. Sometimes they’ll make suggestions as to how they think the project could be improved. If they’re in a really bad mood they might quibble with your punctuation or grammar. Based on their overall perception, they’ll give the proposal a score. It’s often fairly apparent from the nature and tone of the comments and the score awarded whether or not a reviewer thinks the proposal should be funded.

If a proposal receives unanimously poor reviews and low scores from reviewers, and it’s deemed to be beyond salvage, then it may get rejected before going to the full panel. This process helps to avoid the panel’s time being taken up with discussing and considering obvious no-hopers. Otherwise, the lead applicant gets a chance to provide the panel with their response to the reviews. This ‘PI response’ is the applicant’s last chance to boost their proposal’s prospects of success, and for the many proposals that fall into the ‘danger zone’ of not having garnered unreservedly positive reviews and maximum scores it’s a vitally important step in the application process (even proposals with rave reviews aren’t necessarily home and dry – the panel may consider the reviews to be unjustified, or may simply feel that in a highly-competitive round the proposal just isn’t a good strategic fit). All the indications are that a considered, well-written PI response can be the difference between success and failure.

Know your audience – who reads your PI response?

At the grants-committee meeting, panel members won’t look at your proposal in anything like as much detail as the peer reviewers did. They’ll focus mainly on the reviews, and definitely on the PI response to them, to inform their discussions and decision making. Two things to bear in mind here:

1. the panellists probably won’t be experts in your particular field – assume they’re not
2. they’ll have very little time available to discuss your proposal

So when you write your PI response, your primary audience will consist of time-pressed generalists who will have a great deal of business to get through. There’s a highly-important audience sub-category here too – the introducing or presenting member. Their own field of expertise is likely to be closer to yours, but not completely overlapping. They’re likely to have read your proposal in more detail than the other panel members, but they too will look closely at the reviews and your PI response. They’ll have to explain your proposal to rest of the panel, and suggest a score which is then up for discussion. Once again they’ll have a lot to get through, and not very much time.

A question of tone

So your peer reviews have all landed you don’t quite know what to think. They didn’t all hate it at least, which means you’re still in the game. But there are quibbles – some of them a bit left field – and Reviewer Two really did seem to be having a bad day. How to respond?

Probably the single most important thing to bear in mind at this stage is that all of the reviewers have given up their time and effort to review your proposal, in exchange for precisely no recompense. So you’re really grateful to them. No no, really. Even if you’re now thinking that Reviewer Two must be a complete coronavirus, you will take a deep breath, exhale slowly, adjust your face into a beatific smile and direct only kind-spirited thoughts towards your reviewers. The calm, Zen-like state into which you ascend will inform the tone of your response. And if you’re not quite there yet, sleep on it. Life nearly always feels like less of a trial after a good night’s sleep.

And you never know, some of the reviewers may just have a point – at least on the face of it. You only had a few pages in the Case for Support to describe a novel and complex project, and with a lot to try to cover it’s almost inevitable that some of the details fell by the wayside. What may have seemed obvious to you may well be much less obvious to the reviewer, who will certainly know much less about your project and its background. So you now have a chance to fill in the gaps for them, allay their concerns, and perhaps to acknowledge and take on board some good points that you hadn’t actually considered. Try not to see the process as adversarial then, but collaborative – you’re all trying together to make the proposal as robust as possible before it goes in front of the panel.

What follows is a suggested approach to constructing a PI response, along with some key points to bear in mind when you write it.

Constructing your response

First off, I’d recommend completing something of a content-analysis exercise. Going through each of the reviews, look for common themes – points and criticisms raised by more than one reviewer, indeed possibly all of them. Some of these may be easy to spot, some of them a little harder because the points raised by the respective reviewers may be related but not identical. Some of the points may be relatively minor and easily addressed, some may be more serious. You’ll be hoping that none of them are face-palm-inducing deal breakers, but you’ll certainly be looking out for criticisms that will kill the proposal if you don’t address them very convincingly. Rank the points as you find them, with the most serious – and/or hardest to address – points that have been raised by most or all of the reviewers at the top of your list. Once you have identified all the common points, continue working through each of the reviews picking out the points that you’ll need to address in your response and adding them to your ranked list. Be aware that it’s quite possible for a very important point to have been spotted by just one of the reviewers, perhaps because their own field of expertise is closest to the subject of the proposal.

One of the reasons for trying to rank the reviewers’ comments in terms of importance is that you only get a limited amount of space for your response. So it’s important to prioritise the allocation of space in your response document so that you can deal most comprehensively with the most serious points that have been raised. That said, it’s important to try to address all of the legitimate queries and criticisms raised by the reviewers.

When you’re sure you’ve listed all the points that need addressing, share your list with your fellow applicants, along with copies of the reviews themselves for context. Research proposals are always a team effort, and your PI response will be strengthened by the input of the other team members. The tone you adopt for your initial communication with the team will set the tone for their input, and there’s little to be gained from wasting time and emotional energy with a flurry of “how could anyone be so stupid!” emails.

If your proposal’s basically a good’un and the reviewers have done their jobs well, a fair number of their comments, observations and criticisms are likely to focus on the finer points of your science and your proposed methodological approach. Reviewers might also have some concerns about certain aspects of feasibility (can you really recruit all those participants, or get it all done in the available time?), or about how you plan to archive your data and make it available. Perhaps they don’t understand clearly enough exactly who will be doing what in the project. If there’s anything I’ve learned from working with grant applicants to draft their PI response, it’s that I definitely don’t know the answers to questions like these – it’s your project, so it’s over to you. But what I can try to do is provide a bit of help with the approach and the structure of the PI response document. What follows is my two-penn’orth, based on information and insights that I’ve gleaned from those who actually know what they’re talking about in this respect (including past and present panel members and the funders themselves).

Some points to keep in mind

Firstly, just to re-iterate, tone. I have seen panel feedback on unsuccessful proposals to the effect that “yep, we can see Reviewer Two hasn’t done a great job. But they’ve had a go and we appreciate that, so we think your stroppiness is unhelpful and uncalled for.” However good or bad their proposal was, this applicant’s PI response doesn’t seem to have endeared them to the panel. And that can’t have helped their cause. So don’t openly criticise the reviewers, or try to rubbish them. Trust the panel to spot the bad reviewers for themselves and to take their dodgy reviews with a small helping of salt. And remember – responsibility for good communication lies with the communicator (some more thoughts on this here). If someone has failed to grasp something it’s because you failed to communicate it clearly, not because they’re an idiot (you might think this privately, but don’t let it infect the tone of your response).

Politicians regularly infuriate interviewers and the public by refusing to answer the question, perhaps by blatantly answering a different question altogether. And it can be a tempting strategy to adopt when responding to reviews, particularly if some of the comments and questions are awkward to respond to or there are just so darn many of them. Perhaps if you just quietly ignore them, the panel won’t notice? The advice here though is always to try to be thorough and comprehensive in your response, addressing all the material points raised.

Also like politicians, reviewers aren’t always in unanimous agreement. While one reviewer might feel that your statistical power calculation is inadequate, for example, another might indicate that your methodological approach gives them no cause for concern. The response to the first reviewer though cannot simply be that the second reviewer thinks it’s all fine, so nah. Don’t waste the panel’s time by highlighting contrasting views between different reviewers. In this scenario, respond directly to the first reviewer, explaining clearly why you think your approach is appropriately powered – or perhaps even taking on board some of their suggestions. It’s important to be honest and receptive in this respect – if a reviewer has highlighted a limitation of your proposed methodology, or suggested an obvious improvement, then it’s probably counter-productive to dig your heels in and try to defend the indefensible.

When you do respond to a particular point, be specific in what you write and back up what you say with hard evidence, not just opinion and un-quantifiable assertions. This is particularly important if you’re rebutting a criticism, rather than accepting it and taking it on board. Point to peer-reviewed publications or whatever credible evidence you need to back up your statements.

Avoid just referring back to your proposal – if something wasn’t clear enough in the proposal then just repeating it or referring to a particular proposal section is unlikely to wow the panel. Take the opportunity to explain things in a different way, supplemented by a bit more detail if necessary. ‘Opportunity’ is the keyword here; remember how hard it was to fit everything you wanted to say into your Case for Support? Now’s your chance to use a few more pages to flesh out the bones of your proposal, demonstrate your specific knowledge and expertise, and introduce some new supporting data (you could include a link to it if it’s been published). There’s perhaps one exception to all this – if the reviewer has obviously missed something that you did in fact state clearly in the proposal, then tactfully draw the panel’s attention to the relevant material. Employ the same tact if you're correcting a clear factual error that the reviewer has made.

You’ll only be given a limited number of pages for your PI response, so use the space very wisely – be concise and make every word count. Assuming that you do have some important points of criticism to address, don’t go filling up the available space by reproducing all the positive comments. The panel members will have seen them in the reviews, so there’s not really much point in duplicating them if space is tight. While it’s nice to seem appreciative, it’s probably not worth filling up too much of the document with effusive words of thanks.

Similarly, there’s probably no need to reproduce the wording of each point of criticism before you address it. Instead, I’d suggest simply writing your responses under concise sub-headings. So for example, if one or more of the reviewers has suggested that the proposal seems a bit on the over-ambitious side, then you might address their concerns under a sub-heading like ‘Feasibility and scale of ambition’. It would probably be helpful to include the ‘reviewer reference’ number/s as part of each sub-heading, to indicate which reviewer/s have raised that particular point. A further benefit of using clear and descriptive sub-headings is that it will help to order your document and make it much easier to read.

Finally, if at all possible ask someone who’s not involved with the proposal to read through your response before you send it. Ask them specifically if the tone comes across as measured and professional, and whether they feel you have made any statements that don’t seem to be backed up by hard evidence.

It’s never felt more apposite to end a post with the words ‘good luck’.

The views and opinions expressed in this blog are mine alone and are in no way endorsed by my employer. Factual information and guidance are provided on a 'best-endeavour' basis and may become out of date over time. No responsibility can be taken for any action or inaction taken or not in respect of the content of this blog.