Deal or no deal? – The all-important PI response

For this month’s blog post I’ve resisted the temptation to write something about coronavirus. I think everyone’s hearing quite a bit about the ‘c-word’ right now, and there’s just a chance that some are getting a teensy bit fed up with it. So I decided instead to write about something that’s almost equally feared by those on the receiving end of it; grant reviewers’ feedback. As with certain other things, we know it’s coming sooner or later, and part of us just wants to get it over with. But at the same time, when it does finally arrive, we dread the moment when we’ll find out exactly what it’s like. Will it be mild, or severe?

Let’s pause for a moment and think about the purpose of the reviewers’ feedback, and of your response to it as the lead applicant on the proposal. (My focus here is on the UKRI peer-review system, but various other research-funding bodies operate in a similar way.) Peer reviewers will be asked to consider the importance, scientific potential and value for money of the proposed research. In doing so they’ll often raise some questions about points that aren’t clear to them, or where the detail seems insufficient, and they’ll make observations based on their perception of how the proposal stands up against the main evaluation criteria. Sometimes they’ll make suggestions as to how they think the project could be improved. If they’re in a really bad mood they might quibble with your punctuation or grammar. Based on their overall perception, they’ll give the proposal a score. It’s often fairly apparent from the nature and tone of the comments and the score awarded whether or not a reviewer thinks the proposal should be funded.

If a proposal receives unanimously poor reviews and low scores from reviewers, and it’s deemed to be beyond salvage, then it may get rejected before going to the full panel. This process helps to avoid the panel’s time being taken up with discussing and considering obvious no-hopers. Otherwise, the lead applicant gets a chance to provide the panel with their response to the reviews. This ‘PI response’ is the applicant’s last chance to boost their proposal’s prospects of success, and for the many proposals that fall into the ‘danger zone’ of not having garnered unreservedly positive reviews and maximum scores it’s a vitally important step in the application process (even proposals with rave reviews aren’t necessarily home and dry – the panel may consider the reviews to be unjustified, or may simply feel that in a highly-competitive round the proposal just isn’t a good strategic fit). All the indications are that a considered, well-written PI response can be the difference between success and failure.

Know your audience – who reads your PI response?

At the grants-committee meeting, panel members won’t look at your proposal in anything like as much detail as the peer reviewers did. They’ll focus mainly on the reviews, and definitely on the PI response to them, to inform their discussions and decision making. Two things to bear in mind here:

1. the panellists probably won’t be experts in your particular field – assume they’re not
2. they’ll have very little time available to discuss your proposal

So when you write your PI response, your primary audience will consist of time-pressed generalists who will have a great deal of business to get through. There’s a highly-important audience sub-category here too – the introducing or presenting member. Their own field of expertise is likely to be closer to yours, but not completely overlapping. They’re likely to have read your proposal in more detail than the other panel members, but they too will look closely at the reviews and your PI response. They’ll have to explain your proposal to rest of the panel, and suggest a score which is then up for discussion. Once again they’ll have a lot to get through, and not very much time.

A question of tone

So your peer reviews have all landed you don’t quite know what to think. They didn’t all hate it at least, which means you’re still in the game. But there are quibbles – some of them a bit left field – and Reviewer Two really did seem to be having a bad day. How to respond?

Probably the single most important thing to bear in mind at this stage is that all of the reviewers have given up their time and effort to review your proposal, in exchange for precisely no recompense. So you’re really grateful to them. No no, really. Even if you’re now thinking that Reviewer Two must be a complete coronavirus, you will take a deep breath, exhale slowly, adjust your face into a beatific smile and direct only kind-spirited thoughts towards your reviewers. The calm, Zen-like state into which you ascend will inform the tone of your response. And if you’re not quite there yet, sleep on it. Life nearly always feels like less of a trial after a good night’s sleep.

And you never know, some of the reviewers may just have a point – at least on the face of it. You only had a few pages in the Case for Support to describe a novel and complex project, and with a lot to try to cover it’s almost inevitable that some of the details fell by the wayside. What may have seemed obvious to you may well be much less obvious to the reviewer, who will certainly know much less about your project and its background. So you now have a chance to fill in the gaps for them, allay their concerns, and perhaps to acknowledge and take on board some good points that you hadn’t actually considered. Try not to see the process as adversarial then, but collaborative – you’re all trying together to make the proposal as robust as possible before it goes in front of the panel.

What follows is a suggested approach to constructing a PI response, along with some key points to bear in mind when you write it.

Constructing your response

First off, I’d recommend completing something of a content-analysis exercise. Going through each of the reviews, look for common themes – points and criticisms raised by more than one reviewer, indeed possibly all of them. Some of these may be easy to spot, some of them a little harder because the points raised by the respective reviewers may be related but not identical. Some of the points may be relatively minor and easily addressed, some may be more serious. You’ll be hoping that none of them are face-palm-inducing deal breakers, but you’ll certainly be looking out for criticisms that will kill the proposal if you don’t address them very convincingly. Rank the points as you find them, with the most serious – and/or hardest to address – points that have been raised by most or all of the reviewers at the top of your list. Once you have identified all the common points, continue working through each of the reviews picking out the points that you’ll need to address in your response and adding them to your ranked list. Be aware that it’s quite possible for a very important point to have been spotted by just one of the reviewers, perhaps because their own field of expertise is closest to the subject of the proposal.

One of the reasons for trying to rank the reviewers’ comments in terms of importance is that you only get a limited amount of space for your response. So it’s important to prioritise the allocation of space in your response document so that you can deal most comprehensively with the most serious points that have been raised. That said, it’s important to try to address all of the legitimate queries and criticisms raised by the reviewers.

When you’re sure you’ve listed all the points that need addressing, share your list with your fellow applicants, along with copies of the reviews themselves for context. Research proposals are always a team effort, and your PI response will be strengthened by the input of the other team members. The tone you adopt for your initial communication with the team will set the tone for their input, and there’s little to be gained from wasting time and emotional energy with a flurry of “how could anyone be so stupid!” emails.

If your proposal’s basically a good’un and the reviewers have done their jobs well, a fair number of their comments, observations and criticisms are likely to focus on the finer points of your science and your proposed methodological approach. Reviewers might also have some concerns about certain aspects of feasibility (can you really recruit all those participants, or get it all done in the available time?), or about how you plan to archive your data and make it available. Perhaps they don’t understand clearly enough exactly who will be doing what in the project. If there’s anything I’ve learned from working with grant applicants to draft their PI response, it’s that I definitely don’t know the answers to questions like these – it’s your project, so it’s over to you. But what I can try to do is provide a bit of help with the approach and the structure of the PI response document. What follows is my two-penn’orth, based on information and insights that I’ve gleaned from those who actually know what they’re talking about in this respect (including past and present panel members and the funders themselves).

Some points to keep in mind

Firstly, just to re-iterate, tone. I have seen panel feedback on unsuccessful proposals to the effect that “yep, we can see Reviewer Two hasn’t done a great job. But they’ve had a go and we appreciate that, so we think your stroppiness is unhelpful and uncalled for.” However good or bad their proposal was, this applicant’s PI response doesn’t seem to have endeared them to the panel. And that can’t have helped their cause. So don’t openly criticise the reviewers, or try to rubbish them. Trust the panel to spot the bad reviewers for themselves and to take their dodgy reviews with a small helping of salt. And remember – responsibility for good communication lies with the communicator (some more thoughts on this here). If someone has failed to grasp something it’s because you failed to communicate it clearly, not because they’re an idiot (you might think this privately, but don’t let it infect the tone of your response).

Politicians regularly infuriate interviewers and the public by refusing to answer the question, perhaps by blatantly answering a different question altogether. And it can be a tempting strategy to adopt when responding to reviews, particularly if some of the comments and questions are awkward to respond to or there are just so darn many of them. Perhaps if you just quietly ignore them, the panel won’t notice? The advice here though is always to try to be thorough and comprehensive in your response, addressing all the material points raised.

Also like politicians, reviewers aren’t always in unanimous agreement. While one reviewer might feel that your statistical power calculation is inadequate, for example, another might indicate that your methodological approach gives them no cause for concern. The response to the first reviewer though cannot simply be that the second reviewer thinks it’s all fine, so nah. Don’t waste the panel’s time by highlighting contrasting views between different reviewers. In this scenario, respond directly to the first reviewer, explaining clearly why you think your approach is appropriately powered – or perhaps even taking on board some of their suggestions. It’s important to be honest and receptive in this respect – if a reviewer has highlighted a limitation of your proposed methodology, or suggested an obvious improvement, then it’s probably counter-productive to dig your heels in and try to defend the indefensible.

When you do respond to a particular point, be specific in what you write and back up what you say with hard evidence, not just opinion and un-quantifiable assertions. This is particularly important if you’re rebutting a criticism, rather than accepting it and taking it on board. Point to peer-reviewed publications or whatever credible evidence you need to back up your statements.

Avoid just referring back to your proposal – if something wasn’t clear enough in the proposal then just repeating it or referring to a particular proposal section is unlikely to wow the panel. Take the opportunity to explain things in a different way, supplemented by a bit more detail if necessary. ‘Opportunity’ is the keyword here; remember how hard it was to fit everything you wanted to say into your Case for Support? Now’s your chance to use a few more pages to flesh out the bones of your proposal, demonstrate your specific knowledge and expertise, and introduce some new supporting data (you could include a link to it if it’s been published). There’s perhaps one exception to all this – if the reviewer has obviously missed something that you did in fact state clearly in the proposal, then tactfully draw the panel’s attention to the relevant material. Employ the same tact if you're correcting a clear factual error that the reviewer has made.

You’ll only be given a limited number of pages for your PI response, so use the space very wisely – be concise and make every word count. Assuming that you do have some important points of criticism to address, don’t go filling up the available space by reproducing all the positive comments. The panel members will have seen them in the reviews, so there’s not really much point in duplicating them if space is tight. While it’s nice to seem appreciative, it’s probably not worth filling up too much of the document with effusive words of thanks.

Similarly, there’s probably no need to reproduce the wording of each point of criticism before you address it. Instead, I’d suggest simply writing your responses under concise sub-headings. So for example, if one or more of the reviewers has suggested that the proposal seems a bit on the over-ambitious side, then you might address their concerns under a sub-heading like ‘Feasibility and scale of ambition’. It would probably be helpful to include the ‘reviewer reference’ number/s as part of each sub-heading, to indicate which reviewer/s have raised that particular point. A further benefit of using clear and descriptive sub-headings is that it will help to order your document and make it much easier to read.

Finally, if at all possible ask someone who’s not involved with the proposal to read through your response before you send it. Ask them specifically if the tone comes across as measured and professional, and whether they feel you have made any statements that don’t seem to be backed up by hard evidence.

It’s never felt more apposite to end a post with the words ‘good luck’.

The views and opinions expressed in this blog are mine alone and are in no way endorsed by my employer. Factual information and guidance are provided on a 'best-endeavour' basis and may become out of date over time. No responsibility can be taken for any action or inaction taken or not in respect of the content of this blog.