🎙️ Podcast Link 🎙️

I have a massive backlog of #HackingAcademia videos to shoot: right at the top of the pile, prompted by a heap of conversations in recent months, is this one: the art of:

Separating PROCESS from OUTCOMES

This is one of the most important but also most difficult skills to master in many professional careers.

You can do everything right, and get a bunch of bad outcomes (the reverse also applies, but more rarely!).

Some of those bad outcomes are real signal that you need to change things.

BUT

Some of those bad outcomes simply “come with the territory”: they are unavoidable, *even if you’re doing everything right*.

Not differentiating between the two is going to make your professional life infinitely more challenging than it needs to be.

Now, here’s the difficult part – how do you know which situation applies?

Well, that’s what I cover in today’s video:

♣️ my past experience playing online poker, which is arguably the most important example of the need to separate process from outcome

💰 three examples from academia and research grant, project and promotion applications covering “you’re / your group is too successful”, “this is too aligned with what you already do” and “you need to focus on these other areas” feedback

🔍 two ways you can indeed find out which situation applies in any particular circumstance, and

🫂 some reassurance around two ways this problem becomes more navigable as your career progresses!

#careeradvice #feedback #grants #promotion #applications #university #professor #lecturer #academic #academia #fellowships #criticism #strategy #tactics 

Full Video Notes

Many years ago, I engaged in an extended period of playing online poker. This is a card game that you play online, where you are betting money and playing against other human players, or perhaps some bots as well.

I was interested in learning how to play poker at least at an adequate level. I was betting two cents and four cents per hand, so it was extremely low stakes. I played enough hands to get a relatively statistically significant idea of how I was as a player.

I also spent quite a bit of time reading about the art of playing poker, some of the basic ideas of strategy, bluffing, and statistics. One of the key things that is very particular to poker, but which also applies to a lot of other things you do in life, is this idea of separating process from outcomes.

In poker, even if you play many thousands of hands over several days or weeks, it is possible to play a near-optimal strategy, not make any obvious mistakes except those that only become apparent with the benefit of hindsight, and still lose a lot of money.

One of the challenges this poses, especially for new players learning the game, is separating a particular outcome from whether they were actually playing the right way. Those two things are only very weakly connected, especially in the short term.

This core concept of separating process from outcome is a really key one for just about any professional career, including academic and research careers. It is the topic of today’s Hacking Academia video.

Separating process from outcome is, in some ways, an especially challenging concept for academics and researchers because many bring a logical, methodical, evidence-based approach to their careers. This means they are often tempted to take any particular bad outcome and make a bunch of changes without assessing whether that outcome is a meaningful signal or just something that comes with the territory of following a good process.

One way of thinking about this is to consider that you are going to make many decisions, submit many grant applications, and submit many papers throughout your academic or research career. Those actions will attract criticism and critical feedback regardless of how well the grants are written or the papers prepared.

The key is to work out whether the critical feedback you are getting is actually meaningful and something you should change in response to, or whether it is simply something that comes with the territory.

I will give a few concrete examples of this in the context of academic research.

The first example is feedback on a funding proposal. This might be a grant or a fellowship where the criticism is that you are too successful, or that your group already has too much funding and therefore is not a worthy recipient of additional funding.

In this scenario, I am assuming that you genuinely need the funding and are not applying gratuitously. Early in your career, this feedback may be directed at the group you are part of. Later on, it may be directed at you individually, with comments like, you already have lots of grants and do not need any more.

The instinctive reaction for many people is to downplay their success or hide aspects of it to reduce the likelihood of this criticism. That is not looking at the whole picture.

Funding decisions take into account many factors, not just whether you appear to have too much or too little funding. One key factor is the likelihood that you will successfully deliver what you promise in the proposal and meet key milestones. Being experienced, successful, and having evidence of past success helps here.

This creates a tension. If you are too successful and experienced, you may be told you do not need more funding. If you are not successful or experienced enough, funders may not want to risk giving you money or a critical project.

In many research ecosystems, there is a degree of risk aversion. If you have to choose between those two types of criticism, you are almost always better off being seen as too experienced rather than not experienced enough.

So if a proposal is rejected partly because you are perceived as too successful, that is often something that comes with the territory and does not necessarily warrant major changes to how you pitch yourself.

Another common example is when a proposal is rejected because a reviewer says it is too close to your core area of work or overlaps too much with what you are already doing. It is tempting to respond by radically changing your strategy and pitching only in distant areas.

Again, that is not necessarily a good idea. It may be that your process of building on world-class expertise and extending it in novel ways is exactly the right process, even if the outcome on that occasion was negative.

A third example is promotion. Someone who does a lot of applied, industry-focused work may go for promotion and receive feedback that they are too industry focused and need more publications.

This situation plays out every day in academia. The process they followed may have been to double down on their strengths in translational and applied research, which may still be the correct process despite a negative outcome.

One nuance here is that reviewers often provide reasons to justify a rejection, but those reasons are not always deeply thought through or the true drivers of the decision.

So how do you know whether a negative outcome is just something that comes with the territory or a meaningful signal that you should change your process?

The first and most important suggestion is to talk to experienced people around you. They can help you judge whether you should change course or keep backing yourself. Ideally, this person is not conflicted and not directly involved in the decision you are reacting to.

If you do not have such a person, finding one is something you should prioritize, as it will help you throughout your career.

Another thing you can do is role-play the scenario where you take the outcome to heart and make major changes. Think through what those changes would look like, then consider the opposite criticisms you might receive as a result.

This helps avoid knee-jerk reactions and allows you to make a more balanced decision.

Separating process from outcome becomes easier over time for two reasons. First, you gain experience and can make these judgments yourself more confidently.

Second, volatility smooths out. Early in your career, you may have only one critical decision per year, making negative outcomes feel overwhelming. Later on, with many outcomes each year, it becomes easier to take a broader perspective.

Some negative outcomes will genuinely signal that you need to change. Others will simply come with the territory and can be chalked up to bad luck or random variation.