The Goldilocks Difficulty

A couple of weeks ago I posted Failing with Purpose. I had some great feedback, questions, and discussion from that, and have been meaning to post a follow up for a while now. So here it is, finally. Also related: Component Goals – Lessons from a 5K, Look How Tough I Am! (or see the collection at The Decision Boundary).

The main question that arose out of the previous post was, “what is just the right amount of difficulty?” I advocated for setting stretch goals where failure is a likely outcome. I still believe that more benefit can be realized by falling short of a stretch goal than by overachieving on an easy one, but just sending yourself on fool’s errands isn’t very productive. There’s a tl;dr at the bottom of the post if you’d rather skip to the bullet point version.

Brain Power

In grad school my research was on brain-computer interfaces. Basically, I made fancy algorithms that helped translate someone’s thoughts into a control signal for computer cursors, prosthetic arms, etc. Tim Hemmes, shown below, was our first research participant with tetraplegia, and seeing him use a prosthetic arm to reach out and touch his girlfriend for the first time in seven years still might be the most rewarding thing I’ve ever been involved in.

Brain-computer interfaces are a two way learning process, though: the algorithms were trying to learn what the person was thinking but at the same time the person’s brain was trying to learn how to generate the right signals, in much the same way that you first learn to walk or ride a bike. To accelerate the learning process as much as possible we made a series of progressively more challenging tasks. First, the person might just try to control the jumping in Super Mario Brothers. Then, they might try to move a cursor to a target with varying levels of assistance. Finally, they might move to controlling a prosthetic arm.

After 5 years creating neural prostheses and teaching people how to control objects with their minds, I became… a sith lord! No, unfortunately I just graduated and became overqualified for a lot of good jobs. (just kidding, I love my job and the opportunities my degree has created)

Game Flow Theory

We developed our tasks using elements borrowed from video game design (which video game design likely borrowed from decades of psychology research). The primary concept is something called game flow, originally presented by Csikszentmihalyi in 1990 and shown below. It’s a simple concept: if the task is too difficult, someone is likely to get anxious and give up, while if it’s too easy they’ll get bored and quit. As skill increases, difficulty must increase proportionally to ensure that the person remains engaged in the game. “Carrot on a stick” is a saying for a reason. It really works.

Flow Channel Wave (from “The Art of Game Design” book by Jesse Schell)

The difficulty also has a profound effect on the person’s rate of improvement. This has been approximated many times by diagrams like the one shown below. Improvement increases with difficulty, until a point at which the task is completely impossible, where improvement drops off a cliff. The area just before this cliff can be thought of as the Goldilocks zone of difficulty.

Source of Motivation

I believe that for different people the exact shape of the “flow channel” depicted above is different, though. For me, I think there’s a positive y-intercept and probably some non-linearity towards the upper end where I get extra motivation from seemingly impossible tasks. I never have taken too kindly to someone telling me I can’t do something.

You say it’s impossible to look cool with a seafoam green Ford Taurus? Or to take it off-roading? Please, do tell. (but if my parents are reading this, I NEVER took the Taurus up Windrock Mountain. That would be totally impossible.)

There are a lot of factors beyond skill level that go into defining the appropriate level of difficulty. The other primary consideration is motivation, which falls into two categories: intrinsic and extrinsic. For most tasks and for most people, intrinsic motivation is much more powerful.

An illustration shows a person’s upper torso. An arrow on the left begins at the person’s chest and curves around to point inside the head; inside the curve of the arrow are the words “intrinsic motivation (from within)” and three bullet points: “autonomy,” “mastery,” “purpose.” An arrow on the right begins in empty space and curves to a point inside the head. Above the arrow are the words “extrinsic motivation (from outside)” and three bullet points: “compensation,” “punishment,” and “reward.”

In personal experience I’ve found this to be amplified for extremely difficult tasks. I firmly believe that no one who is primarily extrinsically motivated (e.g. “other people will think it’s cool”) will ever finish Barkley. At some point, no matter who you are, the race pushes you to a point that requires an immense amount of motivation to convince yourself to continue, which can only be obtained by relying heavily on internal motives. That’s not to say that external motives can’t be added on top of that and help, but when you’re 50 hours in, sleep-deprived, and nearly every part of your mind and body is screaming at you to quit, somehow “I wonder what will be tweeted about this” isn’t a thought that really comes into play. To be clear, I greatly appreciate all the support, encouragement, and other kind words that have been sent to me throughout the years at Barkley through Twitter or any other medium, but honestly it’s not really on your mind during the actual low points of the race.

You say your mind’s so far gone right now that you’re not sure what reality is? Well, those tweets are real for sure! You’d better get back out there!

Note: the rest of this post is purely my opinion based on my own thoughts and experiences. I have no peer-reviewed literature or double-blind studies to back any of it up.

Long Term Goals vs. Milestones

One thing that’s important to differentiate between is long-term goals that are really driving your actions and shorter term milestones that can serve as guideposts and positive reinforcement along the way. When I set my sights on Barkley, there was no possible way that I could have finished at the time. It was well past the Goldilocks zone and at the bottom of the cliff. But I knew that for future me it might, just maybe, be possible. So I set it as my long term goal. It was the reason I got out of bed to run hill repeats in the snow. When I got sore, tired, or just didn’t want to go for a $@#%! run (as much as you should enjoy any recreational activity you pursue, we all have those days), it was why. With a page out of Jared Campbell’s playbook, I actually started to legitimately view unfavorable conditions as wonderful training benefits.

Snow, fog, single digits, and 6 AM? This is perfect for Barkley! I still hate you Mar Lu Ridge, but thanks!

But, back on topic, Barkley was still initially impossible for me, even that first year that I attempted it (at the time I told myself I could have done it if I had eaten better, but nope… nope I had no idea how much exponentially harder that last loop was). So I charted myself a path to it, building to short term goals (milestones) that were themselves still stretch goals, but possible in the short term. I’ve used my incredible Microsoft Paint skills to illustrate what this path looked like for me on an adaptation of the game flow chart. There’s a lot going on in this chart, so I’ll explain in more detail below.

The upper and lower bounds for difficulty that produce motivation for a given skill level are still in place. I’ve added a third line for what’s achievable at a given skill level. To me, if I draw a vertical line up from my current skill level, an optimal milestone lies somewhere between the achievable line and the max difficulty for motivation line. Above that, and I rage quit and get nowhere. Below that, in the complacency zone, I’m motivated but I won’t improve nearly as fast. Further below that, and I just don’t even care enough to try. Different people and even different goals or other circumstances will mean different margins between achievable difficulty and min/max difficulties for motivation, which is why setting goals is such a personal thing.

Taking a long term view, the optimal end goal is one that intersects my perceived maximum obtainable skill at the achievable difficulty line. This means the goal will truly be pushing me to become the best possible version of myself in order to achieve it. This is where Barkley was for me.

In the short term, Barkley was in my ragequit zone, though. So my strategy was to create short term milestones that were below my max difficulty for motivation but that would ensure I improved as quickly as possible to reach that long term goal. Each time I make an attempt at something (white vertical lines), I believe my improvement (white horizontal lines) is capped by either hitting the line of achievable difficulty or hitting the goal. That’s why to improve as quickly as possible I want my milestone close to my achievable difficulty, but personally I would rather have it a bit over than a bit under. It’s the opposite of the Price is Right: as close to achievable as possible without going under. If you have to build steps to the top, you’ll get there a lot quicker by building fewer but bigger steps, each with its own accompanying sense of accomplishment.

No really, I BQ’d and I’m thrilled. But I did overshoot the goal a bit and probably didn’t improve as much from it as I could have with a more difficult goal…

Essentially, long term goals should motivate you to become the best possible version of yourself, while short term milestones help you get there as quickly as possible. Both of them should reflect the criteria that I discussed in Failing with Purpose: personally meaningful, supported by friends and family, and possible outcomes (positive and negative) evaluated.

Adaptive Goals

Endurance sports are a bit unique from some other endeavors, in that the event itself is long enough that goals can change throughout. In the 2016 JFK 50 Miler, my goal changed at least 5 times during the race because of how my body was responding. I’ll, umm, eventually, add my race report from that, but I’ve laid out the gist of it below in another convenient Paint diagram. For a single event, skill is assumed to be constant, so the x-axis has been replaced by current conditions, which can fluctuate drastically during endurance sports.

My JFK 50 adapting goals that eventually led to the one I started with: a top 10 finish.

I came into the race not having any idea how my body would respond. I was fresh off an Ironman followed by the birth of twins and was just getting my ultra legs back under me. I set a rather optimistic goal for myself of finishing in the top 10 (point 1 in the diagram). As I pushed through the initial trail section of the race, I felt great and was flying near the front. At that point (2) I thought, “wow, I should shoot for top 5 and prize money!” Then, at around mile 20 the wheels fell off completely. The thought raced through my mind that I wasn’t actually prepared for this, and there was no way I could make it another 30 miles. But before I rage quit, I adapted my goal back down to top 10 (3). Things kept getting worse. I thought, “alright, top 20 would still be pretty good” (4). Then, top 25 (5). Eventually, my pace stopped declining and I kept steadily moving along as other people’s paces fell off. Gradually I moved back up. To keep myself from becoming apathetic I began adapting my goals back in the other direction. By the end of the race I found myself once again shooting for top 10 (6) and ended up coming in 9th.

Back in the top 10

One important point for me to make here, is that I don’t believe in setting “fallback goals” before the race. If you already have them in your mind going into it, then it’s far too easy to, well, fall back on them. But being able to adapt during the race to ensure that you don’t stop pushing, either due to despair or apathy, is incredibly valuable in something like an ultra where circumstances can swing so wildly back and forth.

My Future Goals

So, where does that leave me? As I mentioned in my previous post, I’ve struggled a bit to set a new long term goal after Barkley. I have plenty of short term goals, ones that I hope fall out of my training, but not really one that drives me to train.

I’d like to age group podium at Kona, and do well enough in amateur age group standings to race for Team USA next year. I also haven’t raced a marathon in 3 years and would like to see if I can go sub 2:30. But if none of those happen, I’ll get over it and move on. On the other side, I have quite a few goals that I would love to shoot for if I thought they were obtainable even by future me. Some of them have large resource or time requirements that I just don’t know if I’ll have.

There are quite a few long FKTs that I would really love to take a crack at, but being able to dedicate 1-2 months to something like that isn’t something I really see as doable in the next few years.

As I mentioned earlier, extrinsic motivation can add to intrinsic motivation. And in some cases that might just be enough to get past the tipping point. So to maybe get myself past that point, I’m going to go ahead and put it out there that my next major goal is to go sub 24 hours at TWOT 100. It’s an incredible race in an area I love with the kind of tight-knit community around it that I enjoy, and after my experience there last year the goal is quite meaningful to me personally (a requirement for a good goal).

I also believe sub 24 there is right at the edge of what I could achieve by February. It’s not as long term of a goal as Barkley initially was for me, but I really think it can provide the drive I need right now. Not to mention my prize last year was two giant jars of apple butter, and it’s hard to top that in terms of added extrinsic motivation.

Out of apple butter? Only one thing to do…


Long term goals should motivate you to become the best possible version of yourself, while short term milestones help you get there as quickly as possible.

  • Motivation and improvement peak when difficulty is just below the point of failure for a given skill level.
  • Intrinsic motivation is much more powerful than extrinsic, and should be your primary consideration in determining the most difficult goal you can aim for without rage quitting.
  • Choose a long term, driving goal with a difficulty that’s near the limit of what you can achieve in the long term if you increase your capabilities to what you perceive as your maximum obtainable levels.
  • Choose short term milestones that are at the limit of what you can achieve now or in the immediate future.
  • Both goals and milestones should reflect the criteria that I discussed in  Failing with Purpose: personally meaningful, supported by friends and family, and possible outcomes (positive and negative) evaluated.
  • In endurance sports, being able to effectively adapt goals during an event to reflect current conditions is incredibly valuable.
  • I’m not a sith lord, but I am going to go for sub 24 hours at TWOT 100 next year.

15 thoughts on “The Goldilocks Difficulty

  • 2017-07-14 at 5:32 PM

    This is a very fascinating and informative post John. Your infographics and experiences on adaptation during a race is something I have experienced even though my race distances are not quite as tough. I appreciate your advice on “fallback goals”. I am running my 1st 50 miler tomorrow. I will try and get those 2 words out of my mind.

    Like all your posts John, you really put a lot of effort into them. Thank you for sharing! 🙂

    • 2017-07-14 at 8:55 PM

      Thank you, I’m glad you’ve enjoyed them and good luck in the 50 miler! I don’t think there’s anything wrong with having fallback “plans” ahead of time for if things go wrong, but I definitely don’t like to think of them as goals until I find myself in a situation where those plans might really be the optimal outcome given the present conditions.

      • 2017-07-24 at 9:53 PM

        Thanks so much John. My goals certainly did change throughout my 1st 50 miler and reading what you shared about adaptive goals just the day before was really timely. Messed up an ankle and did end up using a fallback plan. Ended up as the 2nd oldest finisher at 59 years old. Thank you so much again! 🙂

        • 2017-07-25 at 7:49 AM

          Congratulations, that’s great! I’m glad it helped and best wishes for a speedy recovery.

  • 2017-07-15 at 6:12 AM

    Thanks for this. A lot of insight that rings true, and plenty to chew over for a few hours on my next long run.

    I’m interested that your JFK 50 goals were to do with race position, something that depends on variables you can’t control. Is that an issue? If you achieve your goal because someone else gets it wrong, or don’t achieve your goal because the field is stronger than you were expecting, does the process still work?

    • 2017-07-15 at 12:09 PM

      My full list of JFK 50 goals were: 1) top 10, 2) 6:30 (which I hit exactly), and 3) within an hour of Walmsley. To your point, Walmsley kept that 3rd goal from happening. At some point you have to decide whether you’re racing racing the clock or racing others. It’s two entirely different strategies, but in any sport where your performance is measured against direct competition, obviously the competition is a factor you can’t completely control (even in running, there are a lot of mental games opponents can play with each other). If the competition is stronger than expected then you can certainly adapt goals during the race like I illustrate, but direct competition is a fierce motivator.

      • 2017-07-17 at 5:57 AM

        That makes some sense, especially at the pointy end of the field, where you are and I’m not.

        I find it tempting to think that for each competitor there’s a pacing strategy that’s the fastest way for them to get to the finish line, and that the best way to race others is to get to the finish line the fastest way you can, and that you should therefore ignore everyone else and run your own race regardless of whether you’re racing the clock or other people.

        I know that’s not quite right, because your mental state affects how fast you can get to the finish line too, and an optimal pacing strategy might be sub-optimal mentally (perhaps because it means letting competitors get away from you and running alone feeling demoralised).

        But I still find it tempting to think like that, probably because I can hang a lot of numbers on the physical side of things (pace, heart rate, calories in/out, etc.), but I can’t do that for the mental side of things.

        I’m about to do my first 40-miler. It’s a low-key event, so the field should be pretty mediocre. Despite that, I don’t expect to win, but there’s a chance I’ll be competitive. My goals for the race are entirely about time taken and maintaining an even pace, because I’m doing it to get the confidence to step up to 100k next spring. But if I find myself ditching the pacing strategy to try to hang on to someone, this discussion is what I’ll be thinking about.

        By the way, a future post on mental games during ultras would be interesting.

        • 2017-07-17 at 9:33 AM

          Thanks, that would make for a good post and is something I had an interesting discussion about a few weeks ago with a sports psychologist that I met.

          For a perfect example of racing others instead of the clock, look at the 2016 Olympics Men’s 1500M. The winning time was a 3:50! There are high schoolers that can run that. They basically jogged 1100M and then raced the last lap.

          Good luck in your race, and if you’re using the race as a stepping stone I’d run it a bit safe and not risk a blowup.

          • 2017-07-17 at 2:26 PM

            Thanks. I’ll try to be sensible, of course…

  • 2017-07-22 at 7:25 PM

    Love this post. Lots to think about. Thanks John!

  • 2017-07-24 at 9:27 PM

    Thanks for this follow up post! Such fascinating thesis research! I found the connections to motivation research fantastic: The U-shaped curve in terms of perceived difficulty in performance is spot on. This works perfectly for anxiety and performance as well. The Goldilocks effect is everywhere!

    I really appreciated your description of how you adjust to conditions mid-race. I just hadn’t my own experience with this last weekend at VT100. I had a very ambitious A goal of 22:30 I was shooting for (probably right on the edge of max difficulty for motivation). At mile 70 that started to slip out of reach, but I was able to hold on for a 23:15 finish. My poor pacers dealt with 30 miles of me mumbling math and pace adjustments to keep as close to that goal as possible, but it kept me present and motivated. I thought about the idea of failing with purpose out there during that time and channeled it into my race mantra of “walk with purpose” as my body failed me. I know I never would have gone as far under 24 without having set a truly ambitious goal at the outset.

    Thanks for sharing and good luck with TWOT100. That looks brutal… but intriguing!

    • 2017-07-24 at 9:36 PM

      Thank you, and congrats on VT100! That’s a fantastic result! I’m glad that the posts maybe helped a bit, and best of luck with your next challenge.

      I do the constant pace calculations too. I actually avoid data fields on my watch that do the calculations for me because doing them in my head is one way I stay focused and alert.

  • 2017-09-08 at 10:05 AM

    Hi John, Interesting post and it’s on a topic that I’m currently trying to grapple with. Last year was the best year of my endurance racing career by a long shot, top 20 in my first 100 miler in Apr., qualified for Boston (been a long term goal for 3.5 years) in Oct., finished ahead of my goal time in my first 100k in Dec. So after that coupled with the birth of my first (child) daughter, I have found myself waaaay over in that blue apathy range. I’m hoping a spectacular failure this next week at the SOS tri and the Fall Classic shake me up and provide some motivation. Like you I’d been looking at FKT’s but they seem too time consuming right now (maybe add to the “long term” list). I had been wondering how you were approaching things after your success at Barkley this year. I’ll probably be coming back to reference this article over the next weeks and months. Thanks and good luck.

    • 2017-09-09 at 6:08 PM

      Congratulations on your recent running accomplishments and the birth of your daughter, what a great year! I hope that you’re able to rediscover your motivation soon, but in the meantime it doesn’t hurt to enjoy the moment for a bit. I’m finding that while I have new goals, I can pursue them while putting a bit less pressure on myself. I’m hoping that I can remain motivated while maintaining a good balance with other things.


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.