i don’t buy it: erythropoietin does benefit cycling performance.

a recent scientific paper gained a lot of attention in the mainstream media, being picked up by outlets such as reuters, espn, and the independent. these media organizations wrote some surprising things, even going as far as saying that lance armstrong was stripped of his title for a drug that “doesn't even work”. given my interests, i was shocked. so I downloaded and read the actual paper. this is what the media (and authors) got wrong.

firstly, it must be acknowledged that the following critique does not negate the necessity or benefit of this study, nor should discourage future studies of this kind. the authors are clear to acknowledge many of the limitations of this study and should be commended for doing so - although there are likely more limitations than those identified and the downplay of said limitations was perhaps too haphazard.

for those who have not yet had a chance to read the study, researchers from the netherlands recruited trained cyclists and randomized them to one of two groups - the groups either received recombinant human erythropoietin (rhepo) or a placebo for 8 weeks. the dosing varied, as the researchers monitored hematocrit and hemoglobin concentrations. the researchers were fairly stringent in their inclusion criteria, ensuring the athletes were actually well-trained cyclists, and managed to include 23 and 24 athletes in rhepo and placebo groups respectively.

before and after the doping regimen, as well as several times throughout the study, athletes completed hematological assessments, as well as maximal exercise tests, with both gases and power output data being collected. before and after the doping intervention, athletes also completed a sub-maximal assessment akin to a time-trial, again collecting data characterizing mean power output and and oxygen consumption. finally, 10-16 days after the last dose, athletes completed a race-style ride up mount ventoux, the french mountain, made famous by the tour de france and widely considered to be one of the most difficult and complex climbs in the world.

my criticisms of this paper lay not with the purpose or intent of the study. rather, the study design employed does not lead to the conclusions espoused by the authors and subsequently spread throughout the media. due to the number of points i wish to raise, i have numbered each point to maintain some structure to what may become an uncontrolled harangue.

1) to start, the authors presented evidence that doping with rhepo improved almost all hematological parameters and performance in the maximal exercise test. out of respect for potential copyrights on the paper i will not present the actual tables here, suffice to say however that vo2max, ventilatory thresholds, and maximal power output all showed statistically significant increases relative to the placebo group. furthermore, even thought the rhepo group started with a lowered lactate threshold, their threshold at the end of the 8 weeks was greater than the placebo group, although this was not statistically significant. in all, almost every performance parameter we associate with cycling fitness increased. yet the authors decided to conclude that rhepo does not work for improving cycling performance, for reasons discussed below. this is already ludicrous, but it gets worse.

2) the sub-maximal protocol showed no effect of rhepo on sub-maximal cycling performance. that is, there were no statistically significant differences between improvements in the groups. however, the data tables show that average power output increased in the rhepo group by 13.55 W and by 7.66 W in the placebo group. so what’s going on here? looking at the sub maximal cycling testing protocol, it appears that cyclists started riding at 80% of maximal power output, and adjusted their power output from there by indicating “up” or “down” to the researchers. The bikes were presumably electronically controlled to produce a constant load. so far, this is fine. however, the riders could only increase the constant-load in 10 W increments! considering the mean change only (13.55 W), this means that riders would have had to push to at least 20W above their baseline value. some riders clearly did, whilst others likely tried to push higher and may have in fact felt comfortable at a 19W increase, but due to the increments of change, had to come back down to only a 10W increase. this interval structure to the power output may also explain the poor correlation with other parameters, since the researchers were artificially limiting the variability around the measure. i hope it is clear why this is a huge problem with this sub-max protocol. i feel almost justified in dismissing most of their analyses of the sub-max protocol.

3) specifically related to the population analyzed, the authors were very clear about the limitations of using ‘well-trained’ cyclists, as opposed to elite cyclists in these types of studies. for obvious ethical and moral reasons, researchers can not dope professional riders, so we must find alternate populations to sample. this is also fine. what i have a problem with is that the authors stated that “our participants were well trained cyclists, and at least closely approached the level of elite cyclists.” this is just after writing that although their subjects had vo2max values of ~55 ml/kg/min (average, for an endurance athlete), and elite athletes can have values closer to the low 70’s and as is common among the top-tier riders, into the 80’s. the wattages produced also show that these riders are not elite riders, with peak w/kg values in the low 4 w/kg range, even though good amateur riders often produce values twice this. a brief look at the mount ventoux segment on strava.com, after adjusting for the length of the ride and being conservative with the gradient, the time these riders achieved up mount ventoux barely puts them in the top 250 riders for 2017 alone. the point is that these are not elite athletes, and they should not be portrayed as such, no matter how much the authors want to extend these findings to elite athletes; even less so for top tour quality riders. potentially more egregious than this however is that the authors then stated that since their findings were no different in their higher performing athletes, even better athletes would likely be similar. i believe they meant that elite cyclists would also not improve sub-maximally - but framed another way, the elite athletes would also supposedly get a vo2max, max power, and lt improvement. this is such a strange claim all around that i am at a loss as to why they would include it.

4) and now we get to the point of the study that likely caused all of the popular media coverage: the ride up mount ventoux.

a) before analyzing these results, it must be pointed out that this ride occurred 10-16 days after the last rhepo administration. the half-life of rhepo is fairly short (in the range of hours, rather than days), and the researchers did not present any hematological data to show that the rhepo group still had favorable blood profiles (i.e. higher hematocrit and hemoglobin). the ommission of these data are curious in the least.

b) without lecturing on study design paradigms, in order to measure change as a result of an intervention, pre- and post-measures are necessary. this is known as a within-subjects design, and limits the variance to be accounted for, because individual variability is accounted for in the design. the researchers clearly understand these concepts, since they pre- post- tested for all other parameters. however the race up mount ventoux in this study was a single event, taking place after the intervention. the researchers recognize this limitation without explicit stating it, because they tried to correct for it by co-varying their analysis with max power output from prior to the intervention. they did this knowing that maximum power output does not have a strong correlation with climbing cycling performance!

c) in short, the authors found no statistical difference between rhepo and placebo groups (actual mean difference has the placebo group being about 17 seconds faster). the authors used this to support the notion that rhepo is not effective at improving cycling performance. however, since we have no baseline performance to compare against, we actually have no idea whether rhepo improved their real world cycling performance! the only statement one could make about these results are that rhepo use does not possess the ability to overcome all of the other racing variables and make winners out of people that otherwise would lose by a significant margin. instead, the authors massively overstate their findings. consider, for example, if a pre-intervention showed that the placebo group had riders that finished the ride minutes ahead of the rhepo group. these post-intenvetion results, even though the rhepo group still would lose by 17 seconds, would clearly show the effectiveness of rhepo. without these data we can not make these claims, even though the authors want to collate this finding with the ‘lack’ of improvement in sub-maximal riding. so no, authors and popular media, these results do not show the rhepo is ineffective at improving real-world cycling performance.

5) finally, the authors recognize that the benefit of rhepo isn’t necessarily related to single-bout exercise performance, but rather to the ability to recovery and re-produce endurance performances, both in training and during stage races. however they felt justified in dismissing this criticism in a very off-hand way, by saying that these effects are probably only of clinical relevance if the effects of rhepo are observed in sub-maximal and road race performance. how the authors possibly contorted their minds to reach this conclusion is beyond me.

in conclusion, this study had many issues - and i have only highlighted the points i think were most severe. again, just to reiterate: limitations in research are unavoidable. it is not the fact that there were limitations in this study that mean we should dismiss it; conversely, in spite of the limitations there are some important conclusions that should be considered. my issue with this study is that the authors over-interpret their data and make claims that are not justified by their findings. and now those interpretations have found their way into the press and will stand forever. although there appears to be knowledgeable exercise physiology or performance researchers that worked on this study, i have to question their input into the findings. there is a reason this was published in a clinical, rather than exercise specific, journal.

as a parting thought, the authors suggested that their interpretation, that cycling performance is not effected by rhepo, may lead to reduce use of the drug by athletes. this is admirable, but i am of the opinion that this interpretation actually harms the anti-doping effort that both they and i serve. athletes are going to continue using substances they believe gives them an edge, no matter how small that edge may actually be. the placebo effect alone is sufficient to continue doping for many athletes. the only minds and behaviors that may shift on account of a research study like this are those in charge of doping control. considering the already underfunded and sparse doping-control efforts, the last thing we need is to be telling these organizations “oh, don’t worry about testing for rhepo, it doesn’t affect performance anyway.” this is wrong, and potentially harmful.

what do you think? do you disagree with my analysis? I would love to hear other interpretations or critiques of the above points. engage with me on twitter: @pedspective