To strike a ball moving at lightning speeds in baseball, tennis and cricket, athletes and coaches are increasingly embracing training techniques involving virtual reality.
Professional baseball players rapidly predict where a ball is heading from their opponent’s body movements. Credit: Norm Hall/Getty
When batters on the Duke University baseball team step to the plate, the opposing team’s pitcher is no stranger. In the days preceding a game, they will have spent hours in virtual reality (VR), watching a 3D avatar of their opponent throw pitch after pitch at them.
With each virtual pitch, the batters call it as a strike or a ball, a fastball or a curveball. Get it right, and the next simulation cuts out a little earlier; get it wrong, and the simulations extend again. Over time, the theory goes, the players will learn to recognize where the ball is heading as early as possible.
“A major element of sports like baseball and cricket is that they’ve been constructed to live right at the edge of human abilities,” says Greg Appelbaum, a cognitive neuroscientist at Duke University in Durham, North Carolina, who splits his time between medical research and working with the university’s sports teams. “If you can pick up a cue earlier, that gives you more time to make a controlled movement,” he says.
The technology used at Duke was developed by WIN Reality, a software company based in Austin, Texas. The version Duke uses does not allow batters to test themselves physically by swinging at simulated pitches or get precise feedback on their virtual shots. But Appelbaum thinks that even the purely perceptual training that the system provides will give athletes an edge.
The heights that these players’ careers reach will depend on many factors, including stroke mechanics, cardiovascular fitness and strength, and the ability to withstand the mental pressures of competition. But in sports such as baseball, cricket and tennis, where balls move so fast that the time to process their flight is minimal, perceptual skills and anticipation are elemental. If a technology such as VR can improve these even fractionally, the benefits on the field could be substantial.
Eyes off the ball
When Bruce Abernethy, a behavioural scientist at the University of Queensland in Brisbane, Australia, began studying high-speed sports in the late 1970s, he encountered an apparent discrepancy between what psychologists said was possible and the reaction times that athletes achieved. “If you did the chronometry,” he says, “it suddenly looked impossible.”
Psychologists said it took at least 250 milliseconds for people to begin moving after seeing a stimulus — and that was if they already knew what to do. If they had to decide what movement to make, reaction times doubled. This seemed simply too slow. In elite baseball, cricket and tennis, the ball regularly travels from the fastest pitchers, bowlers and servers to their opponents in about 400 ms. And to strike that ball, a player must be midway through an exquisitely timed and spatially precise full body motion when it arrives.
“The logical solution”, Abernethy says, “was maybe we hadn’t got the time correct as to when the stimulus onset is.” His hunch was that athletes weren’t merely responding to the ball, but acting on information available before the ball began its journey.
Nowadays, Abernethy says, it’s understood that top competitors use three broad classes of information, of which the flight of the ball is the last.
First, before a play even begins, players assess an opponent’s most likely actions given the state of play and what they know of that opponent’s favoured tactics. These contextual cues enable athletes to begin to prepare for the likeliest potential scenarios.
Next, Abernethy and others have shown that, crucially, players extract invaluable information about where the ball will travel by observing their opponent’s body movements as they prepare to deliver the ball — and that elite players use cues that are imperceptible to novices.
Placing cameras where the batter or receiver stands, researchers filmed cricketers bowling and racket-sport players serving and hitting1. They then played these videos to people with varying degrees of skill and stopped the playback at the moment of ball release or racket contact to ask whether the ball would pitch short or long, and go left or right.
“The probability of making a better than chance judgement was related to skill level,” Abernethy says, “so it seemed we were tapping into something that was important.”
By pausing the videos earlier and earlier, the scientists determined that professionals extract useful information sooner in an opponent’s bowling or serving action than novices do. Then, to uncover exactly which movements were most telling in each phase of action, they blocked out specific body parts in the videos. Abernethy laughs when recalling that, to begin with, he did this by laboriously sticking masking tape directly onto 16-millimetre film.
Cues, it emerged, tend to come first from the trunk, then from progressively more-distal body parts. In tennis, for instance, elbow movement provides the next clue, then, finally, racket motion. “What experts were doing was essentially a very skilled biomechanical analysis,” Abernethy says.
Reading an opponent’s movements allows athletes to begin an action in the right general direction to prepare to strike the imminently arriving ball. But it is not enough to ascertain the precise position they need to be in to make contact. To fine-tune their play, athletes must follow the flight of the ball itself.
Watching something is typically considered a passive act. But visually tracking a fast-moving object requires moving the eyes quickly and accurately: it is as much a motor act as is hitting a ball.
Flight tracking
To move the eyes, the brain must rapidly predict where an object is going — and prediction is something animals do naturally, says Mary Hayhoe, a vision scientist at the University of Texas at Austin. “It’s crucial for survival,” she says. “You have to have your body in the right place at the right time in order to survive.” Fast-ball sports test the limits of that basic neurobiological function.
Eye-tracking studies across various sports indicate a basic pattern whereby players’ eyes follow the ball on release, then produce a rapid movement known as a saccade to look to where the player has predicted the ball is travelling. When the ball enters this new field of view, the brain compares the new sensory data with the prediction it made, and the eyes again try to track the ball.
These saccades rely on a mental model of how a ball will behave given its intrinsic properties and the environment. Hayhoe and her colleagues have shown that such eye movements are highly accurate even in novices2. But she has also seen that in squash, for example, years of playing the game leads experienced competitors to initially track the ball for longer, then to produce more-accurate saccades3.
An influential study from 2000 involving three cricketers, including one full-time professional, suggested that they also tracked then saccaded — although here the better players seemed to saccade earlier — but even a medium-pace ball ultimately escaped their gaze4. However, in 2013, Abernethy’s team examined two of the world’s best batsmen and found that these players’ eyes (and heads) followed the ball all the way to the bat, often using a second predictive saccade and a third tracking phase to do so5.
“The more skilled people are, the more predictive and the further ahead of the game they are than anybody else,” Abernethy says. “It’s that predictive behaviour that allows them to appear to have all the time in the world.”
Contrary to the way in which many people think the brain operates, animals do not sequentially perceive, decide and then act, says Keith Davids, a sports scientist at Sheffield Hallam University, UK. Rather, the relationship is dynamic and reciprocal — perception guides and modifies actions, and actions constantly provide new sensory information. “Perception–action coupling,” Davids says, “is continuous and sophisticated.” This is the key to analysing sporting performance, he adds.
Throughout the biomechanical analysis of an opponent, and then the tracking of the ball, the player is responding to the best-available information. And although a player must commit to a certain shot at some point, that shot’s execution is constantly adjusted according to the continuing stream of information arriving.
A criticism of the work in which participants predicted balls’ trajectories from paused videos was that these people were verbalizing where they thought the ball would go, not moving to strike it. As a result, the researchers might have tested different pathways in the brain from those that directly guide action during sports.
To address this, Damian Farrow, a sports scientist at Victoria University in Melbourne, Australia, with his graduate student David Mann, gave novice and skilled cricketers liquid-crystal glasses that could be triggered to instantaneously black out6. These brave volunteers faced a real bowler, albeit behind a safety net, while the researchers blacked out the participants’ vision at different times, akin to the video studies.
The batters were asked to predict the ball’s direction in one of four ways: by speaking, by moving their legs, by pretending to bat, and finally, by actually attempting to put bat to ball. The study showed that, when the blacking out happened early, the accuracy of predictions made by the professionals increased the closer they got to playing for real. The predictions of novices, meanwhile, did not improve when they attempted to play the ball rather than verbalize.
“The importance of actually intercepting is critical,” Farrow says. Many aspects of sports performance occur at a subconscious level. When the athletes let their bat do the talking — engaging the brain systems used during actual play — an even greater elite advantage than was found in the video experiments is revealed.
Training prediction
The advantages that elite practitioners hold over novices in terms of anticipation are becoming clearer, and it is understood that years of training and immersion in a sport are key to the emergence of these differences. However, the precise long-term learning mechanisms that underpin improvements in anticipatory skills and action-coupling remain opaque. These processes develop gradually, over periods that last longer than standard neuroscience or psychology experiments, making them more challenging to study. Furthermore, scientists are wary of the risk that probing certain skills might bring into conscious awareness processes that are normally executed subconsciously — something that could harm a player’s performance.
Nevertheless, coaches and sports scientists are increasingly confident that the insights already yielded by research into anticipatory skills can guide the development of new — often technology-based — training drills. Such drills fall broadly into one of two camps: those targeted at improving fundamental perceptual skills that are foundational to numerous sports, and those targeted to the demands of specific sports.
In an example of the first approach, Appelbaum published a trial in which baseball players at Duke were put through a combination of dynamic vision-training sessions7. Participants practised catching or hitting a ball under strobe-light conditions — a sort of visual resistance training — as well as working on eye speed by following fast-moving trails of light, and training visual acuity by rapidly focusing near and far. A control group was given similar activities that didn’t directly engage the target skill, such as a task based on static rather than dynamic perception.
After training, neither basic visual tests nor batting average benefited. But during batting practice, the trained group hit the ball farther. The effect was small but statistically significant — and coming off the back of just eight and a half hours’ training, suggested the approach is worth pursuing further.
Sport-specific training, by contrast, focuses on developing the intrinsic knowledge of a sport’s precise demands.
To help junior players develop the skills that will be required of them as adults, Farrow suggests shortening the distance between the person delivering the ball and the person striking it in youth sports. At this level, the ball does not travel as quickly, so developing players have time to watch the ball and base their shots mainly on that information. By reducing the distance that the ball has to travel, Farrow suspects that junior players would have to learn to read their opponents’ movements from earlier ages, better preparing them to work within the time pressures that more-mature opponents bring.
Virtual testing
Virtual-reality software developed by US firm WIN Reality is used to train baseball players.Credit: WIN Reality
Farrow is also one of many sports scientists excited about the potential of VR technology to improve sport-specific training. He sees it as a logical progression from machines that deliver ball after ball to players without tiring the arm of a bowler, pitcher or server. Sometimes, these machines are placed behind a video wall, but the hope is that VR technology will provide a more relevant and tunable training experience than even these methods do.
Farrow is using VR to help developing tennis players get better at following and predicting ball trajectories. Participants are placed inside a simulation of the Rod Laver Arena in Melbourne, Australia, and are subjected to a variety of virtual deliveries that they can attempt to return with a real or mock racket. It’s not a perfect simulation. “If I put [Roger] Federer in there, he’d say, ‘Hang on, that’s not right, I hit the ball a millimetre that way,’ and he’d be right,” says Farrow. But junior players find the virtual environment and hitting experience realistic. They receive haptic (touch-related) feedback and a view of where their shot ended up — and the coaches can even manipulate where players see their shots landing, perhaps to reward them for improvements in shot timing and mechanics by showing them the ball flying down the line for a winner.
The system comes up short because of the appearance of the opponent — the ball is served by a low-resolution avatar that provides no useful pre-serve cues. Conversely, the WIN Reality system that Appelbaum and the Duke baseball team use is strong on avatars but lacks the hitting experience. Not only does the WIN Reality system train batters to look for earlier movement cues by cutting off the simulation earlier with each correctly called pitch, but it can also be made specific to an opponent. All pitchers have idiosyncrasies — variations on the universal biomechanical principles that batters internalize — that can provide extra information about the incoming ball. VR allows batters to learn these specific signals in training. “It seems unfair to the pitchers, right?” says Appelbaum.
Despite the excitement around VR technology that has seen numerous college and major league baseball teams adopt it, sports scientists agree that it’s hard to know for sure which strategies make a difference. Sports coaching is routinely shrouded in secrecy, hyperbole and hearsay, with few methodical studies. “The field itself still leaves a lot to be desired,” Appelbaum says. He is proud that in his dynamic visual-training study, he was able to apply the standards of a clinical trial to sports. In addition to controls, it was pre-registered, and training and control groups were randomized. There are some scientific data indicating that VR training can help baseball players8, but Appelbaum’s plans to rigorously test the system in use at Duke have been delayed by COVID-19. As someone who spends much time testing treatments for psychiatric and neurological diseases, he knows the value of a well-executed study. But Appelbaum says it’s notable that people who know baseball well are already embracing VR. “The scouts, the coaches, the players — they know bullshit, they know snake oil.”
Nature 592, S4-S6 (2021)
References
Abernethy, B. Perception 19, 63–77 (1990).
Diaz, G., Cooper, J., Rothkopf, C. & Hayhoe, M. J. Vis. 13, 20 (2013).
Hayhoe, M. M., McKinney, T., Chajka, K. & Pelz, J. B. Exp. Brain Res. 217, 125–136 (2012).
Land, M. & McLeod, P. Nature Neurosci. 3, 1340–1345 (2000).
Mann, D. L., Spratford, W. & Abernethy, B. PLoS ONE 8, e58289 (2013).
Mann, D. L., Abernethy, B. & Farrow, D. Acta Psychol. 135, 17–23 (2010).
Liu, S. et al. Psychol. Sport Exerc. 51, 101759 (2020)
Gray, R. Front. Psychol. https://doi.org/10.3389/fpsyg.2017.02183 (2017).