Study shows 'video game' more accurate than standard tests at identifying children with attention disorders

By Suzanne Leigh | Originally published on UCSF News

Child playing game on an iPad

In a study, 20 children with ADHD interacted with the Project: EVO™ platform on an iPad for approximately 10 minutes.  [Photo courtesy of Akili Interactive Labs]

A digital assessment platform designed to look and feel like a video game may successfully flag children with attention disorders, according to the results of a study published Sept. 20, 2016, in Translational Psychiatry. Ultimately a therapeutic version of the platform could be used to treat conditions such as attention deficit hyperactivity disorder (ADHD).

To assess the diagnostic capability of the platform, which is being developed as a medical device, researchers studied children with a genetic condition known as 16p11.2 deletion, which is associated with challenges with social communication. Some children may have an autism diagnosis, as well as attention deficits.

Response time to stimuli quicker, more consistent in ‘neurotypical’ participants

“Tests that measure cognitive control deficits in children with autism or ADHD often have only modest sensitivity,” said first author Joaquin A. Anguera, PhD, from the departments of neurology and psychiatry at UC San Francisco. “Here we wanted to see if a device with engaging, game-like functions that uses adaptive algorithms to automatically assess a user’s ability level would be able to measure selective attention with more precision than non-adaptive traditional tests.”

In the study, 20 children with ADHD, whose average age was 10, interacted with the Project: EVO™ platform on an iPad for approximately 10 minutes. The game-like test, which was designed with graphics similar to current consumer video games, consists of three components: visuomotor tracking, where participants steer a character through winding paths avoiding walls and obstacles; perceptual discrimination, where they respond selectively to colored targets by tapping on the screen while ignoring other targets or stimuli; and multitasking, where participants perform both tracking and discrimination simultaneously. The test is designed to adapt in real-time to continually challenge the user.

Lack of engagement may make testing less sensitive

The children also had cognitive assessments using two standard, non-adaptive tools on the iPad: a visual search and a Flanker task. Both tests measured response time to target stimuli while low-level distracting elements were presented. However, Project: EVO™’s multitasking assessment also allows for testing these selective attention abilities under high “interference,” through a set of algorithms licensed from UCSF. The group’s performance on the three tests was compared with 91 children without the deletion, whose average age was also 10.

The results of the tests using Project: EVO™ showed that children with the deletion had longer response times and more variability compared to their peers without the condition. However no such group differences were found on either the Flanker or visual search task, suggesting that the Project: EVO™ test was more robust in characterizing selective attention abilities than traditional tests.

“These standard tests have regularly failed to show clear differences in selective attention abilities between children with autism or ADHD, versus neurotypical children,” Anguera said. “Such null effects may result from a lack of engagement and/or tools that are too coarse to reveal subtle differences in cognitive abilities in populations, especially children that are known to be inherently variable when being assessed.”

Results predictive of school performance, quality of life

The ability to accurately assess cognitive control in children is important because these abilities determine how we reach goals and interact with our environment, according to senior author Elysa Marco, MD, director of the UCSF Sensory Neurodevelopment and Autism Program and associate professor in the UCSF departments of Neurology, Psychiatry and Pediatrics.

“Cognitive control is predictive of academic performance and influences quality of life. But not every child with a 16p11.2 copy number variation will have challenges in attention, as this cognitive ability has multiple contributors,” Marco said. “Knowing who is at risk, so that they can be monitored and treated as needed, is the way to move to a more personalized approach to medicine.”

Project: EVO™ grew from previous work at UCSF led by Anguera and colleagues involving the digital platform NeuroRacer, which was found to enhance multitasking performance in older adults and improve both untreated attention and working memory abilities. Project: EVOTM is currently in development by Akili Interactive Labs, which was co-founded by UCSF neuroscientist Adam Gazzaley, MD, PhD, who was not a co-author on the current paper. The Project: EVO™ platform is being developed to act not only as a diagnostic tool, but also as a personalized treatment for cognitive deficits. However Marco cautioned against optimism that Project: EVO™ would be helpful to all children with attention disorders.

“While we can detect and potentially train children with attention challenges, not all kids with challenges in one skill will end up with overall learning difficulties that need remediation,” she said. “They may have already found work-around abilities or solutions that compensate for them.”

Two researchers in the study, Jeffrey Bower, PhD, and W. Eddie Martucci, PhD, are employees of Akili Interactive Labs, the developer of Project: EVO™, and supplied the product to the study participants. No further conflicts of interest were declared. Research was supported by grants from the Simons Foundation, Wallace Research Foundation, James Gates Family Foundation, and Kawaja-Holcombe Family Foundation.

Co-authors of the study are Anne Brandes-Aitken, Camarin Rolle, Sasha Skinner, Shivani Desai, and Elliott Sherr, MD, all from UCSF; Jeffrey Bower, PhD, and W. Eddie Martucci, PhD, of Akili Interactive Labs, Boston; and Wendy Chung PhD, of Columbia University Medical Center, New York.

Read the paper

Further coverage

About UCSF Psychiatry

The UCSF Department of Psychiatry and the Langley Porter Psychiatric Institute are among the nation's foremost resources in the fields of child, adolescent, adult, and geriatric mental health. Together they constitute one of the largest departments in the UCSF School of Medicine and the UCSF Weill Institute for Neurosciences, with a mission focused on research (basic, translational, clinical), teaching, patient care, and public service.

UCSF Psychiatry conducts its clinical, educational, and research efforts at a variety of locations in Northern California, including UCSF campuses at Parnassus Heights, Mission Bay, and Laurel Heights, the UCSF Medical Center at Mt. Zion, Zuckerberg San Francisco General Hospital and Trauma Center, the San Francisco VA Health Care System, and UCSF Fresno.

About the UCSF Weill Institute for Neurosciences

The UCSF Weill Institute for Neurosciences, established by the extraordinary generosity of Joan and Sanford I. "Sandy" Weill, brings together world-class researchers with top-ranked physicians to solve some of the most complex challenges in the human brain.

The UCSF Weill Institute leverages UCSF’s unrivaled bench-to-bedside excellence in the neurosciences. It unites three UCSF departments—Neurology, Psychiatry, and Neurological Surgery—that are highly esteemed for both patient care and research, as well as the Neuroscience Graduate Program, a cross-disciplinary alliance of nearly 100 UCSF faculty members from 15 basic-science departments, as well as the UCSF Institute for Neurodegenerative Diseases, a multidisciplinary research center focused on finding effective treatments for Alzheimer’s disease, frontotemporal dementia, Parkinson’s disease, and other neurodegenerative disorders.

About UCSF

UC San Francisco (UCSF) is a leading university dedicated to promoting health worldwide through advanced biomedical research, graduate-level education in the life sciences and health professions, and excellence in patient care. It includes top-ranked graduate schools of dentistry, medicine, nursing and pharmacy; a graduate division with nationally renowned programs in basic, biomedical, translational and population sciences; and a preeminent biomedical research enterprise. It also includes UCSF Health, which comprises two top-ranked hospitals, UCSF Medical Center and UCSF Benioff Children’s Hospital San Francisco, and other partner and affiliated hospitals and healthcare providers throughout the Bay Area.