“His smile is awkward and atypical.” That’s a clinical description that many children with autism spectrum disorder receive. Shrikanth Narayanan set out to determine what that really means. 

Narayanan, University Professor, Niki & C. L. Max Nikias Chair in Engineering and research director at the USC Information Science Institute, and his team used motion capture technology to collect data about facial expressions of children with and without autism spectrum disorder, or ASD. They looked at how different parts of the face activate during a smile and the symmetry and synchronicity of facial movements. Using computational analysis, they found that what has been labeled “awkward and atypical” in the past often boils down to “a reduced complexity in dynamic facial behavior primarily in the eye region.”

That is how you measure a smile. And it’s one example of a behavioral biomarker — a measurable quality related to behavior that indicates something about a biological condition — that can help researchers diagnose, understand and treat ASD, a condition caused by differences in the neurogenetic underpinnings that affect communication and behavior.

According to the Centers for Disease Control and Prevention, about 1 in 36 children in the United States are diagnosed with ASD. While this is a complex condition with many variables and open questions, one thing is clear: Early diagnosis and support are key. The National Institutes of Health reports, “Early interventions [ages 2 and younger] not only give children the best start possible, but also the best chance of developing to their full potential. The sooner a child gets help, the greater the chance for learning and progress.”

Narayanan’s Center for Autism Research in Engineering (CARE), part of his Signal Analysis and Interpretation Laboratory (SAIL), aims to provide broader access and scale up support for children with ASD by using analytical tools to derive clinical insights. 

“Present-day AI capabilities can support children with ASD in ways that were not previously possible,” said Narayanan. For example, a relatively small number of clinicians are trained to diagnose ASD. “A caregiver may have to wait for several months to even get a child in to see a clinician,” he added. “When early detection is key, several months is a long time.” Narayanan’s team has found that by analyzing certain behavioral biomarkers, they could obtain clinically relevant insights, a potentially significant step for both earlier diagnoses and broader access.

Working with specialists including the co-developers of the Autism Diagnostic Observation Schedule, or ADOS, which is considered the “gold standard” in ASD diagnosis, Narayanan and the CARE research team are creating computational systems to better understand the disorder, with the goal of producing meaningful, applicable results for clinical decision making that directly benefit human health. 

They do this by developing novel methods to quantify aspects of ASD by identifying, measuring and analyzing behavioral biomarkers. The team leverages AI to level up both their research around these biomarkers and how they put their findings into practice.

The rapid advancement and accessibility of AI technologies is exciting to Narayanan. “How do we even begin to understand ASD in a quantitative way?” he asked. “That’s where new computational techniques including machine learning, novel datasets and using multimodality [e.g., looking at speech and language together, or speech, language, visual information and physiology together] come in. We can do this with AI approaches that are informed by — and can inform — both scientific advances and clinical translation.” He continued, “At the same time, from a scientific and technological point of view, these types of very complex human conditions underscore the need for creating new engineering methods and tools because our knowledge and technological capabilities are still so incomplete. It’s very humbling.”

1. BEHAVIORAL BIOMARKERS

Traditionally, clinicians have used qualitative descriptions of symptoms and characteristics in their assessment and treatment of children with autism spectrum disorder using interview reports and clinical observations. The CARE research team translates these target constructs into behavioral biomarkers — measurable qualities related to behavior that indicate something about a clinical condition — and uses data, engineering, quantitative analytics and artificial intelligence to learn more about ASD.

Prosody: The pattern of stress and intonation in speech. The researchers have broken this characteristic into 24 measurable prosodic features (pitch, volume, rate, etc.) that allow them to quantify and analyze how prosody varies with ASD severity.

Turn-taking: How well does a conversation flow between conversational partners? The researchers measure the percentage of time there is speech, silence and overlapping speech, and the time between turn exchanges during a conversation.

Word count: How many different words does a child use in a conversation, normalized by the total number of words spoken?

Speech Rate: The amount of time it takes to speak a syllable or a word, the number of words per utterance and the duration of pauses are used to help the research team describe vocal affectation.

Facial symmetry: The researchers measure how evenly facial movements happen from left to right across the face.

Facial synchrony: They measure the coordination of different parts of the face during expressions, e.g., when a traditionally developing child smiles, the corners of their mouth lift and the brow crinkles at the same time.

Electrodermal activity: Using wristbands, the researchers track physiological signals linked to stress, affect and cognition.

Conversation Partner Analysis

The researchers have also measured these biomarkers in interaction partners with children with ASD, e.g., a clinician having a conversation with a child with ASD. An interesting finding is that the prosodic, turn-taking and other language features of a clinician indicate the conversational quality degrades when they are interacting with children with greater severity of ASD. The researchers have shown that modeling interaction dynamics provides crucial context for the observed behavior.

2. THE AI OF IT ALL 

Professor Narayanan’s CARE research team uses machine learning to find patterns in the data. They implement AI techniques for behavioral phenotyping and stratification at an individual level, that is, identifying the aspects of ASD behavior that are tied to the child’s condition. They have also developed AI techniques to optimize the administration of well-established clinical instruments to provide efficiency and scale, support personalized technologies for supporting treatment, and track progress to such treatments.