Tech Firms Mine Our Digital Behavior for Medical Signals

Reporting by NATASHA SINGER

Published on nytimes.com

"It could help with understanding the effectiveness of treatments," he said.

The discipline remains so fresh and insufficiently researched that even its backers caution some digital phenotyping efforts might prove no more accurate at flagging health concerns than a crystal ball.

Imagine an outgoing individual who abruptly stops texting companions. That shift could point to depression, suggested Dr. Steve Steinhubl, who directs digital medicine at the Scripps Translational Science Institute in San Diego. Or "it could mean that somebody's just going on a camping trip and has changed their normal behavior," he said.

"It's this whole new potential for snake oil," Dr. Steinhubl said.

That hasn't deterred the surge into the field — by start-ups and major players like Facebook — despite unresolved questions about effectiveness and data privacy.

Hunting for Suicidal Ideation

One of the most sweeping projects is being carried out by Facebook.

The company recently disclosed that it was leveraging artificial intelligence to examine posts and live video feeds on its social network for indicators of potential suicidal thoughts. When the system spots particular linguistic cues — such as friends leaving comments like "Can I help?" or "Are you O.K.?" — it may attach an algorithmic score to the post and notify a Facebook review team.

In some scenarios, Facebook delivers users an encouraging notice with suggestions like "Call a helpline." In emergencies, Facebook has partnered with local authorities to send emergency responders to the user's location. The company said that, over a single month, its response team had collaborated with emergency workers more than 100 times.

Some health scientists praised Facebook's initiative, which ventures into the intricate and delicate territory of mental health, as well-meaning. However, they also expressed reservations. For one, Facebook has not published any study of the system's precision or potential downsides, such as unintentionally intensifying user distress.

"It's a great idea and a huge unmet need," Dr. Steinhubl said. Even so, he added, Facebook is "certainly right up to that line of practicing medicine not only without a license, but maybe without proof that what they are doing provides more benefit that harm."

Furthermore, Facebook is scanning user posts in the United States and several other countries for markers of possible suicidal thoughts without offering users an option to opt out of the scans.

"Once you are characterized as suicidal, is that forever associated with your name?" asked Frank Pasquale, a law professor at the University of Maryland who researches emerging health technologies. "Who has access to that information?"

Will Nevius, a Facebook spokesman, said Facebook removed the algorithmic scores tied to posts after 30 days. Cases involving emergency responders are maintained in a separate system that is not linked to users' profiles, he said.

Facebook said it had consulted with suicide prevention groups while designing the effort. Mr. Nevius added that publishing a useful study would be complex because of the difficulty in removing personal data and "the delicate nature of the posts."

Spotting Depression Through Clicks

Therapists traditionally diagnose depression by observing patients and questioning them about their feelings. Mindstrong Health, a mental health start-up in Palo Alto, Calif., is monitoring people's smartphone usage.

The company has created a research platform that continuously tracks users' phone habits, analyzing variations in taps and clicks for clues about mood and memory changes associated with depression.

"We are building digital smoke alarms for people with mental illness," said Dr. Thomas R. Insel, a Mindstrong co-founder and a former director of the National Institute of Mental Health.

Mindstrong's research app captures 1,000 smartphone-related data points — such as how long it takes someone to scroll through a contact list and select a name. The start-up enlisted 200 volunteers for pilot studies. Dr. Insel said a few of the signals, including shifts in users' keyboard accuracy and speed, aligned with comparable motor skill changes that researchers could gauge through lab tests.

The company is now participating in a large government-funded study of trauma patients. Part of it entails using the Mindstrong platform to investigate whether patients who subsequently develop post-traumatic stress disorder also exhibit corresponding shifts in their smartphone behavior.

"We've got these really interesting statistical signals with very high correlations," Dr. Insel said. "But whether that's going to work in the real world of clinical care is something we're looking at right now."

He added that Mindstrong had brought in law and ethics specialists to help examine the implications of its technology and craft ethical frameworks for using it.

"You want to think through all the unintended consequences early on," Dr. Insel said, "so they don't come back to bite you."

Evaluating Calls for Stress Signals

The traditional use of a phone — talking — is also being analyzed for health clues. Sharecare, a digital health company based in Atlanta, provides a wellness app with an optional feature that assesses users' stress levels during phone calls.

The system applies pattern recognition technology to categorize users' speech, the company said. After each call, the system delivers reports like "you seemed anxious" or "you seemed balanced." It also describes users' relationships with the people they call in terms of attitudes like "dominance" or "affection."

Jeff Arnold, a co-founder of Sharecare, described the voice scan as "an emotional selfie."

"If I can tell you your stress level in real time, it will in itself change your behavior," said Mr. Arnold, who previously founded WebMD.

Health insurers and self-insured employers utilize Sharecare to encourage wellness and control medical expenses. Sharecare is collaborating with the Georgia Institute of Technology to examine the effectiveness of its voice analysis tool.

Jiten Chhabra, a health tech researcher at the university's Interactive Media Technology Center, said volunteers who tried the voice-scanning feature reported feeling less stressed afterward. But he cautioned it was premature to determine whether the stress analysis directly produced the change — or whether volunteers had simply grown more relaxed in their everyday routines.

The company does not record the content of the calls it scans, it said. But the app did gather phone numbers for individuals on the other end of calls from Sharecare users, according to a New York Times examination. The service did not notify people speaking with Sharecare users that their interactions were being evaluated.

Jennifer Martin Hall, a spokeswoman for Sharecare, said the way the company protected data "makes it practically impossible for any Sharecare employees to access a phone number in the call information." She added that characterizing users' voice "analysis in terms of 'relationships' helps contextualize the relevance of their stress and enables them to be more mindful day to day."

Other academics suggested such widespread scanning could backfire — amplifying stress in otherwise healthy individuals.

"It's like we're in school forever," Professor Pasquale said, "and we're being graded in all these ways forever by all the companies that have the most data about us."

Continue reading the main story