New AI for biopsies can analyze microscopic images, talk to pathologists about your tumor

To non-pathologists, the histology slide looked, as all histology slides do, like a sea of mottled lilac and burgundy. Oblong pink spots, like sprinkles on a cookie at a Barbie-themed birthday party, spotted the left side of the image.

To LLaVA 1.5, an open-source artificial intelligence mode, the cells looked like they were from the cheek. LLaVA-Med, a version of LLaVA trained on medical information, told researchers the cells were from breast tissue.

advertisement

“The image you’ve provided appears to be a histological slide of tissue stained with hematoxylin and eosin,” said GPT4 — technically correct, but rather unhelpful.

STAT+ Exclusive Story

STAT+

This article is exclusive to STAT+ subscribers

Unlock this article — plus in-depth analysis, newsletters, premium events, and networking platform access.

Already have an account? Log in

Already have an account? Log in

View All Plans

Get unlimited access to award-winning journalism and exclusive events.

Subscribe