Glyph: An Interpretability Engine for AI-Driven Scientific Discovery in Clinical Data
Abstract
Glyph uses foundation models and sparse autoencoders to extract interpretable clinical concepts from multimodal records and CT data, revealing reproducible predictive associations across patient cohorts.
Automated scientific discovery has the potential to advance understanding of disease mechanisms and accelerate identification of early disease markers. Manual data annotation is infeasible at population scale, motivating methods that automatically extract pre-defined phenotypes in multimodal and unstructured clinical data. Such pre-defined feature sets determine and limit which associations these approaches can discover. This limitation extends to agentic AI systems that use these tools to query population-scale data. In contrast, foundation models (FMs) trained on large volumes of clinical data can perform tasks beyond their training objectives. This suggests their latent embeddings may represent novel or unexplored features that could yield insights into disease mechanisms, yet such latent features remain difficult to interpret. To address this challenge, we introduce Glyph, an automated multistage framework that extracts features from multimodal and unstructured clinical data using FMs and sparse autoencoders, interprets them as coded clinical concepts, and evaluates their value for clinical scientific discovery. We applied Glyph to abdominal CT volumes, CT radiology reports and laboratory measurements from 41,628 patients across different institutions. We identified 397 predictive concepts supported by 4,463 Bonferroni-significant associations. Moreover, a majority of discovered associations were automatically reproducible in held-out cohorts across different sites. Glyph concepts provided complementary associations to literature-derived markers, leading to gains in predictive power. We envision that interpretability can extend beyond a mechanistic understanding of neural networks to connecting implicitly learned features with clinical knowledge and establishing their significance for patient outcomes. We release Glyph at https://github.com/StanfordMIMI/Glyph.
Community
Can model interpretability help us learn something new about human health? Glyph is a framework for AI-accelerated clinical science. It uses sparse autoencoders to uncover patterns in multimodal clinical data hidden within foundation models, turning them into concepts researchers can investigate.
This revealed known and potentially novel risk markers at scale that reproduced across hospitals and improved risk prediction beyond markers drawn from the clinical literature. More broadly, we show how interpretability could move beyond explaining models to expanding clinical knowledge.
Glyph is open source, allowing researchers to extract our concept space from their own data. They can also discover new concept spaces in their own data, share their findings, and test whether those findings reproduce across datasets. We envision this facilitating reproducible clinical discovery at scale.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Interpretable Feature Discovery from Unstructured Data Using LLMs and Statistical Choice Models (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Hi authors, thanks for the work. A quick question: the introduction pairs p = 9.77e-4 and 2.93e-3 with the all-history Phecode rates (49.1%, 63.9%), while the supplement gives 9.77e-4 and 1.95e-3 for the Recent Phecode arm (47.2%, 59.9%); and the "33.8% cohort average at External Site #2" is the report-findings row of Figure 2 (overall 1,701/3,109 = 54.7%). Details in this GitHub issue: https://github.com/StanfordMIMI/Glyph/issues/1
Get this paper in your agent:
hf papers read 2611.08922 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 1
Datasets citing this paper 1
RobbieHolland/Glyph-research
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper