PathBench: A compensive benchmark for pathology foundation models with real practical data from the world.

A Multi-task, Multi-organ Benchmark for Real-world Clinical Performance Evaluation of Pathology Foundation Models

Dataset Information

8,549cases

•

16,159slides

•

9hospitals

•

5organs

Model Ranking Analysis

Top: Average rankings across tasks | Bottom: Detailed rankings per task (Overall)

All Tasks

ACGxHP Detection

Classification

Organ

Stomach

Cohort

Internal

Evaluation Metrics

AUC

Consensus Molecular Subtyping

Classification

Organ

Colon

Cohort

Internal

Evaluation Metrics

AUC

Disease-free Survival (DFS) Prediction

DFS Prediction

Organ

Breast

Cohort

Internal

Evaluation Metrics

C-Index

Disease-free Survival (DFS) Prediction

DFS Prediction

Organ

Colon

Cohort

Internal

Evaluation Metrics

C-Index

Disease-free Survival (DFS) Prediction

DFS Prediction

Organ

Stomach

Cohort

Internal

Evaluation Metrics

C-Index

Disease-specific Survival (DSS) Prediction

DSS Prediction

Organ

Colon

Cohort

Internal

Evaluation Metrics

C-Index

Histopathological Grading Assesment

Classification

Organ

Stomach

Cohort

External-H3

Evaluation Metrics

AUC

Histopathological Grading Assesment

Classification

Organ

Stomach

Cohort

Internal

Evaluation Metrics

AUC

Histopathological Grading Assesment

Classification

Organ

Stomach

Cohort

External-H4

Evaluation Metrics

AUC

HPACG Detection

Classification

Organ

Stomach

Cohort

Internal

Evaluation Metrics

AUC

IDH Mutation Prediction

Classification

Organ

Brain

Cohort

Internal

Evaluation Metrics

AUC

IHC-AR Expression Prediction

Classification

Organ

Breast

Cohort

Internal

Evaluation Metrics

AUC

IHC-C-MET Expression Prediction

Classification

Organ

Lung

Cohort

Internal

Evaluation Metrics

AUC

IHC-CK5 Expression Prediction

Classification

Organ

Breast

Cohort

Internal

Evaluation Metrics

AUC

IHC-CK7 Expression Prediction

Classification

Organ

Lung

Cohort

Internal

Evaluation Metrics

AUC

IHC-ER Expression Prediction

Classification

Organ

Breast

Cohort

Internal

Evaluation Metrics

AUC

IHC-HER2 Expression Prediction

Classification

Organ

Stomach

Cohort

Internal

Evaluation Metrics

AUC

IHC-HER2 Expression Prediction

Classification

Organ

Breast

Cohort

Internal

Evaluation Metrics

AUC

IHC-NapsinA Expression Prediction

Classification

Organ

Lung

Cohort

Internal

Evaluation Metrics

AUC

IHC-PR Expression Prediction

Classification

Organ

Breast

Cohort

Internal

Evaluation Metrics

AUC

IHC-S-100 Expression Prediction

Classification

Organ

Stomach

Cohort

Internal

Evaluation Metrics

AUC

IHC-TTF-1 Expression Prediction

Classification

Organ

Lung

Cohort

Internal

Evaluation Metrics

AUC

Intestinal Metaplasia Detection

Classification

Organ

Stomach

Cohort

Internal

Evaluation Metrics

AUC

Lauren Classification

Classification

Organ

Stomach

Cohort

External-H3

Evaluation Metrics

AUC

Lauren Classification

Classification

Organ

Stomach

Cohort

Internal

Evaluation Metrics

AUC

Lauren Classification

Classification

Organ

Stomach

Cohort

External-H4

Evaluation Metrics

AUC

Molecular Subtyping

Classification

Organ

Breast

Cohort

External-H9

Evaluation Metrics

AUC

Molecular Subtyping

Classification

Organ

Breast

Cohort

Internal

Evaluation Metrics

AUC

Normal vs. Abnormal Classification

Classification

Organ

Stomach

Cohort

Internal

Evaluation Metrics

AUC

Overall Survival (OS) Prediction

OS Prediction

Organ

Breast

Cohort

Internal

Evaluation Metrics

C-Index

Overall Survival (OS) Prediction

OS Prediction

Organ

Colon

Cohort

Internal

Evaluation Metrics

C-Index

Overall Survival (OS) Prediction

OS Prediction

Organ

Stomach

Cohort

Internal

Evaluation Metrics

C-Index

Pathological Subtyping

Classification

Organ

Brain

Cohort

Internal

Evaluation Metrics

AUC

Pathological Subtyping

Classification

Organ

Stomach

Cohort

External-H3

Evaluation Metrics

AUC

Pathological Subtyping

Classification

Organ

Stomach

Cohort

Internal

Evaluation Metrics

AUC

Pathological Subtyping

Classification

Organ

Stomach

Cohort

External-H4

Evaluation Metrics

AUC

Perineural Invasion Detection

Classification

Organ

Stomach

Cohort

External-H3

Evaluation Metrics

AUC

Perineural Invasion Detection

Classification

Organ

Stomach

Cohort

Internal

Evaluation Metrics

AUC

Perineural Invasion Detection

Classification

Organ

Stomach

Cohort

External-H4

Evaluation Metrics

AUC

Polyp Detection

Classification

Organ

Stomach

Cohort

Internal

Evaluation Metrics

AUC

Primary and Metastatic Classification

Classification

Organ

Lung

Cohort

External-H6

Evaluation Metrics

AUC

Primary and Metastatic Classification

Classification

Organ

Lung

Cohort

External-H5

Evaluation Metrics

AUC

Primary and Metastatic Classification

Classification

Organ

Lung

Cohort

Internal

Evaluation Metrics

AUC

Primary Site Prediciton

Classification

Organ

Lung

Cohort

External-H6

Evaluation Metrics

AUC

Primary Site Prediciton

Classification

Organ

Lung

Cohort

External-H5

Evaluation Metrics

AUC

Primary Site Prediciton

Classification

Organ

Lung

Cohort

Internal

Evaluation Metrics

AUC

pTNM Staging

Classification

Organ

Breast

Cohort

Internal

Evaluation Metrics

AUC

TNM Staging

Classification

Organ

Colon

Cohort

Internal

Evaluation Metrics

AUC

TNM-N Staging

Classification

Organ

Colon

Cohort

Internal

Evaluation Metrics

AUC

TNM-N Staging

Classification

Organ

Stomach

Cohort

External-H3

Evaluation Metrics

AUC

TNM-N Staging

Classification

Organ

Stomach

Cohort

Internal

Evaluation Metrics

AUC

TNM-N Staging

Classification

Organ

Stomach

Cohort

External-H4

Evaluation Metrics

AUC

TNM-N Staging

Classification

Organ

Breast

Cohort

External-H9

Evaluation Metrics

AUC

TNM-N Staging

Classification

Organ

Breast

Cohort

Internal

Evaluation Metrics

AUC

TNM-T Staging

Classification

Organ

Stomach

Cohort

External-H3

Evaluation Metrics

AUC

TNM-T Staging

Classification

Organ

Stomach

Cohort

Internal

Evaluation Metrics

AUC

TNM-T Staging

Classification

Organ

Stomach

Cohort

External-H4

Evaluation Metrics

AUC

TNM-T Staging (2 classes)

Classification

Organ

Colon

Cohort

Internal

Evaluation Metrics

AUC

TNM-T Staging (4 classes)

Classification

Organ

Colon

Cohort

Internal

Evaluation Metrics

AUC

Ulcer Detection

Classification

Organ

Stomach

Cohort

Internal

Evaluation Metrics

AUC

Vascular Invasion Detection

Classification

Organ

Stomach

Cohort

External-H3

Evaluation Metrics

AUC

Vascular Invasion Detection

Classification

Organ

Stomach

Cohort

Internal

Evaluation Metrics

AUC

Vascular Invasion Detection

Classification

Organ

Stomach

Cohort

External-H4

Evaluation Metrics

AUC

WHO Grading

Classification

Organ

Brain

Cohort

Internal

Evaluation Metrics

AUC

Task Name	Organ	Type	Cohort	Metrics
ACGxHP Detection	Stomach	Classification	Internal	AUC
Consensus Molecular Subtyping	Colon	Classification	Internal	AUC
Disease-free Survival (DFS) Prediction	Breast	DFS Prediction	Internal	C-Index
Disease-free Survival (DFS) Prediction	Colon	DFS Prediction	Internal	C-Index
Disease-free Survival (DFS) Prediction	Stomach	DFS Prediction	Internal	C-Index
Disease-specific Survival (DSS) Prediction	Colon	DSS Prediction	Internal	C-Index
Histopathological Grading Assesment	Stomach	Classification	External-H3	AUC
Histopathological Grading Assesment	Stomach	Classification	Internal	AUC
Histopathological Grading Assesment	Stomach	Classification	External-H4	AUC
HPACG Detection	Stomach	Classification	Internal	AUC
IDH Mutation Prediction	Brain	Classification	Internal	AUC
IHC-AR Expression Prediction	Breast	Classification	Internal	AUC
IHC-C-MET Expression Prediction	Lung	Classification	Internal	AUC
IHC-CK5 Expression Prediction	Breast	Classification	Internal	AUC
IHC-CK7 Expression Prediction	Lung	Classification	Internal	AUC
IHC-ER Expression Prediction	Breast	Classification	Internal	AUC
IHC-HER2 Expression Prediction	Stomach	Classification	Internal	AUC
IHC-HER2 Expression Prediction	Breast	Classification	Internal	AUC
IHC-NapsinA Expression Prediction	Lung	Classification	Internal	AUC
IHC-PR Expression Prediction	Breast	Classification	Internal	AUC
IHC-S-100 Expression Prediction	Stomach	Classification	Internal	AUC
IHC-TTF-1 Expression Prediction	Lung	Classification	Internal	AUC
Intestinal Metaplasia Detection	Stomach	Classification	Internal	AUC
Lauren Classification	Stomach	Classification	External-H3	AUC
Lauren Classification	Stomach	Classification	Internal	AUC
Lauren Classification	Stomach	Classification	External-H4	AUC
Molecular Subtyping	Breast	Classification	External-H9	AUC
Molecular Subtyping	Breast	Classification	Internal	AUC
Normal vs. Abnormal Classification	Stomach	Classification	Internal	AUC
Overall Survival (OS) Prediction	Breast	OS Prediction	Internal	C-Index
Overall Survival (OS) Prediction	Colon	OS Prediction	Internal	C-Index
Overall Survival (OS) Prediction	Stomach	OS Prediction	Internal	C-Index
Pathological Subtyping	Brain	Classification	Internal	AUC
Pathological Subtyping	Stomach	Classification	External-H3	AUC
Pathological Subtyping	Stomach	Classification	Internal	AUC
Pathological Subtyping	Stomach	Classification	External-H4	AUC
Perineural Invasion Detection	Stomach	Classification	External-H3	AUC
Perineural Invasion Detection	Stomach	Classification	Internal	AUC
Perineural Invasion Detection	Stomach	Classification	External-H4	AUC
Polyp Detection	Stomach	Classification	Internal	AUC
Primary and Metastatic Classification	Lung	Classification	External-H6	AUC
Primary and Metastatic Classification	Lung	Classification	External-H5	AUC
Primary and Metastatic Classification	Lung	Classification	Internal	AUC
Primary Site Prediciton	Lung	Classification	External-H6	AUC
Primary Site Prediciton	Lung	Classification	External-H5	AUC
Primary Site Prediciton	Lung	Classification	Internal	AUC
pTNM Staging	Breast	Classification	Internal	AUC
TNM Staging	Colon	Classification	Internal	AUC
TNM-N Staging	Colon	Classification	Internal	AUC
TNM-N Staging	Stomach	Classification	External-H3	AUC
TNM-N Staging	Stomach	Classification	Internal	AUC
TNM-N Staging	Stomach	Classification	External-H4	AUC
TNM-N Staging	Breast	Classification	External-H9	AUC
TNM-N Staging	Breast	Classification	Internal	AUC
TNM-T Staging	Stomach	Classification	External-H3	AUC
TNM-T Staging	Stomach	Classification	Internal	AUC
TNM-T Staging	Stomach	Classification	External-H4	AUC
TNM-T Staging (2 classes)	Colon	Classification	Internal	AUC
TNM-T Staging (4 classes)	Colon	Classification	Internal	AUC
Ulcer Detection	Stomach	Classification	Internal	AUC
Vascular Invasion Detection	Stomach	Classification	External-H3	AUC
Vascular Invasion Detection	Stomach	Classification	Internal	AUC
Vascular Invasion Detection	Stomach	Classification	External-H4	AUC
WHO Grading	Brain	Classification	Internal	AUC

Dataset Information

All Tasks

ACGxHP Detection

Consensus Molecular Subtyping

Disease-free Survival (DFS) Prediction

Disease-free Survival (DFS) Prediction

Disease-free Survival (DFS) Prediction

Disease-specific Survival (DSS) Prediction

Histopathological Grading Assesment

Histopathological Grading Assesment

Histopathological Grading Assesment

HPACG Detection

IDH Mutation Prediction

IHC-AR Expression Prediction

IHC-C-MET Expression Prediction

IHC-CK5 Expression Prediction

IHC-CK7 Expression Prediction

IHC-ER Expression Prediction

IHC-HER2 Expression Prediction

IHC-HER2 Expression Prediction

IHC-NapsinA Expression Prediction

IHC-PR Expression Prediction

IHC-S-100 Expression Prediction

IHC-TTF-1 Expression Prediction

Intestinal Metaplasia Detection

Lauren Classification

Lauren Classification

Lauren Classification

Molecular Subtyping

Molecular Subtyping

Normal vs. Abnormal Classification

Overall Survival (OS) Prediction

Overall Survival (OS) Prediction

Overall Survival (OS) Prediction

Pathological Subtyping

Pathological Subtyping

Pathological Subtyping

Pathological Subtyping

Perineural Invasion Detection

Perineural Invasion Detection

Perineural Invasion Detection

Polyp Detection

Primary and Metastatic Classification

Primary and Metastatic Classification

Primary and Metastatic Classification

Primary Site Prediciton

Primary Site Prediciton

Primary Site Prediciton

pTNM Staging

TNM Staging

TNM-N Staging

TNM-N Staging

TNM-N Staging

TNM-N Staging

TNM-N Staging

TNM-N Staging

TNM-T Staging

TNM-T Staging

TNM-T Staging

TNM-T Staging (2 classes)

TNM-T Staging (4 classes)

Ulcer Detection

Vascular Invasion Detection

Vascular Invasion Detection

Vascular Invasion Detection

WHO Grading

Real-World Complexity, Zero Compromises

Rigorous, Reproducible Science

Transparency Drives Progress

Submit Your Model

Contribute Data

Website Improvement

Research Partnership