FIX Logo

FIX: Features Interpretable to eXperts

A benchmark for extracting features that are interpretable to real-world experts, ranging from doctors performing gall bladder surgery to cosmologists studying supernovae.


Datasets


Scroll down or click a setting to learn more, explore these settings in ready-to-use notebooks, and easily download these datasets from Huggingface to join the search for interpretable features!

Mass Images

Cosmological Mass Maps

Astronomers seek to use mass maps collected via telescopes to infer cosmological parameters governing the initial state of the universe, such as energy density \((\Omega_m)\) and matter fluctuations \((\sigma_8)\). While deep networks can use cosmological simulations to predict well at this task, explanations thereof can potentially draw attention to new patterns and discoveries if they are understandable to cosmologists. Explanations of mass map predictions that are aligned with cosmological structures (e.g. voids and clusters) can potentially advance our understanding of the formation of the universe.

Supernova

Supernova Identification

The Legacy Survey of Space and Time uses telescopes to search the universe for rare and new astronomical phenomena, such as exploding supernova. Given the size of the universe and the limited view of a telescope, observation schedules need to focus on periods of activity that correspond to astronomical activity to not squander telescope resources. While deep learning models can accurately classify supernova from light observations, aligning explanations with wavelength structures can potentially help astrophysicists in characterizing new and different supernova.

Politeness image

Multilingual Politeness

Why are certain phrases polite in one language and rude in another? Understanding the nuances of politeness across languages can aid in cross-cultural understanding and communication. To this end, communication and psychology research has identified multilingual lexical categories that can capture different aspects of politeness. Aligning explanations of politeness with lexical categories that are common across different languages can help people from different backgrounds understand each other.

Emotion Groups Image

Emotion Detection

Understanding the emotion behind language is a critical component of systems like chatbots that interact with human users. Psychology researchers traditionally explain emotion via the circumplex model of affect, which quantifies emotion with respect to two axes of arousal (i.e. magnitude of intensity) and valence (how negative or positive). Explanations for emotions in language that are aligned with the circumplex model can help guide systems towards more empathetic and adaptable interactions with people.

Chest X-Ray image

Chest X-Ray Pathologies

Medical imaging is a widely used tool for diagnosing problems in the chest. These X-ray images provide doctors with a view into the internal structures and organs, from which diagnoses are made. While deep learning models have seen success in predicting from raw X-ray data, aligning explanations of pathology predictions with bone structures and organs can help doctors better understand the resulting diagnosis.

Surgery image

Cholecys­tectomy Surgery

Cholecystectomy surgey, otherwise known as gallbladder removal, is one of the most common elective abdominal surgeries in the US. AI assistants can potentially identify safe/unsafe regions, which can reduce the risk of major complications like bile duct injuries that cost the US healthcare system $1B annually. To effectively communicate with surgeons, explanations of safety need to aligned with anatomical structures that surgeons use to judge safety (i.e. the critical view of safety).