AI Insight
Researchers developed an interpretable Natural Language Inference system that uses only graph-based representations rather than directly processing text. The system decomposes sentences into atomic propositions, converts them to ConceptNet triples, and feeds three graphs (premise, hypothesis, and retrieved knowledge) into a language model, achieving 89.7% accuracy on SNLI dataset, only 1.9 points below a text-based model. On the more challenging ANLI dataset, performance was comparable to RoBERTa-large on two test rounds but showed larger gaps on others, with the "price of interpretability" ranging from 9-14 points depending on the benchmark.
Why it matters
This approach addresses a critical limitation in AI systems by providing auditable, transparent decision-making processes for natural language understanding tasks. The relatively small accuracy trade-off suggests that interpretable AI systems could be practical for applications where understanding how conclusions are reached is essential, such as medical diagnosis, legal reasoning, or fact-checking.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: While Large Language Model (LLM)-based Natural Language Inference (NLI) systems achieve high accuracy, their decision-making processes lack auditable structures. This paper explores whether NLI can be performed using only interpretable, graph-based representations of evidence. We introduce a fully graph-based pipeline where the classifier never directly processes the input text. Instead, sentences are decomposed into atomic propositions, converted into ConceptNet triples via constrained decoding, and represented as three graphs per pair: premise, hypothesis, and a retrieved ConceptNet subgraph. These graphs are then fed into a fine-tuned 0.8-billion-parameter language model. On the SNLI dataset, our pipeline achieves 89.7% accuracy, just 1.9 points below an identically trained text-based model. On ANLI, it matches the published performance of RoBERTa-large on rounds R2 and R3 (48.0% vs. 48.9% and 44.9% vs. 44.4%) but trails by 16 points on R1, resulting in an overall gap of 9 to 14 points compared to its text counterpart. We term this gap the price of interpretability and demonstrate that it stems from representational limitations rather than data constraints. Ablation studies further reveal that graphs and text are complementary: combining both modalities achieves 92.1% accuracy on SNLI.
Source: Can We Do Interpretable NLI with Graphs Based on Atomic Propositions?