AI Insight
This paper presents a method for interpreting neural network predictions by linearizing the computation process on a per-instance basis. The authors exploit the fact that while neural networks are globally non-linear, each individual prediction follows a locally linear path through the network based on its specific activation pattern. By reformulating convolutional neural network layers as linear matrix multiplications, they derive a simplified equation F(x) = W·x + b that reveals not only which input features are important but exactly how each feature contributes to a specific prediction.
Why it matters
This technique could improve trust and transparency in neural network applications by providing clearer explanations of how models make decisions. The approach has potential applications in both supervised classification tasks and unsupervised learning methods like t-SNE, making it relevant across multiple machine learning domains where interpretability is critical for deployment.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: Neural network have achieved remarkable successes in many scientific fields. However, the interpretability of the neural network model is still a major bottlenecks to deploy such technique into our daily life. The challenge can dive into the non-linear behavior of the neural network, which rises a critical question that how a model use input feature to make a decision. The classical approach to address this challenge is feature attribution, which assigns an important score to each input feature and reveal its importance of current prediction. However, current feature attribution approaches often indicate the importance of each input feature without detail of how they are actually processed by a model internally. These attribution approaches often raise a concern that whether they highlight correct features for a model prediction.
For a neural network model, the non-linear behavior is often caused by non-linear activation units of a model. However, the computation behavior of a prediction from a neural network model is locally linear, because one prediction has only one activation pattern. Base on the observation, we propose an instance-wise linearization approach to reformulates the forward computation process of a neural network prediction. This approach reformulates different layers of convolution neural networks into linear matrix multiplication. Aggregating all layers’ computation, a prediction complex convolution neural network operations can be described as a linear matrix multiplication $F(x) = W cdot x + b$. This equation can not only provides a feature attribution map that highlights the important of the input features but also tells how each input feature contributes to a prediction exactly. Furthermore, we discuss the application of this technique in both supervise classification and unsupervised neural network learning parametric t-SNE dimension reduction.
Source: Instance-wise Linearization of Neural Network for Model Interpretation