Today, with the advancement of artificial intelligence models and the development of models such as deep neural networks, understanding the cause of the results of these models has become impossible. These models, which are generally called black boxes, are used in many artificial intelligence activities because of their very good results. Although these models perform very well in many AI activities, it is not clear how they work to produce a specific prediction. Now we need to find a way to make these models transparent, which helps to understand how they make decisions.
Explainable Artificial Intelligence
In the past few years, a hot and important concept called artificial intelligence has been expressed, which generally refers to the methods and techniques that can be used to explain how artificial intelligence works and how to clarify the decision-making of black box models for humans. In general, these explanations provide an answer to the question of why a model predicts an outcome. For example, consider the following model:
As shown in the figure above, the process of many machine learning models is that a black box is first trained by the dataset and that black box is used to make predictions on new inputs. It is clear that the function of the black box is not obvious, therefore, explainable artificial intelligence adds clarity to such models by providing a variety of explanations and turning them into a transparent model like the one below:
Explanations and their types
As mentioned earlier, providing explanations is a way to add transparency to various AI black box models. Computer scientists have tried for decades to open these black boxes and increase the transparency of such models by providing explanations. A recent research review shows that these efforts have yielded good results. Therefore, explanations can be considered as information about the results and model predictions, which helps to better understand the performance of the model. These explanations can be presented in the following types:
Textual explanations: In this type of explanation, a textual explanation is provided that shows the function of the model. Texts in this type of explanation can be simply presented or generated by the learning process. For example, the output of \(x_1\) is equal to \(y_1\) because \(x^3>20\).
Model simplification: In this type of explanation, a completely new model is reconstructed based on the previously trained model. This new and simplified model tries to optimize the previous complex model and reduce its complexity by maintaining its performance. For example, part a of the figure below shows this type of explanation.
Visualization: Visualization means being able to display and visualize the behavior of the model and its complex interactions through a visual display, for example, in the form of charts. For example, part b of the figure below shows an example of this explanation.
Local explanations: This explains the part performance of the entire system by dividing the model space into smaller subspaces and examining them.
Feature importance: This explanation calculates the importance of a feature on the model prediction, comparing and checking the importance of this feature in different models can give very good information about the performance of those models. Part c of the figure below refers to this type of explanation.
Explanation by example: This means that the sample extracted data from the model by which we can have a better understanding of the model performance. For example, input 3 leads to output 10.
Conclusion
By utilizing these explanation techniques, Explainable AI (XAI) not only enhances the transparency of AI models but also builds trust in AI-driven systems, making them more reliable for critical decision-making processes. As AI continues to integrate into various aspects of our lives, the importance of explainability cannot be overstated. It ensures that AI is not just powerful but also understandable and accountable.
For practical examples of XAI in action, you might be interested in exploring the following articles:
These articles delve into specific applications of XAI, offering insights into how explainability is applied in real-world scenarios.
