Understandability in AI refers to how easily the workings and decisions of an AI model can be comprehended by people, generally making sure that AI systems convey their operational logic in human-accessible terms.


Imagine going to a magic show and the magician does lots of cool tricks. But wouldn’t you want to know how he does these tricks? Understandability in AI is like that magician telling you how he does his tricks. So, it’s about making sure that AI systems can explain in easy-to-understand language how they make their decisions.

In-depth explanation

The complexity of AI models, especially deep learning ones, often makes their internal workings difficult to interpret. Without a clear comprehension of these models, it’s like using a black box where inputs are given and outputs are received, but the process in between is unknown. Hence, the idea of understandability in AI is essentially trying to turn that black box into a glass box, so to speak.

Understandability can sometimes be considered a part of interpretability, but where interpretability is about the model being understandable at a macro- level, understandability zooms in further at a micro-level. Understandability allows the consumer of the AI model’s output to grasp why a specific decision was made by that model. It is about giving a clear cause-and-effect kind of explanation where, when a certain factor changes in the input, the impact on the output can be explicitly explained. In essence, it ensures that the model provides enough information to enable informed decisions and engender trust in its workings.

The techniques used to enhance understandability can vary depending on the kind of model. For simpler models such as linear regression and decision trees, they are inherently understandable due to the explicit representation of the decision making process. However, with complex models like neural networks, it can be complicated due to the large number of hidden layers and non-linear transformations. Specific techniques have been devised for them, such as saliency maps or feature importance charts.

Various factors can influence the understandability of an AI system, including the complexity of the model, variance in interpretations by different users, the nature of data used, and the specific scenario in which the AI system is deployed. Therefore, ensuring understandability should be a focus throughout the design and implementation of AI systems, embedded at every stage from problem formulation to model validation and user interactions.

As AI systems continue to drive decisions in diverse sectors from finance to healthcare, the ability to understand these systems is becoming increasingly critical. Improving understandability can reduce the risks associated with misuse of AI and bias in decisions, and can aid in ethical audits and regulatory compliance.

Interpretability, Explainability, Transparency, Salient Features, Trustworthy AI, Ethical AI, Saliency Maps, Feature Importance