_

What does AI explainability in cybersecurity entail?

The concept of “AI Explainability”

Explainability is defined as the ability of a system to make its reasoning process and results intelligible to humans. In the current context, sophisticated models often operate as “black boxes,” concealing the details of their functioning. This lack of transparency raises issues. Without a clear understanding of the decision-making process, it becomes challenging to identify, let alone correct, potential errors. Moreover, it’s difficult for humans to trust AI that delivers results without apparent justification.

The Significance of Explainability

In domains where decision-making is critical, it is essential to understand how AI operates for us to trust them. The absence of explainability and transparency is currently a barrier to integrating AI into these sensitive sectors. Security analysts for example need to know why a behavior was classified as suspicious and receive in-depth attack reports before taking a significant action, such as blocking traffic from specific IP addresses.

However, explainability doesn’t only benefit end users. For AI system engineers and designers, it simplifies the detection of potential ML model errors and prevents “blind” adjustments. Explainability is thus central in designing reliable and trustworthy systems.

How to Make AI Explainable 

• ML models like decision trees are naturally explainable. While generally less efficient than more sophisticated ML techniques like deep neural networks, they offer complete transparency.
• Some “post hoc” techniques, such as SHAP and LIME, have been developed to analyze and interpret “black box” models. By modifying inputs and observing corresponding output variations, these techniques allow the analysis and deduction of the functioning of many existing models.
• The “explainability-by-design” approach goes beyond post hoc techniques by integrating explainability from the inception of AI systems. Rather than seeking to explain models after the fact, “explainability-by-design” ensures that every step of the system is transparent and understandable. This may involve the use of hybrid methods and enables the design of appropriate explanations.

How Custocy Applies This?

Custocy actively explores the most sophisticated explainability techniques. In its current version, Custocy’s NDR integrates a report generated with SHAP, which identifies the elements our AI models have considered. This helps analysts save time and assess the relevance of our model’s detections.

Custocy is also investigating a cutting-edge approach to network security explainability: “explainability-by-design.”

This means that transparency and explainability are at the core of the design of our solution developed in our research laboratory. This approach allows for selecting the most suitable techniques for each objective, identifying and correcting errors, and generating data to challenge the models. The “explainability-by-design” approach also aims to provide relevant explanations tailored to security analysts. Our “explainable-by-design” systems produce explanations at multiple levels: visual representations for quick network status awareness and detailed attack reports for a deeper understanding.

In Conclusion

Explainability in AI is not a luxury but a necessity, especially in sensitive domains like cybersecurity. It not only builds user trust but also continuously improves detection systems. Therefore, it is an essential factor to consider when choosing a security solution.