Why organisations are struggling with AI risks

As AI systems become more deeply embedded in products, workflows, and customer-facing experiences, the risks have grown more visible. Incidents involving hallucinations, flawed code generation, misinformation, and misaligned outputs have shown how unpredictable AI can be in real-world environments.

These challenges are not theoretical. Industry examples across sectors show how models can inadvertently reveal sensitive information, generate misleading outputs, or create vulnerabilities that can be exploited. Attackers are also leveraging AI to scale fraud, impersonation, and content manipulation, including deepfake-enabled social engineering attempts.

To build trust, organisations need greater clarity, stronger controls, and consistent methods for evaluating AI behaviour. This is why many have turned to emerging toolkits and frameworks to benchmark model performance, test for bias, assess robustness, or simulate adversarial attacks.
 

A diverse but fragmented toolkit ecosystem

The market now includes government-led initiatives, commercial open-source packages, and academic benchmarking suites. More than 80 percent of the toolkits surveyed are open-source and focus on principles such as fairness, explainability, safety, and security. Examples include:

• AI Verify – A WebUI developed by Singapore’s IMDA to evaluate AI systems along multiple dimensions.
• Project Moonshot – A benchmarking and red-teaming toolkit for Large Language Models.
• Veritas Toolkit – A MAS initiative to support fairness and transparency in financial AI systems.
• MLCommons Safety Benchmarks – Industry benchmarking for safety-related model performance. 

Beyond these government-led initiatives, several widely used open-source toolkits address specific evaluation needs. Microsoft’s Fairlearn helps teams measure and mitigate model fairness issues, while IBM’s AIF360 and AIX360 provide comprehensive libraries for fairness testing and explainability. Meta’s Cyberseceval focuses on cybersecurity risks in LLMs, and Microsoft’s PyRIT enables automated red-teaming for generative AI systems. Academic tools such as HELM and AIR-Bench benchmark model safety and performance using standardised datasets, while the EleutherAI LM Eval framework provides a unified framework for testing generative models across multiple tasks.

This diversity of tools is useful but can also be overwhelming. Each toolkit approaches AI evaluation differently, uses different metrics, and is built for different stages of the AI lifecycle.
 

Regulatory expectations continue to rise

Governments are moving beyond voluntary guidelines into enforceable regulation. Examples include:

• EU AI Act – A risk-based regulatory framework governing AI deployment.
• NIST AI Risk Management Framework – Guidance for structured evaluation and governance.
• ISO/IEC 42001:2023 – A global standard for AI management systems.
• Singapore’s Model AI Governance Framework and Generative AI Governance Framework – Guidelines for governing AI responsibly.
• CSA Guidelines on Securing AI Systems – Practical guidance for securing AI pipelines. 

As regulatory expectations tighten, organisations need clarity on how to measure compliance across the AI lifecycle, from data management to model training, deployment, monitoring, and incident response.
 

Gaps in today’s AI evaluation tools

Despite the growing number of frameworks, toolkits and benchmarks available, most organisations still struggle to obtain a clear, comprehensive view of AI risk. Several structural gaps explain why.

1. Difficulty interpreting results

Many toolkits generate scores or metrics that can be hard to interpret without deep technical expertise.

For example, fairness libraries often offer multiple statistical measures, each with different implications. Benchmarking tools may produce performance or risk scores but provide limited clarity on what those scores actually mean for a specific use case.

Tools such as Moonshot, AIR-Bench and LM Eval often generate capability or safety scores without accompanying guidance, making it hard for teams to interpret real-world implications or translate results into operational decisions.

The result is that business and compliance stakeholders often receive outputs they cannot easily translate into operational decisions.

2. Narrow or incomplete evaluation coverage

Many tools were designed for depth rather than breadth.

• Some focus only on fairness.
• Some specialise in explainability.
• Some only perform adversarial red-teaming.
• Some benchmark capabilities but not safety.

This pattern appears across the broader ecosystem. Fairlearn focuses primarily on fairness metrics, AIX360 on explainability, while AIF360 offers depth in statistical fairness but limited coverage of other principles. Benchmarking tools such as HELM, AIR-Bench, and LM Eval measure capability or safety but do not assess areas like privacy or transparency. Conversely, red-teaming tools such as PyRIT, Giskard, or Cyberseceval identify harmful behaviour or cybersecurity risk but do not provide broader lifecycle evaluation.

Even broader tools like AI Verify offer multiple dimensions but may provide fewer specialised metrics than focused libraries. This makes a “single toolkit” approach unrealistic; organisations must combine tools to achieve full lifecycle coverage.

3. Inconsistent methodologies across tools

Different toolkits measure similar principles—fairness, robustness, safety—using different datasets, thresholds, or mathematical definitions.

This inconsistency can lead to contradictory conclusions when evaluating the same model. Without a harmonised evaluation standard, organisations struggle to compare results across toolkits in a meaningful way.

4. Technical complexity and usability issues

Some toolkits require advanced configuration, dependency management or specialised knowledge to deploy effectively. Examples include:

• Python package dependencies
• Command-line setup
• Limited documentation for niche tools
• Dataset formatting requirements

Tools like HELM, for example, can require complex environment setup. Other tools face similar usability challenges. For instance, PyRIT and Cyberseceval require familiarity with security-oriented workflows, Giskard relies on automated test suite configuration, and libraries like AIF360 and Fairlearn demand expertise in selecting and interpreting statistical fairness metrics. These factors can increase the learning curve for teams without deep technical backgrounds.

5. No guidance on “what to do next”

Even when an organisation successfully evaluates an AI model, most toolkits stop at presenting the results. They do not advise on:

• Prioritising risks
• Selecting mitigation strategies
• Incorporating findings into governance workflows
• Aligning with regulatory frameworks
• Deciding which lifecycle areas require re-testing

This leaves AI teams with a gap between measurement and action, thereby increasing the need for expert interpretation and structured governance.

 
How to choose the right AI evaluation tools: a practical, lifecycle-based approach

Choosing the right AI evaluation tools starts with understanding what you need to test and when you need to test it. No single toolkit covers every principle: capability, fairness, robustness, safety, security. The key is matching the right tool to the right stage of the AI lifecycle. Here are the steps for how organisations can confidently select the appropriate benchmarks, principle-based tests, and red-teaming tools based on the AI model’s maturity, the risks involved, and the outcomes they need to validate.

1. Identify the AI lifecycle stage

Start with clarity on where the model sits:

• Data preparation
• Model development
• Evaluation
• Deployment
• Monitoring
• Incident response

Different stages require different tests and controls.

2. Define the right metrics for that stage

Each lifecycle stage has its own relevant principles:

• Fairness during data processing
• Explainability during model development
• Robustness before deployment
• Red-teaming during pre-production testing
• Safety monitoring post-deployment

Without clear metrics, tool selection becomes arbitrary.

3. Select the most suitable toolkit(s) for each goal

Rather than forcing one toolkit to cover everything, organisations should adopt a modular approach:

• Benchmarking tools to understand model capability
• Principle-based tests for fairness, explainability and robustness
• Red-teaming tools for stress testing and safety evaluation

Most comprehensive evaluations require combining 2–4 toolkits. For example, organisations may combine Moonshot or AIR-Bench for benchmarking, AI Verify, Fairlearn, or AIF360 for principle-based testing, and PyRIT, Giskard, or Cyberseceval for red-teaming and adversarial risk assessment. No single toolkit provides complete coverage, but together they offer a more holistic evaluation across capability, fairness, robustness, safety, and security dimensions.

4. Combine results into an integrated governance workflow

This “multi-tool” approach must be tied back to organisational governance:

• Evaluation documentation
• Risk registers
• Model cards
• Compliance mapping to NIST, ISO 42001, IMDA frameworks
• Continuous monitoring dashboards

This ensures evaluation insights become operational safeguards.