What is AI governance – and why do organisations need it?

Artificial intelligence is becoming a natural part of working life. Employees use ChatGPT for content production, developers use AI assistants for programming, and a growing number of IT systems are incorporating built-in AI features.

For many organisations, this is happening faster than the establishment of guidelines, accountability and control mechanisms. The result is that AI is being adopted without management necessarily knowing how, where or for what purposes the technology is being used.

“AI governance” has become one of the most important terms in artificial intelligence. In simple terms, it concerns how organisations manage, control and follow up on their use of AI.

Governance is a term used in areas such as quality, information security, data protection and corporate management. It describes how an organisation establishes accountability, rules, processes and control mechanisms to ensure that an area is managed in the desired way.

When we talk about AI governance, it is therefore not about the technology itself, but about how the organisation manages its use of artificial intelligence.

AI governance is about establishing rules, accountability, processes and control mechanisms for how artificial intelligence is used within the organisation.

The goal is not to limit innovation or prevent the use of AI. On the contrary, AI governance is about facilitating safe, effective and responsible use of the technology.

An AI governance programme should help the organisation to:

• use AI in a controlled manner
• manage risks associated with AI
• meet relevant legal requirements
• protect information and personal data
• ensure quality and reliability in AI-based processes
• document how AI is used and followed up

In short, AI governance is about managing artificial intelligence in the same way that organisations already manage quality, information security and data protection.

In just a few years, AI has moved from being a niche technology to a tool used by employees across almost every function.

Leaders use AI for analysis and decision support. Marketing teams use AI for content production. HR uses AI in recruitment and competence development. Developers use AI for code generation and testing.

At the same time, both the risks and the requirements for control are increasing.

The need for AI governance is growing rapidly, driven in part by the following.

Even where the organisation has not introduced its own AI solutions, employees often use AI tools on their own initiative. This can create challenges relating to security, quality and compliance.

When AI is used for decision support or to automate processes, it becomes important to understand how results are generated and what limitations the technology has.

AI systems often process large volumes of data. Without clear guidelines, sensitive information can be shared with external services or used in ways that are not in line with the organisation’s requirements.

EU AI Act introduces new requirements for organisations that develop, supply or use AI systems. Although the requirements vary according to risk and use case, the direction is clear: AI must be governed and documented.

More organisations are already beginning to ask questions about how their suppliers use artificial intelligence. Just as information security and data protection have become part of procurement processes, AI governance will become an increasingly important topic.

AI can create significant value, but uncontrolled use can also lead to problems.

Employees may inadvertently share confidential information with AI services without being aware of how the data is processed.

AI can generate content that appears correct but contains errors or fabricated information. If used uncritically, this can lead to poor decisions or reduced quality.

In many organisations, there is no overview of which AI tools are in use, who is using them or what tasks they are being used for.

AI models can produce outputs influenced by biases in training data or algorithms. This can have consequences for, amongst other things, recruitment, customer handling and decision-making processes.

Without clear rules, different departments may develop their own practices for using AI. This makes it difficult to ensure consistent governance and control.

An AI governance programme does not need to be complex, but it should cover a number of key areas.

The organisation must define who is responsible for AI-related decisions, follow-up and risk management.

Employees need clear frameworks for what AI can be used for, which tools are approved and how information should be handled.

The use of AI should be assessed in the same way as other technology solutions. Risks relating to security, data protection, quality and compliance must be identified and managed.

Employees must understand both the opportunities and the limitations of AI. Many of the risks are linked to a lack of competence and incorrect use.

The organisation should establish mechanisms for following up on how AI is used and ensuring that guidelines are adhered to.

It should be possible to document which AI solutions are in use, for what purposes and what assessments have been made.

For organisations using external AI solutions, it is important to assess suppliers in the same way as other critical system suppliers.

ISO 42001 is the world’s first international standard for artificial intelligence management systems.

The standard describes how organisations can establish, maintain and continually improve an AI management system.

Many of the elements that make up AI governance can be found in ISO 42001, including:

• roles and responsibilities
• risk management
• management of AI systems
• documentation
• monitoring and improvement
• competence and training

For organisations seeking a structured and recognised approach to AI governance, ISO 42001 provides a concrete framework to build on.

Many organisations already have a solid starting point.

If you have established management systems based on ISO 9001, ISO 27001 or similar standards, there are often processes and ways of working that can be extended to cover AI.

A good first step is to map:

• which AI tools are currently in use
• which processes are affected by AI
• which risks are relevant to the organisation
• which guidelines are missing

The organisation can then establish the necessary roles, guidelines and control mechanisms as part of the existing management system.

As a technology company with experience in both management systems and the practical use of AI, we often find that the challenge is not solely about documentation. Equally important is understanding how AI is actually being used within the organisation, which processes are affected, and how the technology can be governed in a way that creates value without introducing unnecessary risk.

Our consultants have extensive experience in establishing and improving management systems based on, amongst others, ISO 9001 and ISO 27001. At the same time, we work in a technology company where AI is used actively in development, operations and internal work processes. This combination enables us to connect the requirements of the standards with the practical reality organisations face.

We help organisations with advisory services, establishment and further development of AI management systems. Get in touch to find out how we can help you!