Context
The blog explains that agentic AI can bring major productivity benefits, such as automating complex workflows and freeing people for higher-value work, however organisations must manage the risks of autonomous systems behaving unexpectedly. It recommends using safeguards, sandboxing, and active oversight to reduce unintended or unauthorised activity, especially given recent incidents involving AI models and agentic systems.
Key points to note and next actions
- Advice and effective practice will continue to evolve as the technology matures and evidence bases are built.
- Firms need to understand built in safeguards and protections, consider additional safeguards and assess how much autonomy is needed.
- Be clear on the level of autonomy, and risk you are willing to tolerate, as this will inform the design of controls.
- Understanding safeguards available, their efficacy and limitations, will enable determination of additional controls that are needed. Irrespective of inbuilt controls, all deployments should be subject to robust observability, operational monitoring and response procedures.
- Identify what could go wrong – Before deployment, carry out threat modelling to identify the failure scenarios during the agent’s activities.
- Prompt carefully – instructions and context given to an agentic AI system and its underlying model will influence the activities it carries out.
- Set the right level of oversight – this should inform your risk tolerance and the potential consequences if the agent behaves unexpectedly.
- Control the AI agent’s environment with a robust sandbox.
- Observability: log, audit and monitor agentic AI activity as part of security operations.
- Make your AI activity easy to attribute – If your AI agent communicates with third-party systems, make it as easy as possible for those organisations to identify that the activity originates from you.
- Emergency shutdown: maintain the ability to ‘pull the plug’.
Further reading: The advice in this blog should form part of a wider approach to securing AI models and systems. Further information is available in: