Assessing and Enhancing the Trustworthiness of Autonomous AI Code Agents
Project Title: Assessing and Enhancing the Trustworthiness of Autonomous AI Code Agents
Background
The software development landscape is experiencing a significant shift from traditional autocomplete tools to advanced AI coding agents. Unlike earlier models, these conversational agents possess a higher degree of autonomy, capable of executing complex tasks, interacting with environments, and modifying codebases. While this autonomy boosts productivity, it introduces a critical "Trustworthiness Gap". As these agents take on more independent roles, ensuring they operate safely, reliably, and securely within real-world software development workflows becomes a pressing challenge.
Project Aim
This project aims to systematically evaluate and enhance the trustworthiness of autonomous AI coding agents. By analyzing the behavior of these agents under various software engineering scenarios, the overarching goal is to understand their limitations and develop generalized guardrails that ensure safe and robust human-AI collaborative coding.
Research Objectives
- Behavioral Safety: To evaluate the behavior of AI coding agents when interacting with complex software environments and to mitigate the risks of executing unsafe or unintended actions.
- Capability Reliability: To investigate the consistency and accuracy of autonomous agents during multi-step coding tasks, with a focus on identifying and reducing agent-specific errors (e.g., capability limitations or hallucinations) within the agentic loop.
- Code Security: To assess the security implications of code generated autonomously and explore proactive strategies to prevent vulnerabilities from being introduced into the software lifecycle.
Project Lead
Project Team
Prof John Grundy, Dr Xiaoning Du, Dr Kla Tantithamthavorn, Dr Tingting Bi