Resumen de Automated Test Data and Test Oracle Generation for REST APIs
Juan Carlos Alonso Valenzuela
Web Application Programming Interfaces (APIs) enable communication between heterogeneous software systems over the network. Most applications we use daily rely on one or more web APIs to deliver their functionality. These APIs—fundamental components powering services from companies like Google, Meta, and Microsoft—typically adopt the REpresentational State Transfer (REST) architectural style and are known as REST or RESTful APIs. By offering external access to their APIs, companies and independent developers allow third-party developers to integrate their services into their applications. This widespread openness and interoperability have made REST APIs a central pillar in the modern software ecosystem, driving seamless integration across platforms and services.
Given the critical role REST APIs play in software integration, thoroughly testing them is essential: a single fault in an API can compromise the functionality of all systems that depend on it. In recent years, the automated testing of REST APIs has emerged as a vibrant research area, with numerous approaches leveraging API specifications to automatically generate test cases. To support this need, many companies now provide Testing as a Service (TaaS) solutions, offering developers a practical way to test APIs without the overhead of managing dedicated testing infrastructure.
The main objective of this dissertation is to address two major limitations commonly found in existing automated testing approaches for REST APIs: the lack of mechanisms for generating realistic input values automatically, and the limited capability of their test oracles, which often focus solely on detecting server errors or validating response syntax. To overcome these challenges, this work introduces three novel contributions: ARTE, an approach that leverages the specification of an API and execution feedback to generate realistic input values; AGORA+, a technique that infers test oracles from previous API executions; and SATORI, a method that analyzes the specification of an API to generate test oracles, without requiring prior API execution. These contributions have been implemented as tools designed to integrate seamlessly with existing automated testing frameworks, thereby enhancing their ability to generate valid API requests and detect a broader range of failures.
We conducted extensive evaluations of our proposed approaches in real-world scenarios, focusing on industrial APIs with millions of active users. Our techniques uncovered over 40 previously undetected failures in widely used commercial platforms such as Amadeus Hotel, YouTube, Vimeo, Foursquare, GitHub, and GitLab. These findings have led to multiple bug fixes and improvements in API documentation, demonstrating the practical impact and effectiveness of our contributions.
