Publications

Research in testing and dependable software.

Selected journal articles, conference papers, books, tutorials, and talks covering cyber-physical systems, software testing, and trustworthy AI.

2026

Irshad, A., Valle, P., Ayerdi, J., & Arrieta, A. (2026). “Hype Meets Reality: Large Language Models as Mutators in Search-based Automated Program Repair of Simulink-Stateflow Models.” 37th IEEE International Symposium on Software Reliability Engineering (ISSRE), Replication and Negative Results (RENE) Track (accepted). [Scholar] [Preprint]
Arrieta, A. (2026). “An Empirical Evaluation of White-box and Black-box Test Case Prioritization Techniques in CPSs Modeled in Simulink.” Empirical Software Engineering, 31(5), Article 141. [Scholar] [Article] [PDF]
Saurabh, P., Valle, P., Arrieta, A., Ali, S., & Arcaini, P. (2026). “VISOR: A Vision-Language Model-based Test Oracle for Testing Robots.” Proceedings of the 3rd ACM International Conference on AI-Powered Software (AIware), Main Track. [Scholar] [Article]
Yousaf, M., Arrieta, A., Ali, S., Arcaini, P., & Wang, S. (2026). “Assessing Vision–Language Models for Perception in Autonomous Underwater Robotic Software.” 19th IEEE International Conference on Software Testing, Verification and Validation (ICST), Industry Track, pp. 64–75. [Scholar] [Preprint]
Valle, P., Ali, S., & Arrieta, A. (2026). “Exploring the Potential of Large Language Models in Simulink-Stateflow Mutant Generation.” Proceedings of the 2026 IEEE/ACM Third International Conference on AI Foundation Models and Software Engineering (FORGE), pp. 99–109. [Scholar] [Preprint]
Valle, P., Segura, S., Ali, S., & Arrieta, A. (2026). “Metamorphic Testing of Vision-Language Action-Enabled Robots.” 19th IEEE International Conference on Software Testing, Verification and Validation (ICST), Research Track (accepted). [PDF]
Panichella, S., Arcaini, P., Cohen, M. B., & Arrieta, A. (Eds.). (2026). Roadmap for DevOps in Cyber-Physical Systems: Challenges and Future Directions. Springer Singapore. [Book]
Romero-Arjona, M., Parejo, J. A., Alonso, J. C., Sánchez, A. B., Arrieta, A., & Segura, S. (2025). “Meta-Fair: AI-Assisted Fairness Testing of Large Language Models.” Information and Software Technology (accepted).
Wu, J., Lu, C., Arrieta, A., Ali, S., & Peyrucain, T. (2026). “Vision Language Model-based Testing of Industrial Autonomous Mobile Robots.” 48th IEEE/ACM International Conference on Software Engineering (ICSE), Software Engineering in Practice Track (accepted).
Arrieta, A., Valle, P., & Ali, S. (2026). “FlowRepair: Search-based Automated Program Repair of CPS Controllers Modeled in Simulink-Stateflow.” Information and Software Technology, 108010. [Article]

2025

Valle, P., Riccio, V., Arrieta, A., Tonella, P., Arratibel, M. “An Industrial Experience Report on Applying Search-based Boundary Input Generation to Cyber-Physical Systems”, In Empirical Software Engineering Journal (EMSE) 30(4): 112 (2025)
Ugarte, M., Valle, P., Illarramendi, M., Arrieta, A. “Enhancing multi-objective test case selectionthrough the mutation operator”, In Automated Software Engineering 32 (1), 18
Valle, P., Arrieta, A., Han, L., Ali, S., Yue, T. “Defining and generating multi-level and uncertainty-wise test oracles for cyber-physical systems”, In Software and Systems Modelinsg (SoSym) 1-16, 2024 [PDF]
Ugarte, M., Valle, P., Parejo, J.A., Segura, S., Arrieta, A. "Astral: Automated safety testing of large language models" in the 6th ACM/IEEE International Conference on Automation of Software Test (AST 2025)
Nylænder, K., Arrieta, A., Ali, S., & Arcaini, P. (2025, July). Search-based Generation of Waypoints for Triggering Self-Adaptations in Maritime Autonomous Vessels. In Proceedings of the Genetic and Evolutionary Computation Conference (pp. 1453-1461).
Miguel Romero-Arjona, Pablo Valle, Juan C Alonso, Ana B Sánchez, Miriam Ugarte, Antonia Cazalilla, Vicente Cambrón, José A Parejo, Aitor Arrieta, Sergio Segura (2025). Red teaming contemporary ai models: Insights from spanish and basque perspectives. Sistedes 2025

2024

Ayerdi, J., Terragni, V., Jahangirova, G., Arrieta, A., Tonella, P. “GenMorph: Automatically Generating Metamorphic Relations via Genetic Programming”, In IEEE Transactions on Software Engineering, 2024 (Accepted for publication) [PDF]
Ayerdi, J., Iriarte, A., Valle, P., Roman, I., Illarramendi, M., Arrieta, A. “MarMot: Metarmorphic Runtime Monitoring of Autonomous Driving Systems”, In ACM Transactions on Software Engineering and Methodology, 2024 (Accepted for publication) [PDF]
Aldalur, I., Arrieta, A., Agirre, A., Sagardui, G., & Arratibel, M. (2024). A microservice-based framework for multi-level testing of cyber-physical systems. Software Quality Journal, 32(1), 193-223.
Wu, J., Lu, C., Arrieta, A., Yue, T., & Ali, S. (2024). Reality Bites: Assessing the Realism of Driving Scenarios with Large Language Models. In First Special Event on AI Foundation Models and Software Engineering [PDF]
Wang, X., Ali, S., Arrieta, A., Arcaini, P., & Arratibel, M. (2024). Application of Quantum Extreme Learning Machines for QoS Prediction of Elevators' Software in an Industrial Context. In Proceedings of the 32nd ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering [PDF]

2023

Arrieta, A., Valle, P., Agirre, J.A., & Sagardui, G. “Some Seeds are Strong: Seeding Strategies for Search-based Test Case Selection”. In ACM Transactions on Software Engineering and Methodology (TOSEM), 2022, 32 (1), 1-47.
Han, L., Ali, S., Yue, T., Arrieta, A., Arratibel, M. “Uncertainty-aware Robustness Assessment of Industrial Elevator Systems”. In ACM Transactions on Software Engineering and Methodology (TOSEM), 2023, 32(4), 1-51.
Ayerdi, J., Valle, P., Segura, S., Arrieta, A., Sagardui, G. & Arratibel, M. “Performance-Driven Metamorphic Testing of Cyber-Physical Systems”. In IEEE Transactions on Reliability, 2023, 72(2): 827-845.
Michelton, G.K., Martinez, J., Sotto-Mayor, B., Arrieta, A., Assuncao, W.K.G., Abreu, R., Egyed, A. “Spectrum-based feature localization for families of systems”, Journal of Systems and Software, 2023, 195, 111532. [PDF]
Valle, P., Arrieta, A., Arratibel, M. “Automated Misconfiguration Repair of Configurable Cyber-Physical Systems with Search: an Industrial Case Study on Elevator Dispatching Algorithms”. In Proceedings of the 45th International Conference on Software Engineering (ICSE), Software Engineering In Practice (SEIP) Track, 2023, (pp. 396-408) [PDF]
Valle, P., Arrieta, A., Arratibel, M. “Applying and Extending the Delta Debugging Algorithm for Elevator Dispatching Algorithms (Experience Paper)”. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (pp. 1055–1067) [PDF]
Ugarte, M., Jee, E., Liu, L., Valle, P., Arrieta, A., Illarramendi, M. “Search-based Test Case Selection for PLC Systems using Functional Block Diagram Programs”. In 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE) (Accepted for publication). IEEE.
Arrieta, A., Illarramendi, M. “A Novel Mutation Operator for Search-based Test Case Selection”. In International Symposium on Search Based Software Engineering (pp. 84-98). Cham: Springer Nature Switzerland.
Arrieta, A., & Illarramendi, M. (2023, December). A Novel Mutation Operator for Search-Based Test Case Selection. In International Symposium on Search Based Software Engineering (pp. 84-98). Cham: Springer Nature Switzerland. Best Paper Award

2022

Garciandia, A., Arrieta, A., Ayerdi, A., Illarramendi, M., Agirre, A., Sagardui, G. & Arratibe, M. “Machine Learning-based Test Oracles for Performance Testing of Cyber-Physical Systems: an Industrial Case Study on Elevators Dispatching Algorithms”. In Journal of Software: Evolution and Process, 2022, (Accepted for publication).
Han, L., Yue, T., Ali, S., Arrieta, A., Arratibel, M. “Are Elevator Software Robust Against Uncertainties? Results and Experiences from an Industrial Case Study”. In Proceedings of the 30th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (pp. 1331-1342).
Ayerdi, J., Arrieta, A., Pobee, E., & Arratibel, M. “Multi-Objective Metamorphic Test Case Selection: an Industrial Case Study”. In 2022 IEEE 33rd International Symposium on Software Reliability Engineering (ISSRE) (pp. 541-552). IEEE.
Arrieta, A. “On the Cost-Effectiveness of Composite Metamorphic Relations for Testing Deep Learning Systems”, at Metamorphic Testing Workshop (MET2022), 2022, pp. 42-47.
Arrieta, A. “Is the Revisited Hypervolume an Appropriate Quality Indicator to Evaluate Multi-Objective Test Case Selection Algorithms?”, at Genetic and Evolutionary Computation Conference (GECCO 2022), 2022, (pp. 1317–1326).
Arrieta, A. “Multi-Objective Metamorphic Follow-up Test Case Selection for Deep Learning Systems”, at Genetic and Evolutionary Computation Conference (GECCO 2022), 2022, (1327–1335).
Arrieta, A., Otaegi, M., Han, L., Sagardui, G., Ali, S., Arratibel, M. “Automating Test Oracle Generation in DevOps for Industrial Elevators”. In 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) (pp. 284-288). IEEE.
Valle, P., Arrieta, A. “Towards the Isolation of Failure-Inducing Inputs in Cyber-Physical Systems: is Delta Debugging Enough?”. In 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) (pp. 549-553). IEEE.

2021

Markiegi, U., Arrieta, A., Etxeberria, L., & Sagardui, G. “Dynamic test prioritization of product lines: An application on configurable simulation models”. In Software Quality Journal (2021), 29(4), 943-988.
Galarraga, J., Arrieta, A., Ali, S., Sagardui, G., Arratibel, M. “Genetic Algorithm-based Testing of Industrial Elevators under Passenger Uncertainty”. In 2021 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW) (pp. 353-358). IEEE.
Ayerdi, J., Terragni, V., Arrieta, A., Tonella, P., Sagardui, G., Arratibel, M. “Generating metamorphic relations for cyber-physical systems with genetic programming: an industrial case study”. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (pp. 1264-1274).
Michelton, G.K., Sotto-Mayor, B., Martinez, J., Arrieta, A., Abreu, R., Assuncao, W.K.G. “Spectrum-Based Feature Localization: A Case Study using ArgoUML”. In 2021 Software Product Line Conference (SPLC) – Challenge track.
Arrieta, A., Ayerdi, J., Illarramendi, I., Agirre, A., Sagardui, G. & Arratibe, M. “Using Machine Learning to Build Test Oracles: an Industrial Case Study of Elevators Dispatching Algorithms”. In 2021 2nd ACM/IEEE International Conference on Automation of Software Test (AST), 2021.
Gartziandia, A., Ayerdi, J., Arrieta, A., Ali, S., Yue, T., Agirre, A., Sagardui, G. & Arratibel, M. “Microservices for Continuous Deployment, Monitoring and Validation in Cyber-Physical Systems: an Industrial Case Study for Elevators Systems”. In 2021 IEEE 18th International Conference on Software Architecture Companion (ICSA-C) (pp. 46-53). IEEE.
Garciandia, A., Arrieta, A., Agirre, A., Sagardui, G. & Arratibe, M. “Using Regression Learners to Predict Performance Problems on Software Updates: a Case Study on Elevators Dispatching Algorithms”. In 2021 ACM/SIGAPP Symposium on Applied Computing (SAC), 2021.
Zafar, M. N., Afzal, W., Enoiu, E., Stratis, A., Arrieta, A., & Sagardui, G. “Model-Based Testing in Practice: An Industrial Case Study using GraphWalker”. In 14th Innovations in Software Engineering Conference (formerly known as India Software Engineering Conference), 2021 ACM.
Arrieta, A., Bellido, G., Da Silva, J., Markiegi, U., & Gago, J. (2021). De la simulación a la realidad: metodología para el aprendizaje de sistemas ciber-físicos. Actas de las Jenui, 6, 283-286.

2020

Arrieta, A., Agirre, J. A., & Sagardui, G. “Seeding strategies for multi-objective test case selection: an application on simulation-based testing”. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference (pp. 1222-1231). Best SBSE PAPER and Nominee to conference best paper award
Ayerdi, J., Segura, S., Arrieta, A., Sagardui, G., & Arratibel, M. “QoS-aware Metamorphic Testing: An Elevation Case Study”. In 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) (pp. 104-114). IEEE.
Ayerdi, J., Garciandia, A., Arrieta, A., Afzal, W., Enoiu, E., Agirre, A., Sagardui, G., Arratibel, M. & Sellin, O. “Towards a taxonomy for eliciting design-operation continuum requirements of cyber-physical systems”. In 2020 IEEE 28th International Requirements Engineering Conference (RE), 2020, (pp. 280-290). IEEE.
Arrieta, A., Agirre, J. A., & Sagardui, G. “A Tool for the Automatic Generation of Test Cases and Oracles for Simulation Models Based on Functional Requirements”. In 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), 2020, (pp. 1-5). IEEE.

2019

A. Arrieta, S. Wang, U. Markiegi, A. Arruabarrena, G. Sagardui and L. Etxeberria. “Pareto Efficient Multi-Objective Black-Box Test Case Selection for Simulation-Based Testing” in Information and Software Technology, 2019, 114, 137-154.
A. Arrieta, S. Wang, G. Sagardui and L. Etxeberria. “Search-Based Test Case Prioritization for Simulation-Based Testing of Cyber-Physical System Product Lines” in Journal of Systems and Software, 2019, 149, 1-34.
Markiegi, U., Arrieta, A., Etxeberria, L., & Sagardui, G. “Test case selection using structural coverage in software product lines for time-budget constrained scenarios”. In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing (pp. 2362-2371), 2019. ACM.
Markiegi, U., Arrieta, A., Etxeberria, L., & Sagardui, G. (2019, September). “White-box and black-box test quality metrics for configurable simulation models”. In Proceedings of the 23rd International Systems and Software Product Line Conference-Volume B (p. 92). ACM.

2018

A. Arrieta, S. Segura, U. Markiegi, G. Sagardui and L. Etxeberria. “Spectrum-Based Fault Localization in Software Product Lines” in Information and Software Technology, 2018, 100, 18-31. (DOI)
A. Arrieta, S. Wang, U. Markiegi, G. Sagardui and L. Etxeberria. “Employing Multi-Objective Search to Enhance Reactive Test Case Generation and Prioritization for Testing Industrial Cyber-Physical Systems” in IEEE Transactions on Industrial Informatics, 2018, 14(3), 1055-1066.
A. Arrieta, S. Wang, A. Arruebarrena, U. Markiegi, G. Sagardui, and L. Etxeberria. “Multi-Objective Black-Box Test Case Selection for Cost-Effectively Testing Simulation Models” in GECCO2018: Genetic and Evolutionary Computation Conference, 2018, 1411-1418.

2017

A. Arrieta, G. Sagardui, L. Etxeberria, and J. Zander. “Automatic Generation of Test System Instances for Configurable Cyber-Physical Systems” in Software Quality Journal, 2017, 25(3), 1041-1083.
G. Sagardui, L. Etxeberria, J. Agirre, A. Arrieta, C.F. Nicolás, and J.M. Martín. “A Configurable Validation Environment for Refactored Embedded Software: an Application to the Vertical Transport Domain” in ISSRE 2017 (Industry Track): IEEE International Symposium on Software Reliability Engineering, 2017, 16-19.
L. Etxeberria, F. Larrinaga, U. Markiegi, A. Arrieta, G. Sagardui. “Enabling Co-Simulation of Smart Energy Control Systems for Buildings and Districts” in ETFA2017: IEEE 22nd Conference on Emerging Technologies and Factory Automation, 2017, 1-4.
U. Markiegi, A. Arrieta, G. Sagardui, and L. Etxeberria. “Search-based product line fault detection allocating test cases iteratively” in SPLC2017: Software Product Line Conference, 2017, 123-132.
X. Perez, O. Berreteaga, L. Etxeberria, A. Arrieta, and U. Markiegi. “Modeling Systems Variability with Delta Rhapsody” in JISBD2017: XXII Jornadas de Ingeniería del Software y Bases de Datos, 2017.
A. Arrieta, U. Markiegi, and L. Etxeberria. “Towards Mutation Testing of Configurable Simulink Models: a Product Line Engineering Perspective” in JISBD2017: XXII Jornadas de Ingeniería del Software y Bases de Datos, 2017.
A. Arrieta, S. Wang, U. Markiegi, G. Sagardui, and L. Etxeberria. “Search-Based Test Case Generation for Cyber-Physical Systems” in CEC2017: IEEE Congress on Evolutionary Computation, 2017, pp. 688-697.
G. Sagardui, J. Agirre, U. Markiegi, A. Arrieta, C.F. Nicolás, and J.M. Martín. “Multiplex: A Co-Simulation Architecture for Elevators Validation” in ECMSM 2017: IEEE International Workshop of Electronics, Control, Measurement, Signals and their application to Mechatronics, 2017, 1-6.

2016

A. Arrieta, S. Wang, G. Sagardui, and L. Etxeberria. “Search-Based Test Case Selection of Cyber-Physical System Product Lines for Simulation-Based Validation” in SPLC2016: Software Product Line Conference, 2016, pp. 297-306.
A. Arrieta, S. Wang, G. Sagardui, and L. Etxeberria. “Test Case Prioritization of Configurable Cyber-Physical Systems with Weight-Based Search Algorithms” in GECCO2016: Genetic and Evolutionary Computation Conference, 2016, pp. 1053-1060.

2015

A. Arrieta, G. Sagardui, and L. Etxeberria. “Variability in Test Systems: Review and Challenges” in VALID 2015: The Seventh International Conference on Advances in System Testing and Validation Lifecycle, 2015, pp. 15-22.
A. Arrieta, G. Sagardui, and L. Etxeberria. “Cyber-physical systems product lines: Variability analysis and challenges,” in VI Jornadas de Computación Empotrada, 2015.
A. Arrieta, G. Sagardui, and L. Etxeberria. “Test control algorithms for the validation of cyber-physical systems product lines,” in SPLC 2015: Software Product Line Conference, 2015, pp. 273-282.

2014

A. Arrieta, G. Sagardui, and L. Etxeberria. “A Configurable Test Architecture for the Automatic Validation of Variability-Intensive Cyber-Physical Systems” in VALID 2014: The Sixth International Conference on Advances in System Testing and Validation Lifecycle, 2014, pp. 79-83.
A. Arrieta, G. Sagardui, and L. Etxeberria. “A Model-Based Testing Methodology for the Systematic Validation of Highly Configurable Cyber-Physical Systems” in VALID 2014: The Sixth International Conference on Advances in System Testing and Validation Lifecycle, 2014, pp. 66-72.
A. Arrieta, G. Sagardui, and L. Etxeberria. “A Comparative on Variability Modelling and Management Approaches in Simulink for Embedded Systems” in V Jornadas de Computación Empotrada, 2014, pp. 26-33.
A. Arrieta, G. Sagardui, and L. Etxeberria. “Towards the Automatic Generation and Management of Plant Models for the Validation of Highly Configurable Cyber-Physical Systems” in ETFA2014.

2013

A. Arrieta, I. Agirre, and A. Alberdi. “Testing Architecture with Variability Management in Embedded Distributed Systems” in IV Jornadas de Computación Empotrada, 2013, pp. 12-19.

Tutorials and Talks

Arrieta, A. “Variability modeling and management of MATLAB/Simulink models”. In Proceedings of the 23rd International Systems and Software Product Line Conference-Volume A (p. 54). ACM.
A. Arrieta, G. Sagardui, and L. Etxeberria. “Efficient virtual testing of variability-intensive automotive cyber-physical systems” in 9th Graz Symposium Virtuellez Fahrzeug, 2016.
A. Arrieta “Test Optimization of Configurable Cyber-physical Systems with Search Algorithms” in First International Summer School on Search-Based Software Engineering, 2016.