The following Great Innovative Idea is from Sebastian Elbaum, Professor of Computer Science and Engineering at the University of Nebraska-Lincoln and David S. Rosenblum, Dean of the School of Computing at the National University of Singapore.
Their paper Known Unknowns: Testing in the Presence of Uncertainty won second place at the Computing Community Consortium (CCC) sponsored Blue Sky Ideas Conference Track series at the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE), November 16-22, 2014 in Hong Kong.
The Innovative Idea
Uncertainty is present in most systems we build today, whether introduced by human decisions, machine learning algorithms, external libraries, or sensing variability. This uncertainty leads to occasional misbehavior or incorrect output that is deemed to be acceptable. In the context of software testing, this uncertainty makes it difficult to distinguish acceptable from unacceptable misbehaviors, and to determine when there are faults in the system that are being masked by acceptable misbehaviors. Existing approaches to deal with uncertainty in testing have been partial and of limited scope, leaving the systematic treatment of uncertainty in testing still open. In the paper, we explore the use of Hidden Markov Models and statistical reasoning about behaviors observed during testing in order to distinguish between acceptable misbehavior and behavioral errors that are due to latent faults.
Impact
The idea will allow us to deal better with uncertainty that arises in modern complex software systems in many different forms, such as imprecision in readings from hardware sensors or imprecision in machine learning-based classifiers, and correspondingly in a broad range of applications from human activity recognition to quadcopter stabilization.
Other Research
We both work in testing and analysis of complex software systems. Sebastian has interests in the development and assessment of automated techniques to improve software dependability. David has interests in the design, verification and testing of distributed systems and context-aware ubiquitous computing systems.
Researcher’s Background
Sebastian is a Professor of Computer Science and Engineering at the University of Nebraska – Lincoln (UNL). He is a co-founder of the E2 Software Engineering Group at UNL, and the Nimbus UAV Lab at UNL. He is the Program Co-Chair for the 2015 International Conference in Software Engineering, and a member of the ACM Transactions on Software Engineering and Methodology Editorial Board.
David is Professor of Computer Science and Dean of the School of Computing at the National University of Singapore (NUS). He previously held academic positions at University College London and the University of California, Irvine, and he also was a research scientist at AT&T Bell Laboratories (Murray Hill) and CTO at a technology startup called PreCache, Inc. He is the Editor-in-Chief of the ACM Transactions on Software Engineering and Methodology (ACM TOSEM) and is Past Chair of the ACM Special Interest Group on Software Engineering (ACM SIGSOFT).
Links
Sebastian’s homepage is at http://cse.unl.edu/~elbaum/.
David’s homepage is at http://www.comp.nus.edu.sg/~david/.
To view more Great Innovative Ideas, please click here.
