Congratulations to Ben Smith of Realsearch, who passed his oral preliminary examination!
Title: “A Pattern Catalog to Guide Black Box Testers with Security Testing”
Date: September 12, 2011
Place: EBII, Room 3211
Dr. Laurie Williams (Chair& Advisor)
Dr. Annie Antón
Dr. Ting Yu
Dr. Mladen Vouk
Dr. Jacqueline Halladay
The United States is suffering from a shortage of software security experts. The software development community needs a vehicle for knowledge transfer with which security experts can proliferate their insights. We have adapted the notion of a software design pattern to the domain of black box security testing. Expressing proven security testing techniques as patterns makes them more accessible to people who are not experts in security. Patterns also make it easier to reuse successful testing strategies in different systems, and in different areas of the same system. The goal of this research is to help security experts proliferate their knowledge by introducing and evaluating a software security test pattern catalog. In this paper, we present the first six test patterns based on tests that target a list of common vulnerabilities. These patterns contain keywords, that when found in software system’s natural language artifacts, guide testers toward the appropriate pattern. We created test cases from these patterns using 284 functional requirements from a public specification to generate 137 black box tests. We then executed these tests on five electronic health record systems (685 test results in total), which are currently used to manage the clinical records for approximately 59 million patients, collectively. Thirty-seven percent (37%) of these tests revealed 253 vulnerabilities in the five systems. Our evaluation shows that our patterns target different vulnerabilities than automated techniques like automated penetration testing and static analysis. Our preliminary pattern catalog provides a foundation for disseminating security expertise that can be expanded with contributions from the community.