Archive for the ‘Testing and Reliability’ Category

Modifying Without a Trace: General Audit Guidelines are Inadequate for Electronic Health Record Audit Mechanisms

September 1st, 2011 No comments

J. King, B. Smith, L. Williams, “Modifying Without a Trace: General Audit Guidelines are Inadequate for Electronic Health Record Audit Mechanisms”, Proceedings of the International Health Informatics Symposium (IHI 2012), to appear, 2012.


Without adequate audit mechanisms, electronic health record (EHR) systems remain vulnerable to undetected misuse. Users could modify or delete protected health information without these actions being traceable. The objective of this paper is to assess electronic health record audit mechanisms to determine the current degree of auditing for non-repudiation and to assess whether general audit guidelines adequately address non-repudiation. We derived 16 general auditable event types that affect non-repudiation based upon four publications. We qualitatively assess three open-source EHR systems to determine if the systems log these 16 event types. We find that the systems log an average of 12.5% of these event types. We also generated 58 black-box test cases based on specific auditable events derived from Certification Commission for Health Information Technology criteria. We find that only 4.02% of these tests pass. Additionally, 20% of tests fail in all three EHR systems. As a result, actions including the modification of patient demographics and assignment of user privileges can be executed without a trace of the user performing the action. The ambiguous nature of general auditable events may explain the inadequacy of auditing for non-repudiation. EHR system developers should focus on specific auditable events for managing protected health information instead of general events derived from guidelines.

Systematizing Security Test Case Planning Using Functional Requirements Phrases

April 22nd, 2011 No comments

B. Smith, “Systematizing Security Test Case Planning Using Functional Requirements Phrases“, Proceedings of the International Conference on Software Engineering Doctoral Symposium (ICSE Doctoral Symposium), Honolulu, Hawaii, pp. 1136-1137, 2011.


Security experts use their knowledge to attempt attacks on an application in an exploratory and opportunistic way in a process known as penetration testing. However, building security into a product is the responsibility of the whole team, not just the security experts who are often only involved in the final phases of testing.  Through the development of a black box security test plan, software testers who are not necessarily security experts can work proactively with the developers early in the software development lifecycle.  The team can then establish how security will be evaluated such that the product can be designed and implemented with security in mind.  The  goal of this research is to improve the security of applications by introducing a methodology that uses the software system’s requirements specification statements to systematically generate a set of black box security tests.  We used our methodology on a public requirements specification to create 137 tests and executed these tests on five electronic  health record systems. The tests revealed 253 successful attacks on these five systems, which are used to manage the clinical records for approximately 59 million patients, collectively. If non-expert testers can surface the more common vulnerabilities present in an application, security experts can attempt more devious, novel attacks.

Using SQL Hotspots in a Prioritization Heuristic for Detecting All Types of Web Application Vulnerabilities

April 22nd, 2011 2 comments

B. Smith, L. Williams, “Using SQL Hotspots in a Prioritization Heuristic for Detecting All Types of Web Application Vulnerabilities“, Proceedings of the International Conference on Software Testing, Verification and Validation (ICST 2011), Berlin, Germany, pp. 220-229, 2011.



Development organizations often do not have time to perform security fortification on every file in a product before release. One way of prioritizing security efforts is to use metrics to identify core business logic that could contain vulnerabilities, such as database interaction code. Database code is a source of SQL injection vulnerabilities, but importantly may be home to unrelated vulnerabilities. The goal of this research is to improve the prioritization of security fortification efforts by investigating the ability of SQL hotspots to be used as the basis for a heuristic for prediction of all vulnerability types. We performed empirical case studies of 15 releases of two open source PHP web applications: WordPress, a blogging application, and WikkaWiki, a wiki management engine. Using statistical analysis, we show that the more SQL hotspots a file contains per line of code, the higher the probability that file will contain any type of

Challenges for Protecting the Privacy of Health Information: Required Certification Can Leave Common Vulnerabilities Undetected

August 17th, 2010 No comments

B. Smith, A. Austin, M. Brown, J. King, J. Lankford, A. Meneely, L. Williams, “Challenges for Protecting the Privacy of Health Information: Required Certification Can Leave Common Vulnerabilities Undetected“, Proceedings of the Security and Privacy in Medical and Home-care Systems (SPIMACS 2010) Workshop, co-located with CCS, Chicago, IL, pp. 1-12, 2010.


The use of electronic health record (EHR) systems by medical professionals enables the electronic exchange of patient data, yielding cost and quality of care benefits. The United States American Recovery and Reinvestment Act (ARRA) of 2009 provides up to $34 billion for meaningful use of certified EHR systems. But, will these certified EHR systems provide the infrastructure for secure patient data exchange? As a window into the ability of current and emerging certification criteria to expose security vulnerabilities, we performed exploratory security analysis on a proprietary and an open source EHR. We were able to exploit a range of common code-level and design level vulnerabilities. These common vulnerabilities would have remained undetected by the 2011 security certification test scripts from the Certification Commission for Health Information Technology, the most widely used certification process for EHR systems. The consequences of these exploits included, but were not limited to: exposing all users’ login information, the ability of any user to view or edit health records for any patient, and creating a denial of service for all users. Based upon our results, we suggest that an enhanced set of security test scripts be used as entry criteria to the EHR certification process. Before certification bodies spend the time to certify that an EHR application is functionally complete, they should have confidence that the software system meets a basic level of security competence.

Does Hardware Configuration and Processor Load Impact Software Fault Observability?

January 24th, 2010 No comments

R.A. Syed, B. Robinson, L. Williams, “Does Hardware Configuration and Processor Load Impact Software Fault Observability?,” Proceedings of Third International Conference on Software Testing, Verification and Validation (ICST 2010), To Appear.


Intermittent failures and nondeterministic behavior complicate and compromise the effectiveness of software testing and debugging. To increase the observability of software faults, we explore the effect hardware configurations and processor load have on intermittent failures and the nondeterministic behavior of software systems. We conducted a case study on Mozilla Firefox with a selected set of reported field failures. We replicated the conditions that caused the reported failures ten times on each of nine hardware configurations by varying processor speed, memory, hard drive capacity, and processor load. Using several observability tools, we found that hardware configurations that had less processor speed and memory observed more failures than others. Our results also show that by manipulating processor load, we can influence the observability of some faults.

Idea: Using System Level Testing for Revealing SQL Injection-Related Error Message Information Leaks

January 10th, 2010 No comments

B. Smith, L. Williams, A. Austin, “Idea: Using System Level Testing for Revealing SQL Injection-Related Error Message Information Leaks“, Lecture Notes in Computer Science, vol. 5965, Engineering Secure Software and Systems (ESSoS 2010), pp. 192-200, 2010.


Completely handling SQL injection consists of two activities: properly protecting the system from malicious input, and preventing any resultant error messages caused by SQL injection from revealing sensitive information. The goal of this research is to assess the relative effectiveness of unit and system level testing of web applications to reveal both error message information leak and SQL injection vulnerabilities. To produce 100% test coverage of 176 SQL statements in four open source web applications, we augmented the original automated unit test cases with our own system level tests that use both normal input and 132 forms of malicious input. Although we discovered no SQL injection vulnerabilities, we exposed 17 error message information leak vulnerabilities associated with SQL statements using system level testing. Our results suggest that security testers who use an iterative, testdriven development process should compose system level rather than unit level tests.

Using Grouping of Static Analysis Alerts to Identify Files Likely to Contain Field Failures

December 1st, 2009 No comments

Sherriff, M., Heckman, S., Lake, M., Williams, L., Using Grouping of Static Analysis Alerts to Identify Files Likely to Contain Field Failures, ACM SIGSOFT Foundations of Software Engineering Short Paper, to appear.

Should software testers use mutation analysis to augment a test set?

August 11th, 2009 No comments

BB.H. Smith and L. Williams, Should software testers use mutation analysis to augment a test set? Journal of Systems Software, vol. 82, no. 11, pp. 1819-1832, 2009.

Mutation testing has historically been used to assess the fault-finding effectiveness of a test suite or other verification technique. Mutation analysis, rather, entails augmenting a test suite to detect all killable mutants. Concerns about the time efficiency of mutation analysis may prohibit its widespread, practical use. The goal of our research is to assess the effectiveness of the mutation analysis process when used by software testers to augment a test suite to obtain higher statement coverage scores. We conducted two empirical studies and have shown that mutation analysis can be used by software testers to effectively produce new test cases and to improve statement coverage scores in a feasible amount of time. Additionally, we find that our user study participants view mutation analysis as an effective but relatively expensive technique for writing new test cases. Finally, we have shown that the choice of mutation tool and operator set can play an important role in determining how efficient mutation analysis is for producing new test cases.

Personal Statement Writing

August 11th, 2009 No comments

Personal statement writing is a difficult task for many professionals and students because it involves writing about you. Personal statement writing involves presenting to the reader information about one’s abilities, weaknesses, achievements and even aspirations in life. There are many types of personal statement writing which cover application letter, personal philosophy, graduate application letters or scholarship application among others. Personal statement writing gives the student of even a non student the opportunity to market oneself. There are two main types of application processes which are part of personal statement writing. First, there is the comprehensive or general personal statement which gives the applicant the freedom of the content to be included in the personal statement an example of this are the application forms for law schools and for medical schools. As second category of application processes involves responding to a specific asked question this is common when a student is applying to a graduate or business school and the personal statement writing should respond to the specified question alone.

This is the online writing company which provides students and professionals to seek assistance and support they need in personal statement writing. This is the online writing company which has a team of highly qualified and creative writers to help you in your personal statement writing. The eligible writers know that your personal statement is the gate way for you top get into that profession or organization you dream of or that school you want to be associated with., personal statement writing from our writers involves using the factual information you have given us and stitching the information in creative manner which will capture the readers attraction. They do not just write for the sake of writing but they ensure that your personal statement stand out among the numerous applicants seeking the same job or school opportunity you also want.

As a legal and trustworthy online writing service we have a collection of samples from our personal statement writing team which have earned students their chances to their dream jobs and learning institutions. You can also look at the comments they have posted on our website concerning the kind of good and extraordinary work we do at our personal statement writing. Unlike other companies we ensure that authenticity of personal statement writing is achieved. We do not plagiarize or copy paste information from the internet. We instead use your specific instructions to write for you your unique personal statement to suit your specific needs.

Our personal statement writing service is very affordable to all people. We charge students lower cost than non students and what we focus on is the high quality service you expect from our company. We ensure that your personal statement writing is done in a keen manner with critical consideration on lexical choice, grammar rules and language style required to capture the interest of the reader. Be sure that from our personal statement writing we will never let you down. Even when you need revisions on aspects you will need to be rectified, do not hesitate to let us know. We do not charge for revision of your personal statement writing because we are a pocket friendly company. Those online writing services which require you to pay for plagiarism report and revisions f in their personal statement writing are not genuine companies.

Categories: Testing and Reliability Tags:

Does Calling Structure Information Improve the Accuracy of Fault Prediction?

April 28th, 2009 No comments

Yonghee Shin, Robert Bell, Thomas Ostrand, and Elaine Weyuker,
Does Calling Structure Information Improve the Accuracy of Fault Prediction?,
The 6th IEEE Working Conference on Mining Software Repositories (MSR 2009), co-located with ICSE 2009, May 16-17, 2009, Vancouver, Canada (To appear)