A Comparison of Nano-Patterns vs. Software Metrics in Vulnerability Prediction

Kazi Zakia Sultana, Byron J. Williams, Amiangshu Bosu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

Context: Software security is an imperative aspect of software quality. Early detection of vulnerable code during development can better ensure the security of the codebase and minimize testing efforts. Although traditional software metrics are used for early detection of vulnerabilities, they do not clearly address the granularity level of the issue to precisely pinpoint vulnerabilities. The goal of this study is to employ method-level traceable patterns (nano-patterns) in vulnerability prediction and empirically compare their performance with traditional software metrics. The concept of nano-patterns is similar to design patterns, but these constructs can be automatically recognized and extracted from source code. If nano-patterns can better predict vulnerable methods compared to software metrics, they can be used in developing vulnerability prediction models with better accuracy. Aims: This study explores the performance of method-level patterns in vulnerability prediction. We also compare them with method-level software metrics. Method: We studied vulnerabilities reported for two major releases of Apache Tomcat (6 and 7), Apache CXF, and two stand-alone Java web applications. We used three machine learning techniques to predict vulnerabilities using nano-patterns as features. We applied the same techniques using method-level software metrics as features and compared their performance with nano-patterns. Results: We found that nano-patterns show lower false negative rates for classifying vulnerable methods (for Tomcat 6, 21% vs 34.7%) and therefore, have higher recall in predicting vulnerable code than the software metrics used. On the other hand, software metrics show higher precision than nano-patterns (79.4% vs 76.6%). Conclusion: In summary, we suggest developers use nano-patterns as features for vulnerability prediction to augment existing approaches as these code constructs outperform standard metrics in terms of prediction recall.

Original languageEnglish
Title of host publicationProceedings - 25th Asia-Pacific Software Engineering Conference, APSEC 2018
PublisherIEEE Computer Society
Pages355-364
Number of pages10
ISBN (Electronic)9781728119700
DOIs
StatePublished - 2 Jul 2018
Event25th Asia-Pacific Software Engineering Conference, APSEC 2018 - Nara, Japan
Duration: 4 Dec 20187 Dec 2018

Publication series

NameProceedings - Asia-Pacific Software Engineering Conference, APSEC
Volume2018-December
ISSN (Print)1530-1362

Conference

Conference25th Asia-Pacific Software Engineering Conference, APSEC 2018
CountryJapan
CityNara
Period4/12/187/12/18

Keywords

  • Vulnerability
  • nano-pattern
  • software metrics
  • software security

Fingerprint Dive into the research topics of 'A Comparison of Nano-Patterns vs. Software Metrics in Vulnerability Prediction'. Together they form a unique fingerprint.

  • Cite this

    Sultana, K. Z., Williams, B. J., & Bosu, A. (2018). A Comparison of Nano-Patterns vs. Software Metrics in Vulnerability Prediction. In Proceedings - 25th Asia-Pacific Software Engineering Conference, APSEC 2018 (pp. 355-364). [8719530] (Proceedings - Asia-Pacific Software Engineering Conference, APSEC; Vol. 2018-December). IEEE Computer Society. https://doi.org/10.1109/APSEC.2018.00050