Abstract
Inspections are a proven approach for improving software requirements quality. Owing to the fact that inspectors report both faults and non-faults (i.e., false-positives) in their inspection reports, a major chunk of work falls on the person who is responsible for consolidating the reports received from multiple inspectors. We aim at automation of fault-consolidation step by using supervised machine learning algorithms that can effectively isolate faults from non-faults. Three different inspection studies were conducted in controlled environments to obtain real inspection data from inspectors belonging to both industry and from academic backgrounds. Next, we devised a methodology to separate faults from non-faults by first using ten individual classifiers from five different classification families to categorize different fault-types (e.g., omission, incorrectness, and inconsistencies). Based on the individual performance of classifiers for each fault-type, we created targeted ensembles that are suitable for identification of each fault-type. Our analysis showed that our selected ensemble classifiers were able to separate faults from non-faults with very high accuracy (as high as 85-89% for some fault-types), with a notable result being that in some cases, individual classifiers performed better than ensembles. In general, our approach can significantly reduce effort required to isolate faults from false-positives during the fault consolidation step of requirements inspections. Our approach also discusses the percentage possibility of correctly classifying each fault-type.
| Original language | English |
|---|---|
| Title of host publication | iSOFT - Proceedings of the 11th Innovations in Software Engineering Conference, ISEC 2018 |
| Publisher | Association for Computing Machinery |
| ISBN (Electronic) | 9781450363983 |
| DOIs | |
| State | Published - 9 Feb 2018 |
| Event | 11th Innovations in Software Engineering Conference, ISEC 2017 - Hyderabad, India Duration: 9 Feb 2018 → 11 Feb 2018 |
Publication series
| Name | ACM International Conference Proceeding Series |
|---|
Conference
| Conference | 11th Innovations in Software Engineering Conference, ISEC 2017 |
|---|---|
| Country | India |
| City | Hyderabad |
| Period | 9/02/18 → 11/02/18 |
Fingerprint
Keywords
- Ensemble
- Fault types
- Inspection reviews
- Machine learning
- Supervised learning
Cite this
}
Validating requirements reviews by introducing fault-type level granularity : A machine learning approach. / Singh, Maninder; Anu, Vaibhav; Walia, Gursimran S.; Goswami, Anurag.
iSOFT - Proceedings of the 11th Innovations in Software Engineering Conference, ISEC 2018. Association for Computing Machinery, 2018. a10 (ACM International Conference Proceeding Series).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Research › peer-review
TY - GEN
T1 - Validating requirements reviews by introducing fault-type level granularity
T2 - A machine learning approach
AU - Singh, Maninder
AU - Anu, Vaibhav
AU - Walia, Gursimran S.
AU - Goswami, Anurag
PY - 2018/2/9
Y1 - 2018/2/9
N2 - Inspections are a proven approach for improving software requirements quality. Owing to the fact that inspectors report both faults and non-faults (i.e., false-positives) in their inspection reports, a major chunk of work falls on the person who is responsible for consolidating the reports received from multiple inspectors. We aim at automation of fault-consolidation step by using supervised machine learning algorithms that can effectively isolate faults from non-faults. Three different inspection studies were conducted in controlled environments to obtain real inspection data from inspectors belonging to both industry and from academic backgrounds. Next, we devised a methodology to separate faults from non-faults by first using ten individual classifiers from five different classification families to categorize different fault-types (e.g., omission, incorrectness, and inconsistencies). Based on the individual performance of classifiers for each fault-type, we created targeted ensembles that are suitable for identification of each fault-type. Our analysis showed that our selected ensemble classifiers were able to separate faults from non-faults with very high accuracy (as high as 85-89% for some fault-types), with a notable result being that in some cases, individual classifiers performed better than ensembles. In general, our approach can significantly reduce effort required to isolate faults from false-positives during the fault consolidation step of requirements inspections. Our approach also discusses the percentage possibility of correctly classifying each fault-type.
AB - Inspections are a proven approach for improving software requirements quality. Owing to the fact that inspectors report both faults and non-faults (i.e., false-positives) in their inspection reports, a major chunk of work falls on the person who is responsible for consolidating the reports received from multiple inspectors. We aim at automation of fault-consolidation step by using supervised machine learning algorithms that can effectively isolate faults from non-faults. Three different inspection studies were conducted in controlled environments to obtain real inspection data from inspectors belonging to both industry and from academic backgrounds. Next, we devised a methodology to separate faults from non-faults by first using ten individual classifiers from five different classification families to categorize different fault-types (e.g., omission, incorrectness, and inconsistencies). Based on the individual performance of classifiers for each fault-type, we created targeted ensembles that are suitable for identification of each fault-type. Our analysis showed that our selected ensemble classifiers were able to separate faults from non-faults with very high accuracy (as high as 85-89% for some fault-types), with a notable result being that in some cases, individual classifiers performed better than ensembles. In general, our approach can significantly reduce effort required to isolate faults from false-positives during the fault consolidation step of requirements inspections. Our approach also discusses the percentage possibility of correctly classifying each fault-type.
KW - Ensemble
KW - Fault types
KW - Inspection reviews
KW - Machine learning
KW - Supervised learning
UR - http://www.scopus.com/inward/record.url?scp=85044398564&partnerID=8YFLogxK
U2 - 10.1145/3172871.3172880
DO - 10.1145/3172871.3172880
M3 - Conference contribution
T3 - ACM International Conference Proceeding Series
BT - iSOFT - Proceedings of the 11th Innovations in Software Engineering Conference, ISEC 2018
PB - Association for Computing Machinery
ER -