A Quantitative Study on the Receipt of Formalized RMF Training and Perceptions of RMF Effectiveness, Sustainability, and Commitment in RMF Practitioners.
By P. Devon Schall, Ph.D., CISSP, RDRP
Over the past year, I have conducted research on the relationship between the receipt of formalized RMF training and perceptions of RMF effectiveness, sustainability, and commitment in RMF practitioners. I am very pleased to announce, I have completed the study and have some interesting results to report. This article will provide an overview of my research methods and research study findings.
Research Methods
Quantitative data on the perceived confidence, compliance commitment, and sustainability ratings for RMF were collected and used in this research. Survey research was implemented, and data were collected through a questionnaire. The intended participants in the study were those who work in the U.S. Government or serve as U.S. Government contractors with requirements of cybersecurity compliance in their job roles. The survey questionnaire was provided to the members of the LinkedIn group titled Risk Management Framework (RMF) Resource Center via a survey link posted in the group as well as a private message sent to each member of the group with an explanatory invitation. This group consists of 1779 members and was established to provide its members with the opportunity to connect in understanding RMF. The survey was presented to all group members without any prior research or bias regarding their previous RMF training received or years of experience. The data were analyzed utilizing statistical methods of descriptive statistics, analysis of variance (ANOVA) and Pearson’s Correlations.
Findings
Based on the results of this study, a significant, positive relationship exists between the receipt of formalized RMF training and perceptions of RMF effectiveness. Statistical significance can be seen in ANOVA tests where there was a significant difference in the mean effective Perceived Competency Scales (PCS) Scores among those with varied levels of formal RMF training (MS = 5.388), (F [2,78] = 3.645, p < .05). Pearson’s Correlation also indicated that there was a significant positive association with the Effective PCS Score and the Amount of Training Received Category, (r = .253, n = 81, p = .023).
Breaking It All Down
I conducted a quantitative (based on math and statistics) research study which delivered a survey through a LinkedIn Group titled Risk Management Framework Resource Center. The survey presented Likert-type scales which asked respondents on a 0-7 scale how strongly they identified as being effective in implementing RMF, felt committed to RMF, and felt RMF was a sustainable framework for the U.S. Government. The participants were also asked how many hours of formalized RMF training they had received.
For those who are not experts in statistical analysis, I will try to explain simply how the data were analyzed. After collecting the results of the survey, I split the data into three groups. Those groups were low (0-32 hours of formalized RMF training received), medium (32-40 hours of formalized RMF training received), and high (40+ hours of formalized RMF training received).
To establish if any statistically significant data existed, I utilized a statistical method called an Analysis of Variance (ANOVA). The ANOVA tests relates to groups (for this study my three RMF formalized training hours categories) and it indicated if a significant difference existed in any of the groups as they related to the participants answers to the 0 – 7 Likert-type scales. In this scenario, the ANOVA test indicated that one of the three groups were significantly different from the other two.
I then used another statistical method called Duncan’s Multiple Range Test to dig deeper into the data and learn that the biggest difference was between the medium group (32-40 hours of formalized RMF training received) and the high group (40+ hours of formalized RMF training received). The conclusion from the ANOVA paired with Duncan’s Multiple Range Test was that RMF practitioners who receive 40+ hours of formalized RMF training showed a statistically significant increase in their confidence in being proficient and effective in the implementation of RMF.
To support the ANOVA results, correlation analyses were conducted and showed a significant positive relationship existed on a linear basis between the receipt of formalized RMF training and RMF practitioners’ perceptions of being effective in the application of RMF. A weak trend was observed in the relationship between the receipt of formalized RMF training and perceptions of RMF commitment and no significant relationships were observed between the receipt of formalized RMF training and perceptions of RMF sustainability.
Future Research
I plan to conduct future research studies which explore the relationships between the receipt of formalized RMF training and increased RMF project efficiency and cost savings. I am confident that by showing conclusive data that formalized RMF training reduces overall project costs the RMF community can get away from the idea that anyone can learn RMF by reading NIST policy documents in their free time. As an RMF practitioner, I am committed to improving the real-world application of RMF with the goal of mitigating the idea that RMF is failing.
The entirety of my research study can be found below:
I hope I didn’t you lose you in this article! Please let me know if you have any questions.
Dr. RMF