Abstract
In today’s rapidly evolving digital landscape, ensuring the security of government data is vital. The Federal Risk and Authorization Management Program (FedRAMP) plays a crucial role in safeguarding commercial systems by providing a standardized framework for assessing and authorizing cloud service providers (CSPs). A vital component of the assessment process is the penetration testing of the CSP Cloud Service Offering (CSO), the intent of which is to identify potential vulnerabilities in the CSO applications and services and to validate the integrity of these offerings prior to use by government agencies. Fortreum’s personnel have been in the FedRAMP space since the program’s inception, and we believe there are significant issues on why the penetration testing portion performed as part of these assessments might not be up to par, or in simple terms, just plain sucks.
Scoping is Inadequate or Limited
One of the greatest impacts to the assessment’s level of success starts with scoping the CSO. Some third-party assessment organizations (3PAOs) or a CSP will limit what is in scope due to their own lack of understanding of each of the mandatory attack vectors for all authorized systems outlined in the FedRAMP Penetration Test Guidance. At the time of this writing, the following attack vectors are considered mandatory and reflect current best practices in penetration testing (in a future post we will break these attack vectors down and describe them in detail):
- External to Corporate
- External to CSP Target System
- Tenant to CSP Management System
- Tenant to Tenant
- Mobile Application to Target System
- Client-side Application and/or Agents to Target System
Without a clear and comprehensive scope, critical components and functionality may be overlooked leaving potential vulnerabilities undiscovered or potential attack paths available to malicious users. For instance, a poorly scoped penetration test might exclude a mobile application (and perhaps the mobile backend application programming interface (API) is exposed and different from a core API) that might be in-scope for the assessment, potentially leaving an entire attack path available to an attacker. We encourage you to have an in-depth scoping discussion with your 3PAO on the penetration testing activities, ask questions about the attack vectors and what should be considered in the boundary for testing. We understand it can be difficult to pull all the right resources together, however, it’s essential in determining the level of effort and ensuring the full boundary of the offering is covered as part of the penetration testing activities within the applicable attack vectors.
3PAO Testing Methodology Missed the Mark
A shallow or poorly followed penetration testing methodology can limit the identification of complex or deep-rooted security issues, undermining the overall efficacy of the penetration test. Here are some things to consider when selecting a 3PAO for penetration testing services:
- Is your 3PAO effectively conducting the penetration test or just relying on vulnerability scans to identify vulnerabilities and/or exploit systems?
- Does your 3PAO have a standardized methodology they can share with you beforehand? Not all methodologies are created equal, so be sure to make a comparison and select the 3PAO that fits your needs and offers a best-in-class service.
- Are the methodologies based off best practices known in the industry? For instance, testing for OWASP Top 10 when conducting web application assessments.
- Does the 3PAO outsource the penetration testing to a third party or with inhouse (employees)?
The methodology should serve as the foundation for conducting comprehensive security assessments, so when the methodology is flawed or comes up short, it will likely yield less than ideal results and potentially leave attack avenues open to potential exploitation.
Lack of Effective Coordination and Communication between the 3PAO and CSP
Proper coordination and communication between the 3PAO and CSP is crucial for a successful engagement. Insufficient communication and/or collaboration can result in misunderstandings, incorrect scope, and potentially poor results or outcomes. Additionally, cooperation within the CSP’s involved teams is just as crucial to ensure a smooth penetration test starting with the request for information through completion of the engagement and report delivery. Being able to communicate the intent of the penetration test and the requirements to the applicable service teams along with the needed 3PAO access for penetration testing is essential to a successful engagement. Without this level of collaboration, the access or permissions granted to the penetration testers may be incorrectly provisioned. The additional time and resources that may be required to resolve these kinds of issues may result in constraints on the actual testing portion of the assessment, causing limited or inadequate testing timeframes and limiting the thoroughness of the penetration test.
Your 3PAO Might Just Be Running Vulnerability Scans
We consistently have conversations beginning with CSPs asking about how we “perform our scans”. It is important to understand that here at Fortreum we do use automated tools to assist our penetration testing efforts, however, much of our testing is manual and hands on. We have seen some ”penetration tests” conducted by other 3PAO organizations and the outputs look closer to a “vulnerability assessment” than a penetration test. While automated tools are great at catching the low hanging fruit during an assessment, an experienced tester should not depend solely on those results. When performing a penetration test, Fortreum thinks like a threat actor and approaches the target employing that mindset. A potential set of scenarios may include employing sophisticated attack techniques such as vulnerability chaining, manual attack techniques, post-exploitation, and data exfiltration activities to gain a foothold within the CSP infrastructure or access government data. Penetration tests by inexperienced 3PAOs may result in missed testing of certain functionalities within the CSO scope. They may not understand the business logic or the environment that the service offering sits upon nor how a threat actor may leverage a vulnerability or series of vulnerabilities to gain a foothold within the CSP environment. An easy way to identify these shortcomings is by taking time to read the penetration test report. Does the report contain extraneous pages of tool output and canned findings, or does it contain comprehensive details around findings, attack narratives outlining what and how a vulnerability was exploited, and actionable recommendations on resolving the identified issues? Here at Fortreum, we strive to collaborate with our clients by outlining the positives of the engagement, highlighting key areas for improvements, and offering a technical debrief at the end of every engagement to walk through the detailed report of findings and actionable recommendations.
Conclusion
By acknowledging and addressing these challenges, organizations can work collaboratively with experienced 3PAOs to maximize the effectiveness of the required FedRAMP penetration test(s). CSPs should dive in and explore the full CSO scope definition, review the 3PAO penetration testing methodologies, ensure appropriate access and permissions to test environments, allocate sufficient time and resources during the engagement, and employ effective communication. All these activities can lead to more robust security assessments which in turn maximizes the money invested into the assessment and bolsters the security of your offerings to more securely house government information in the face of continually evolving cyber threats. It is important to understand the inherent limitations of penetration testing, which can only provide a point in time view into your service offering. While a penetration test provides valuable insights, it may not fully uncover all potential vulnerabilities. Because of this we encourage you to engage in a more regular cadence or a continuous cycle of offensive security testing. Fortreum can work with you to establish an ongoing partnership to meet your needs in the offensive security space. See our service offerings for additional information: Penetration Testing.
