Operational disruption can impact financial stability, threaten the viability of individual firms and financial market infrastructures (FMIs), or cause harm to consumers and other market participants in the financial system. Firms and FMIs need to consider all of these risks when assessing the appropriate levels of resilience within their respective businesses. Dealing with cyber risk is an important element of operational resilience and the CBEST framework is intelligence-led penetration testing which aims to address this risk.
CBEST is part of the Bank of England and Prudential Regulation Authority’s (PRA’s) supervisory toolkit to assess the cyber resilience of firms’ important business services. This prioritised and focused assessment allows us and firms to better understand weaknesses and vulnerabilities and take remedial actions, thereby improving the resilience of systemically important firms and by extension, the wider financial system. Continued use of CBEST has confirmed its use as a highly effective regulatory assessment tool, which can now also be conducted on a cross-jurisdictional basis, in collaboration with other international regulators and frameworks.
This latest version of the CBEST Implementation Guide builds upon the previous framework and contains improvements learned from the extensive testing which has taken place. In particular, we have analysed and implemented changes with the aim of clarifying CBEST roles and responsibilities as well as regulatory expectations for different CBEST activities. While the underlying intelligence-led penetration testing approach remains the same, we have reviewed and updated the technical guidance for most activities, prepared new templates (eg Penetration Testing Report) and incorporated important references to cross-jurisdictional assessments. Another key element is the increased focus on the Threat Intelligence and Detection & Response capability assessments.
As the threat from cyber evolves, we keep CBEST and our overall supervisory approach under review and will continue developing them, in order to set clear expectations for firms and provide tried and tested tools to assess firms’ cyber resilience.
Head of PRA Operational Risk and Resilience Division, Bank of England
This CBEST Implementation Guide has been developed by the Prudential Regulation Authority (PRA) for the benefit of CBEST participants which are firms and financial market infrastructures (FMIs). This guide explains the key phases, activities, deliverables and interactions involved in a CBEST assessment.
Because CBEST is a guiding framework rather than a detailed prescriptive methodology, this guide should be consulted alongside other relevant CBEST materials available from the Bank of England (CBEST (2020a,b)). These can be found at Financial sector continuity.
Firms, FMIs or service providers can ask questions or provide feedback on the CBEST process to the PRA at: CBEST@bankofengland.co.uk.
Further information on the CBEST process is also available on the Council for Registered Ethical Security Testers (CREST) CBEST website.
© 2020 Bank of England
This work is licensed under the Creative Commons Attribution 4.0 International Licence.
To view a copy of this licence, visit Creative Commons or send a letter to Creative Commons, 444 Castro Street, Suite 900, Mountain View, California, 94041, USA.
Figures 3 and 6 cannot be reproduced in any format without the permission of Shutterstock.
Organisations that form part of the UK’s financial services sector must remain resilient to cyber attacks. To help organisations achieve this goal, the Bank of England has implemented the CBEST security assessment framework, which regulators (eg PRA and Financial Conduct Authority (FCA)) have now integrated into their supervisory strategies.
CBEST promotes an intelligence-led penetration testing approach that mimics the actions of cyber attackers’ intent on compromising an organisation’s important business services (IBS) and disrupting the technology assets, people and processes supporting those services.
Collaboration, evidence and improvement lie at the heart of CBEST, as well as a close liaison with the relevant regulators. For those organisations that form part of the Critical National Infrastructure (CNI), liaison with the National Cyber Security Centre (NCSC) may also be required.
What differentiates CBEST from other security testing regimes is its intelligence-led approach. This is the ‘golden thread’ that runs throughout the entire length of a CBEST assessment. This approach means that an organisation’s activities are traceable to their role in supporting the wider economy, and the credible threats that the organisation faces in undertaking that role. This is summarised in Figure 1.
Figure 1: Intelligence-led ‘golden thread’
2.1: Structure of this document
The remainder of this document is structured as follows:
- Section 3 provides an overview of CBEST, including a description of the relevant stakeholders, their roles and responsibilities.
- Section 4 provides information about the CBEST accreditation process.
- Section 5 presents the CBEST risk management process and relevant activities that the Control Group (CG) should consider to manage the assessment.
- Sections 6, 7, 8, 9 and 10 provide an overview of the CBEST process and outlines the four phases of CBEST in more detail, including their planning and project management considerations.
2.2: Legal disclaimer
The information and opinions expressed in this document are for information purposes only. They are not intended to constitute legal or other professional advice, and should not be relied on or treated as a substitute for specific advice relevant to particular circumstances. The sponsors and authors of this document shall accept no responsibility for any errors, omissions or misleading statements in this document, or for any loss that may arise from reliance on the information and opinions expressed within it.
3: CBEST overview
3.1: When should CBEST be carried out?
A CBEST assessment should be carried out only if the firm/FMI meets one of the following criteria:
- The firm/FMI is one of the ‘core’ target group for the regulator who are required to undertake a CBEST as part of the supervisory cycle. The core list is reviewed and agreed by the PRA and the FCA on a triannual basis in line with thematic focus and supervisory strategy.
- The firm/FMI has requested to undertake a CBEST as part of its own cyber resilience programme, and consultation/agreement has been sought with the regulator.
- An incident or other events have occurred which has triggered the regulator to request a CBEST in support of post incident remediation activity and validation, and consultation/agreement has been sought with the regulator.
3.2: Stakeholder and information flow
The following stakeholders are involved in a CBEST assessment:
- Control Group of the participant firm/FMI;
- Threat Intelligence service provider (TISP);
- Penetration Testing service provider (PTSP); and
- National Cyber Security Centre (NCSC).
More details on the key actions and related responsibilities are described in the RACI matrix in Annex B. The flows of information between the above stakeholders is summarised in Figure 2.
Figure 2: Stakeholders and information flow
3.2.1: Control Group (CG)
The CBEST participant is the firm/FMI conducting the CBEST assessment. They are responsible for selecting a CG and appointing a Control Group Co-ordinator (CGC) who will co-ordinate all the test activities for the firm/FMI.
The CG is responsible for the management of the CBEST assessment and its main responsibilities include ensuring that:
- all CBEST minimum criteria (Annex A – CBEST minimum criteria) and requirements described in the CBEST Implementation Guide are met during the implementation of the assessment;
- an overall project plan is defined during the Initiation Phase and systematically updated during the project;
- the CBEST assessment is conducted in a controlled manner, implementing a risk management process to identify, assess and mitigate risks related to CBEST activity during all the phases;
- the secrecy of the CBEST assessment is preserved during the whole implementation. If this is compromised, or there is a suspicion that it has, the CG must report this immediately to the regulator;
- the scope of the CBEST assessment is representative of the IBSs of the firm/FMI. The important systems underpinning the IBSs and compromise actions in scope of CBEST are identified by mean of impact assessment;
- the co-ordination, communication and engagement with all the external parties (TISP, PTSP, regulator, etc) is effective;
- the TISP and PTSP engaged for the assessment are accredited CBEST service providers; and
- deliverables are produced in line with CBEST guidelines/templates and shared with the regulator on a timely basis.
The CGC is responsible for the CG observance responsibilities, the governance, quality assurance (QA) and project management of CBEST. The CGC is responsible for CBEST project management and stakeholder co-ordination.
The CG should comprise of a select number of senior individuals at the top of the security incident escalation chain. The CG should include only members, who are strictly required to:
- provide essential information and knowledge to implement CBEST (eg on IBSs, asset, processes, etc), usually one for each system being tested as part of the CBEST scope to provide subject matter expertise; and
- ensure an effective CBEST risk management process is in place. CG members should have authority to take relevant decisions, but membership is not necessarily limited to, roles such as the Chief Operating Officer, Chief Information Officer, Chief Technology Officer, Chief Information Security Officer.
There is not a fixed number of members for the CG since this will depend on different organisational aspects of the firm/FMI. However CG membership should be as limited as possible and information shared only on a ‘need to know’ basis.
It is also possible that third parties need to be part of the CG (eg important systems underpinning IBS are outsourced). In this case, the firm has to engage with the third party during the early stages of the project and take all the required actions in order to ensure the integrity of the assessment.
3.2.2: The regulator
CBEST is a regulatory-led assessment; regulators provide guidance and direction throughout the assessment, verifying the exercise runs in accordance with the CBEST framework. For simplicity, the term ‘Regulator’ will be used in this document even where there are multiple regulatory bodies involved in the assessment.
Either the PRA, the Financial Market Infrastructure Directorate (FMID) of the Bank of England or the FCA will lead the CBEST assessment. For dual regulated firms/FMIs, both the PRA and the FCA will be required to set up a team with cyber expertise and project management. For cross-jurisdictional CBESTs, UK regulators will collaborate with regulatory bodies from other countries as agreed at the beginning of the assessment.
Regulatory teams will include relevant personnel from both supervisory and cyber specialist teams. The regulator is responsible for using the deliverables from the CBEST assessment to form a view of the participant’s cyber security position. They will monitor the status of risk mitigation activities implemented to manage the process and maintain the secrecy and integrity of the process.
Regulator’s responsibilities will also include:
- exercising oversight of CBEST outcomes and remediation plans throughout the entire process (eg planning, execution and review);
- receiving and acting upon immediate notifications of issues that have been identified, and that would be relevant to their regulatory function; and
- reviewing the CBEST assessment findings in order to produce sector specific thematic reports.
The regulator is also responsible for liaising with NCSC during CBEST. This will include notifying NCSC of CBEST initiation and ensuring NCSC provides input to the Threat Intelligence (TI) validation workshop.
3.2.3: Threat Intelligence service provider (TISP)
The Threat Intelligence service provider (TISP) is an independent company, which will be hired by the firm/FMI in order to plan and execute a threat intelligence analysis of its organisation.
The TISP must be CBEST accredited. The TISP will implement the TI analysis following the best practice described in the CBEST Cyber Threat Modelling guideline.
At minimum, the TISP should complete the following tasks in order to satisfy the CBEST minimum criteria:
- provide an external threat intelligence assessment of the firm/FMI, which features evidentially supported profiles of cyber threat actors that could potentially target the firm/FMI;
- provide information that potential threat actors could uncover about the IBSs and key systems identified as within the CBEST Scope;
- create threat scenarios based on the outcomes of the Targeting assessment and Threat Intelligence;
- complete the Threat Intelligence Capability Assessment of the firm/FMI’s TI function based on the CBEST guidelines;
- provide further intelligence and direction during the Penetration Testing (PT) phase and input to the final PT Report, as appropriate; and
- feedback on the CBEST execution during the Debrief session with the regulator.
During the CBEST engagement, the TISP should work collaboratively with both the firm/FMI and the Penetration Testing service provider (PTSP). This should include:
- ensuring the TI analysis is aligned to the PT plan during the TI phase; and
- continuing to provide further intelligence that may enhance implementation of the scenarios, during the PT phase.
The primary day-to-day contact within the TI/PTSPs are the Project Managers, the CREST Certified Threat Intelligence Manager (CCTIM) (CREST (2020a)).
3.2.4: Penetration Test service provider (PTSP)
The Penetration Test service provider (PTSP) is an independent company, which will be hired by the firm/FMI in order to plan and execute the penetration testing activity on the base of the threat scenarios identified during the TI phase.
At minimum, the PTSP should complete the following tasks in order to satisfy the CBEST minimum criteria:
- design and plan the PT execution in line with the target actions agreed in the scope and the threat scenarios identified in the TI phase;
- agree a PT risk management process with the firm/FMI in order to run a controlled assessment and minimised the risks inherent in a CBEST assessment;
- execute the threat scenarios identified by the TISP and approved by the firm/FMI, using an ethical red teaming testing methodology;
- provide updates on the key target actions implemented and the results during the PT phase;
- complete the Detection & Response (D&R) capability assessment of the firm/FMI based on the CBEST guidelines;
- draft the PT Report in line with the CBEST guidelines; and
- provide feedback on the CBEST execution during the Debrief session with the regulator.
During the CBEST engagement, PTSP should work collaboratively with both the firm/FMI and the TISP. This will include:
- providing comments during the TI phase to improve the analysis and ensure that the proposed threat scenarios will be executable during the PT phase; and
- adapting the assessment by integrating further intelligence details provided by the TISP during the PT phase.
The primary points of day-to-day contact within the PTSPs are the Project Managers and the CREST Certified Simulated Attack Manager (CCSAM) (CREST (2020b)).
3.2.5: National Cyber-Security Centre (NCSC)
The UK National Cyber-Security Centre (NCSC) is a UK Government organisation that provides advice and support for the public and private sector in how to avoid cyber security threats.
During the Threat Intelligence Validation, NCSC will comment on the threat scenarios and other elements described in the Threat Intelligence Report and Targeting Report.
4: Accredited CBEST service providers
CBEST service providers are professional cyber security services suppliers that have gone through an accreditation process that is undertaken by the Bank of England. Service providers must be accredited in order to conduct the threat intelligence, penetration testing and reporting elements of the CBEST.
Accredited service providers must also be members of the cyber security membership body CREST and service providers are obliged to abide by strict and enforceable codes of conduct, underpinned by a code of ethics. These codes can be found at: CREST Company Complaints and Resolutions and Individual Complaints and Resolutions.
It is important that the integrity of the CBEST process is maintained, therefore any actions taken by the service providers that are designed to manipulate the process or the results must be reported to CREST for investigation by the participant firm/FMI.
It is the responsibility of the service providers to report to the regulator if they suspect that the process has been manipulated by the firm/ FMI to provide a more positive response to the regulator. This could include such actions as manipulation of the scope to exclude vulnerable or important systems, inappropriate preparation for the test through informing system owners of the test, manipulation of the final reports, or undue pressure on the service provider to present a positive outcome.
4.1: Certified individuals
As a pre-condition for accreditation onto the CBEST scheme, CBEST service providers are required to employ certified individuals who have demonstrated appropriate standards of proficiency that allow them to operate under the CBEST scheme.
For TISPs, CREST has developed a CREST Certified Threat Intelligence Manager (CCTIM) qualification (CREST (2020a)). The CCTIM qualification validates the candidates’ knowledge and expertise in leading a team that specialises in producing threat intelligence.
For PTSPs, CREST has worked with the regulators and industry to develop the CREST Certified Simulated Attack Manager (CCSAM) (CREST (2020b)) and CREST Certified Simulated Attack Specialist (CCSAS) (CREST (2020c)) qualifications.
The CCSAM certificate is designed to demonstrate competence in penetration testing, project management and management of risks to operational systems during the assessment. The CCSAS certificate demonstrates that the individual is very experienced in simulated attack techniques.
These examinations have been assessed by the regulator as being a demonstration of skill, knowledge and competence in the relevant disciplines. The combination of these roles ensures that the highest level of testing can be provided in a safe controlled environment. Certified individuals sign off all major activities and deliverables on behalf of the service provider. Credentials can be checked by emailing email@example.com.
4.2: CREST accreditation body
Although not directly part of the CBEST process, the CBEST accreditation body CREST performs a very important function. The regulator has reviewed the CREST company accreditation processes, Codes of Conduct and Ethics adopted by CREST and augmented their standards with additional requirements specifically for the finance industry.
Any complaints raised during a CBEST between the firm/FMI and the CBEST service providers or those employed on the assignment, can be referred to CREST, who will act as the point of contact; see Complaints and resolution measures for CREST member companies.
5: Risk management
The CG is responsible for running CBEST in a controlled manner. This means the CG should identify and analyse risks that could affect the CBEST implementation during the whole project. For each of the risk identified, the CG should plan and implement actions to mitigate it. Risks are reduced by advanced planning, clear definition of the scope and predefined escalation procedures.
The CG should complete an accurate CBEST risk assessment prior the CBEST kick off and the identified mitigating measures should be regularly reviewed by the CG and iterated to ensure they remain appropriate throughout the process.
The CBEST risk assessment aims to keep the CG in control of CBEST during all its phases. The assessment should scope all the CBEST phases and not limited to the PT phase.
The PT phase requires particular attention. Penetration testing of live systems delivering IBSs will mean that there will always be an inherent level of risk associated with a CBEST assessment.
The CG remains in control of CBEST for the whole implementation of the assessment and at any time it can order a temporary halt if concerns are raised over damage (or potential damage) to a system or disruption to IBSs. The use of a CG positioned at the top of the security incident escalation chain also helps prevent miscommunication and protect the confidentiality of the CBEST assessment.
The following paragraphs present tools that the CG should consider during CBEST implementation.
TISP and PTST procurement: Risk is also managed through contracts with the TISP and PTSPs. In order to reduce risk, advanced planning is required. The procurement process should include specific clauses on:
- minimum security and confidentiality requirements;
- scope specification; and
- agreement on issue escalation and disruption.
The use of accredited providers is another measure designed to further mitigate the risk of damage to important live systems (see Section 4).
Project code name: The CG should assign a project code name (unrelated to the organisation’s name) and use this for referencing the organisation within CBEST communications and documentation. This provides confidentiality to the assessment, which may contain sensitive information, such as identification of vulnerabilities in the delivery of IBSs.
CBEST deliverables (eg reports) contain highly sensitive information and therefore they must be managed accordingly during their lifecycle. The deliverables shared with the regulator must not contain sensitive information, which is not necessary for the regulatory analysis. Specifically, the CG should make sure that Personal Identifiable Information (PII) and technical details (such as IPs, system names, emails, configuration details, etc) are removed from the reports before sharing with the regulator.
Project Initiation Document (PID): Responsibility for ownership of overall project and risk management plans sits with the CG. The recommendation for the CG is to use appropriate tools, such as a PID detailing the risk assessment and the mitigations.
TISP and PTSP produce plans respectively for the TI phase and the PT phase and they will share these with the CG, so they can be factored into the overall CBEST risk management plan.
The following figure shows how the PID and the CBEST risk management should be co-ordinated by the CG.
Figure 3: PID and CBEST project and risk management
The PID should also include the CBEST project management plan. The CG organises all activity including regulatory meetings and engagement with the TISP and PTSP.
A summary of the structure of the core project teams across the firm/FMI and the TI/PTSPs, and how they interact with one another, is given in Figure 4.
Figure 4: Project team structure and interaction
- (a) CREST Certified Threat Intelligence Manager.
- (b) CREST Certified Simulated Attack Manager.
- (c) CREST Certified Simulated Attack Specialist.
We provide more details about CBEST project management practice in Section 6.1.
Collaboration: The overall project management approach of CBEST has to be collaborative for it to work effectively. Promoting and maintaining a collaborative approach is the responsibility of the all the stakeholders involved in the assessment and the TISP and PTSP project managers in particular. In detail:
- during the TI phase, once approved by the CG, the TISP should share its deliverables with the PTSP for information purposes;
- the PTSP should provide early reviews of the draft TI deliverables and make sure all required information is available to ensure an effective handover;
- during the PT phase, the TISP should remain available to provide any further support required; and
- the CG, TISP and PTSP should also exchange information freely with the regulator upon request.
The collaborative approach will enable identification and mitigation of any service issues which could impact the firm/FMI.
6: CBEST process
The CBEST assessment process consists of four phases of work, which is summarised in Figure 5:
- Phase 1: Initiation Phase during which the CBEST assessment is formally launched, the scope is established and TI/PTSPs are procured;
- Phase 2: Threat Intelligence Phase during which the core threat intelligence deliverables are produced, threat scenarios are developed into a draft Penetration Test Plan and control of the assessment is handed over to the PTSP;
- Phase 3: Penetration Testing Phase during which an intelligence-led penetration test against the target systems and services that underpin each IBSs in scope is planned, executed and reviewed. The firms Threat Intelligence and Detection and response capabilities are assessed; and
- Phase 4: Closure Phase during which the firm/FMI’s Remediation Plan is finalised, the TI/PTSPs are debriefed and the regulator supervises the execution of the Remediation Plan by the firm/FMI.
Figure 5: CBEST assessment process model