The Assurance Review Process

1. Onboarding/Scoping

The review for an organisation’s CAF self-assessment return should be commissioned either in advance of the self-assessment or in parallel. It is important that the assessor understands early on in the process the scope of the self-assessment activity to be in a position to understand the likely level of scale and effort. At this point you may want to provide a copy of your GovAssure Scoping Document to the provider ahead of the planning stage.

As part of the onboarding process and in good time ahead of the review itself, it is sensible to engage the assessor early on and hold an on-boarding / scoping call to agree logistics around agreeing arrangements for access to IT equipment and provisioning access to any systems (e.g. setting independent assessors up with ‘Assessor’ profile access on WebCAF) or file shares that may be required for the assessor to gain access to evidence and artefacts as part of the review.

2. Planning for the independent assurance review

As part of the planning stage, the organisation should meet with the assessor to brief them on the context of the organisation, systems being assessed and the assigned target government CAF profile, their approach to completing the CAF self-assessment. The organisation’s GovAssure Scoping Document should be referred to throughout this planning phase.

At the initiation stage of the review, individuals who are involved in the work – those who completed the CAF self-assessment return, as well as those who can help with the evidence supporting the assertions in the return – should be identified and put in contact with the assurance provider. At this meeting, or immediately afterwards, the provider should be sent the details necessary to log onto the WebCAF portal to access the return as well as the collection of evidence supporting the self-assessment.

We recommend that the assessor prepares an operational terms of reference document for the review that includes how the review will be managed, timelines and key contacts and that this document is agreed between the organisation and the provider.

3. Assessment and analysis

Once dates have been agreed for the commencement of the review, the review can begin. Assessors are advised to follow a similar process to that of the organisation. This is usually approached in the following way:

Initial high-level desk-based review

The assessor will typically perform an initial high-level assessment self-assessment submission. Consideration will be given to the following:

• Quality – overall initial assessment as to the quality of the self-assessment
• Completeness – the assessor will review progress within WebCAF to confirm that all contributing outcomes and IGPs have been answered and the rating and comments are showing as complete.
• Evidence – availability, quality and consistency of supporting evidence and the extent to which it is clearly cross-referenced back to the CAF
• High level review of contributing outcomes – an initial run through of CO’s to identify any that may need closer attention, for example where they are identified as ‘Not Applicable’ and those that do not meet the target profile against which the return is being assessed. Contributing outcomes marked as ‘Achieved’ and ‘Partially Achieved’ are likely to be subject to greater scrutiny than those marked as ‘Not Achieved’.
• Understanding the approach taken to completion of the CAF objectives – This will consider the level of commonality in responses and its appropriateness. For contributing outcomes under objectives, A and D specifically, gaining an understanding as to the level of commonality in responses and whether these have been described at an organisational level and the level of application at a systems level. Whereas the majority of the CO’s in Objectives B and C are likely to be system-specific and should be reviewed on a per system basis.

Detailed IGP assessment in support of contributing outcomes

Following an initial high-level review, the assessor will take an in-depth view of contributing outcomes, their assessed state compared to the target CAF profile and the supporting IGPs. Consideration will be given to the following:

• Review of achievement against the target CAF profile - The assessor will reflect on the contributing outcome level and the organisations assessed view versus the target CAF profile for each contributing outcome. The organisation should have completed the CAF self-assessment return with a focus on the target CAF profile. Therefore, the assessor may see that where a contributing outcome has a target of ‘Partially Achieved’, organisations may have chosen to review only the ‘Not achieved’ and Partially Achieved’ IGPs rather than seeking evidence to demonstrate achievement over and above the target CAF profile. Conversely it may be possible for organisations to exceed the set target CAF profile.

• Review and assessment of IGP statements supporting the overall contributing outcome - The assessor will be required to provide an expert judgement on whether the specific IGP statements apply to the system at both the IGP and contributing outcome level based on the commentary, evidence and workshop discussions. They will use the “Yes/No/Not Assessed” questions on WebCAF and their judgement as to whether each individual statement describes the organisation or system on its own merit. To assess each IGP statement, the assessor will use their expert judgement to decide whether they agree with the statement from the drop-down selection, by selecting:

  • ‘Yes’ – Yes, the IGP describes the state of the system given the evidence provided.
  • ‘No’ – No, the IGP does not describe the state of the system given the evidence provided.
  • ‘Not Assessed' – This IGP is not applicable or exempt. If selecting this option, justification for why should be made clear in the comment box.
  • The IGPs supporting a contributing outcome are not intended to be exhaustive, and organisations may implement additional good practice or compensating controls that would otherwise return an "Achieved" or "Partially Achieved" contributing outcome. Where alternative good practice is implemented, this should be reflected in by the assessor with supporting narrative.

• Provision and review of evidence. The assessor will review supporting evidence in parallel with reviews of both IGP statements and contributing outcome narrative. The organisation is responsible for managing their own evidence and providing it to the assessor in an organised manner that can be readily cross referenced to their evidence entry reference on WebCAF. It is critical that the organisation provides the assessor with appropriate access to the necessary evidence that has been collated to support your self-assessment, to enable the assessor to be able to complete the assurance review. The assessor may ask questions of the organisation attendees to clarify any points that arise from the review.

• Reviewing contributing outcomes. WebCAF makes it mandatory for assessors to select both an achievement rating for each contributing outcome statement and supporting narrative as to their conclusions based on the organisation’s assessment and evidence. Following an assessment of the individual IGPs comprising the contributing outcome, the assessor will choose from the following achievement ratings in the same way as the organisation did as part of the Stage 3 CAF self-assessment – the assessor will review your ‘Achieved’, ‘Partially Achieved’ and ‘Not Achieved’ status and review them on the following basis:

  • “Achieved” - For the assessor to mark a contributing outcome as "Achieved", they must agree with all the ‘Yes’ to every “Achieved” IGP (except where an IGP has been identified as “not assessed”).
  • “Partially Achieved” (where present) - this system is partially achieved for this specific contributing outcome. To mark a contributing outcome as "Partially Achieved", the organisation must have answered ‘Yes’ to every “Partially Achieved” IGP, based on your judgement of the organisation meeting every “Partially Achieved” IGP (except where you mark an IGP as “not assessed”). Note - not every contributing outcome will have associated “Partially Achieved” IGPs.
  • “Not Achieved” - this system has not achieved this specific contributing outcome. If the organisation has answered ‘Yes’ to any “Not Achieved” IGP, you must mark this contributing outcome as “Not Achieved”.
  • Assessors will be required to justify their assessment of the contributing outcome achievement rating in the box provided. This is a required field that requires them to provide an overall rationale for their assessment of the contributing outcome achievement status.

• Narrative detail - The independent assessor will be required to provide a narrative at the contributing outcome level but may provide detail at the IGP level if there is a difference in the assessor’s view compared with the organisation’s evaluation, or, to justify the use of ‘Not Assessed’, to explain the justification for the factors preventing a clear assessment. Where assessor and organisational evaluations are the same, they will not be required to provide supporting narrative.

Question and query resolution

The general approach is that the assessor will collate their initial comments and queries and then work through these as part of dedicated workshops, generally, on a ‘per Objective’ basis, you should work in a way that suits you whilst not introducing delays. Some providers may prefer to provide a list of questions and queries back to the organisation for response from the appropriate technical contacts; this may make a more efficient use of time. We recommend not providing repeated loops around this process as this may delay the work. It is better to have a single, focused ‘Request for comment’ round.

Workshops

Working with the organisation lead, the independent assessor will schedule a series of workshops covering objectives A – D. The size of Objective B might mean that it needs to be assessed over more than one workshop session. The workshops may cover the following:

• Consideration to each CO at a holistic level before drilling down into underlying IGPs and any queries the assessor may have at the IGP level.
• Consider achievement against the target CAF profile.
• Consideration to whether there are any IGPs outliers? For example, most are ‘partially achieved’ but one is ‘not achieved’. If any IGPs are shown as being ‘not applicable’, and whether this is justified?
• Whether there is a consistent level of evidence for those IGPs that require justification?
• Completing these three steps should enable areas of focus to be determined – any IGPs that are not appropriately justified by the evidence, or which do not fit with the overall assessment of the CO.
• Questions or queries relating to evidence and its completeness.

Development of a list of control gaps / observations

Summarise the findings of any control gaps at a CO level, providing an overall achievement level and justification for this (especially if it differs from the original level). There should be an understanding of the IGPs leading to the contributing outcome not achieving its target CAF profile.

4. Reviewing and communicating results

The arbitration process is required if the workshops and reviews identify conflicting views that the department and the assurance organisation cannot resolve. In the first instance, the areas of dissent should be identified in a single list and a separate workshop arranged to focus on their resolution. This should include the senior stakeholder from the department. Similarly, any final lists of observations and recommendations should be reviewed and agreed between the organisation and the provider to allow the Independent Assurance Review Report (IARR) to be produced.

If there are still remaining unresolved points following this workshop, they can be escalated to the Cabinet Office team within the Government Security Group (GSG).

GovAssure WebCAF External Assessment System Report

The assessor is expected to document their results in WebCAF but they may want to support working through the assurance review with the auto-generated report containing outputs from an organisation’s self-assessment including the assessor assessment of an individual system. This will be available to the organisation and the assessor following the completion of the stage 4 independent assurance review and will provide a full ‘download’ report covering assessors’ comments across the CAF on a system by system basis.

The report provides the following:

• Summary radargram graphics for each CAF objective showing the underlying contributing outcomes of the assessor results for the system vs the target government CAF profile (Baseline or Enhanced).
• A summary table comparing the assessor achievement results of each contributing outcome vs the Government CAF profiles.
• A list of the contributing outcomes under each objective that assessor ratings have highlighted did not meet the required rating under the target government CAF profile (Baseline or Enhanced).
• A table of the IGP comments and achievement levels as determined by the assessor clearly referencing any IGPs that are causing the contributing outcome to not meet the target Government CAF profile for the system.

This auto-generated report will be used by assessors to generate the Independent Assurance Review Report (IARR) which will be shared with organisations. The data and graphs produced in this report should be used by assessors in the IARR.

Next Steps

By the end of Stage Four, organisations will have agreed a list of observations with the independent assurance provider. This will be developed into the Independent Assurance Review Report (IARR) as part of the final assessment feeding into the Targeted Improvement Plan (TIP), to be delivered as part of Stage Five.