Threat Led Penetration Testing (TLPT) in DORA: The final text

Last week the European Supervisory Authorities (ESA’s) EBA, EIOPA and ESMA published the second batch of policy products for DORA. This includes the final text about Threat Led Penetration Testing (TLPT). In this article I will explain the major differences with the consultation version.

If you are not aware of TIBER I recommend to read my blog on what is the TIBER framework? And if you missed my previous blog on the differences between DORA TLPT and TIBER-EU this will give you a quick overview on their main differences. Shortly said TIBER-EU is a red teaming framework that sets out how to run a threat intelligence led red teaming test on live systems of entities in the financial (or any other) sector. DORA TLPT aims to enhance the digital operational resilience of the European financial sector using Threat Led Penetration Testing in accordance to TIBER-EU.

The main changes in DORA TLPT

As the complete document is 184 pages I cannot highlight all the changes and still provide a readable blog. Therefore I will highlight the main changes and post follow-up blogs on specific topics around DORA TLPT at a later stage.

Clarification on the timelines

Based on the public consultation the ESA’s adapted some of the timelines in the testing process. For example the Blue Team now has 10 weeks to deliver their blue team test report.

This seems positive to those that asked the ESA’s for more time, but while going through the document it becomes clear multiple deliverables are running simultaneously and therefore can cut into the time given. For example while the Blue Team gets 10 weeks for the Blue team report the Red Team also still has to deliver their Red Team report. In my experience the Blue Team has a hard time to deliver a Blue Team report without the Red Team test report. Therefore of the 10 weeks given to the Blue Team only 6 remain if the Red Team provider uses all the allocated time to deliver the Red Team report.

Similarly the replay exercise and purple teaming are only useful if the Blue team has gone through the Red Team test report and has created its own timeline, but the replay and purple teaming exercise are also within this 10 week window. Some of these final timelines can become problematic in practice.

Below are diagrams about the preparation, testing and closure phases. There are no changes made to the timelines in the testing phase and the active red team testing time is still a minimum of 12 weeks.

Clarification on ‘pooled testing’ and ‘joined testing’

One of the more unclear topics in the consultation version was ‘pooled testing’ as this is not part of the TIBER-EU framework and this was not really well explained in the consultation version. This has been adapted in the final version.

Pooled testing is meant for multiple, possibly unrelated, financial entities and their external ICT third party provider (TPPs). The idea is that a third party provider that (primarily) provides externally hosted services (like AWS/Azure/GCP) can be tested in combination with multiple financial entities. The DORA TLPT text describes now how such a test with possible cross border aspects will more involving multiple TLPT authorities.

Joint testing on the other hand is meant for multiple financial entities that belong to the same group and use the same shared ICT systems and can therefore be tested jointly.

Requirements for external testers

Multiple changes have been made in the final text to the requirements put on external testers (both red team providers and TI providers):

• The number of years of experience that are required for the red team can now be gained by red teaming testing and penetration testing. Before only the number of years of experience in red teaming were counted.
  
• For the threat intelligence provider the final text seems to have dropped the requirement for the TI provider to have: “… three years of collecting, analysing and producing threat intelligence for the financial sector“. But on the side of the testers (red team provider) they are still required to have: “knowledge about the business of the financial entity“
  
• A good addition in my view is that both the TI and red team provider (testers) cannot perform blue team tasks or other services that might present a conflict of interest.
  
• There is a clarification that the TI and RT services can be procured from the same provider, but the teams must be separated and have different reporting lines. This is same standard used by most TIBER authorities.
  
• A new major deviation is that “In exceptional circumstances” the financial entity can hire a threat intelligence provider and/or external testers that do not fulfil one or more of the requirements listed in the DORA RTS. They then have to “adopt appropriate measures to mitigate the risks relating to the lack of compliance with such points and record them.”
  
  The question then becomes if the TLPT authority can object to this. The RTS gives the TLPT authority in in Article 8 (10) the option to object to hiring the TI providers and/or the testers if the “exceptional circumstances” have not been met or the appropriate measures have not been taken.

Requirements for internal testers

DORA TLPT introduced the ability to use internal testers where TIBER-EU at this moment doesn’t allow for this. Although this was not completely clear in the consultation text, DORA has the same requirements for internal as external testers. Internal testers therefore have to show the same number of years of experience as their external colleagues. This has been clarified.

A change made in this final version is that internal testers now only need to be working at the entity for one year instead of two before they can function as an internal tester for DORA TLPT.

Changes to the entities in scope for DORA TLPT

• The final text changed the criteria for which entities in the insurance and reinsurance sector have to perform TLPT under DORA. I have made a separate article on the entities in scope of DORA TLPT based on the final text.
  
• The consultation text already gave the competent authority the option to exclude entities that officially are in scope, but are deemed not mature enough to do testing on live systems. It now also highlights that TLPT authorities should consider (in the future) to include other entities than those that already in scope of DORA TLPT. Crypto asset service providers are explicitly mentioned as an example.
  
• The final text has made it explicit that competent authorities have the option to go beyond the requirements in the RTS for the “largest, most significant and most advanced entities“. The RTS have to be seen as the minimum standard for TLPT under DORA. This gives the TLPT authority the option to require more than is described in the RTS.

Conclusion

The final text incorporates a lot of the feedback given by 111 responses in the consultation round. It made a lot of clarifications like on timelines, internal testers, on pooled and joined testing, but it also introduced some changes like deviating from the requirements of external testers in exceptional circumstances.

DORA TLPT is not perfect and it is not complete, but remember that the ESA’s have been given the assignment to make the TLPT RTS in accordance with TIBER-EU. So let’s use the experience gained from TIBER-EU in the domain of DORA TLPT. It would be shame if all of the lessons learned and the cooperation that has been build up between financial entities and authorities in TIBER is lost because of the introduction of this law. Let’s continue on the path that TIBER-EU set out to improve the cyber resilience of the financial sector as a whole using testing together!