Comparison between Penetration Tests and Red Teaming Engagements

In IT security, there are various approaches to assess the resilience of systems and organizations against cyberattacks. However, when it comes to improving cyber resilience against threats, we often pose the question: "How can the deployed software solutions or individual systems be efficiently examined and evaluated for vulnerabilities?"

To solve this issue, we offer our clients a variety of services, including security audits, penetration tests and red teaming engagements.

The aim of this article is to provide an overview of selected security-related services to facilitate their classification and benefits in specific use cases. The following section examines the fundamental characteristics of an internal penetration test and a red teaming engagement based on an "Assumed Breach" scenario. Additionally, a short introduction into security audits is given. While penetration tests can also be performed on single systems and applications, the focus of this article is on comparing tests against internal IT infrastructures and their evaluation from the perspective of an "Assumed Breach" scenario.

What is a Penetration Test?

In a penetration test, the security level of individual systems and applications, as well as entire infrastructures is assessed. During a "scoping" meeting, the systems that should be included in the penetration test are formally defined in coordination with the client.

Essential Characteristics

• Focus on defined systems: Assessment of specific systems, infrastructure components, or applications to identify vulnerabilities.
• Technically oriented: Detailed technical analysis to evaluate the security level.
• Clearly defined objectives: Specification of target systems and the timeframe for the testing process.
• Transparent execution: Open communication with the client's internal IT department throughout the entire penetration test.

Objectives and Convenience

The objective of the penetration test is to identify existing vulnerabilities and assess them based on their exploitability and the impact of a successful attack. The resulting measures aim to minimize the attack surface against potential threats. After identifying vulnerabilities, an evaluation is conducted based on the following technical criteria:

• Attack vector
• Attack complexity
• Required privileges
• User interaction
• Impact on the confidentiality, integrity, and availability of the respective software solutions

Furthermore, critical vulnerabilities are communicated to the client immediately after their identification, so that appropriate measures can be initiated.

Documentation and Communication

The identified vulnerabilities and corresponding remediation measures are documented in a comprehensive report which is submitted to the client upon completion of the penetration test. Following this, a debriefing takes place with the client, where the identified vulnerabilities are explained in detail and any remaining questions are addressed.

What is a Red Teaming Engagement?

In red teaming, realistic attack simulations (also referred to as engagements or campaigns) are conducted against organizations and their infrastructure. These simulations are typically carried out covertly and employ attack techniques similar to those used by real attackers or ransomware groups (also known as TTP, or "Tactics, Techniques, and Procedures"). For each red teaming campaign, "Rules of Engagement" are established in coordination with the client. They essentially define the campaign objectives, termination conditions, and specific systems that must be excluded due to their criticality.

Essential Characteristics

• Holistic approach: Conducting a realistic attack that takes technology, processes, and people into account.
• Testing responsiveness: No prior information provided to the defenders (Blue Team) to realistically assess detection and response capabilities.
• Responsibilities: Defined roles for the White Team (client), Blue Team (client's IT department / Security Operations Center), and Red Team (attacker).

Objectives and Convenience

During a red teaming engagement, vulnerabilities that are essential for achieving the campaign objectives are identified by the Red Team. The focus is on evaluating the Blue Team's detection and response capabilities, as well as the associated reporting chains. If critical vulnerabilities are identified during a red teaming engagement, they are typically communicated to the client immediately. However, to avoid disrupting the campaign, communication between the White Team and the Red Team is necessary to ensure that access for further analysis is not restricted by the remediation of the vulnerability.

Documentation und Communication

In contrast to a penetration test, all activities conducted during a red teaming engagement are documented in the so-called "Operator Log". This allows for easier attribution of alerts detected by the Blue Team during the red teaming engagement. Additionally, the testers maintain oversight to determine if any activities may have been overlooked by the Blue Team.

The final report details the course of the campaign, describing both successes and failures. Due to the increased documentation requirements and the more careful approach, red teaming engagements typically require a longer time for execution compared to penetration tests.

Another characteristic of a red teaming engagement are the regular exchange meetings between the Red Team and the White Team, which are defined in advance by the White Team during the "scoping" meeting. The testing team prepares an ongoing presentation detailing the progress of the engagement. This meeting is also intended to address any immediate questions.

After the completion of the red teaming engagement, the testing team finalizes the report document and is usually available to clarify any open questions, after the report is sent.

What is a Security Audit?

In a Security Audit, a comprehensive review of systems, configurations, and processes is conducted based on established standards, best practices, and regulatory requirements. The aim is to harden the IT environment by identifying misconfigurations, outdated components, or missing security mechanisms. It is important to note that during a security audit, no active attack attempts are made.

Essential Characteristics

• Focus on configurations and processes: Review of policies and procedures to ensure security standards.
• Benchmark comparison: Alignment with recognized standards such as CIS, NIST CSF, or ISO 27001.
• No attack simulation: Emphasis on identifying vulnerabilities without active testing.
• Basis for hardening and compliance: Provides the foundation for enhancing security and regulatory compliance.

The security audit will not be examined in detail, as this article focuses on the differences between the two services “penetration testing” and “red teaming engagement.”

Overview: Comparison between Penetration Tests and Red Teaming Engagements

Summary

Due to the differences between the various test approaches, the following recommendations arise:

• Security Audits: Security audits are advisable for hardening the configurations of systems or applications, as they identify vulnerabilities in configurations, review security policies, and provide recommendations for improving the security level.
• Penetration Tests: Regularly conducting a penetration test is recommended to identify vulnerabilities in the IT infrastructure as well as in individual systems/web applications and to assess the attack surface. Vulnerabilities can arise over time in individual systems due to software updates and/or configuration changes.
• Red Teaming Engagement: This engagement is particularly useful after security audits and penetration tests have already been conducted, ensuring a fundamental hardening in advance. The Red Team seeks to identify and exploit additional previously unknown paths to achieve the campaign objectives. The focus is on evaluating the detection and response capabilities of the Blue Team, as well as their reporting chains.

The three security services create a complementary security concept; however, they are not substitutes for one another. A well-hardened system reduces the potential attack surface for penetration tests and Red Teaming engagements. Furthermore, penetration tests and Red Teaming engagements help identify security gaps that regular security audits may not capture.