Inside the Hacker's Playbook - How We Stopped a €20M Ransomware Payout - Joseph Carson

Inside the Hacker's Playbook - How We Stopped a €20M Ransomware Payout

Speakers: Joseph Carson, Chief Security Evangelist, Sigura

Conference: Disobey

YouTube: https://www.youtube.com/watch?v=IeGaoJ-e-AE

Overview

Joseph Carson, a seasoned security researcher and Chief Security Evangelist at Sigura, delivered a compelling talk at Disobey, taking the audience on an immersive journey through a real-world ransomware incident. Titled "Inside the Hacker's Playbook - How We Stopped a €20M Ransomware Payout," Carson meticulously dissected an attack that brought an organization to a complete standstill, detailing the attacker's tactics, techniques, and procedures (TTPs) from initial compromise to the brink of a massive ransom payment. The talk not only highlighted the technical intricacies of the attack but also the profound human impact and organizational failures that often precede such catastrophic events.

Carson, an incident responder who personally assisted the victim organization, received rare permission to share the full, end-to-end demonstration of the attack, complete with real-world scripts and techniques used by the adversaries. This transparency provides an invaluable learning opportunity for security professionals, offering a granular look at how a sophisticated ransomware-as-a-service (RaaS) operation unfolds. The presentation culminated in the dramatic revelation of how the organization narrowly avoided a €20 million ransom demand, albeit at a significant cost, underscoring the critical importance of resilience and proactive security measures.

The talk serves as a stark reminder that even organizations with traditional, modern security controls can fall victim if those controls are not actively monitored, managed, and complemented by a robust and practiced incident response plan. Carson's objective was to empower attendees with the knowledge to fortify their own defenses, emphasizing that the lessons learned from this incident are crucial for building a safer digital world.

Background

▶ Watch: Talk introduction and ransomware case overview (0:00)

Joseph Carson's extensive career, spanning over 30 years in cybersecurity, provides a deep historical context for understanding the evolution of cybercrime. He humorously referenced the first person ever convicted of a cybercrime, Ian Murphy, also known as Captain Zap, in 1981, whose "crime" involved "time traveling" by changing system dates at Bell Labs to secure cheaper internet rates. This historical anecdote underscores a persistent motivation for cybercriminals: financial gain, a theme that remains dominant in modern ransomware attacks. Carson drew a parallel between the early forms of extortion by shipping pirates in Somalia and the emergence of ransomware, suggesting that attackers adapted the concept of holding ships hostage to holding data and systems hostage.

The specific case detailed in the talk involved the Krylock ransomware, a variant that evolved from Kryakle (active 2016-2019) into Krylock 2.0 and 3.0. Krylock is notable as one of the first Ransomware-as-a-Service (RaaS) models, where its creators provided the ransomware as an affiliate service to hands-on keyboard attackers, receiving a percentage of successful ransom payouts. Carson mentioned that the creators of Krylock, Vadim and Elena, were eventually convicted in 2023, extradited to Belgium, and sentenced to 13 and 7 years respectively, with millions in assets recovered.

The organization in question discovered they were a victim of ransomware on a Sunday morning around 9:00 AM. Interestingly, the attackers, frustrated by the lack of immediate response, resorted to finding the IT manager's personal contact details and directly calling them to inform them of the compromise. This highlights a shift in attacker tactics, moving beyond passive ransom notes to active, aggressive communication.

During the incident investigation, Carson uncovered other intriguing details. Two years prior to the ransomware attack, an employee had installed crypto mining software, leveraging company servers to mine cryptocurrency for 24 months, resulting in thousands of euros in energy costs. This discovery, though outside the immediate scope of the ransomware, revealed a separate internal security breach. More directly related, Carson found evidence of another victim embedded within the attack scripts—passwords, usernames, and infrastructure details of a large entertainment organization. Despite Carson’s attempts to warn them, this organization initially denied the breach, only admitting to paying a ransom after being informed that evidence would be handed to law enforcement. To this day, they have not publicly disclosed the incident. These auxiliary findings underscore the complex and often interconnected nature of cyber investigations.

The immediate aftermath of the ransomware attack plunged the victim organization into extreme stress and anxiety. With all critical data—payrolls, contracts, inventory, supply chain details—encrypted and unavailable, the prospect of business failure and job losses loomed large. Carson emphasized the importance of empathy in incident response, noting the severe emotional burnout experienced by employees during this period. The organization's initial attempts to activate their Incident Response (IR) plan were met with a critical failure: the plan itself was stored on an encrypted server and inaccessible. This common oversight, combined with a lack of practice and simulation of the IR plan, highlighted fundamental weaknesses in their preparedness. They lacked defined ownership, alternative communication channels (as attackers were monitoring emails and Slack), an updated contact list, clear response definitions for different attack types, internal resource assessments, and strategies for public communication, legal exposure (GDPR, PCI), clean environment recovery, and lessons learned. The absence of basic readiness, such as a "ready bag" for responders or a plan for mass credential rotation, further exacerbated their predicament. Ultimately, the organization made the critical decision to shut down all systems, going completely offline and dark to contain the spread and sever attacker access.

Key Findings

▶ Watch: Historical anecdote: first cybercrime conviction (Captain Zap) (3:00)

The primary finding from this incident was the successful prevention of a €20 million ransomware payout, reducing the total cost of the incident to approximately €1 million. This significant achievement, however, was not due to sophisticated security controls, but rather an accidental stroke of luck: the organization had neglected to decommission old ERP hardware and systems after a migration a year prior. These "dusty servers" contained an older, unencrypted snapshot of critical data, which became the lifeline for recovery.

Key findings regarding the attack and organizational posture included:

• Extended Attacker Presence: The attackers had "hands-on keyboard" access for approximately 14 days prior to deploying the ransomware. However, the initial evidence of compromise—a single successful login from a suspicious IP address—dated back seven months before the attack, indicating a prolonged reconnaissance and preparation phase. Subsequent suspicious logins from NordVPN and Tor VPN exit nodes further confirmed early access.
• Fundamental Security Failures: The incident exposed a cascade of basic security shortcomings:
• Weak Password Policy: An 8-character password policy allowed attackers to crack most user passwords within 25 minutes from hashes.
• Poor Credential Management: Passwords were stored insecurely on desktops (e.g., in a file named "important stuff") and within unmanaged web browsers.
• Excessive Privileges: The "patient zero" accounting user had local administrator rights on their system, and a critical scheduled backup task ran under a Domain Administrator account.
• Lack of Monitoring: Despite having antivirus, EDR, and MFA, the organization lacked the resources and processes to actively monitor and react to the deluge of alerts, allowing attackers to operate undetected for months.
• Attacker Sophistication and Automation: The adversaries demonstrated a blend of opportunistic exploitation and methodical execution:
• They utilized a Ransomware-as-a-Service (RaaS) model, choosing Krylock due to its ability to bypass existing AV/EDR solutions.
• They employed legitimate tools like GMER for reconnaissance (identifying security products) and specialized hacking tools like Mimikatz for credential harvesting and RDP+ and Network Scanner for lateral movement.
• Attribution clues pointed to Russian-speaking attackers, with command-and-control (C2) infrastructure in an Amsterdam data center and an IP address traced to occupied Ukraine.
• Carson highlighted the modern trend of attackers using Large Language Models (LLMs) to rapidly analyze exfiltrated data, accelerating the discovery of valuable information like new credentials or personal data from weeks to seconds.
• Recovery Challenges: Despite avoiding the ransom, the recovery was arduous, requiring 2.5 months and the hiring of 20+ data analysts to manually recreate a year's worth of digital data from the older systems and archives. The total cost of investigation and recovery exceeded €1 million.

Technical Deep Dive

▶ Watch: Important disclaimer for live hacking demonstrations (4:00)

The attack chain meticulously reconstructed by Joseph Carson illustrates a clear progression from initial access to domain compromise and eventual ransomware deployment.

Initial Access (Patient Zero)

The breach originated with an accountant's RDP server, which was publicly exposed due to direct pressure from the accountant on a hosting provider. This circumvented the organization's existing VPN and multi-factor authentication (MFA) controls. The initial compromise likely stemmed from several common vulnerabilities:

1. Weak Credentials: The company's 8-character password policy was a significant weakness. Carson demonstrated that most passwords could be cracked from their hashes within 25 minutes. Attackers likely obtained credentials through:

• Password Reuse: The accountant reusing weak passwords across multiple systems.
• Phishing: A successful phishing campaign targeting the accountant.
• Responder/LLMNR Poisoning: For a remote-working accountant, tools like Responder could capture NTLM hashes from network shares in public Wi-Fi environments (e.g., hotels, cafes).

1. Credential Validation: Attackers used tools like Crowbar to validate harvested credentials against public-facing RDP services, checking for successful logins without triggering alerts.
2. Insecure Data Storage: Once inside the accountant's RDP session, attackers discovered critical vulnerabilities:

• A file named "important stuff" on the desktop contained usernames and passwords for accounting software and databases.
• The unmanaged web browser stored numerous credentials and SaaS tokens (e.g., for Office 365), providing immediate access to further accounts.

Privilege Escalation & Persistence

The accountant's machine was a goldmine for the attackers due to misconfigurations:

1. Local Administrator Rights: The accountant had local administrator privileges on their system, a critical failure of the principle of least privilege.
2. Command and Control (C2) Access: Attackers typically established sessions lasting 4 to 8 minutes to minimize detection. They would access their C2 server (located in a data center in Hutland, Amsterdam) to download tools and scripts. The C2 hosted various ZIP files (A.zip to F.zip) containing automation scripts, a network scanner ("scan"), and different ransomware payloads ("zap").
3. Disabling Security & Evasion: The first downloaded script, A.zip, contained:

• clean.bat: A script to delete log files and hide tracks.
• disable_security.bat: This script would disable local security controls for approximately 15 minutes by clearing the recycle bin and stopping critical services. This short window explained the attackers' brief session durations, as Windows protected services would reactivate after this period.

1. Persistence Mechanisms:

• Sticky Keys Backdoor: Attackers modified the registry to replace the sethc.exe (Sticky Keys utility) with cmd.exe. This provided a persistent backdoor to a command prompt, even if the user's password changed. This technique is often only detectable via registry scanning or during the brief 30-second window when the utility helper is launched.
• Mimikatz Deployment: After disabling security, attackers downloaded and executed Mimikatz. They enabled credential capture in memory by modifying the registry, ensuring that any user logging into the system would have their credentials (including cleartext passwords) harvested.

1. Domain Administrator Compromise: The crucial mistake that led to domain compromise was a misconfigured scheduled task. A daily backup of the accounting server, set to run at 12:30 AM, was executed under a Domain Administrator account. When this task ran, its credentials were captured by Mimikatz, which the attackers then harvested on a subsequent login. This gave them the keys to the kingdom.

Lateral Movement & Reconnaissance

With domain administrator credentials, the attackers rapidly expanded their foothold:

1. Security Product Discovery: Attackers used GMER, a legitimate rootkit and backdoor detection tool, to identify the organization's security products (EDR, AV, etc.) running in the background. This allowed them to understand how to evade detection, a tactic Carson noted seeing in other concurrent cases.
2. Domain Controller Access: They used RDP+ (a non-standard RDP client, possibly to evade logging that standard RDP might trigger) to log onto the Domain Controller. Carson stated that once attackers gain domain admin access, ransomware deployment or extensive data exfiltration is typically 4 hours away.
3. Active Directory Backdoor: On the Domain Controller, they created a new, persistent user account in Active Directory to maintain access.
4. Network Mapping and Deployment: They downloaded and launched Network Scanner. This tool, customized with Cyrillic (Russian) language translations in its interface, allowed them to map the network and deploy scripts (for cleaning logs, creating backdoors) across all servers. This Cyrillic customization was a key attribution indicator, suggesting Russian-speaking attackers.
5. Remote Execution: Attackers utilized tools like PSexec and Evil-WinRM for remote execution and lateral movement throughout the network.

Data Exfiltration & Ransomware Deployment

The final stages of the attack involved data exfiltration and the execution of the ransomware:

1. Data Exfiltration: Attackers extracted gigabytes of data. Carson highlighted a disturbing trend: the use of Large Language Models (LLMs) by attackers to rapidly query stolen data for sensitive information like credit card details, personal data, or new credentials, reducing analysis time from days/weeks to seconds.
2. Ransomware Deployment: The chosen ransomware variant was Krylock, specifically selected because it was not detected by the organization's existing antivirus and EDR solutions, having been "obfuscated" to bypass their security controls. The initial ransom demand was €10 million, with a threat to double it to €20 million if not paid within two days.

Demo / Proof of Concept

▶ Watch: Ransomware evolution and the Krylock variant (5:00)

Joseph Carson's presentation included a compelling live demonstration that walked the audience through the exact steps of the attack. While the initial access phase was based on assumptions (weak credentials, phishing, or Responder for NTLM hash capture), everything from the remote access onwards, including the scripts and technical details, was an accurate recreation of the actual incident. Carson used Kali Linux for his attack machine, noting that the original attackers were more comfortable with Windows environments.

The demonstration vividly illustrated the critical failures that enabled the attack:

• Initial Access: Carson showed how an attacker, masquerading as the accountant, could log into the publicly exposed RDP server. He simulated the use of Responder to capture NTLM hashes, emphasizing how easily weak passwords could be cracked.
• Insecure Data Storage: The demo highlighted the shocking discovery of the "important stuff" file on the desktop, containing plaintext credentials. It also showed how an unmanaged web browser could expose stored passwords and sessions, giving attackers access to the accountant's personal email and Office 365 accounts.
• Privilege Escalation: Carson demonstrated the download of A.zip from the C2 server, showing the contents including disable_security.bat and the Sticky Keys backdoor setup. He then ran Mimikatz to enable credential harvesting in memory and dump credentials, showing how the Domain Administrator account used for the daily backup task would eventually be captured.
• Lateral Movement: The demonstration progressed to using RDP+ to log into the Domain Controller, creating a persistent backdoor user in Active Directory. Carson then showcased Network Scanner, revealing its Cyrillic (Russian) interface elements, which were a key indicator of the attackers' origin. He also mentioned the printer redirect mistake by the attackers in RDP+, which exposed an IP address tracing back to occupied Ukraine.

Carson concluded the demo section by sharing his ongoing experiment to AI-automate the entire attack chain. He illustrated how he could prompt an AI to create wordlists and even directly crack hashes, highlighting the increasing automation capabilities available to attackers and the potential for complete end-to-end autonomous attacks. This showcased the future direction of offensive security, where attackers leverage AI to accelerate their operations.

Defensive Implications

▶ Watch: Krylock creators convicted and asset recovery (6:00)

The incident detailed by Joseph Carson provides a comprehensive blueprint for strengthening an organization's defensive posture against sophisticated ransomware attacks. The key takeaway is that resilience and proactive measures are paramount, often more so than simply deploying security products without active management.

1. Robust Credential Management & Multi-Factor Authentication (MFA):

• Enforce Strong Passwords: Implement and strictly enforce password policies that mandate complexity, uniqueness, and length beyond the easily crackable 8-character minimum. Regularly audit password strength.
• Eliminate Password Reuse: Educate users on the dangers of reusing passwords across personal and professional accounts.
• Universal MFA: Deploy MFA for all remote access, sensitive applications, and internal systems, even if it's perceived as inconvenient. This would have prevented the initial RDP access even with a compromised password.
• Credential Rotation: Have a plan for mass credential rotation in the event of a breach.

1. Principle of Least Privilege:

• Remove Local Admin Rights: Never grant local administrator privileges to standard users, especially on workstations. This would have significantly hampered the attacker's ability to deploy Mimikatz or modify system settings.
• Service Account Isolation: Ensure scheduled tasks and services run with the absolute minimum necessary privileges. The use of a Domain Administrator account for a backup task was a critical failure.

1. Secure Backup and Recovery Strategy:

• Offline/Immutable Backups: Implement a 3-2-1 backup strategy (three copies of data, on two different media, with one copy offsite and offline/immutable). Ensure backup systems are isolated from the production network to prevent ransomware from encrypting them.
• Regular Testing: Routinely test backup and recovery procedures to ensure data integrity and operational readiness.
• Disaster Recovery Plan: Develop a comprehensive disaster recovery plan that includes manual fallback procedures and ensures business continuity even if primary systems are compromised.

1. Proactive Incident Response (IR) Planning and Practice:

• Accessible IR Plan: Store IR plans offline, in hard copy, or on an immutable, uncompromised system. Ensure it's accessible during a complete system outage.
• Regular Simulations: Conduct frequent, realistic IR simulations and tabletop exercises to identify gaps in processes, communication, and resource availability.
• Defined Ownership and Communication: Clearly define roles, responsibilities, and decision-making authority within the IR team. Establish alternative, out-of-band communication channels (e.g., dedicated secure messaging apps, burner phones) that are not monitored by attackers.
• "Ready Bag" for Responders: Equip incident responders with essential tools and resources (e.g., clean drives, ear muffs, offline documentation) for rapid deployment.

1. Enhanced Monitoring and Log Analysis:

• Centralized Log Management: Implement a Security Information and Event Management (SIEM) system to centralize logs from all systems (endpoints, network devices, applications, identity providers).
• Active Threat Hunting: Don't just collect logs; actively analyze them for anomalous activities, such as unusual login times, IP addresses (e.g., NordVPN, Tor exit nodes), or tool usage (e.g., GMER, Mimikatz). The initial login seven months prior should have been flagged.
• Endpoint Detection and Response (EDR): While the organization had EDR, it was not actively monitored. Ensure EDR alerts are investigated and acted upon, and that solutions are configured to detect known TTPs like Sticky Keys modification or Mimikatz execution.

1. Secure Software Development & Configuration Management:

• Managed Browsers: Implement group policies or MDM solutions to manage browser security settings, preventing users from saving passwords and enforcing secure configurations.
• Secure Coding Practices: Address vulnerabilities that could lead to data exposure (e.g., storing plaintext passwords on desktops).
• Asset Management and Decommissioning: Maintain an accurate inventory of all assets. Strictly enforce decommissioning procedures to ensure old, vulnerable systems are removed and not left as accidental backups or potential backdoors.

1. Supply Chain and Third-Party Risk Management:

• Vet Hosting Providers: Thoroughly vet third-party service providers (e.g., hosting providers) and ensure they adhere to strict security policies, preventing unauthorized RDP exposure.
• Contractual Obligations: Include clear security and incident response clauses in contracts with third parties.

1. Employee Security Awareness Training:

• Continuous Education: Regularly train employees on phishing awareness, safe browsing habits, the importance of strong passwords, and proper data handling.
• Reporting Suspicious Activity: Empower and encourage employees to report any suspicious activities without fear of reprisal.

By addressing these defensive implications, organizations can significantly reduce their attack surface, improve detection capabilities, and enhance their resilience against ever-evolving ransomware threats.

Key Takeaways

• Ransomware is a Sophisticated, Automated Business: Attackers operate with a business mindset, leveraging Ransomware-as-a-Service (RaaS) models, advanced tooling (Mimikatz, Network Scanner), and increasingly, AI/LLMs for rapid data analysis and attack automation.
• Basic Security Hygiene Remains Critical: The foundation of the attack was built on fundamental weaknesses: weak passwords, insecure credential storage (desktop files, unmanaged browsers), and excessive user privileges (local admin for accountants, domain admin for backup tasks). Addressing these basics is paramount.
• Incident Response Plans Must Be Practiced and Accessible: An IR plan is useless if it's encrypted, inaccessible, or never simulated. Organizations must regularly test their plans, establish alternative communication channels, and ensure key personnel know their roles.
• Resilience and Recovery are as Important as Prevention: While prevention is ideal, the ability to recover is existential. The accidental retention of old, undecommissioned hardware ultimately saved this organization, highlighting the value of diverse, isolated recovery options.
• Active Monitoring and Log Analysis are Non-Negotiable: Deploying security tools (AV, EDR, MFA) is insufficient without a dedicated team to actively monitor alerts, analyze logs, and hunt for persistent threats. The initial access seven months prior was detectable but unacted upon.
• The Human Element is Central to Security and Recovery: Empathy for victims and clear communication are crucial during an incident. Conversely, human errors like weak passwords, storing credentials insecurely, or pressuring IT for shortcuts often create the initial vulnerabilities attackers exploit.

About the Speaker(s)

Joseph Carson is a highly experienced security researcher and the Chief Security Evangelist at Sigura. With over 30 years in the cybersecurity industry, Carson has a deep understanding of both offensive and defensive security strategies. He is known by his hacker aliases "Wiretrap," "Wire," and "Rogue One," often participating in capture-the-flag events from his lab in Estonia. Carson is also the host of the "Security by Default" podcast, which aims to promote the idea that security should be a fundamental, always-on aspect of technology for everyone. His extensive background provides a unique perspective on real-world cyber threats and incident response.