Why you need a plan in advance, not during the attack
At the moment of a breach, an administrator loses a third of their effectiveness to stress and makes decisions that destroy evidence: rebooting the server, deleting suspicious files, changing passwords before the logs are analyzed. An incident response plan is a sequence of actions written in advance for the first hour, when decisions are made fast but with no room for error.
The goal of the first 60 minutes is not to investigate the incident fully but to stop further damage and preserve everything the investigation and recovery will need. A full investigation takes days, but the decision to isolate the server must be made in minutes.
Minute 0-10: detection and isolation
The first step is to confirm this is really an incident, not a monitoring false positive. Once confirmed:
- Isolate the server at the network level: disable the public interface through the provider's panel or a firewall rule, but do not power off the server itself — RAM holds traces of the attacker's process.
- Do not delete or overwrite files — any change destroys evidence needed for later analysis.
- Record the exact detection time and first observations in text — an hour later, details fade from memory.
iptables -I INPUT 1 -j DROP
iptables -I OUTPUT 1 -j DROP
These two rules block all new traffic while keeping existing sessions alive for later memory analysis.
Minute 10-25: collecting initial evidence
While the server is isolated but still powered on, capture state snapshots for later analysis:
ps auxf > /mnt/evidence/ps.txt
netstat -tulpn > /mnt/evidence/netstat.txt
who -a > /mnt/evidence/who.txt
last -50 > /mnt/evidence/last.txt
cp /var/log/auth.log /mnt/evidence/
Copy evidence to an externally mounted volume, not the same disk — the attacker may have left a log-cleanup script that triggers on reboot. If the organization runs Wazuh SIEM or auditd, export events from the last 24-48 hours as a separate archive before powering the server down.
Minute 25-40: assessing the scope
Determine exactly what was affected:
| Question | Where to look for the answer |
|---|---|
| Which accounts are compromised | the auth.log journal, command history, new SSH keys in authorized_keys |
| What data may have been copied | outbound traffic in netstat, web server logs from the last hours |
| Is there attacker persistence | crontab, systemd timers, new systemd units, autostart entries |
| Are other servers affected | outbound SSH sessions, shared accounts, a shared database |
Do not rush to conclusions: a sign of compromise on one server often means the whole network needs checking, including backups made after the breach.
Minute 40-60: notification and first decisions
Report what happened using a contact list prepared in advance: the technical lead, the business owner, and, for a personal data leak, the data protection officer. A short notification template:
Incident: [date, time detected]
Affected system: [host/service]
Status: isolated / under investigation
Preliminary scope: [what is known so far]
Next step: [for example, restore from backup N-1]
At this stage, decide on the next path: restoring from a clean backup, bringing in an external incident response specialist, notifying a regulator about a personal data leak. The detailed cleanup process is described in the article hacked site: diagnosis and recovery.
First-hour checklist
- The server is isolated at the network level but not powered off — memory is preserved for analysis.
- Evidence is copied to an external volume before any changes are made on the server.
- The exact time and first observations are recorded.
- The scope of the incident is assessed at least preliminarily: affected accounts, data, other servers.
- Responsible people are notified using a contact list known in advance.
After the first hour, move on to a full investigation: use osquery for inventory and investigation to systematically compare the server's state against a known baseline.