Incident

Start with Creating a Computer Security Incident Response Team: A Process for Getting Started. Review DataLossDB, the research project aimed at documenting known and reported data loss incidents world-wide. You want an Incident Response procedure and an Incident Response team created and tested before you need it. Members of the team should have copies of the procedure, and Lenny Zeltser’s cheat sheets. Test your plan regularly. Learn the gaps and revise. Search for “Incident Response Plan” and you will find many examples of published Incident Response Plans. See ENISA’s Support For CERTs/CIRTs for examples and practice exercises.

However, which came first: the incident or the declaration of the incident? The incident. If an incident occurs in a forest, and there’s no one there to notice it, does it still occur? Yes.

From the U.S. Army Information Assurance Security Officer (IASO) Training

Signs of an incident fall into one of two categories: indications and precursors. A precursor is a sign that an incident may occur in the future. An indication is a sign that an incident may have occurred or may be occurring now. Too many types of indications exist to exhaustively list them, but some examples are listed below:

• The network intrusion detection sensor alerts when a buffer overflow attempt occurs against an FTP server.
• The antivirus software alerts when it detects that a host is infected with a worm.
• The Web server crashes.
• Users complain of slow access to hosts on the Internet.
• The system administrator sees a filename with unusual characters.
• The user calls the help desk to report a threatening e-mail message.
• The host records an auditing configuration change in its log.
• The application logs multiple failed login attempts from an unfamiliar remote system.
• The e-mail administrator sees a large number of bounced e-mails with suspicious content.
• The network administrator notices an unusual deviation from typical network traffic flows.

One should not think of incident detection as being strictly reactive. In some cases, the organization can detect activities that are likely to precede an incident. For example, a network IDS sensor may record unusual port scan activity targeted at a group of hosts, which occurs shortly before a DoS attack is launched against one of the same hosts. The intrusion detection alerts regarding the scanning activity serve as a precursor of the subsequent DoS incident. Other examples of precursors are as follows:

• Web server log entries that show the usage of a Web vulnerability scanner
• An announcement of a new exploit that targets a vulnerability of the organization’s mail server
• Information stating that the Unit will receive a cyber attack

Not every attack can be detected through precursors. Some attacks have no precursors, whereas other attacks generate precursors that the organization fails to detect. If precursors are detected, the organization may have an opportunity to prevent the incident by altering its security posture through automated or manual means to save a target from attack. In the most serious cases, the organization may decide to act as if an incident is already occurring, so that the risk is mitigated quickly. At a minimum, the organization can monitor certain activity more closely—perhaps connection attempts to a particular host or a certain type of network traffic.

A Mandiant presentation at Black Hat in 2006 reported that organizations detected attacks:

• Sometimes through anti-virus alerts, although most alerts are uninvestigated. (Note that this is my point as well; investigation of anti-virus alerts yields results.)
• Sometimes through an Intrusion Detection System (IDS), although attacks through SSH, HTTPS and VPN escape IDS detection. (Note that with a largely mobile or remote workforce, relying upon VPN and public network availability, IDS does not involved.)
• More often through client reports, outside the company.
• Sometimes through end user problem reports.

End user problem report symptoms that may be indicators of an incident:

• Continual termination of anti-virus software.
• Installing new applications does not work.
• Commonly used applications will not run.
• You cannot “Save As”.
• Task Manager closes immediately after you open it.

You do not know if an incident should be declared until you are well into an investigation (the Identification stage of Incident Response). An unexpected restart in an event log might be an incident. A port scan might be an incident. You won’t know if the incident response procedure should be invoked until you have learned more about the specific restart or more about the specific port scan. (In the military? Unexpected restarts and port scans are ALWAYS incidents. Report them.)

Many sources will help make the determination that an incident has occurred. Get LennyZeltser’s Security Incident Survey Cheat Sheet for Server Administrators now, and be comfortable with it. Read John D. Howard & Thomas A. Longstaff’s A Common Language for Computer Security Incidents.

Once you have determined that an incident has occurred, your (previously established and tested) incident response procedure must be invoked.

Grab a bound notebook (not spiral bound, you don’t want to be accused of tearing out pages), preferably with numbered pages and write down the events which have occurred so far. Use details. Imagine that you will be reading this seven years from now and need to reconstruct the events. That is, mentally rewinding and replaying the steps prior to declaring an incident needs to be the first task of your incident response procedure. This is particularly problematic if a criminal investigation may be involved. When that is the case, the forensic process, including establishing a chain of custody, must be followed. There will be steps you wish you had done differently prior to the declaration of an incident. That happens; that’s prudent. Once you recognize an incident, however, your actions must adapt.

The steps of incident response. Most lists include only the PICERL steps. The full team events of notification and post-incident review should also be included.

1. Preparation
2. Identification or Assessment
3. Notification
4. Containment
5. Eradication or Remediation
6. Recovery
7. Post-Incident Review
8. Lessons Learned (or Follow Up)

Determining if an incident should be declared is part of the Identification step in incident response.

Note that this list assumes that preparation and declaration of an incident are not within the scope of the incident response. Once a specific incident has been declared, the specific incident life cycle begins. General preparation is outside the scope of specific response.

Regarding Notification: there’s internal notification and external notification. Internal notification is to the Incident Response team. A member of the Incident Response team should be responsible for external notification. There are incidents which require external notification, which is why a lawyer is on the Incident Response team. They may not be the person who makes the notification, but they would review the incident for notification requirements.

If you find yourself in the extremely unusual situation of knowing that a notification is required but is not being done, you may believe that your choice is between making the notification (and losing your job) or not making the notification. Get legal advice. Consider the long term possibilities. Possible outcome if you make the notification: lose job. Possible outcome if you choose to not make the notification: lose job, face prosecution, lose savings, serve time, and become unemployable.

That notebook … is it for all incidents, or only security incidents? A burst pipe in the building or a natural disaster are also incidents. Without the notebook, Lessons Learned becomes difficult.

Test: City of Norfolk hit with code that takes out nearly 800 PCs. What lessons do you take away?

Cluff’s team noticed that computers were taking longer than normal to shut down around 4:30 p.m. on Feb. 9. Those machines could not then be restarted. After investigating, his team discovered that a virtual print server was pushing out malicious code. The team pulled the virtual server offline, scrubbed it and reverted it to a previous instance of the print server software, he said.

Regrets: Did not preserve the virtual print server for further investigation. How did this happen? What could have prevented this? Evidence is lost.

Online Trust Alliance Data Breach and Incident Readiness Planning Guide

Privacy Rights  Clearinghouse including actual reported breaches

Mozilla InvestiGator (MIG) is a platform to perform investigative surgery on remote endpoints. It enables investigators to obtain information from large numbers of systems in parallel, thus accelerating investigation of incidents and day-to-day operations security. Masche adds memory forensics capabilities to MIG.

On TTPs