UNIX System Administration Handbook - Evi Nemeth [404]
This is also a good time to remind yourself that some studies have shown that 60% of security incidents involve an insider. Be very careful who you discuss the incident with until you’re sure you have all the facts.
Here’s a quick 9-step plan that may assist you in your time of crisis:
Step 1: Don’t panic. In many cases, a problem isn’t noticed until hours or days after it took place. Another few hours or days won’t affect the outcome. The difference between a panicky response and a rational response will. Many recovery situations are exacerbated by the destruction of important log, state, and tracking information during an initial panic.
Step 2: Decide on an appropriate level of response. No one benefits from an over-hyped security incident. Proceed calmly. Identify the staff and resources that must participate and leave others to assist with the post-mortem after it’s all over.
Step 3: Hoard all available tracking information. Check accounting files and logs. Try to determine where the original breach occurred. Perform a backup of all your systems. Make sure that you physically write-protect backup tapes if you put them in a drive to read them.
Step 4: Assess your degree of exposure. Determine what crucial information (if any) has “left” the company, and devise an appropriate mitigation strategy. Determine the level of future risk.
Step 5: Pull the plug. If necessary and appropriate, disconnect compromised machines from the network. Close known holes and stop the bleeding. The Compromise FAQ from ISS provides some good technical suggestions on what to actually do with the systems that were broken into. It can be found at
http://xforce.iss.net/library/faqs/compromise.php3
Step 6: Devise a recovery plan. With a creative colleague, draw up a recovery plan on nearby whiteboard. This procedure is most effective when performed away from a keyboard. Focus on putting out the fire and minimizing the damage. Avoid assessing blame or creating excitement. In your plan, don’t forget to address the psychological fallout your user community may experience.
Step 7: Communicate the recovery plan. Educate users and management about the effects of the break-in, the potential for future problems, and your preliminary recovery strategy. Be open and honest. Security incidents are part of life in a modern networked environment. They are not a reflection on your ability as a system administrator or on anything else worth being embarrassed about. Openly admitting that you have a problem is 90% of the battle, as long as you can demonstrate that you have a plan to remedy the situation.
Step 8: Implement the recovery plan. You know your systems and networks better than anyone. Follow your plan and your instincts. Speak with a colleague at a similar institution (preferably one who knows you well) to keep yourself on the right track.
Step 9: Report the incident to authorities. If the incident involved outside parties, you should report the matter to CERT. They can be reached by fax at (412) 268-6989 or by email at cert@cert.org. Provide as much information as you can.
A standard form is available from www.cert.org to help jog your memory. Here are some of the more useful pieces of information you might provide.
• The names, hardware types, and OS versions of the compromised machines
• The list of patches that had been applied at the time of the incident
• A list of accounts that are known to have been compromised
• The names and IP addresses of any remote hosts that were involved
• Contact information if you know it for the administrators of remote sites
• Relevant log entries or audit information
If you believe that a previously undocumented software problem may have been involved, you should report the incident to your vendor as well.
1. A number of commercial security auditing tools, such