Disaster planning and recovery are often viewed as the need for backup systems to safeguard an organization’s data. Here’s a little hint for the survival of your company … it’s not just about the data. The IT and data elements of disaster preparedness are critical components to overall preparedness, but this is the first part of the story.
In today’s technological and global environment, banks and financial institutions – and more importantly, their customers – rely heavily on access to their data 24x7x365. Any interruption of that access can be catastrophic to the organization. Increasingly, this data is not only for internal and customer use, but also for use by business partners and vendors supporting the organization. The availability of this information may literally be the difference between an organization’s survival and demise, and can reduce the disaster’s impact and recovery time.
In today’s information-hungry world, reducing vulnerabilities of data access can impact an organization and customers’ ability to recover, as was documented in New Orleans following Hurricane Katrina. Yet according to a survey conducted by AT&T and the International Association of Emergency Managers (IAEM), nearly one-third of U.S. companies did not have a business continuity plan (BCP) in place. After action reports continue to underscore gaps in preparedness by businesses of all sizes, even with lessons learned from the Sept. 11 terrorist attacks and multiple catastrophic hurricanes, floods, and wildfires.
Large-scale disasters expose the quantity of organizations that are unprepared while demonstrating just how damaging that lack of preparation can be. The tragedies of 9/11, Hurricane Katrina, the earthquakes in Haiti and Japan, Superstorm Sandy, and recently Hurricanes Harvey, Irma, Maria, and Michael have placed business continuity planning and disaster recovery in the spotlight. It is important to note that localized disasters such as a fire and even brief disruptions such as fiber cuts and utility outages, can be equally damaging. Cybersecurity threats are also a growing concern with a simple Internet virus, a worm, or other attack spread through a single laptop can bring an organization’s operation to a grinding halt and cause untold harm to their image and brand. Credit card systems, human resources data, and other critical systems contain the “pot of gold” cybercriminals are seeking, including passwords, account and social security numbers, and other personally identifiable information (PII).
If an organization has been impacted by a major natural or technological disaster, it is more likely they have invested some effort and budget in strengthening their business continuity and disaster recovery postures. However, as a result of the catastrophic disasters of the last two decades, industry standards and regulations such as OCC and Sarbanes-Oxley compliance also act as drivers for implementing better plans, policies, and procedures. These regulations are designed to protect organizations by enforcing the implementation of business continuity and disaster recovery plans.
When updating his company’s continuity and disaster recovery plans, Jason Dobronz, vice president of corporate emergency management and business continuity for the FNB Corporation, turned to guidance provided by the Federal Financial Institutions Examination Council (FFIEC).
“FFIECs Business Continuity Planning Guide provided clear examples of standards and best practices in building a comprehensive business continuity program,” said Dobronz. The guide provides standards on a wide range of issues including business continuity, physical security, information security, and information technology.” Dobronz added, “The guide can provide a solid foundation for a business continuity framework but then requires a collaborative effort across the enterprise to ensure the plans are customized for your organization and how you do business.”
Dobronz’s comments make it clear a plan written to comply with regulations does not ensure the organization will survive. The regulations are imposed as minimum standards. Even the application of national best practices does not mean they are best practices for your organization. Because banks and financial institutions are a part of the nation’s critical infrastructure, these regulations help to ensure the survival of the infrastructure. However, this comes with a caveat. Regulators will often reference the FFIEC manual on business continuity and other standards; but to work in a disaster or crisis, plans must align with the capabilities, resources, and business processes of their organization.
Immediately following a disaster, there is tremendous effort on behalf of the private sector organizations and government agencies that have been directly impacted by the event to develop new plans, implement new policies and procedures, and make investments in technology, equipment, and facilities. However, as time passes without another similar incident, we develop “institutional amnesia” and give less attention and resources to the type of emergency in which we were trying to prepare.
Organizations must remember the consequences of a major disaster can be the same regardless of the causal event. Data and facilities may be damaged or compromised, lives lost, and the organization, its employees, customers, and their families are affected. Planners must use worst-case scenarios to plan and anticipate the consequences caused by any hazard or threat. It is far easier to ratchet down response and recovery activities for a smaller incident than it is to scale up if plans have not been built to be flexible.
During a catastrophic event, many organizations are forced to put their business continuity and IT disaster recovery plans into action. Many more organizations, however, find the lack of a plan or failover capabilities make an already difficult task insurmountable. In a hopeless situation, the presence of an actionable plan can create hope. While this type of safeguard is absolutely a necessity to protect valuable data and reduce the amount of time your organization will need to recover from an incident, this is only part of the solution. A true disaster plan goes far beyond firewalls, backup servers, and drives.
It’s not just about the data – the rest of the story
When disaster strikes, what plan does your organization have in place to communicate internally and externally? How will your organization ensure it can get your personnel to your hot site? An IT disaster recovery plan is only a small part of an overall business continuity plan or holistic emergency management program. The most important aspect for successful disaster response and recovery is preparedness, which must take place far ahead of an incident when the sun is shining and the birds are singing. Preparedness, and more specifically, a planning process which engages all stakeholders and addresses all hazards is far more valuable than the completed plan.
Organizations that have not promoted a culture of preparedness and resiliency typically delegate the writing of their plans to one or two people within their organization. The failure is in the plan itself. After months of writing the company’s disaster plan, they distribute the document to all departments where it sits on bookshelves collecting dust. The next time the plan is looked at is when the disaster strikes. In this scenario, assumptions have not been grounded, there is no buy-in from stakeholders, and the plan is destined to fail. Organizations can avoid this fate and survive by using standards as the planning foundation and then follow four simple steps:
Step 1: Involve everyone.
Disaster planning must involve all stakeholders in the process. Just as data recovery takes into consideration different priorities and timeframes for bringing systems back online, different priorities and timelines exist for bringing services back to a pre-disaster level. Additionally, it is critically important for government, private industry, and community organizations to plan together, as there is a reliance on each other for components of response and recovery. During a widespread disaster such as a hurricane, all stakeholders will be in harsh competition for the same limited supplies and resources, including food, water, computers, and clean-up services. The impact of the disaster may be so widespread that local relocation may not be an option. Transportation (and fuel) options may be limited or nonexistent for days or weeks depending on the disaster. All organizations must work together to survive the event.
Step 2: Plan for the worst-case scenario.
While you cannot plan for every contingency when developing your plan, consider not only your organization’s vulnerabilities, which can include location, security threats, etc., but also your company’s capabilities. Once you have determined the most likely disasters to affect your organization and the impacts on your organization, the plan must be grounded in reality. When you write your plan, consider the capabilities and resources that exist today – not what you are planning toward or will eventually purchase. That is not to say you shouldn’t build remediation steps into the plan. Determine what resources will be needed to respond and recover based on a threat and vulnerability analysis. If they don’t exist, the list will serve as a road map for strategic and budgetary planning, or at least indicate to senior leadership and shareholders what will be required at the time of the disaster. Set expectations in advance of the disaster.
Step 3: Train and exercise the plan.
Even if the plan is developed with the input of all stakeholders, the planning process is still not complete. The plan will only work if every employee (from the receptionist to the CEO) is aware of the plan and how to use it. One of the most critical steps in the planning process is to test the plan. Short of an actual disaster, the easiest and most efficient way to test a plan is through a training exercise.
An exercise serves as the “final exam” at the end of a planning cycle, fosters communication between business units, trains users on the employment of the plan and what is expected from them following a disaster, and provides a “no-fault” environment to identify gaps. The time to find out if the plan will work is not when people are standing in a pile of rubble. Having the plan fail during the exercise is actually a good thing, as long as changes are immediately made, updates are communicated, employees are made aware, and the plan remains a dynamic work in progress.
Step 4: Put it all together (don’t forget the people).
The plan is developed, personnel are trained, and the plan has been tested. Then a disaster strikes and the plan still fails. Some plans have ignored or forgotten the most important aspect of the disaster planning process: All disasters affect people. Disasters leave victims in their wakes, and some of these victims may be the very personnel organizations were counting on for response and recovery. If the plan has not considered employees’ personal and family needs during and after a disaster, they will not be there for the organization when disaster strikes. If there is advanced notice of the disaster, such as a hurricane or blizzard, allow time for your employees to address protection of their families and property. Once all is safe on the home front, they are more likely to be available for their employers. After Hurricane Andrew struck south Florida, firefighters and police in Homestead walked off the job or never showed up for work because their homes were damaged; their lives were in chaos. Once city officials brought in crews to assist in the cleanup and temporary repair of their homes, they felt secure enough to return to duty. Also take into consideration the hardship on a family if you relocate your business operation out of the area. Your plan will only work if every member of your response team is familiar with it and if post-disaster expectations and roles are clearly defined.
Disaster planning is really about the process and less about the technology for disaster recovery. The plan should serve as the framework and general direction to follow, but it cannot take into account every scenario or contingency. There is no way to plan and train for all disasters. However, having a strong core plan with policies and standard operating procedures to guide management and employees during a disaster will ensure your organization’s survival. Remember, plans do not tell your personnel how to do their jobs, but rather how to do their jobs in a compressed timeframe, under stress, and possibly without all of your organization’s resources in place.
Adam Montella is vice president of planning and analysis for the Olson Group, Ltd. Montella has more than 33 years of direct experience in government, private, and non-profit sectors. He is recognized as one of the top disaster planners in the country and possesses a wealth of experience supporting the banking and financial services industry in developing and testing plans and building crisis management programs. Montella has served as the disaster recovery advisor to the New York Stock Exchange, continuity of operations manager for the U.S. House of Representatives, and was the on-camera disaster expert for the Discovery Channel series, “The Colony.” Montella has also served on the Federal Emergency Management Agency’s National Advisory Council. He holds a master’s degree in public administration and more than 40 certifications in business continuity, disaster recovery, emergency management, and homeland security. Montella can be reached at [email protected]