One of my favorite subjects to discuss and write about is the issue of realistic testing when conducting an exercise. While the subject of this article might seem like a simple play on words, the differences and implications of each statement are quite important to your continuity efforts.
Reflecting back on my 35-plus years in our industry, I have participated directly in, overseen, or audited several hundred continuity exercises. That experience has provided me with a tremendous base of information to draw upon regarding the various approaches to testing and the varying degrees of success people have achieved.
Throughout all of these tests and across all of the various approaches I have seen, two fundamental flaws which have a serious impact on your program’s success are the following:
- Most tests are overly planned and not truly representative of the recovery process they are intending to address.
- Most people never test their actual recovery plan; they only test their test plan!
I’ve written before on the subject of actionable plans – another hot button with me – but I think this issue of realistic exercises is worth a fresh discussion, especially in light of how these two flaws play off of each other.
First, let’s discuss the issue of overly planned tests!
Reflecting on the lack of warning or preparation time in an actual disaster event, let’s look at a typical scenario for testing, and along the way, think about how accurate this is in your own environment.
Many companies start planning their tests a number of months before the actual event is to take place. Some are shorter and some longer, but let’s use three to four months for this discussion.
So, we get the team together and start discussing the test objectives and timeframe. We contact the alternate site, whether that is an internal, colocation site, or vendor site for an IT exercise (or a BC exercise) and schedule our time to ensure we have access to the alternate location on specific dates. As a part of that test, we inform our user community and staff of the dates, making sure people will be available, allowing for vacations, holidays, and any other known conflicts.
Along the way we discuss the test plan, laying out the timeframes, roles, and responsibilities. We tell everyone when and where they need to show up. If travel is involved, we may need to book hotel rooms, flights so we get the best airfares and cars – everything geared toward making the test go as smoothly as possible. See any problems yet?
Next, we determine exactly which data we will need to test the applications. We arrange for required copies of data of an IT-related exercise, back it up several days before the test and send it to the recovery center so the alternate site staff can unpack and inventory them – or we create a copy of our replicated data so we have it ready to go. We certainly don't want missing or corrupt data to impact our test. For our business staff and end users, we make sure all their required equipment is in place and functioning to avoid any delays. Finally, the test date arrives. We may get to the alternate site the day before, get a good night’s sleep, double check our plans, and as a result of all of our planning, we most often have a successful test.
A successful test? It was but it was also an unrealistic view of what will happen in a real world situation, which is exactly how testing should prepare you.
The bottom line is that I believe too many of our tests are orchestrated for the purposes of having a successful test. As a result, we don’t get an accurate view of what would happen in an unplanned, unstructured situation. Planned tests are good when you are just starting out, but do not make that the norm!
If you see this issue play out in your organization, ask yourself if this puts you in a stronger position with your executive management team or a weaker position. A successful test is good news all around, but if a real event should occur then what expectations have you set?
Second, let's discuss the issue of testing the plan!!
Now some of you might really be surprised by this one, but most people do not use their continuity plans to actually run their test. One of the objectives of conducting a test should be to test your actual recovery plan, not your test plan. Remarkably, many companies never actually test the plan itself. How do I know this? By seeing company after company come into recovery centers to test without their recovery plans.
There is one primary reason this situation occurs. If most people really reviewed their plan, they would find it to be a great repository of important information, but that it lacks one critical value. I call this “actionability.”
What do I mean by that? Most plans do not tell you “who does what when?” (at least to the degree required to actually direct the recovery process).
I’'ve written separate articles on this issue, but it is one of the basic reasons most people don’t test their plan. They lack the confidence that the plan can be used to direct an actual recovery.
This specific issue is also a huge problem for companies when they consider automating the execution of their plans. Are the steps really actionable, considering dependencies, staff and skill constraints, etc.? If not, you have a reference document instead of an actionable plan.
While the executive management team may be under the impression that a solid, executable plan is in place, consider the expectations which have been set and the unpleasant surprises that could occur.
How do we fix this issue? Try utilizing the actual plan in the whole process of scheduling, preparing for, and conducting the exercise. Use the declaration process in the plan to notify staff and schedule the test. See if the plan allows for booking hotel rooms, cars, and airline tickets. Validate the plan's timeline against the test results to see if the planned recovery times can actually be met. See if it tells people what to do, when to do it, and has sufficient detail that a non-familiar person could effectively support your recovery effort.
While evaluating the performance of people, processes, and technology during the test, consider the usefulness of the actual plan itself.
The bottom line here is that the more one can involve the actual recovery plan in the exercise process, the more someone will know how effective it will be in a real situation.
Let’s reflect on both issues together!
My purpose in discussing these issues is to create a renewed focus on making sure the recovery exercise process is as realistic and beneficial to an organization as it can be. Pre-planned tests have their place but should not be the only approach that is taken. While test plans might be required to address specific scenarios, one should reflect on the results to see what aspects of the test plan should be incorporated into the actual plan that is intended to use during a crisis.
After all, our objective is to find as many issues during the testing process and fix them before an event occurs. While there is often pressure to have a successful test as the best proxy for demonstrating preparedness, you need to make sure those observing the test understand that problems are best identified before they can have a material impact on the actual recovery from a disaster.
I would like to hear from you as one of my intentions with these articles is to spur discussion and ideas. Please feel free to reach out to me via e-mail at JJ@FusionRM.com
John A. Jackson FBCI Hon., CORE, is an executive advisor for Fusion Risk Management and on the board of directors for Business Continuity Institute (BCI). Jackson has more than 30 years of continuity industry leadership and is widely regarded as an early and current visionary in enterprise business continuity, crisis management, risk management, and disaster recovery. Jackson’s industry experience includes executive management of Comdisco, HP, and IBM’s business continuity organizations. Jackson co-founded Fusion Risk Management in 2005 and today actively contributes to the industry by serving on the Disaster Recovery Journal Executive Advisory Board and the Disaster Resource Guide Editorial Advisory Board.
