Your Monthly Security Report Is Worthless

I am doing my best to keep up my blogging momentum. It’s been easier for me to share experiences than to tutorialize tactics, so I present to you this tidbit…

This post is directed at anyone who works in security and is responsible for communicating security information to business leadership, board members, and / or anyone (for that matter) who gives a damn.

It is quite common that I meet with a client and they want to communicate the health (if you will) of their security program. Typically, by handing me a report that they regularly draft and distribute or present to upper management or other members of the board (if I am working directly with an officer). Some reports are very pretty. They have charts, graphs, and a nice layout. Others… well… let’s not focus on what they look like… yet. Let’s talk about what they tell us. In most cases… NOTHING.

But I made it visual! It is easy to follow!

Yeah… with what? your vast array of Crayolas? In many cases, I come across reports that are laden with screenshots from any number of security tool dashboards, provide executive summaries that highlight a bunch of fluff, and that don’t accurately measure the success of the overall program.

Three Principles

Keeping it short and sweet… if you want a report that has any value then follow these three principles.

  • The outline of the report should highlight your security program(s)
  • Within each program you must outline the objectives of that specific program (otherwise, the reader doesn’t have the slightest idea what you are trying to tell them)
  • Following the objectives, you need metrics and trends that provide actionable intel to your audience.

Metrics and Trends

This is fundamentally very easy, but often overlooked. Metrics are a measurement of something.  A trend is the relationship between a metric and time. Your metrics will demonstrate objective success vs failure, and your trends will demonstrate improvement or the lack thereof. All of this is vital information for program changes, objective changes, budgeting, staffing, etc.

What do I measure?

Simple. You measure how you are meeting the objectives of your program. If one of your objectives is to have a 90 minute time to respond, then your metrics would reflect that by showing the number of incidents vs time to respond. (To elaborate on that information you would want to include the types of incidents [better communicates where you are succeeding / struggling]). The trend would demonstrate the improvement or decline in meeting that objective over 30 days, within a year, time between reports (whatever allowable threshold is decided by leadership).  You get the point.

Pretty Pictures DO Matter

Data visualization is very important. You can’t effectively communicate the information mentioned above in tabular format. You have to create some graphs and pretty charts that someone can quickly look at to evaluate the effectiveness of the program. This is key. Business leaders are always having information thrown at them. To prevent cross contamination, we give them a straight answer in a language that anyone can understand. Pictures.

Write Your Executive Summary Last

The heading says it all. It’s a summary of the report. Therefore, it should be the last thing that you write. At a high level include details of notable events (incidents, program changes, staffing changes, etc). Again… this is all valuable and this is the spot where you can write the “skinny” that may fill in some of the gaps that better explain your data. The majority of folks will tell you to keep an executive summary to a maximum length of one page. This is not always possible if something serious has happened. Aim to keep it one page. No more than two. If there are notable incidents that have occurred during the month you should have plenty of supporting documentation. Append it to the report, reference it in the summary, and make sure that it is clearly represented in the table of contents.

Ref:

Security Metrics: Replacing Fear, Uncertainty, and Doubt (2007 – Addison-Wesley Professional) Andrew Jaquith

Creating a Monthly Information Security Scorecard for CIO and CFO (2010 – SANS) Michael Hoehl

Leave a comment

Leave A Reply