Security: A Top Five Trend in Analytics for 2015


The Institute for Operations Research and the Management Sciences (INFORMS) recently named security one of the top five trends in analytics for 2015.

As stated in the INFORMS announcement:

With the number of security breaches on major corporations being reported almost weekly, such as at Target, The Home Depot and more recently at Sony Pictures, there will be a significant increase in investment across the board in safeguarding commerce and privacy on the Internet. The importance of applying analytics methods – from using decision analysis to guide investment choices, to statistical methods, to detect-and-anticipate breaches and optimization models, to improve infrastructure design for safety, reliability and performance – will accelerate and continue to grow in 2015.

As we wrote in a recent post, situational intelligence provides a crucial, comprehensive platform for use in preventing, preparing for, detecting and responding to physical and cyber security threats.



Situational Intelligence and Security


To be continually vigilant, asset-intensive organizations need to prepare for and prevent the possibility of physical and cyber attacks, and then quickly and accurately detect and respond to any attacks that do occur. Preparation for, prevention and detection of, and response to attacks each introduces degrees of complexity to the security challenge. The typical utility has many siloed systems independently evaluating security measures, making it difficult to connect isolated but possibly related events and making the security challenge even more complex.  As a result, potentially dangerous situations may be ignored while investigation into false alarms consumes time and money

Preparing for and preventing attacks

When preparing for attacks a utility should assess both the likelihood of attack and the consequences of an attack on each asset. The probability of attack on organizational assets often comes down to motive and location.

  • Are physical systems and assets safe from authorized field workers, contractors, and ex-employees?
  • Are cyber systems and assets safe from third party and external threats such as denial-of-service (DoS) attackers?
  • Is a physical asset easily accessible to those wishing to cause damage or disruption?
  • Is a cyber asset – such as a computer, control system or communication network component – indicating unexpected connections, failed logins or uncharacteristically extended response times?

Organizations also need to assess the consequences should an asset be attacked. An asset might be very vulnerable to attack, but the consequence of losing that asset might be negligible, reducing the priority of safeguarding that particular asset over other, more important assets.

By correlating data from disparate assets across the organization to calculate the probability and consequence of attack, situational intelligence solutions provide an accurate ranking of security risks that can be proactively addressed.

Detecting attacks

The number of assets at large organizations can run from the tens of thousands to the tens of millions. Such scope means a huge area to patrol and lots of data to protect. To separate actual attacks from malfunctions or flukes, it is useful to correlate multiple data points that occur close to each other in both space and time.

In a large scale event, dozens of alarms may trigger at once. For instance, if an entire building is somehow damaged, all alarm-equipped assets in that building will send out signals. Instead of receiving dozens of individual notifications, operators would benefit from a system that correlates those individual items in real-time into a single, larger, more meaningful alert.

Using situational intelligence solutions, data and alarms from multiple, disparate sources can be correlated and presented to users in a single view, drawing attention to anomalous conditions and facilitating fast, informed decision-making.

Responding to attacks

Your situational intelligence system has detected an attack—now what? First, you need to understand exactly what has happened. Because situational intelligence correlates data across the dimensions of space, time and network node, operators can quickly close in on the root cause of an event. They can also see at a glance the network impact upstream and downstream of the event.

Next, you need to know who should be notified, which repair crews should be dispatched where, which first responders to contact, and what reports need to be filed. The period immediately following an attack is critical for controlling damage, preventing injury, collecting evidence and apprehending suspects.

Once an attack has been resolved, it’s good to review process and procedures, to improve security and prevention and to better prepare for the next possible attack. Situational intelligence systems can capture spatial-temporal-nodal information for later analysis. This helps operators, administrators and investigators study, assess and revise responses to attacks.

For more information about situational intelligence and security, see this white paper.


Can Situational Intelligence Help Address Data Quality Issues?


Organizations often wait until they believe they have complete and perfect data before undertaking data analytics projects. This hesitation is understandable. No one wants to waste effort analyzing the wrong data. But the day of complete and perfect data may never arrive.

On the other hand, a situational intelligence solution can actually improve data quality.

By visualizing data in its spatial, temporal and network context, situational intelligence solutions quickly highlight data problems that might otherwise be hard to spot among millions of rows of data. Likewise, by correlating data from multiple siloed systems into a single display, situational intelligence solutions quickly highlight inconsistent data across your organization.

For instance, if you visualize all your utility’s substations and feeders, you can easily see if your data has incorrect relationships between substations and feeders. This would be laborious to discern by crawling through tables of data to double check associations. Having the correct asset relationships in your data is critical to analyzing network impacts related to failing equipment, crisis situations, capital expenditure planning and more.

Situational intelligence solutions also speed the process of colleagues working together to fix data quality issues, by bringing multiple sources of data together into single, unified display. Continuing with the substation and feeder example, having colleagues from operations, GIS, asset planning and other departments all working with a single display is a faster way to resolve bad or missing network model data compared with multiple people trying to manually correlate disparate data to pinpoint the problem.

Waiting for perfect data postpones getting value from the data that you currently have and perpetuates existing data quality issues. Beginning to implement situational intelligence solutions helps you realize value from your data today, while also improving the quality of your data for tomorrow.


ISO 55000 and Situational Intelligence


A recent Utility Analytics article on ISO 55000 explores the connection between analytics and asset management standards.

The ISO 55000 standard guides organizations such as utilities in putting “a risk management process in place that will enable utilities to assess the likelihood of asset failure, and determine those assets whose failure would produce the greatest consequence.”

We have written about this two-sided nature of risk—probability and consequence—and how situational intelligence constantly analyzes the fluctuation and interplay between these two forces.

Standards like ISO 55000 are more descriptive than prescriptive, outlining the general steps organizations should take, but leaving room for adapting those steps to each organization’s unique circumstances.

ISO 55000

Asset management best practices outlined by the ISO 55000 standard

Adaption leaves room for each organization to blend the six domains of situational intelligence to suit their needs and goals in implementing the standard.

As the article notes, ISO 55000 is very new, having been ratified in January 2014. Companies in North America are using the standard as a guide to best practices, while in other regions such as the Middle East and parts of Asia, companies are pursuing certification of their ISO 55000 implementation. Either way, situational intelligence can be a useful tool to bring the rigor of ISO 55000 to your asset management practices.


Data Scientist: Job of the Year


As the Wall St. Journal wrote in a recent article, “People call them unicorns.” Data scientists have a rare combination of in-demand skills that make them crucial players in development and implementing situational intelligence.

Building mathematical models that analyze data across spatial, temporal and network dimensions requires knowledge of statistics, computer science, business and communications. A Harvard University blog post describes the data scientist role as performing five essential functions:

  • Collect and clean data
  • Manage data
  • Explore data to create hypotheses
  • Make predictions based on the data
  • Communicate insights gained from the data

Workers with current and relevant skills to perform all five of these areas are hard to find, and thus are commanding a premium in the labor marketplace.

A McKinsey report says that, “By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills typical of a data scientist, as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions.”

Excellent user experience design in situational intelligence solutions can mask some of the complexity of analytics and help managers and analysts make effective, data-driven decisions. That might reduce the need for data scientists among end users, but such design requires data scientists to help create the user experience in the first place.

All this means the demand for data scientists will likely continue, making it the job of the year for years to come.

Given the high demand for data scientists, should you hire your own, or employ consultants?

You might want to hire your own data scientists if

  • You are building your own situational intelligence solutions from scratch.
  • You are buying a situational intelligence platform and need to customize it.
  • You want to do your own data integration and analysis from existing, siloed IT, operations and external data sources.

On the other hand, you might want to employ consultants if

  • You are implementing a situational intelligence solution from a vendor who will configure your analytics and visualizations.
  • Your near- and mid-term analytics requirements are not extensive, such as you are just starting work on proof-of-concept projects.

Visualization: The Power of Lassos


Maps and diagrams offer an intuitive and compelling way to organize information.  Maps of countries, diagrams of city transit systems, floor plans of buildings, schematics of electrical systems and more convey large amounts of information effectively and in context.

Often, a whole map or diagram provides far more information than we want or need. And the categories of information on the map may not suit our needs. To benefit from situational intelligence, you want to perform analysis in real-time according to spatial and network relationships, not preset and arbitrary designations.

For instance, in working with a GIS system, you may need to compile demographic information for a neighborhood, only to find that the map is organized instead by postal codes.

The ability to select arbitrary polygons of area on a map or diagram lets you filter and analyze only the data that interests you. This ability, often called lassoing or rubberbanding, gives you an easy way to convert big data to little data by screening out data that’s unnecessary for your analysis.

Lassos can be quite sophisticated. For instance, a situational intelligence solution may let you create multiple lassos and then perform analysis on the data represented in the areas either included in or excluded by the lassos. You could also do the same with the intersection of multiple lassos. Saving lassos and sharing them with colleagues speeds up analysis and collaboration.

Don’t be limited by the built-in categories in maps and diagrams. Lasso your own insight.


Is Visualization Just a Commodity?


We now have GIS-quality maps in our browsers, on our phones, and on the dashboard screens of our cars. Infographics arrive daily via email, Twitter, LinkedIn, and Facebook. We make charts instantly in Microsoft Office, and free images abound on the web. Is visualization becoming a commodity, or does it play an important role in understanding and acting on enterprise data?

Representing data visually is arguably as old as writing itself. It makes trends, patterns and outliers immediately and intuitively apparent in a way that lists and tables often can’t. It remains the best, and really only, tool for bringing together spatial, temporal and node information, a requirement for situational intelligence. It provides easier ways to interact with large amounts of data from multiple sources, helping to make big data into little data.

All of this is to say that visualization leads to visual analytics that improve, simplify and speed data-driven decision making.

For all these reasons, visualization cannot be a commodity. It matters how decisions are made, how users interact with data, how trends and patterns and outliers are shown. When techniques and processes matter, then the related tools cannot be commodities, unspecialized and widely distributed.

Ultimately, we’re not concerned with simply seeing large amounts of data. We need insight about the data that we can act upon.


Situational Intelligence and Regulatory Compliance in Great Britain


In 2010, the Office of Gas and Electricity Markets for England and Wales (Ofgem) adopted a revised utility regulatory framework that will tie a utility’s revenues to incentives for delivering innovation and outputs. The framework soon took on the abbreviation RIIO. The main aim of RIIO is to motivate utilities to deliver total value for the money they collected and to pursue innovation such as integrating low carbon and renewable energy sources into their generation portfolios. Under RIIO, companies could realize additional revenues and enjoy financial bonuses for meeting or exceeding output goals, and for working with greater efficiency. They could also face penalties for falling short of goals or being less efficient. Could situational intelligence provide tools for complying with, and possibly benefiting from, this new framework?

The new RIIO framework immediately raised several difficult-to-answer questions with high stakes:

  • How can I define, track, and improve efficiency?
  • What are my best ways for integrating low carbon and renewable energy sources?
  • Which investments in innovative technologies will yield the greatest return?
  • What is my risk for penalties?

Fortunately, these are the types of questions at which solutions based on situational intelligence excels. By correlating, analyzing, and visualizing data from multiple silos, situational intelligence allows collaboration across the utility for identifying and implementing answers to such broad but important questions. The important difference is that situational intelligence enables these regulated companies to constantly analyze and visualize their progress and “operationalize” their performance monitoring – rather than expend significant manual effort to produce snapshots of the situation to support intermediate reporting or future negotiations with Ofgem.

In particular, situational intelligence excels at pinpointing asset risk. From a thorough understanding of risk, utilities can predict and proactively prevent future asset failures, which leads to key actionable insights in safety, reliability and customer satisfaction. Understanding risk also supports modeling capital investments to realize the greatest operational improvement or risk reduction for the money invested.

As Electricity North West alluded to in a recent SmartGrid GB meeting, situational intelligence also supports new ways to identify additional capacity from the existing network, by modeling the effects of new commercial conditions with large energy consumers in times of network stress or outage. Given the dynamic nature of commercial agreements, with commercial users continually coming on and off the grid, implementing solutions based on situational intelligence could deliver significant cost savings and risk reduction benefits.

A recent blog post from analysts at DNV GL suggests that utilities that can deliver outputs and innovations under the RIIO framework can receive “a steady stream of revenues, and a steady, predictable return on capital.” Those utilities that exceed goals for outputs and innovations could earn “significantly greater revenues.” That sounds like a strong incentive to pursue situational intelligence solutions.

For more information about situational intelligence and RIIO regulatory compliance, see this white paper.


What is the Ultimate Goal of Big Data?


Here’s an illuminating excerpt from a recent Utility Analytics interview with Carol Stimmel, author of Big Data Analytics Strategies for the Smart Grid.

Utility Analytics: What is the ultimate goal of big data?

Stimmel: Utility big data analytics is all about revealing insights; we want to use data analytics to predict, explain, and find hidden opportunities, not just to have more glorified dashboards. What we are really looking for is real world situational awareness, and we don’t always know what that will look like and we can’t always guess, even with years of industry experience.

We couldn’t agree more (except maybe by changing “situational awareness” to “situational intelligence“). Dashboards have their place for summarizing historical data or offering a snapshot of current conditions but actionable insight and prediction comes from intuitive and interactive visualizations of deep analysis.


Converting Big Data to Little Data for Big Payoff


The secret to benefiting from Big Data lies not in accessing all the data, but in identifying, analyzing and acting on the right subset of data.

The primary goal of situational intelligence is to simplify access to high volumes of heterogeneous data and transform it to actionable information. With situational intelligence, users have the flexibility to hide or display the data they want to see on-the-fly, view performance over time, get a birds-eye view of a situation and drill-down to the details of specific assets to troubleshoot root causes, interface to related documents and applications to follow defined procedures, examine diagrams and asset documentation, or take corrective actions.

For instance, it is difficult to identify infrequent errors or combinations of factors within millions of data records by using traditional display formats such as tables and charts. However, the combination of geospatial visualizations, temporal displays and anomaly detection models can alert users immediately to the fact that a problem occurred and pinpoint precisely where and when it happened (and might happen again).

How would this work in the real world? Consider the 2+ million vehicle fleet of the United States Postal Service and the associated need for tires. Simply tracking all those vehicles simultaneously on a GIS application, waiting to identify vehicles with tire problems, is pointless and inefficient. Similarly, searching through a tabular report on all 2+ million vehicles looking for evidence of old or risky tires is so overwhelming as to be useless.

What if a USPS district purchasing manager wanted to avoid towing bills and downtime from tire failures by building a forward-looking budget for purchasing new tires for the coming 12 months?

Correlating and analyzing data on the types of vehicles, dates of tire purchases, miles traveled per month, typical weather and road conditions or other location information can give more precise information about current tire wear. This combination of data can identify and rank the vehicles that will potentially need new tires.

Analyzing this information with contracts and price lists from tire suppliers and budget projections for the coming year, the district purchasing manager can easily make data-driven decisions about which few vehicles out of thousands in her fleet require new tires. Plotting those vehicles and their routes on a map shows which delivery areas will be affected by tire replacement, and how many substitute vehicles might be needed to cover for those out of service for tire replacement. That keeps mail arriving on time and might even postpone the next price increase for stamps.