Operational Analytics, Business Intelligence and The Internet of Things


IoT 01

The Internet of Things (IoT) is rapidly changing the way business operations are monitored and managed. Connected devices detect and communicate the status of essentially any aspect of manufacturing, warehousing and distribution. Many of these same devices are also able to receive commands such as to open or close a switch or valve. As this digital transformation pervades throughout operations the speed at which adjustments and corrections can be made to improve processes, throughput and cost efficiency is becoming faster.

The increased speed of process throughput and improvement now exceeds the capabilities of traditional Business Intelligence (BI) systems offering “descriptive analytics” that are inherently retrospective. The traditional BI modus operandi was to review the output from analyses and then take corrective measures. The cycle time typically spanned days to more than a month. Nowadays with IoT, the cycle time is reduced to mere hours, minutes or even seconds.

This sea change poses challenges for BI solutions that were not designed for fast cycle times, much less immediate real-time processing of streaming data. Just about every operation today is awash in data and crunched for time.

The data problem will continue to pose ever greater challenges because:

  • The Internet of Things is expanding, which means that smart sensors will soon be almost everywhere, creating additional streams of continuous data.
  • New technology will measure data at ever finer intervals, such as synchrophasors used in the transmission and distribution of electricity that measure voltage up to 30 times a second
  • Lean operational processes, such as Kanban and flow, improve operations and just-in-time production and inventory, and generate large volumes of data in the process.
  • Digital customer service increases the number of touch points between customers and vendors, generating still more data.

For all this data to make an immediate impact on your operations, you need to be able to capture it, normalize it, and in many cases analyze it immediately.

This is where traditional (BI) solutions fall down. BI was not and is not designed for real-time analytics of large volumes of high-velocity data. It enables users to ask questions by querying their data, but leaves it to the user to convert the data-out responses to usable and actionable answers and then decide how to apply them. More specifically, BI systems were originally designed for producing data and reports, organized and visualized in presentable formats (e.g., tables, graphs). This was and still is a very useful and valuable, but it’s not the same as enabling ongoing and in many cases real-time operational process management.

To take a data-driven approach to improving operational efficiency, what you need is a more comprehensive analytics approach that integrates and analyzes multiple sources of data both in batches and in real-time to deliver insights that you can act on immediately to drive and/or fully automate business operations.

The need for a more comprehensive solution that transcends the now limiting capabilities of BI systems has been met by a new category of enterprise software solutions referred to as “situational intelligence” (SI). Situational intelligence is a superset of BI capabilities that adds analysis of operational systems with purpose-built advanced analytics that can consume any type of data: internal, external, structured, unstructured, big, streaming and more.

With access to all this data and an understanding of its contribution to the big picture, situational intelligence illuminates the what, where, when, why and how of every asset and situation to provide context needed to make fast and confident business decisions that lead to optimal actions and outcomes.

I strongly recommend that organizations not only adopt and operationalize advanced analytics, but do so within the context of SI solutions to thrive and survive as the IoT transformation continues to unfold.

That’s a bold statement, I know. In coming posts I’ll discuss specific use cases to show how situational intelligence optimizes operations, helps handle uncertainties that arise, and detects and corrects anomalies as they occur.


Platforms Make Visual Analytics More Accessible


Analytics software and the computing power to run it are becoming increasingly affordable, yet organizations thus far have not availed themselves to analytics. A recent Forrester Research report states that 88% of organizational data is not being analyzed. The lack of analysis for data coming from the Internet of Things (IoT) is more acute – according to a 2015 IoT report from McKinsey & Co., only 1% of IoT data is used for any actionable purpose.

Simply put, there’s a lot of opportunity out there for analytics software. What’s the best way to bring analytics into your organization?

There are generally two ways to implement software solutions. The first way is to identify a specific need and then develop a specific product, or so-called “point solution”, that addresses and solves that need. The second way is to develop a platform that provides resources to run multiple solutions. Once a platform is available in the marketplace, both the platform vendor and a community of developers can create and market solutions.

Solutions that run on a platform possess the following advantages over point solutions:

  • With a platform, innovation happens faster and for a larger number of consumers and industries, especially if the platform achieves strong support from a community of developers who can expand their business opportunities.
  • Organizations that use a platform can experiment in ways that a solution or a variant of a solution can be used for another purpose. Once a technology solution is operationalized and people begin to realize its benefits, end users naturally begin to form ideas about additional features and capabilities (just ask any product manager, who is the typical recipient of feature wish lists). Advanced analytics with its predictive capabilities is particularly susceptible to this type of wish list explosion. After all, if an analytics solution can predict which assets are likely to have the shortest or  longest remaining useful life, it should be able to predict other likelihoods too. Because of its relative ease in customizing and extending solutions, a platform makes it possible to clone an application and alter it to support a similar but different use case.

Because there’s so much opportunity out there for analytics, a platform adds greater value to your organization. A recent survey and report from Salesforce states that high performing organizations are 3 times more likely that others to be deriving value from analytics in more than 10 different use cases.  One platform supporting 10 different use cases is much more cost effective and efficient than acquiring or developing, and then maintaining, 10 different point solutions.

Making analytics more accessible through platforms seems like the fastest way to start tapping into the 88 to 99 percent of data that is not being analyzed.



Democratizing. Operationalizing. Institutionalizing. Systematizing.


Every person in your organization comes from a different background, different training and different perspective. Without some sort of social and operational glue, chaos could reign when these different people work together on a common task. This is especially true when working on analytics.

The glue that holds analytics organizations together has four components:

  • Democratizing means the ability to spread the use and benefits of a solution throughout the organization. It is a way to remove silos and other barriers, or at least bridge them and foster more collaboration and correspondingly more productivity.
  • Operationalizing means the ability to integrate a new tool, skill or approach into organizational processes. Successfully operationalizing something necessitates communicating its rationale and business value, making it easily available and training personnel in how to effectively employ it.
  • Institutionalizing means the ability to enshrine a consistent task, process, policy and other approach as “the way we get things done in our organization.”
  • Systematizing is similar to institutionalizing. It means having consistent and systematic processes and methods of conducting business; in other words, a collection of related, institutionalized approaches.

Situational intelligence lends itself to democratizing, operationalizing, institutionalizing and systematizing in  some important ways. Intuitive visualization of analytics results make them more accessible to more people, smoothing the way to democratic use and enshrinement as “the way we get things done.” Because spatial-temporal-nodal analytics delivers relevant context, a broad range of workers from the back office to the field can participate in and benefit from analytics. The focus on users taking action based on analytics (rather than on users performing analytics) means situational intelligence moves quickly into operations.

When all the different line workers, field workers, knowledge workers, managers and executives in your organization directly see and benefit from new and improved analytics processes, those processes are more likely to become institutional, systematic and part of your organizational culture. Interpersonal and interdepartmental collaboration rises, and everyone enjoys getting things done efficiently, optimally, anytime and anywhere.


Water, Capital Efficiency and Situational Intelligence


Modern urban wastewater treatment plant.

In a previous post, I discussed the counter-intuitive notion that water utilities can gain more value from their tight budgets by increasing energy efficiency. A more straightforward approach to freeing up money is improving capital efficiency, of which situational intelligence plays a part.

Capital efficiency is much like energy efficiency—getting more results or value from the same resources. Capital efficiency questions a water utility might ask include

  • Where in my distribution network can I get the most leak reduction for the least amount of construction cost?
  • Considering demographics, land prices, construction budgets, and projected future demand, which site is best for a new water storage facility to cost effectively serve future demand?
  • Can I postpone a new water treatment plant for 10 more years without jeopardizing safe, reliable and affordable service?

These complex questions regarding where and when to take action regarding water infrastructure are ideally suited for analysis using situational intelligence, due to three distinguishing features of situational intelligence:

  • Data integration breaks down silos across the utility to represent the full range of capital-related challenges and considerations
  • Spatial-temporal-nodal analytics accounts for the full context of capital investment decisions
  • Intuitive visualizations present the results of analytics in ways that people across the utility can immediately grasp and act on

Capital efficiency is important because water utilities need to invest a lot of money, and soon, to maintain the services that customers expect.

In 2013, the American Society of Civil Engineers’ report card gave the United States a grade of D+ for drinking water and a D for waste water systems. They cite a 2007 report from the Environmental Protection Agency that the U.S. water system needs $334.8 billion invested over 20 years.

Saving just one percent of that amount through capital efficiency would yield $3.5 billion in savings. Those billions would go a long way to keeping water utility margins healthy and water rates affordable as capital expenditures rise to fund needed work. Situational intelligence can deliver that one percent savings by providing fast, reliable answers to specific capital efficiency questions related to repairing, refurbishing, replacing and expanding water infrastructure.

To read more about water and capital efficiency, see this article in World Water magazine.


The Relationship Between Analytics and Situational Intelligence


This blog contains categories and posts for situational intelligence and for analytics.  Outside of this blog, these terms sometimes are used interchangeably, so I thought it would be worthwhile to describe the relationship between analytics and situational intelligence. Generally speaking, analytics is a component of a situational intelligence solution.

Generically, analytics is a broadly used term that describes a type of computational software.  More specifically, the term analytics describes algorithms, models and an entire category of software applications. Specific types of analytics generate outputs and results that range from insightful details about past events to recommended responses to predicted future events.

Because we typically use analytics to obtain a detailed understanding of past events and to predict future events, the output of the analytics must be readily understandable and acted upon. This is one of the reasons why analytics algorithms and models are generally embedded within an application program that makes the output available and actionable to users and to other systems.

Algorithms and models operate on data, so analytics must somehow have access to systems and sources of data (generally in predetermined formats). This is another reason that analytics is embedded within an application program – to seamlessly integrate data access with analytical capabilities.

Analytics can be delivered in several different forms: as native algorithms (e.g., as an R package), as specific models and point solutions, and as a salient component of “intelligence” solutions, such as situational intelligence solutions.

Situational intelligence is the latest generation of intelligence solutions that accesses data from many systems and sources then, depending upon the use case and solution, correlates, analyses and presents the results of analytics in contextually relevant and intuitively actionable ways.

As an example, consider a situational intelligence solution that generates an optimal work schedule based on rules, constraints and changing conditions. While the solution may use stochastic optimization, an analytical method, to generate the optimized output, the complexity of this particular analytical method is hidden from end-users who receive output familiar and actionable to them – schedules and work orders.

Embedding analytics within intuitive and easy–to-use application software removes barriers to use and consumption. Conversely, this approach also extends and operationalizes analytics throughout the organization by delivering information to people that is easily consumed and comprehended (at-a-glance) when and where they need it to drive and affirm decisions and actions.  This approach spreads the power of analytics beyond the IT “glass house” and into the hands of the people taking action to achieve organizational goals.

Seamlessly integrating analytics into situational intelligence applications that elegantly handles the data input and output makes analytics accessible to more people to drive more favorable outcomes. This method of embedding analytics is among the best ways to democratize and operationalize analytics, and it also clarifies the relationship between analytics and situational intelligence.