Business Intelligence: Decision-making fed by real-time data

Business Intelligence: Decision-making fed by real-time data
By Jessica Twentyman

Published: March 11 2009 17:41 | Last updated: March 11 2009 17:41

At the London-based same-day delivery company, eCourier, it is not enough for managers to know the average number and value of recent orders placed by a particular client. That type of historical analysis can easily be obtained using traditional business intelligence tools.

Technology now has to go further, identifying problems the moment they arise.

Given eCourier’s growing customer numbers and high volume of daily transactions, that is not a problem weekly reports can solve, says Jay Bregman, chief technology officer of eCourier (now part of SAP Business Objects).

Early attempts to sift through reports on each customer manually were too time-consuming, he says. Instead, eCourier switched to technology from SeeWhy, a real-time BI specialist, to monitor and analyse customer booking behaviour in real time.

As soon as a client places an order on the company’s website, the transaction is added to a unique profile of their behaviour and analysed against data already held. If the new order does not match the pattern of previous orders, an automatic e-mail alert prompts the client’s account manager to get in touch to resolve problems or to sell extra services, such as overnight or international delivery.

This is business intelligence at its most responsive, but it is far from the norm. Today, most corporate BI environments focus on running queries and reports against a data warehouse that holds a recent snapshot of data collected from different transactional systems.

Before it can be analysed, that data must be cleansed and organised – which can take from a few hours to several days. By the time data is available for analysis, it may be out of date and getting less relevant to decision-making.

The enterprise data warehouse can either be a key enabler or a painful bottleneck, says James Kobielus, an analyst with Forrester Research, the IT market research company.

“Decisions must keep pace with the urgency of modern business. Some business decisions are just too time-sensitive, and the stakes too high, to rely entirely on days-old analytics. For the most time-critical decisions, enterprises require ‘really urgent’ analytics,” he says.

His company defines this as “any business intelligence, metrics or other operational data that is updated from the source continuously or, at the very most, every few minutes”.

Typically, this approach may be useful, for example, in providing customer insights to the contact centre (as at eCourier); managing logistics based on external conditions such as weather and traffic volumes; or enabling highly responsive marketing campaign management.

The growing need for more immediate visibility into data has prompted a number of approaches to BI. SeeWhy’s approach, for example, is based on sophisticated event-stream processing (ESP) technology.

Each “event” – typically a message generated by an operational system – is analysed as it occurs, rather than in the aggregated, historical fashion of traditional BI. ESP typically means looking for scenarios of events, such as patterns and combinations of events in succession, that are significant indicators of a business problem, explains Charles Nicholls, chief executive of SeeWhy. Mastercard, for example, uses the technology to detect fraudulent use of credit cards.

“The outputs of these events are usually real-time metrics and alerts and the automatic initiation of immediate actions in other applications, instead of reports, which require analysis and interpretation by humans before any decision can be made,” he says.

In research conducted by his company, he claims more than three-quarters of operational managers surveyed said they were frequently forced to make business decisions before data became available, and two-thirds said that by the time it did arrive, data was already out of date or no longer useful.

A similar, event-based approach is deployed by Tibco Spotfire, formed in 2007 when Tibco, a provider of middleware, which connects applications, acquired business analytics company Spotfire.

The combined business has married the two technologies to create complex event processing technology that monitors the progress of key business processes and issues alerts and fixes when pre-set thresholds are breached. Tools are also provided to enable business analysts to focus on data relating to a specific process, especially when it’s gone awry, so problems can be resolved.

“In a downturn, the efficiency of your business processes is everything,” says Rock Gnatovich, Tibco Spotfire chief operating officer. “Where they deviate from the norm, the events that are generated by your computer systems represent both opportunities and emergencies for your organisation, so it makes sense to keep tabs on them.”

Other BI vendors are hardly blind to the opportunity that real-time – or at least near-real-time BI – represents. Most enterprise BI tools now seek to help customers tackle the problem of data latency – the period between the time data is entered on a company’s systems for the first time and the time it is available for analysis.

IBM Cognos, for example, uses streaming technology in its IBM Cognos Now product, aimed at organisations looking for a simpler approach to operational monitoring. This is offered either as a BI appliance (a pre-configured, software-hardware hybrid) or via a software-as-a-service model.

But despite the hype surrounding real-time BI, not every situation requires this approach, says Graham Mackintosh, business unit executive for IBM Cognos. “Real-time BI only makes sense in situations where you can respond to its findings in real time. Extreme low latency is only helpful if there is measurable business benefit and return on investment to be gained from quick responses,” he says.

“It’s more about delivering data to operational managers within an actionable time frame,” agrees Royce Bell, chief executive of Accenture Information Management Services. “Organisations need to prioritise if they are to avoid overloading managers with information – otherwise, it just becomes noise. And that can only come about through regular, thoughtful conversations with them about their data needs.”

Nor may it be necessary to invest in new technology to close the data latency gap. Some organisations already have the tools they need to get a grip on “really urgent” analytics, says Forrester’s Mr Kobielus.

“For starters, your enterprise data warehouse administrators can shorten end-to-end data latency by reducing the batch update cycle from once every evening, to once every hour, every 15 minutes or even every few seconds,” he says.

“Clearly, you can take measured, incremental steps into real-time analytics, or bet the business on a bleeding-edge futuristic approach. The path you take depends on how key real-time responsiveness is to your strategy.”

SAP Business Objects: one year on

German software giant, SAP, bought Business Objects, the business intelligence company, just over a year ago and the combined business now faces the task of convincing a huge tranche of customers that have already invested in SAP BI technology to migrate to the Business Objects BI platform.

Last year, SAP released its post-acquisition BI product roadmap that showed some SAP products – including its Business Explorer reporting tool – would be phased out. The company has insisted it will continue to invest in its own SAP Netweaver BI tools, but it is clear the primary focus will be on Business Objects’ tools.

Convincing customers could take time, according to a recent report by Gartner analyst Kurt Shlegel.” Generally speaking, the majority of mature NetWeaver BI customers are loyal to SAP and the investments they made in the product,” he writes. “However, they are also in no hurry to quickly adopt the Business Objects product roadmap.”

But Mr Aman insists there are big advantages for customers ready to take that step. “There is already tight integration between SAP and Business Objects technology – although we are, of course, working to make it more seamless – but more importantly, there’s a lot of capability within the Business Objects portfolio that is unmatched by current SAP capabilities,” he says.

One example, according to David Meyer, vice president of emerging technologies at SAP Business Objects, is Polestar, an end-user tool that enables users with little or no BI experience or training to search for and explore data. He said it aims to offer them the simplicity and speed of internet search.

Polestar users, for example, can use keyword searches to find data and the tools automatically generate a visual representation of the results in the form of charts or graphs.

“This technology will enable SAP customers to put BI in the hands of everyone and anyone in their company in a way that they have never been able to achieve before,” adds Franz Aman, vice president of intelligence platform product marketing at SAP, and the person with responsibility for promoting the combined companies’ portfolio of information management products, including BI capabilities and related tools, including data profiling, metadata management and data quality.

In coming months, he adds, the company will release a product that combines Polestar with SAP’s Business Warehouse Accelerator, a high-performance analytics appliance.

Extending the reach of BI is more important than ever in the current economic climate, he says. “True competitiveness is about people understanding goals and targets and knowing where they stand in relation to those goals and targets. That’s what BI can deliver and the more people you can extend BI to, the better. In fact, I’d say it’s a business must-have for 2009.”

Copyright The Financial Times Limited 2009