So Much Data, So Little Room
Publishers and brands are having serious data issues. The problem isn’t about where to get it from, because it’s pouring in from all sides. There’s sentiment analysis, social listening, social engagement indexes, even catalogs of data claiming to measure “emotional currency”, among dozens of other, probably quite valid, measures of consumer status.
In order for all that data to be useful, it has to fit in the proverbial data warehouse, somewhere, without overloading the entire structure. Just like your attic, there’s a limit to how many lifetimes worth of stuff you can fit into an existing structure without causing a big, messy spill the next time that it’s accessed. Most companies, even smaller ones, aren’t just analyzing multiple data streams from a few hundred customers, they’re looking at thousands, with ever-increasing sources of data and elements of analysis.
Too Big to Fail : Critical Data Analysis
While retailers and publishers fight for each customer, accurate analysis, a whole lot of it, becomes even more imperative. That means big data keeps getting bigger, more detailed, and less manageable without the proper tools. Still brands and publishers need rapid access to that data for strategy performance analysis and for making the data-driven, day-to-day decisions that drive industry.
Since the 1970’s, companies wanting to analyze trends loaded big data into “OLAP” cubes, which are intermediary databases. OLAPs were designed to facilitate complex queries and return results quickly, but instead were often found to be slow, having to refresh frequently, and became quite expensive.
Thinking Outside the Cube
Recently Kognitio released a new product which may be a game-changning innovation in the field. Kognitio’s Pablo product claims to be the industry’s first analytic database that renders virtual “cubes” at the data’s source without negatively impacting performance. That means faster access to critical data that marketers and brands need.
“(Pablo) allows you to build very large (multi-terabyte) OLAP cubes, in memory, without physically instantiating those cubes. This is a big advantage.”, writes Phillip Howard, Research Director for Bloor Research. “It can often take hours to (re)build a cube and many companies have dozens, hundreds and even thousands of cubes, all of which makes the whole environment unresponsive and unwieldy. Using WX2 should make a lot of this pain go away.”
What does that mean on a real-world level? Big data has just shrunk considerably. “Users can now do ‘train-of-thought’ analysis where they continuously probe and query the data as soon as questions come to mind.”, said John Coppins, Product Director at Kognitio. “And they can do it with easy analytical tools like Excel that they already know how to use.”
Under the Hood
Pablo communicates with Kognitio’s companion WX2 database to combine in-memory images of a company’s databases with a metadata layer and an MDX connection. This permits all transactional data like that from a CRM or a company’s real-time financials to be stored directly in the WX2 database, cutting out the time normally needed to create “cubes” for new data in the old OLAP style. This means users can potentially eliminate the need for some of the more elaborate front-end applications that cover this territory.
“The need for diverse types of analytics is the strongest trend in data warehousing and business intelligence today,” said Philip Russom, senior manager of research at TDWI in a press statement. “To get the most out of analytics, users are looking for new platforms that are built for analytics, in that they support in-memory databases, MPP architectures, and – of course – very, very large analytic datasets.
The Verdict: Worth a look. Cost-cutting and ease of access to massive data via familiar MDX tools like Excel is a good thing. Companies that are most comfortable with, and can afford, glossy front-end apps that use OLAP-based solutions will continue to do so, but they would do well to check out viable alternatives, like Pablo.
Timeline: Pablo will be available June 1st.
More in Media
Media Briefing: European publishers speak out on advertisers’ punishing brand safety practices
This week’s Media Briefing recaps what publishers had to say behind closed doors during last week’s Digiday Publishing Summit Europe about the brand safety/suitability practices that are penalizing their ad businesses.
At the Digiday Publishing Summit Europe, publishers move away from trend chasing
Despite the publishing industry’s grim outlook, many outliers are simultaneously fighting to change and reluctantly accepting the status quo.
Inside Dow Jones’s AI governance strategy, with Ingrid Verschuren
During the Digiday Publishing Summit Europe, Dow Jones’s evp of data and AI detailed the role that the publisher’s AI steering committee plays in its use of generative AI technologies.