Are BI Appliances Simply 30 Year Old Databases?

In a thought-provoking blog post published by WIT, a business intelligence consulting company in the U.S., the author writes of latest acquisitions relating to Business Intelligence appliances.

BI Appliances

It got me thinking. I’ve been seeing and hearing the term ‘BI appliance’ a lot recently, and whenever I do - I find myself struggling to understand what it means.

One characteristic that seems to be commonly identified with BI appliances is that they are a combination of software and hardware that form specific functions that have to do with analytics (i.e business intelligence). WIT’s article lists a few examples, including HANA (SAP), HP Business Decision Appliance (Microsoft), Netezza (acquired by IBM) and Greenplum (acquired by EMC).

But is proprietary hardware really required for a so-called BI appliance? No, it’s not. And indeed, I have noticed numerous references to Vertica (acquired by HP) and ElastiCube (by SiSense) as BI appliances. Interestingly enough, both are software-only solutions (i.e. software appliances).

It makes sense, as it shouldn’t matter if your ‘appliance’ runs on proprietary hardware or commodity hardware, if it essentially does that same thing.

The BI Appliance Wars

In a recent interview and in response to quips made by Netezza’s CEO regarding HP’s latest acquisition, Vertica CEO Chris Lynch had this to say about Netezza:

“Their tag line is ‘The power to question everything'. So the first question is: why do they need proprietary hardware? The second question is: why are they using a database engine that’s based on technology from 1982?”

He is obviously angry, but I agree with the premise of his argument. If you’re in the analytics business and you require proprietary hardware – there’s something seriously wrong with your database software technology. Commodity hardware is so powerful today with 64-bit computing and multi-core CPUs, that it’s hard to imagine what type of BI solution would require proprietary hardware.  That is, if your technology was engineered in the 21st century.

The established vendors are not oblivious to this, but rewriting their entire codebase is not something they are willing to do. So some are partnering and/or merging with hardware companies as an alternative. But at some point, scraping this codebase will be unavoidable, or customers will flee due to availability of much better and cheaper alternatives.

BI Appliance or BI Tool?

As if to toss a little more confusion into the mix, the WIT author asks:

"Though I wonder - with memory becoming cheaper and cheaper and with 64 bit platform, why do you have to have a special appliance? Why not use an in-memory tool with tons of RAM ?"

The question itself indicates a misunderstanding of why appliances exist in the first place, and there are a several answers to this question.  Here are a few:

1. RAM is cheaper, but it's not cheap. Disk was and always will be cheaper than RAM.
2. The price of a computer jumps significantly beyond 64GB.  A PC with 64GB of RAM costs significantly less than a server machine with 65GB of RAM, even though there is supposedly just a single GB of memory difference.
3. In-memory databases assume that the main bottleneck is I/O.  However, when dealing with large amounts of data, this is no longer true.  At such volumes, bottlenecks are between RAM and CPU.

For more information about this, please read In-Memory BI is Not the Future, It's the Past.

---

By: Elad Israeli | The ElastiCube Chronicles - Business Intelligence Blog

BI vs. Big Data - Watch a Columnar Database in Action! (Video)

I was recently fortunate to speak at one of the database technology conferences held in Israel. Big parts of this conference revolved around ‘Big Data’ and I was asked to give the Business Intelligence perspective on this fascinating subject.

As part of my presentation, I attempted to show the impact of columnar database technology on the basic premise of business intelligence - the ability to have business users perform ad-hoc analytics and reporting tasks over as much data as possible.

In order to do that, I represented a business user building a report over a very large operational database containing 13 tables, the largest of which hold 100 million and 40 million rows. While databases of this size were once rare - now, any company who has a properly tracked website quickly accumulates even more data than that.

To demonstrate, I was using a front-end analytics tool (SiSense Prism) to create reports that query the database directly - a feat not advisable with a relational database. So instead of querying the source database, the data was replicated (but unmodified) into a columnar database that was designed specifically for ad-hoc analytics - ElastiCube.

The computer holding the ElastiCube was a $1200 off-the-shelf PC with 6GB of RAM, 100GB of disk space and a single quad-core CPU (64-bit). The Prism front end could be installed on any computer, as it does not process the queries or hold data - only requests query results.

For your convenience, here is the video of this demonstration.



Interesting Points
One thing you should understand from this video is how simple drag-and-drop operations of a business user (or multiple users) within his or her desktop tool turn into complex database operations that would choke any relational database (joining, grouping, aggregating), yet are handled by a columnar database without any difficulty.

Which brings me to my final, and perhaps most important point -

Whichever business intelligence front end tool you pick, dealing with issues like this (and their subsequent side effects) is in fact 90% of the life-time cost of a BI solution, and often why the solution stops being used all together.  This is because BI solutions which rely on relational back-end technology must be designed to assume there had been significant trimming, de-normalizing and pre-aggregation of the data conducted prior to being delivered to business users.  This process never ends, and only becomes more and more difficult to maintain over time.

Columnar databases change this reality entirely, and combining them with 64-bit and multi-core computing makes for a dramatic evolution in BI development.

---

By: Elad Israeli | The ElastiCube Chronicles - Business Intelligence Blog

Chatting with Albert Einstein about Business Intelligence

Albert Einstein is arguably one of the most intelligent people who have ever lived.  Have you ever wondered what it would be like to chat with him about business intelligence?  I have.

---

Mr. Einstein, thank you for taking the time to talk to me today. I work for a business intelligence vendor called SiSense and I would love to hear your thoughts on a subject you know more about than anyone else – intelligence.

"Whoever undertakes to set himself up as a judge of truth and knowledge is shipwrecked by the laughter of the gods.”

Modest as always! However, instead of talking about personal intellect, I would love to get your perspective on what we call ‘business intelligence’.

“One may say the eternal mystery of the world is its comprehensibility.”

That is exactly why I’m talking to you. While great technological and functional strides have been made in the business intelligence space, no one has really been able to figure it out yet. Maybe you can help us out?

“Technological progress is like an axe in the hands of a pathological criminal. It has become appallingly obvious that our technology has exceeded our humanity”

Mmm, right. But business intelligence technology is not exactly the atom bomb.

“Concern for man and his fate must always form the chief interest of all technical endeavors. Never forget this in the midst of your diagrams and equations.”

Tough crowd. Okay, let me try to clarify. Business intelligence describes the ability of a business to make informed decisions based on facts inherent in its data. Experience shows that the challenge in achieving this can be easily tracked down to the technology being used behind this ability. Innovators in the space place a lot of focus on this, but I think they find it challenging to explain to the average person why it is so important.

“Most of the fundamental ideas of science are essentially simple, and may, as a rule, be expressed in a language comprehensible to everyone.”

So what you’re saying is that the technology doesn’t matter, if you cannot explain its benefits in simple terms. Fair enough. So what if I told you that modern business intelligence technology simplifies the ability of decision makers to gain knowledge and insight into their business operations and through that make better informed decisions?

“The true sign of intelligence is not knowledge but imagination.”

But you cannot disagree that following business metrics is a good idea.

“Not everything that can be counted counts, and not everything that counts can be counted.”

I guess that’s true. So, are you saying we should give up on the whole idea of business intelligence being realistic?

“Reality is merely an illusion, albeit a persistent one… Learn from yesterday, live for today, hope for tomorrow. The important thing is not to stop questioning.”

But that is exactly what business intelligence means - never stop questioning. However, without getting the occasional answer, this could prove to be quite a tedious challenge, right?

“The devil has put a penalty on all things we enjoy in life. Either we suffer in health or we suffer in soul or we get fat.”

I guess I could lose a few pounds, but what part would technology play in all this?

“We can't solve problems by using the same kind of thinking we used when we created them.”

That is why new technology is constantly being thrown into the business intelligence mix. Modern chipsets like CPU and RAM, Columnar databases, NoSQL databases, etc. These technologies did not exist before, and they're all being used to make business intelligence more approachable.

“Everything should be as simple as it is, but not simpler.”

Having things simple is important, no doubt, but there is also the issue of business intelligence costs going through the roof.

“Sometimes one pays most for the things one gets for nothing.”

We indeed encounter people dropping loads of cash on something that could be easily done in Excel. Isn’t that insane?

“Insanity is doing the same thing over and over again and expecting different results.”

Interestingly, it indeed seems like many business intelligence vendors just re-wrap their old products based on technology invented before you were born. You end up paying for the same thing again and again, without really getting any value back. How can new business intelligence vendors succeed where the others have failed?

“You have to learn the rules of the game. And then you have to play better than anyone else.”

This was a fascinating chat. I’d like to thank you for your time.

“The only reason for time is so that everything doesn't happen at once."

---

By: Elad Israeli | The ElastiCube Chronicles - Business Intelligence Blog

* Einstein quotes are the courtesy of BrainyQuotes.