It is well established that the technology Vertica is proposing — a grid-enabled, column-oriented relational database — can provide a huge performance boost for data analytics. My company, Sybase, makes one such product, the Sybase IQ analytics server. It’s already available, with nearly 1,000 customers experiencing tremedous performance acceleration and significant ROI.

As an example: Nielsen Media Research implemented Sybase IQ for their audience data warehouse. Sybase IQ has provided 10-100x increase in response time for even the most complex of queries, along with a 70% compression ratio (which is allowing them to save quite a bit on hard storage.)

http://www.sybase.com/detail?id=1035802

As a whole, analytics servers -– both emerging products like SAND and enterprise-class products like Sybase IQ (which includes advanced features like encryption) — are experiencing very high growth. Dr. Stonebraker’s company could ride this wave of success, so it may not matter that their technology isn’t really new.

Ah, Vertica. We actually had Michael Stonebraker in about a year and a half ago to talk about (convince us to use) Vertica. I was in on that meeting and was less than impressed with the presentation. The product seems sort of interesting, the presentation just struck me as less than honest. That meeting was a while ago so I don’t remember a lot of the details, just what I put down in some sketchy notes. Also, I have not been following Vertica closely so I don’t know what changes have been made in the last year and a half.

There are a number of people here want to try it out (I would really like to do some testing wrt Oracle, time permitting. Hah. If anyone has some interesting tests in mind let me know, I’ll see what I have time for). We supposedly even have a copy to play with, though I don’t think much is happening in that direction as of yet. Of course I bet a lot of the attraction for all the MIT AI lab hackers here is having a product in beta, that isn’t big bad Oracle.

Vertica came out of c-store (http://db.csail.mit.edu/projects/cstore/vldb.pdf)
This product seems to be designed for DW and query intensive databases. Storage is by column, not tuple. This allows retrieval solely of the attributes you want instead of all other attributes also stored in the block with tuple based storage. The column values are also sorted and compressed. Multiple sorts can be stored to help querying speed.

There is a lot of redundancy in this design. It is a shared nothing architecture, with redundancy (at least as far as the data, not the sorting) across multiple nodes. It seems availability and recoverability are combined.

There is also a weird way of mixing in OLTP transactions. They are done in a separate work area and a tuple mover then puts them into the read store. There appears to be a time delay between committing data and having it visible from a query. Strange.

A few additional comments from my notes:
–It seems as if the High Availability (HA) and recoverability mechanisms are intertwined. Recoverability happens by accessing a replicated site. This implies a bunch of synchronous data transfer.
–If the above is true, I would really like to see query vs. write performance. At times it sounded as if the product was designed for improving DW queries, but M. Stonebraker also claimed OLTP writes were fine. He really pushed both. Even with multiple synchronous copies? And redundant data tables?
–We are reentering the shared nothing vs shared disk database cluster debates.
–A bit disingenuous about some of the references to “the elephants”. A lot of references were either 5 years old or referring to default behaviour. Granted, given the default setup,a standard table, and simple SQL Oracle will usually store in order entered (DELETES notwithstanding), but there is no requirement in the model to always do so. Of course this doesn’t mean Vertica can’t do some things better, just that we need to be educated consumers. (the disingenuousness, especially from an academic, made me slightly squirmy)
–I would still like to know more about backups.

Hey, just check my email, and M. Stonebraker will be giving a talk at my work Tuesday afternoon. Anybody have any questions for him?

Well that talk was today. If there is any interest I’ll post some notes/comments later. Too tired right now.