Last week, Michael Copeland of Andreessen Horowitz interviewed me and Peter Levine, General Partner at Andreessen Horowitz, on the topic of big data moving from on-prem infrastructure to the cloud.
It was a fun chat after we literally took over Peter’s office while he was attending a board meeting! In the interview, Peter, Michael and I discuss how:
- Big Data 1.0 evolves to Big Data 2.0 when you have rapid and frictionless access to data. Cloud enables this. We call it Big Data on Demand.
- The Big Data stack is complicated because it has a diverse set of technologies that are being driven by a thriving open-source community. Enterprises should not look for one silver bullet for all workloads. Each technology has a sweet spot in terms of price-performance.
- The move to the cloud requires two big issues to be overcome, friction (see #1 above) and complexity (see #2). Big Data as a Service is the new way to do so.
- For the cloud to be the next platform for analytical data processing, data must “land” on the cloud and process multiple workloads. Moving back and forth is painful. Avoid this.
- Big Data 2.0 helps pool data and spread it around faster. It will amplify the signal from the noise. For organizations large and small, it will create a data-driven culture.
- Peter has an interesting vision of application-aware Big Data 3.0 that ultimately derives from Big Data 2.0.
See the full interview below, read the full transcript, or download the podcast.
