Series: The Hidden Challenges of Putting Hadoop and Spark in Production
A recent Gartner survey estimates that only 14% of Hadoop deployments are in production. We’re not surprised. We’ve been in many conversations with companies that have been piloting Hadoop to bolster their analytic capabilities beyond relational databases. Common challenges fall into a few important categories, which we explore in this blog series:
- Infrastructure: Choosing and configuring servers for Hadoop
- Performance Optimization: Scaling and tuning Hadoop for price-performance
- To Cloud or Not: Selecting, configuring and new challenges
- Security: What to consider
Cloud Security for Big Data as a Service: What to Consider
When we started the Cazena journey, we commissioned a big data survey and research analysis from GigaOm. Our goal was to discover the greatest challenges for organizations adopting cloud services for analytics, data science or big data. The top challenge? Security, unsurprisingly.
Our early research and conversations validated and fueled Cazena’s commitment to develop the most comprehensive approach to security and compliance in the cloud services category. We got feedback from our beta testers and advisors, hired the top market experts, brought in advisors and put in the time to get it right. Cazena was truly built for security from the ground-up. And those aren’t empty words for the web, we prove this out to skeptical CISOs and security teams on a regular basis.
Cazena often spends just as much time with security and compliance departments, as we do with the analytic teams that want to use Cazena’s platform. Security teams have many, many questions. It’s understandable. We hear that many cloud services can’t be used in production by enterprises, because basic security and compliance is an afterthought, handled inconsistently through hastily-created processes.
After many meetings and lots of research, I suggest these questions upfront to give you sense of a cloud provider’s stance on Big data as a Service security.
- Which resources are provided to secure my cloud tenant and ensure only authorized parties can access it?
- How will we access resources through our corporate firewall?
- Is there a common way of performing user authentication across all services?
- How is end to end (in-motion and at-rest) encryption handled?
- What capabilities are available to store and manage keys?
- Does the service offer data-level security?
- Which compliance certifications do you have?
- How can the service be audited by a 3rd party?
- Is there constant monitoring of access and usage from a security perspective?
- Is there an explicit policy for patch and vulnerability management?
We’re not suggesting right or wrong answers (for now), but getting feedback on these will give you an idea of the provider’s expertise in security and compliance. To give you a sense of Cazena’s approach, our strategy is two-fold:
- Architect for security. Build everything with security and data privacy top-of-mind, so that compliance being “easy” (or easier!) is merely the end result of us already having a secure platform. We did not want to have to make significant architectural changes later on because of compliance requests or because someone took a shortcut.
- Compliance is about process + technology. We started the compliance journey early by hiring and consulting with experts from a variety of backgrounds. All agree: Compliance is not just about the technology, though that is important, too. We must also have the right processes for software development, release and operations – as well as ongoing operations, patching, upgrades…the list goes on.
This combination of process and technology gives us confidence when we promise an organization that they can trust us with their crown jewels. Our approach employs best practices such as layer-based security, which is described in more detail here: Cazena’s Big Data as a Service Security. Our platform is fully-managed, with our team of experts providing monitoring, patching and other ongoing processes. And we have an ongoing security roadmap to ensure that we continue to lead the market. We have designed not just our platform, but our entire company, to deliver the most secure Big Data as a Service.
