Hadoop is moving forwards at a rather remarkable pace. It seems like there are almost daily announcements of new partnerships, new deployment options and regular customer success stories.

What follows is a round up of some of those I think are interesting, and important, from the past month with a few of my thoughts by vendor. Let me know in the comments if I missed something you think is important!

Cloudera:

• A new partnership with Blue Talon. This allows the use of the Blue Talon  Policy Engine in conjunction with CDH.
  • My Take: Cloudera has been putting a lot of emphasis on security. This partnership lets customers optionally use Blue Talon instead of Apache Sentry. Blue Talon is apparently far superior to Apache Sentry to by partnering, and certifying, Cloudera has an extra step forwards in its enterprise security message.
• A new partnership with EMC.  EMC announced this with both vendors at around the same time. First came Cloudera and then a week or so later Hortonworks was added.
  • My Take: EMC is looking to deliver integrated systems including things like Isilon, Hadoop and more to deliver customers a one stop shop for the Big Data requirements. Partnering with the two big Hadoop vendors makes sense and it provides a new channel to both Hortonworks and Cloudera.
• A new deployment option announced with added support for the Google Cloud Platform with their Cloudera Director technology. This adds on to the already supported AWS capabilities.
  • My Take: I believe cloud is very much the future. Cloudera has expanded its capabilities in this space catching up with Hortonworks that had this support in Jan 2015 (link). This is a very smart move providing more options for organizations looking to not host their own clusters.
• New customer announcements:
  • British Telecom in the UK. Deployed a 50 node CDH cluster and replaced legacy batch ETL processes (with an exhausted batch window) with Hadoop based processes. Next steps outlined in the article.

Hortonworks:

• A new partnership with Ernst & Young to work together to deliver big data projects for their mutual clients.
  • My Take: Ernst and Young have a very good footprint in Financial Services in particular with a lot of C level access. This partnership could unlock some fairly big opportunities through a new channel for Hortonworks.
• A new partnership with NEC in Japan to give them access to the Japanese Market. As per the agreement NEC can resell  Hortonworks Hadoop software and support services
  • My Take: This is a smart move as it will give Hortonworks a channel via a well trusted third party in Japan. In effect it gives them a new channel.
• A new partnership with EMC.  EMC announced this with both vendors at around the same time. First came Cloudera and then a week or so later Hortonworks was added.
  • My Take: EMC is looking to deliver integrated systems including things like Isilon, Hadoop and more to deliver customers a one stop shop for their Big Data requirements. Partnering with the two big Hadoop vendors makes sense and it provides a new channel to both Hortonworks and Cloudera.
• An acquisition of Onyara and releases the Hortonworks DataFlow powered by Apache NiFi.
  • My Take: As the Internet of things takes hold I think handling streaming data, in BOTH directions and securely, will become ever more important. This acquisition moves Hortonworks down the path of being able to offer solutions in that space and gives them people with deep expertise.
• New customer announcements:
  • Open Energi in the UK. Will be using HDP to collect, store and analyse sensor data focused on real-time electricity consumption and control processes. Essentially this looks like a SMART grid use case.

MapR:

• A new deployment option announced with added support for AWS.
  • My Take: As I stated previously I believe cloud is very much the future. MapR has now caught up with the other larger commercial vendors by offering their distribution on AWS. I think this is a very smart move providing more options for organizations looking to not host their own clusters.
• New customer announcements:
  • Harte Hanks announced they are leveraging the MapR Hadoop distribution as part of their big data solutions.
  • Terbium Labs was announced as leveraging MapR. This is a really interesting use case. They are going to be trying to identify stolen data on the “dark web”.

Other News:

• SAP unveiled “SAP HANA VORA“. I am still digesting all the various articles so watch out for some thoughts on this going forwards.
• Datameer secured $40 Million in funding to continue their expansion around their Hadoop based BI platform which shows, as much as anything, that this is where investors expect to see a lot of future data analysis and reporting happening.
• HP announced native file support (ORC and Parquet) for Vertica on Hadoop as well as tighter integration with Apache Kafka.
• WanDisco, who help deal with Hadoop issues around active-active replication etc,  announced a new version and shared information about a new sale with Nokia. WanDisco is going to become increasingly important as enterprise deployments of Hadoop take hold. Watching their announcements might give you a hint as to who are moving mission critical workloads to Hadoop.

In conclusion

This post is just scratching the surface. Keeping up with Hadoop, and its vast ecosystem, is a full time job. I am sure I missed something important and if so I invite you to add it to the comments. I hope that this overview helped bridge any recent gaps you have.

From my perspective the Onyara acquisition and  the SAP HANA Vora announcement stand out. I look forward to fining out more about them both.