It seems a long time ago that the magic 3 V’s of volume, variety and velocity was unleashed on the world to describe the evolution of data organizations were about to see. In fact it has been a whole 17 years since Doug Laney first used the term.
The story was that we needed to get ready for a new wave of data. We needed to be ready to store more data, take data that might not look like we had traditionally from operational systems (such as textual unstructured data) and handle data arriving more quickly. This was in the web era long before mobile and social took off.
Then we had the Big Data storm where all the V’s got bigger, faster and more diverse. Social Media had arrived and the use of external data to help make decisions was on the rise. I went on record at the Gartner summit in 2014 saying I thought the buzz word would die out. It turns out it is still around but we hear it much less frequently nowadays.
Back at the end of August I put together a post where I stated that “Hadoop adoption seems to be accelerating“. Now, just a few months later, it is clear that the Hadoop juggernaut just keeps on rolling and that many organizations are adding it to their IT architectures.
Safe Harbor is no more. Well Safe Harbor still exists but following a ruling in the European Court of Justice it has now been ruled as an invalid mechanism to protect the data and privacy of European Union citizens personal data when it is moved to the USA from the EU. In essence this is a damning statement that says European Citizen personal data was never safe from certain parties in the USA.
TThis is down to the fact that the US government is perceived as having used that transmitted data to spy on citizens (as alleged by Edward Snowdon). There is no suggestion of bad behaviour by US organizations themselves who generally offer much better protection than many companies could develop and maintain themselves.
Exactly what this means is still being worked on. I suspect it will mean a lot of scrambling and head scratching for the many organizations (estimated at about 4500) who use Safe Harbor. Those organizations use it to enable them to run a centralized internal IT platform which supports their entire global operations, to offer centralized platforms for consumers hosted and run totally out of the US and for companies who move data around as part of their general business approach.
This week I had a number of non-IT friends ask me what this was all about. Mainly these questions are coming as the ruling is making news headlines around Europe. This post is a summary of what I tried to explain to them. Safe Harbor is something most people never knew existed in the first place and it is amazing it has taken over our televisions.
When Hortonworks announced they were going to buy Onyara I have to admit I had never heard of Onyara and had no idea what technology or value they would be bringing to the Hadoop table. To be acquired there had to be something interesting I was missing. So, Starting with a basis of no knowledge, my natural curiosity kicked in. For the past few days I have been trying to learn about Onyara and the technology they were involved with. This blog reflects a little of what I have learnt. I welcome any comments or additional information to help all of us learn more.
Hadoop is moving forwards at a rather remarkable pace. It seems like there are almost daily announcements of new partnerships, new deployment options and regular customer success stories.
What follows is a round up of some of those I think are interesting, and important, from the past month with a few of my thoughts by vendor. Let me know in the comments if I missed something you think is important!
The story was that we needed to get ready for a new wave of data. We needed to be ready to store more data, take data that might not look like we had traditionally from operational systems (such as textual unstructured data) and handle data arriving more quickly. This was in the web era long before mobile and social took off.
