Data integration is going through a Renaissance at the moment just like application integration did a few years ago. The primary driver for these changes is the movement to the cloud. When you turn back the clock, application integrations used to be simple. There were a few ERP systems, and as things started moving to the cloud via SaaS, application integration changed quite a bit. Now we’re seeing analytic applications move to the cloud as well. The biggest force of gravity is the mass movement to the cloud. That is also driving data integration. By definition, we’re now replicating platforms as much as we are moving into them. Enterprises that are moving into the cloud need a parallel data application integration strategy to make use of your data as an asset and to make sure the overall plan will work.
Data is frustrating to deal with. The problems seem easy and the reality is hard. Normalizing data, merging it together so you can get meaningful results out of it, making sure your SLAs are met in terms of freshness and quality are all easy to describe but are frustratingly hard to do. One of the challenges we have for on-premise systems is there has always been a goal to democratize data. The challenge is access. It sounds easy, and the cloud inherently breaks this democratization barrier. There is now a public place where people can go get access to data. That centrality is driving a lot of innovative capabilities we are seeing emerge.
Going from legacy applications to cloud requires a different approach to application and data integration. One starting point is choosing a cloud platform and trying to pick the right one. This can be the toughest decision a customer faces. Within each cloud platform there are also myriad choices, from which query technology to use, to whether to use Hadoop implementations on Azure, and more. There are so many choices and it’s easy to feel like you’ve made the wrong one.
Working in enterprise IT, there is a shift in the view of how data should be managed, accessed, and manipulated. In the early days, it was a developer’s job. IT was the team who knew how to access data and application systems. Self-service pushes some of the responsibility out to the edges. It’s not about wanting to move away from IT, but it’s about empowering those who who have the domain knowledge and will eventually be using the data. How we access these systems is different now. The structure of the data is also different. It used to be row and column-oriented. Nowadays, with REST and JSON, being able to handle the formats and not create new barriers so you are not manhandling the tool into dealing with data in unnatural ways. That will make developer lives easier and make it easier for vendors to add connections to the data.
Databases used to be relatively simple. These days it’s often special-purpose databases such as in-memory, object-based, hierarchical, and relational databases. In some ways this makes things more complex, but with layers of abstraction it can also make things easier. When we only had relational, you had to make data look relational whether it was meant to or not. Now you do not need to force anything to fit different data sets.
SnapLogic is beginning to leverage AI and Machine Learning. Their focus is on making computers work for you instead of making you work for computers.
We discuss the birth of Kubernetes at Google and how it started as part of Google’s own best practices and infrastructure, then was created into an open-source project and shared externally. It was a brand new code-base, but built based on Borg, which was an internal system at Google for over 10 years. That is why Kubernetes has a lot of design aspects of a more mature project. It is a combination of Google’s past work on containers and the new container ecosystem related to Docker so that Kubernetes can work outside of Google in any environment. Anyone can get involved and start contributing to the system. There is no enterprise version of Kubernetes, there is just Kubernetes and the different vendors and providers add value to it.
We look at the announcements from Google NEXT. There were no Kubernetes surprises because it is developed in the open, so everything you want to know about it is actively tracked on GitHub. Instead, at Google NEXT, they tried to add clarity about why Google is working on specific features like dynamic storage provisioning and custom schedulers. It’s clear Google is evolving into the enterprise cloud space and they are committed to staying ahead of the curve. For Google Cloud, that means opening up some of its core technologies such as TensorFlow and Google Cloud Spanner so that enterprises can begin to consume Google technology.
We look at Amazon’s recent decision to allow customers with Reserved Instance contracts to subdivide some of their Linux and UNIX virtual machine instances and still keep their capacity discounts. This decision was made so that Amazon could keep up with Google on flexible pricing. These types of changes are evidence of a healthy marketplace. A lot of times people try to mimic the spending habits of traditional enterprise IT which require a three-year roadmap, which is the opposite of what cloud should be. Cloud should be a dynamic world where you can scale-up and scale-down, which is why discounts should kick in based on behavior and actual usage. It’s also why Google has per-minute billing. In this way the pricing structure matches the dynamic needs of cloud computing.
Last, we discuss how small to medium-sized businesses in India can get subsidies for cloud adoption, while the US wants to tax its usage. This may be an opportunity for the US government to offer tax breaks for cloud adoption to incentivize change so that we can use these resources more positively. Some enterprise clients view cloud computing as a complex expensive endeavor. In essence, they’re right, they have archaic infrastructures and it takes a lot of work to move them to cloud, so some of the work it takes can remove the incentive to make the switch. In order to get them over the hump, they need to understand the cost savings, but it seems like a step in the right direction to provide tax incentives much like buying a Tesla can be subsidized. That should at least send the right message to companies in the US. Cloud adoption reduces data center space, saves electricity, and helps build businesses faster, which seems like a win-win for large enterprises and the US government.
Our guest on the podcast this week is Ray Young, VP at Cloud Technology Partners.
We discuss the AWS outage and how it should not come as such a surprise because cloud provider outages should be expected from time to time. For a lot of companies, the outage was a wake up call not to move away from cloud, but that they need to make the right architectural decisions about the cloud to prevent these sorts of outages. worrying about outages is not the answer – it’s all about management, monitoring, and the ability to have self-healing capabilities. Cloud providers will have outages from time to time, and your enterprise’s cloud architecture will make the difference.
We also discuss digital innovation in the cloud. There are many ‘me too’ players jumping into the cloud, following others with the same solutions. Not a lot of people are thinking out of the box and doing new, innovative things. For instance, it seems like AI as an industry is not innovating anymore, just implementing the same technology to solve similar problems.
Traditionally, when IT has talked about cloud adoption the focus has been on how to shift from data center mindset to infrastructure as code. But more and more, there is a real driver for cloud adoption to be innovative. The primary way cloud makes innovators is by allowing them to stay relevant and survive in today’s technology world. There is also a shift from loyalty of a brand based on the merits of a brand versus brand loyalty based on customer experience and engagement. The cloud is where innovation and agility is happening on customer engagements. Customers are now making investments in the cloud for cost savings as well.
Large enterprises now see their competition as small, nimble, emerging startups. These startups are able to mature their capabilities through their investment in cloud, SaaS, and custom-builds on the cloud to be seen as an equal in the marketplace. For large enterprises trying to stay relevant in a marketplace of startups who can move faster than they can, cloud is the only way to equalize the playing field.
We discuss containerization orchestration technologies like Kubernetes, and the healthcare industry’s complex relationship with cloud computing.
We look at the reasons to use Kubernetes as a containerization orchestration tool. Kubernetes represents about 80% of the orchestration tools that exist. The market position is known, skill-sets exist for it, it can scale, and it can provide a one-stop shop to do something effective with containers. In all, companies are not doing a lot of containers today, it is still a small part of the cloud, but it will continue to expand rapidly.
We also look at the healthcare industry’s complex relationship with the cloud. Healthcare is behind the curve in terms of how they’re leveraging technology. The amount of automation that can occur and good that can be done with technology in the healthcare industry is immense. They can move a lot faster with moves to the cloud.