Taking the complexity out of the cloud - birth of "Super Clouds"
The “big three” cloud providers, Amazon Web Services, Microsoft Azure and Google Cloud Platform provide the most extensive cloud infrastructure, platform and services solutions worldwide. These and other cloud providers, have their respective cloud data centers located in multiple countries and work with local governments to ensure they adhere to data regulation and privacy laws while meeting the scalability, service and uptime(99.9% ) needs of any organization, large or small.
The industry across domains has no doubt made rapid strides leveraging cloud services with order of magnitude improvements as evident in their accelerated speed of deployments in production, provisioning at scale, higher business agility and higher customer satisfaction besides helping in transforming their digital landscape.
Cloud Complexity
The enterprise perimeter has evolved and is now difficult to demarcate like a convoluted inaccessible border. Cloud adoption has created a mesh architecture that looks more like a spaghetti with multiple enterprise islands fawning up. All this has led to complex challenges including: pricing complexity, realization of cost savings, performance issues, customization ease, potential vendor lock-in, data transfer costs, specific regulatory considerations, gray areas in service level agreements(SLAs), multi-cloud interoperability, absence of multi-cloud SLAs, cloud architectural drift, latency in business critical applications, legacy applications migration beyond lift and shift … the list seems long enough.
A lot of these challenges can be handled by having a proper cloud management and governance team defining policies on roles and access, monitoring alerts and metrics. For example, hourly checks may be appropriate for detecting role changes in identity access management(IAM), while daily check could suffice for less critical cloud services. However, some of these relate to the complexity of cloud and can still manifest in any enterprise architecture.
In every enterprise, there are cloud workloads that undergo frequent changes as more workloads and services are deployed to the cloud, resulting in more developers and authenticated services interacting with the infrastructure across various cloud environments and providers. Architectural drift or infrastructure drift occurs and just like ticket incidents, are part of the operations life cycle, more so in complex enterprise environments. Architectural drifts in your enterprise applications on cloud are difficult to detect unless a breakdown happens much later, when all hell breaks loose severely impacting business.
Multi cloud environment today is used for deploying different workloads rather than splitting the same workload. This brings another complexity of ownership of traceability of a breakdown or deterioration in service in such an environment. It therefore becomes crucial to be able to easily and quickly detect and possibly revert the drift or breakdown.
Yes, the industry has offered multiple solutions to provide scalability on demand, abstraction of infrastructure etc to address some of these complexities but there is still a long way to go. Some solutions include:
- Hyperscalers, for customization at will at component level including bare metal components for performance
- Hyperconverged infrastructure(HCI) as an integrated system that combines compute, networking, storage, and virtualization into a single platform and can help reduce costs and complexity
- Pre-configured data workloads like datalakes, data lakehouses/data mesh/data fabric
- Service mesh for ease of handling APIs for microservices
- Edge computing clouds for low latency
- Bringing cloud to your data center (Google Anthos, AWS Outposts, Azure Stack) ensuring no data leaves your center for regulatory compliance needs and performance
- Tools like CloudFormation with built-in infrastructure drift detection feature
- Vendors providing private data center solutions helping standardize thereby easing public cloud migration
- Evolving patterns like strangler for legacy applications migration and gradual conversion to microservices architecture
Looking ahead one must be able to build truly distributed multi-cloud applications as well that avail best in class services or resources from multiple cloud vendors. The application architecture will need to handle synchronization of threads, shared services with failsafe switchovers in real time. On the cloud front the vendors need to evolve interoperability standards enabling uninterrupted operations including seamless migration if required. While standards like Kubernetes exist but do the cloud players have the will to work together to allow seamless migration like the telecom industry guaranteeing universal number portability? Is there a way to holistically address these challenges?
AI Super Cloud
Taking the complexity out of the cloud we believe will need the industry to move towards “AI Super Clouds”. These super platforms will make working across clouds seamless, providing monitoring with traceability alerts right down to the component, container or pod level in a Kubernetes cluster. These super platforms would themselves need to work across multiple players providing both supercloud and cloud level abstraction. Beyond standards, regulations will need to enforce this.
The AI or generative AI models in the super cloud will predict and auto heal some potential faults or service deviations before they occur. There will be large or small action models (LAMs/SAMs) prescriptively implementing solutions for other faults or regressions, moving us to an era of auto resiliency across or within any distributed workloads. Multiplicity of such platforms, whenever they become a reality, will prevent vendor lock ins and hopefully help us realize our cost savings beyond rapid provisioning/de-provisioning!!!
Super Architects
AI Super clouds will need Super Enterprise Architects! Enterprise architecture is so intertwined with cloud architecture and data (along with AI) now, that it needs a different growth trajectory which needs to be addressed by the organization’s learning and development department.
The potential evolution paths will need to transform the enterprise architect into an aspirational super architect role!
Architecture Description Languages
Additionally, enterprise architecture has become very complex, more so on the cloud, with rapid changes and deployments being the order of the day. Super clouds allowing distributed workloads will only accentuate this problem. Prevention of architectural drift needs to be addressed in an automated way through software-based definition of these architectures using Architecture Description Languages(ADLs). Their coming of age will spur a revolution like its HDL(hardware description languages) counterpart did for the chip industry. Automated regression suites will detect and revert unintended architectural drifts and ensure architectural compliance at all times!
This will hopefully meet the leadership expectations from their cloud deployments in consonance with their strategic plans on business agility, scalability and cost!