Cloud Infrastructure Automation Shows Promise in Infancy

Many of today’s retail companies are no longer interested in hosting and maintaining their own digital storefronts, preferring instead to focus their resources on innovation that drives a channel-less experience for their end-consumers.  This has created a rapidly expanding opportunity for technology implementation companies to provide Managed Services for their customers.  In fact, it has virtually become an expectation of retailers that their implementation partner be capable of hosting and maintaining the eCommerce solution that they implement.  In an effort to differentiate themselves these organizations have developed proprietary cloud offerings for platforms like SAP Hybris and Oracle Commerce (ATG), the most advanced of which enable auto-scaling and auto-healing at all tiers.  While these solutions pose major benefits to retailers the Managed Services organizations often struggle to operate at scale and quickly on-board new customers.

Infrastructure Automation

Amazon Web Services (AWS) pioneered a solution to these challenges by introducing the concept of “Infrastructure as Code” and developing their own utility for automating the formation of infrastructure called CloudFormation.  This concept is now referred to as “Infrastructure Automation”.  Many Managed Services organizations have developed custom Infrastructure Automation solutions using a combination of AWS CloudFormation, the AWS Command Line Interface (CLI), and procedural scripting tools (like Apache Ant or Apache Maven).  Their solutions enable the creation of entire environment landscapes and dramatically reduce the time to on-board clients.  However, they generally fall short in a few areas:

1. Custom Solution: Since they are generally custom solutions there is little investment made in supporting, troubleshooting, and evolving the utility.  Additionally developers may need to learn a multitude of scripting and definition languages to use and support the custom solution.
2. No Maintenance Support: The onboarding solutions generally only accommodate the initial formation of infrastructure.  Once it is formed, any adjustments to the infrastructure still need to be performed via the AWS CLI or the AWS Management Console.
3. Platform-Specific: The solutions typically rely on specific IaaS technologies and, as such, are platform-specific and not easy to port to another IaaS offering.

Thankfully, a couple of Infrastructure Automation utilities have been released that do a great job in solving these shortcomings.

Infrastructure Automation Utilities

In the past couple years a number of Infrastructure Automation Utilities have been released that serve as an abstraction layer on top of platform-specific infrastructure code.  These utilities provide a common language enabling the formation of infrastructure that may span multiple underlying IaaS platforms without concern for any IaaS-specific syntax.  While these solutions are still in their infancy, they show incredible promise.  We’ve attempted to capture some areas that we think highlight the benefit of these Infrastructure Automation Utilities and some roadblocks that you might encounter if converting to them from a custom solution today.  While specific to one of the more mature offerings, Terraform, our hope is that they generally highlight the current maturity of the space.

Benefits

1. Much more compact than traditional JSON and provides the ability to specify lists of items in resource assignments instead of one at a time.
2. Capabilities have been introduced that make these tools a hybrid between traditional definition languages and scripting languages. For example,
   1. Utility functions have been developed that make referencing dynamic attributes of other resources possible.
   2. Loop-like capabilities enable the creation of multiple resources of the same type with only 1 resource definition.
   3. Ability to define an unlimited number of loosely-typed variables and data structures such as arrays, maps, and lists.
3. In addition to supporting providers for the top IaaS offerings (AWS and Azure), the providers themselves offer a much more intuitive way of defining many resources than with the IaaS offerings.

Roadblocks

1. Variables cannot reference other variables.
2. No support for nested-interpolation.
3. No support for conditional creation of resources.
4. While fairly comprehensive, syntax validation has trouble in “here doc” statements.
5. State is not always accurately managed.
   1. If a script fails and you destroy resources, many times not all resources are destroyed.
   2. Running updates causes integrity issues on select resources.

So What?

Infrastructure Automation Utilities, like Terraform, are a great alternative to custom automation solutions and platform-specific automation solutions but still not suitable for maintenance.  While in their infancy, they show great promise and are only going to get better.  If you’re willing to develop a couple workarounds while the offerings mature, they’re a great solution for anyone’s Infrastructure Automation needs.