This post is for admins and DevOps engineers looking to solve the common problems that come with deploying applications to the cloud at scale. When dealing with many different types of users across diverse projects, it gets harder to maintain user profiles, accessibility to resources, and standardize the way users access infrastructure.
Business agility is one of those trendy business phrases that you’d struggle to get a definition for, like ‘synergy’ or ‘digital transformation’. But in that buzzword is some real value - like how enterprises can manage their cloud infrastructure to deliver better products, faster.
Business agility is how companies approach operations and resources in a flexible manner. It’s the organizational ability to deliver products, tackle new trends, and make better use of resources for everyone. In other words, the company’s ability to quickly and safely change its mind.
On the operations side we ask ourselves: how do we enable our developers to operate quickly? This is where the Agile movement became popular: using lean teams that focus on delivering specific features in strict amounts of time. Instead of looking at IT divisions as lumbering, large-scale release operations, we take it at a day-by-day or weekly release cycle. Add that with the ability to quickly react to market forces and the winners emerge every quarter.
The most recent example of this being how Instagram was able to adopt Snapchat’s product features in months and deploy them to bigger audiences. Then with the feedback of those bigger audiences, they tweaked the product to boost interactions. It’s not just about having better features these days - it’s about delivering those features to your audiences in record time.
But it gets tricky - more developers and more business units and more products naturally makes it difficult for enterprises to move at the same level. Despite that, Agile works at scale. Many big companies have seen revenue, customer satisfaction, and company culture all get a boost by adopting the Agile management framework. Spotify has a great way of tackling this problem through organizing teams through tribes - groups of developers with like minded ideas who can operate with minimal overhead to drive their ideas forward.
On the resources side there’s a few things that play into business agility. The first is creating continuous delivery/continuous integration pipelines to get products out smoothly, which is why DevOps has become so important in the modern tech company. The second is empowering your engineers to innovate and try new things. The third is leveraging services across cloud providers. A majority of companies have multi-cloud strategies and use each provider for different reasons. Companies can visualize data using Amazon RedShift, horizontally scale databases across the globe using Google’s Cloud Spanner, or deploy Kubernetes on OpenStack to create clusters using pools of resources. Different projects and applications have different needs. So how companies effectively manage and use resources across clouds is vital.
Here’s a real life example of a lack of business agility: my friend worked for a financial services company that dealt with massive surges of traffic during tax season. They were processing large amounts of data throughout the night and receiving heavy traffic to their public websites during the day. They had spun out bits of their monolithic architecture into microservices on the private cloud, but even so - every inch of this company’s resources were maxed out. On top of that, because of the financial and security policies behind setting up new servers, they had to order new racks for the seasonal surge six months in advance. Today they’re getting crushed by other companies that have migrated those data crunching workloads to the cloud to stop worrying about micromanaging costs, over-provisioning servers or having sysadmins rushing to put out network fires.
But this company was concerned because security could be a liability, and so that risk aversion hurt their business and stifled innovation. One year later, they’re just starting to dip their toes in OpenStack based environments in production, but by next tax season it might be too little too late. You may say it’s impossible in 2017, but changes at scale take a lot of convincing across business units (i.e. what Accounting cares about is different from what DevOps finds necessary versus Security).
That perspective is understandable - when user data is sensitive and there are a lot of entry points to access infrastructure, strict security and network controls are necessary. But for that control, here’s what you lose on: slower delivery of customer value, longer time to revenue, employees are less engaged, and it’s harder to change priorities to try new things.
People quit, and the ones that stay go behind the company’s back to get work done. Instead of dealing with convoluted ordering portals and lengthy cycles, I’ll have my team use an AWS account hooked up to a corporate card. This is where costs usually get out of control - people doing good work but going behind the scenes to make it happen.
So is there a solution? Well, there is a middle ground between enabling flexibility and maintaining control which can help solve those problems.
On the operations side, business units, regardless of their size, should be able to identify opportunities, then plan strategies and deploy them quickly. When I worked as a developer we used Pivotal to operate with the Agile methodology. Every morning we would say what we were working on, say if we had any blockers, and focus on pushing code to production every Friday. We were encouraged to plug our datasets into Redshift and track trends, then voice any interesting notes or concerns we had. If there was an opportunity or a problem, we could easily shift directions.
Now that I’m in Marketing for Scalr, we use Asana to do the same thing. We break up our different projects: public website updates, Blog management, Social Media, and use that to map our strategies and track ROI. The best part is that like Pivotal, everyone can see our tasks, make suggestions, and visualize how we divide and conquer. If the Sales team notices that customers keep asking questions about certain tools or services, we can make sure to improve our messaging to make that clear.
Because what we’re working on is transparent to the rest of the company and constantly evolving, it’s easier to make suggestions. It may not be as vital as the cloud provider you use, but when everyone is on the same page it’s easier to guide the ship.
On the resources side, there’s a million tools and workflows that help companies deploy applications faster and more efficiently. So why don’t enterprises use them? Well, you need champions that’ll drive adoption, plus the work that goes into updating how processes is daunting to companies that are resistant to change. Because standardization is necessary in big companies, pushing people to new tools isn’t easy.
But if there’s one place to start, it’s organizing the resources inside your cloud providers (AWS, GCP, Azure, etc.) to focus on high deliverability and giving users the freedom to experiment within particular environments.
Here’s an example of organizing cloud resources using our own product, which works as an abstraction layer on top of clouds. We can enable developers in testing environments to spin up servers in whichever cloud they choose and as an admin we can control what they can change at deployment time, like instance size, security groups, and SSH keys. We can even automate that servers within the testing environment get spun down after a few days. Beyond servers, you can also design and deploy complete application stacks, regardless of their cloud provider. Our hierarchical model also allows you to easily enforce changes at scale, migrate apps, or make them available to users. And because we tap into pricing from public cloud providers, we have detailed billing and cost reporting across clouds. Developers, admins, finance, and DevOps all get what they need. Most importantly - we try to make all of this automation and standardization happen behind the scenes, so everyone stays happy and focused.
For example, let's say Amazon introduces a new generation of instance types, and you want to offer up those instances to a subset of the applications your app teams use. Scalr allows you to do this at a higher scope, and have the action propagated to all relevant applications.
No matter what tool you use to improve business agility, the outcome is that there’s fewer limits on how your developers can use cloud resources. In addition, finance is less concerned about taking financial risk if you set up preventative financial controls, DevOps teams knows that the workflows they’ve set up can’t be ruined by users going off the rails, and admins know they can see everything from one UI. With Scalr, we contribute to business agility by enabling companies to change these policies safely and at scale. In freely using cloud resources teams can identify opportunities, change product strategies, then design products and then deploy them in shorter cycles. And isn’t that what it’s all about?
If you have any questions, feel free to email me directly at firstname.lastname@example.org.