A New Workflow for Cloud Development

Leverage the benefits of containerization without the headaches & hassle
John Amaral
Apr 13, 2021

We’ve heard how developers struggle with the current Docker toolset and why container best practices don’t work. What can we do differently?

At Slim.AI, we’re engineering a new workflow that puts developers in charge of how they build and optimize their containers, leading to fewer issues down stream, faster development loops, and better security.

Our goal is to reduce the friction between application developers and DevOps teams by providing tools that automate the process of building and shipping containerized applications. We want the cloud-native development process to keep shifting left.

We know that DevOps teams want small, secure, lightweight images that are easy to handle in the CI/CD pipeline and run smoothly in production. And we know that developers want full-featured images that have the tools and libraries they need to speed development. But how do we get there?

Let’s take a look at how the developer workflow has changed since the advent of Docker and modern DevOps workflows.

In the pre-Docker era, we would play an endless game of ping-pong between Development and Operations. Teams would request resources, script deployment, and enter the “commit-and-patch” cycle as they triaged dependency issues or requested Ops support. We called this “dependency hell”.

h/t coolabnix.com for original diagram

These iterative loops were not only annoying, they were time consuming and open to security breaches. Most detrimental, in the view of DevOps philosophy, they made frequent, continuous deployments nearly impossible.

Docker promised to counter these iterative dependency loops. Dependency issues could now be sorted out in local build-test cycles and ops teams could focus on providing configuration details necessary to deploy through the CI/CD pipeline.

h/t coolabnix.com for original diagram

What this workflow doesn’t show, however, are the critical pain points that developers face even in this streamlined process:

  • Building a safe and optimized container is difficult, labor intensive, and requires “tribal knowledge”
  • “Dependency hell” hasn’t been removed, simply moved to the application development process, reducing some of the chuck-it-over-the-wall dynamic.
  • Developers often don’t know what’s required for an image to work, so they leave unnecessary files, libraries, and dev tools in their containers, causing bloat and slow startup times.
  • Generic downstream systems don’t have a reliable way of optimizing images. This becomes particularly nefarious in the increasingly popular serverless deployment environment in a managed Kubernetes cluster, where your container may take longer starting than it does actually running.

Let’s look at how it works today, using one of our favorite cloud platforms, Amazon ECR, as an example:

  1. Dev writes code, packages into Docker, and commits
  2. System ships code through pipeline
  3. Automated processes compress images and run automated tests
  4. Container gets pushed to cloud environment (be it dev, staging, or prod)

It’s not a bad state of the world if you make two massive assumptions: 1) the developer is able to find the right Docker container and build it correctly, securely, and in a timely fashion; and 2) the system knows what’s necessary and what’s not for optimization and security at each state of the SDLC.

In our experience, neither of these are true.

The workflow we are proposing at Slim.AI takes the manual effort out of both the Build and Deploy steps above.

Our guiding principles to achieve this vision are simple:

  • We need to change our mindset on container optimization: It is not just about size, performance, and security. Containers also need to be developer-friendly.
  • Developers need tools that automate container optimization across all stages of the SDLC, reducing manual efforts to move from dev to production.
  • Our tools must also simplify common tasks like debugging, security, compliance, and continuous improvement, allowing devs to focus on creating great apps.

We think this new workflow will change the game when it comes to how we build apps. It will give developers more control, make systems safer, and increase throughput in the system.

Interested? Stay tuned for the the fourth and final article in our series for a description of the tools we’re building right now. If they sound like something you’d like to test out, get in touch and we’ll send you an invite to our private beta.

Related Articles

5 Most Commonly Asked SlimToolkit Questions

We enlisted SlimToolkit expert and Slim.AI Developer Experience Engineer to dive into how container slimming works.

Primož Ajdišek

Technical Staff

5 Ways Slim Containers Save You Money

Do slim containers really save you money on your cloud bill? Are there cost advantages to smaller containers? Find out here.

Chris Tozzi

Automating SlimToolkit in Your CICD Pipeline

Using GitHub Actions, you can refine container images automatically making them smaller, faster to load, and more secure by default – all without sacrificing any capabilities.

Nicolas Bohorquez

Contributor

Building Apps Using Cloud Native Buildpacks

Getting started with this innovative technique

Vince Power

Contributor

Building SlimToolkit into a Jenkins Pipeline

A step by step tutorial on building SlimToolkit into your CI/CD pipeline.

Clarifying the Complex: Meet Ivan Velichko, Container Dude at Slim.AI

Ivan recently joined the team at Slim.AI, and we sat down with him to learn more about the path that led him here.

Ivan Velichko

Container Dude

Container Insights: Dissecting the World's Most Popular Containers

Join Ayse Kaya in this series, as she creates her 2022 Container Report Chalk Full of Important Security Findings for Developers.

Ayse Kaya

Analytics & Strategy

Container of the Week: Python & Flask

Our weekly breakdown of a popular container

What We Discovered Analyzing the Top 100 Public Container Images

Complexity abounds in modern development

Ayse Kaya

Analytics & Strategy

2022 Public Container Report

Vulnerabilities continue to increase and developers are struggling to keep up.

Ayse Kaya

Analytics & Strategy

Containerizing Python Apps for Lambda

A tutorial on deploying AWS Lambda using containers, Python edition.

Docker Containers for Your Raspberry Pi

Compact PCs need compact apps

Martin Wimpress

Community

Explore and Analyze a Docker container with SlimToolkit's X-Ray

Understanding container composition

Martin Wimpress

Community

Five Things You Should Never Ship to Production in a Container

Here is our take on five things to avoid when creating a container or shipping it to production.

Chris Tozzi

Five Proven Ways to Debug a Container

When Things Just Are Not Working

Theofanis Despoudis

Contributor

Increasing Your CI/CD Velocity with Slim Containers

We’ll explain what Slim Containers are, how they speed up the build process, and how they can improve the efficiency of your testing.

Mike Mackrory

Contributor

Integrate Testing into Your Container Pipeline

A closer look at testing within container pipelines, CI/CD, software delivery, and containerization.

Faith Kilonzi

Software Engineer

Reducing Docker Image Size - Slimming vs Compressing

Know the difference

Pieter van Noordennen

Growth

Serverless Applications and Docker

How to Scale the Latest Trend in Infrastructure

Pieter van Noordennen

Growth

Slim.AI Docker Extension for Docker Desktop

How to access our Docker Extension and try it for yourself.

Josh Viney

Product

Slimming a Rails Application with SlimToolkit

Dissect a simple Rails application container using SlimToolkit to analyze, optimize, and deploy your product more quickly.

Theofanis Despoudis

Contributor

Where Do You Store Your Container Images?

Container Registry Options are Growing in Number and Complexity

Pieter van Noordennen

Growth

Using AppArmor and SecComp Profiles for Security Audits

Conduct better container security audits using tools like SecComp, NGINX, and Docker.

What’s in your container?

Why Docker Layers matter for container optimization

Pieter van Noordennen

Growth

Why Developers Shouldn't Have to Be Infrastructure Experts, Too

Simplifying processes required to containerize and deploy cloud-native apps.

Chris Tozzi

Why Don’t We Practice Container Best Practices?

Container best practices are easy to understand, hard to do

John Amaral

CEO

5 Best Practices Production-Ready Containers

Knowing what’s in a container is critical to securing your software supply chain.

Martin Wimpress

Community

Better Security Audits with AppArmor and SecComp via SlimToolkit

Combine the power of tools like SecComp, NGINX, and Docker.

Improving the Developer Experience When Adopting Cloud-Native Best Practices

PaymentWorks Case Study

Josh Viney

Product

The 4th S of Software Supply Chain Security

An approach to Front Line Software Supply Chain Security (SSCS).

John Amaral

CEO

Where Shift Left Goes Wrong

As if cloud infrastructure were not complex enough, there’s a whole new complication to contend with: Shifting left.

John Amaral

CEO

Why We Founded Slim.AI

Developers Just Want to Write Code. How Slim.AI is Helping.

John Amaral

CEO

Cloud Development Is Still Too Manual & Complex

Lessons we learned from interviewing more than 30 developers

John Amaral

CEO

Getting Started with Multi-Container Apps

Up your container game with Docker Compose

Nicholas Bohorquez

Contributor

The Squeak Interview

CEO John Amaral joins Chris on his livestream