Struggling with CI and CD

Continuous Integration and Continuous Delivery are two practices that are supposed to boost the development, integration and delivery of features, bug fixes, etc.

Someone may claim that they actually improve only the QA/release side of it, but I like to take the position according to which “done is released” - this is certainly not something I invented -  I can’t remember which book I have taken this from (maybe Continuous Delivery?).

Software development has changed, it has become an industry, still we struggle to reach common definitions. We are full of best practices, de-facto standards, and so forth, yet it’s so easy to end up with people without a clear understanding about certain topics, about the reasons things could be done in a certain way instead of doing it “because it has always worked for us”.

However, I have noticed a particular pattern: when something is a “practice” it gets often misunderstood. The best example is a REST web service. I really don’t want to get into this discussion again, because REST is an interesting concept, yet it has a million implementations. So many that developers in the end wind up frustrated with different ideas of REST.

Pragmatism is certainly important in our field, yet sometimes it’s important to have a common dictionary, so that we can refer to roughly the same concept, especially within the same company. This is unfortunately not always the case. CI and CD are only the tip of the iceberg, together with many other examples, because I think they are difficult to implement.

Why so? Because it’s actually difficult to understand how you want to do things and what you want to do with them. It’s a practice, not a tool, therefore it needs to be understood first, it can be adapted to your needs, why not, but it should not be seen as a savior, because it’s not going to solve all your problems.

If you want to “do” CI and CD correctly, you need to have the right problem, the right mindset, and the right tools. I would personally not do CD with medical software or space components - my experience with these fields is too little, so I can’t say much - it’s just a rough idea.

However, given the right problem (which is difficult to define!), you need to have people with the right mindset: we need to do something new, and this is going to have a severe impact on how we do things here - you won’t be anymore a QA or a DEV, but you will take care of everything from A to Z. Literally.

And, of course, you need the right tools, otherwise it becomes a pain in the *** to manage all these pipelines, tasks, failures, rollbacks, etc. Fortunately, nowadays we have plenty of them, even open source.

So, what is the sort of problems that CI/CD try to help us solve?

I would say that the first and foremost problem they help us with is the time to market - which is essential in business. Then everything else comes almost as a consequence, like “batteries included”:

• code is always in a deployable state
  • it has passed all the QA rounds of testing, etc.
    • it offers a feeling that things are safe/green, which is always good to have
  • it was built, so it’s ready to be installed, etc.
• tendency to have metrics-based pipelines
  • for example, if some component doesn’t reach X% of coverage etc., then it won’t be promoted to next stage

What kind of mindset does it require?

It asks people to take what they have always done, wrap it up, and throw it away. Sometimes it even asks them to wipe their *** with it. Pardon my French.

It requires people to think in a way that is deterministic, repeatable, stateless, yet in units that have to be integrated to make “the whole greater than the sum of its parts” (just to mention Aristotle).

It’s not enough to say “we need to commit on a daily basis”. It just doesn’t work that way. Same applies to “we need to achieve X% coverage so that the builds are self-testing”, where X is a ridiculously high number (considering that now the coverage is below the sea level). That will be not only counter productive, but will end up with frustration.

Depending on where you work, this may be harder or easier to implement. Having the right tools here helps a lot, educating people helps even more. In my opinion, the best way to achieve something here is by taking the time to explain the value this new approach offers, compared to what has been done so far, together with all the challenges this implies.

What’s the lesson here?

I think progress is never easy to achieve. It takes courage, an open mindset, some stubbornness, and sometimes also the honesty to say “ok, it doesn’t work this way, maybe it needs some improvements”.

Further Readings

There’s plenty of material to learn about CI/CD, however some of the most important articles about these two practices are from Martin Fowler:

Continuous Delivery Continuous Integration