Monday, February 9, 2015

"From complete chaos to Octopus Deploy" Part 1: The trigger of change

Introduction to this blog series

How do you automate the non-existing deployment routines of an organization with over 100 different customers, each having their own environments? How do you convince the leaders, developers and customers to give you the resources needed in order to automate everything?  Is it really possible to introduce a routine that works for everyone?
Part 1: The trigger of change
Part 2: Where to begin?
Part 3: The basic concepts of Octopus Deploy
Part 4: Required application changes
Part 5: The deployment process
Part 6: The hosting vendors
Part 7: Load balancing
Part 8: Lessons learned and useful resources

The trigger of change

At the end of 2013, I had been working on a project for some time when I realized I dreaded every new change and feature the customer requested, but I couldn't quite understand why. I usually love developing new functionality, so I found it quite strange that I now felt the urge to slow the customer down. Suddenly it hit me, it wasn't the new functionality I dreaded, it was the deployment of the functionality I dreaded.

The project completely lacked deployment routines. Everything was done manually, and none of the developers felt they had full control of the responsibility of the applications, how the applications interacted with each other and how the hosting vendor had set all this up to work as it should. Every deploy felt like walking through a minefield, the risk of blowing something up was huge and there were regularly fires that needed to be put out because a deploy had gone wrong.

When dealing with manual deployments, you have no immediate knowledge of when the last deployment was done or who did it. It's time consuming and error prone, not to mention tedious. Developers don't want to spent their time copying files from one server to another, especially not when it comes to rollbacks.

I love the quote from Lao Tzu: "If you do not change direction, you may end up where you are heading." 

We were heading in the direction of flushing an otherwise great project down the drain because we weren't able to create routines for ourselves. I complained a lot about this to my fiancée who is also a developer, and he gave me the same answer every day: "Why don't you just automate everything?" I shrugged the answer away every time. Then I realized, that's what all the developers on the project were doing, we were all avoiding the issue instead of tackling it head on. So I decided it was time to take charge and change the direction we were going in.

The next time my fiancée said: "Why don't you just automate everything?" I answered: "But how? Where do I start?" He pointed me in the direction of a colleague of his who had introduced Octopus Deploy to one of his projects and I started looking at this product they all were so excited about. It didn't take me long to see that Octopus Deploy could deliver exactly that we needed, it was almost too good to be true!

Convincing the customer

Now that we had a solution to our issue, we had to convince the customer that the cost of automating the deployment routines would in fact reduce their overall cost of development. I realized our chances of convincing the customer would be larger if the hosting vendor was supporting our case, so I went to the hosting vendor with my plan. They were very positive as they had the same feeling of walking through a minefield as we had, and together we set up a draft of how this could be done.

We decided it would be wise to automate the deployment routines one step at the time, starting with the most basic application in the test environment before we moved on to the more complex applications. After successfully automating the test enviroment, we would move on to the production environment where we'd have to face the challenge of load balancers, SLAs and a more complex infrastructure in general. We were able to estimate the initial part of the job, automating the most basic application in the test environment, but beyond that we really couldn't give them any useful estimates as none of us were in complete control of how everything was set up.

That's a bit ironic, isn't it? In order to get complete control, we needed the customer to accept a guesstimate as we didn't have enough control to estimate accurately. Luckily, the customer trusted us and knew we would never try to fool them in any way, so they accepted our guesstimate and a start date for the automation project was agreed upon. I was thrilled!

When everything fails

But not for long. Unfortunately, things don't always work out as planned. As the automation project was about to begin, we recived orders that all development for this customer had to be paused indefinitely due to their financial situation. I was extremely disappointed, I'd been dreaming of Octopus Deploy for months and now I'd have to move on to some other project instead.

The same day, the CEO of the consultancy I work for (Epinova) asked to speak to me. I'd discussed my intent to automate the deployment routines with him earlier on and he knew how motivated I was for this task. His question baffled me: "Is there any way you could introduce Octopus Deploy to all our customers?"

Just to clarify what he was asking me. Epinova is a consultancy with (at the time) approximately 30 consultants. We had more than 100 different customers and for each customer we were responsible for at least 1 development project. The projects were spread across approximately 30 different hosting vendors.

So what he was actually asking me was: "Are we able to create an automated deployment routine that will suit all our customers? Can you spare our developers time by making them use Octopus Deploy? And are you able to convince all the hosting vendors to play along?"

My answer? "Yes. Yes. Yes."

How would I do all that? I had no idea... But I knew I'd love the challenge! Stay tuned for tomorrows post: Where to begin?

No comments:

Post a Comment