Tuesday, February 17, 2015

"From complete chaos to Octopus Deploy" Part 7: Load balancing

Introduction to this blog series

How do you automate the non-existing deployment routines of an organization with over 100 different customers, each having their own environments? How do you convince the leaders, developers and customers to give you the resources needed in order to automate everything?  Is it really possible to introduce a routine that works for everyone?

Load balancing

I've received some questions as to how we handle load balanced scenarios in Octopus Deploy, so I decided to dedicate a post to the topic. 

Not all hosting vendors allow us to interact with their load balancers for a variety of reasons, where the main reason seems to be that they don't enjoy easing up on their control of the infrastructure. And that's fine. I mean, we've already 'forced' Octopus Deploy on them (although I believe they should be grateful), so I think it's only fair to let them hold on to their load balancers until they've gotten used to the idea and seen that Octopus is not an evil monster corrupting their servers.

Let's talk about the load balancers we have been allowed to play with instead! The first one out was the Windows Network Load Balancer (not a very good one, but that's a different story) that has it's own set of PowerShell cmdlets. As we were able to script against it we added two script steps to our deployment process, one removing a server from the load  balancer and one for adding the server back: 



If you're not familiar with the concept of child steps in Octopus Deploy, what it does in short is "allow you to wait for a step to finish on one machine before starting the step on the next machine." Read the Octopus Deploy documentation on Rolling Deployments for more information. So in the screenshot above, steps 1.1 to 1.4 will be finished on one machine before they are executed on the next.
You might be curious of what the "Remove from load balancer" and the "Add back to load balancer" steps look like. These both run a PowerShell script towards the Network Load Balancer using a couple of Octopus variables, and I've created these scripts as Gists: 
But what about those load balancers that don't have PowerShell cmdlets available, what to do then? Our hosting vendors have several creative solutions to these scenarios. 
Example 1
The load balancer monitors a port on the server and a site in IIS is set up with a binding towards that port. When the load balancer notices that the site is unavailable it drains all traffic to the server. When the site goes back up, the load balancer starts directing traffic to the server again. 
In this scenario, the PowerShell scripts for removing the server from load and adding it back simply have to stop and start a site in IIS.
Example 2
The load balancer monitors a file on the server. When the load balancer notices that the file is removed it drains all traffic to the server. When the file appears again, the load balancer starts directing traffic to the server again. In this scenario, the PowerShell scripts for removing the server from load and adding it back simply have to move a file to an alternative location and move it back to its original location afterwards.
So there are creative ways of interacting with the load balancer without actually scripting towards it directly. The main challenge with the two examples above is knowing when the load balancer is finished draining the traffic and the server has been removed from load, and knowing when the server is back in load. In these cases, I find that the hosting vendors have a lot more knowledge about how this can be done than I do. So my main advice is to talk to your hosting vendor and ask them for advice on how you can make this possible.
In tomorrows post it's time to wrap this blog series up with Lessons learned and useful resources

No comments:

Post a Comment