We recently migrated all of our KARL Hosted Clients to a new infrastructure. Six Feet Up is the exclusive provider of hosting services for KARL, a knowledge management solution. Previously, we provided each of our KARL clients with a separately-managed Xen (a virtual server) to host their KARL Application.
The new infrastructure we deployed no longer requires a separate Xen for each client. This substantially has increased the efficiency of our hosting support efforts. This reduces the number of maintenance requirements of items such as operating system patches, dependency patches and other issues outside of KARL for each client. These improvements allow us to provide much more attention to the maintenance of the KARL application itself for each client. Additionally, it allows us to better utilize the system and disk usage requirements needed to run our clients' KARL sites.
Once discussions began about moving to a new infrastructure, Six Feet Up and the KARL Development Team worked together to create a pilot project to evaluate what the scope of a migration and how to evaluate the performance of the pilot. Once the pilot project was completed and its results were evaluated, the decision was made to move forward with the new infrastructure. To do this we migrated all of our existing clients to a new single hosted platform free from Xens.
After the migration, I became the release manager for our clients' KARL Hosted Sites. I will manage the updates and the deployment of new clients' KARL sites. By managing the releases for KARL updates, Six Feet Up now is providing more support for our KARL clients and allowing the KARL Development Team to spend more time on KARL application improvements. With the KARL Development Team, we implemented new methods to deploy multiple production KARL Sites on one machine and worked to automate as much as possible.
With the new infrastructure, I have set guidelines for standardization. Because of the decision to put more standardization in place, we now use a common directory layout, setup and configuration style on each KARL client. Since we have that more uniformity, we found it much easier to automate more of our processes. Before we actually performed the migration, Chris Rossi of the KARL Development Team and I wrote scripts and developed processes to automate the release of an update. This has been a great launching point for me to begin to automate as much as I can to manage our KARL client sites.
During the week of the migration, we accomplished almost no down time for each client. We achieved this by spending the necessary time on each client's configurations and standardizing them as much as possible. Next, we deployed to the new server using our scripts. By updating our /etc/hosts files, we could verify that the site was working correctly while the world was still going to the production site on the old Xen.
When we were ready to go live, we did the following steps in rapid succession per client:
- Sync data to the new server for this client
- Put up a maintenance page on the new server for this client (The world was still going to the site on the old Xen at this point)
- Turn off mail processing on the new server for this client
- Redirect mail from the old Xen to the new server and watch the logs to verify the e-mail was successfully making the redirect to the new server.
- Restart Apache on old Xen with a proxy redirect to the new server. (The world sees the maintenance page on the new server by being redirected there from the old Xen)
- Final Data Sync from old Xen to new server (This took from <1 sec to < 1 minute typically)
- Put client site on new server in production mode (Users could use KARL site again on new server)
- Restart mail processing on the new server for this client
- Update DNS to new server/Notify client if they manage their own DNS that they were ready to update their DNS
It was a resounding success. We migrated all of our clients without any major surprises and with very little down time.
We are excited about the new infrastructure. We expect to be able to roll out more new features for our clients in a faster, more standardized fashion to benefit all of our KARL clients. In addition, this migration allows us to deploy KARL to new customers more easily.