I have been reading recently about the “Lean Startup” concept, which is funny since I have been working with and been part of more than a few without knowing there is a nifty catch phrase for them. So now I am all up on the lingo and hash tag #leanstartup I can speak on it. Today is the perfect time to take a gamble and dive into a lean startup. Being that there are so many free tools and services you can use along with very inexpensive solutions to infrastructure that were not available even a few years ago. By using the power of Open Source software, free online services, and cloud computing one can build a scalable and very cost effective lean startup.
Let’s first look at your new companies hosting and servers. being that you are lean you may most likely be working from home. So let’s look at hosting and services you will need in order to host some of the Open Source applications will will talk about in a moment. In my experience the basic things you need to get going in terms of setting up shop are the basics:
I wanted to touch again on the use of Subversion (SVN) to populate the /var/www of app servers on Scalr. Basically the issue is how to add your web content to a new instance once it has automagically launched a new instance due to high load. So Scalr will launch another app role once the server reaches a load threshold you have previously set. So the issue is I can have the instance started, but once it has launched the /var/www needs to be populated for that server to be able to serve content via load balancer.
This is where SVN and Scalr Scripting come into play. I keep all my site content in a SVN repo. So I link to whatever production tag I want to be live at that time. In order to get the directory populated I make a simple script to do an svn checkout of that tag to /var/www. A simple bash script is added to do the checkout and is added to the “OnHostUp” option. This way once the server sends its SNMP trap saying it is up the script will be executed. This is also a helpful means of updating your servers to a newer build. I DO NOT checkout the tag directly into /var/www instead I make a symlink to /var/svn where the tags are checked out. So when it is time to roll out a new production tag I simply checkout the new tag to /var/svn and redo the symlink to point at new tag. This way if there is an issue that was not forseen in the QA process I can roll back to known good tag by redoing symlink. This is an easy but very effective way of using Scalr scripts and SVN to manage content loading on servers.
I am a big fan of using Subversion for things other than just versioning code. In the past I have used SVN to manage configuration files across many servers. Making it easy to deploy and (if needed) roll back changes. It is also a big help on development servers for PHP developers to commit changes and see them live on the development server. This is easily accomplished using SVN hooks. There are plenty of HowTos on this topic if seeking that information.
Now that I am up to my eyeballs in Amazon Web Services I am looking to use SVN to help me leverage the new found power of the cloud. Now I am not saying it is a good thing to use SVN for things other than code versioning. But it has always worked for me in many other ways as well.
Some of the things I am thinking of using SVN:
- Update DEV web server using hook scripts for devs to see changes to trunk.
- Maintain Apache and other config files for AMIs.
- Maintain code repository for versioning along with take advantage of S3 for backup and processing power of EC2.
This is a work in progress so I am looking to perfect the design to my liking soon.
I have been working on some new projects since leaving my last job about 6 months ago. One is to build an entire infrastructure that is highly available and redundant. With Amazon Web Services this is a snap and almost makes my job obsolete. With EC2, S3, EBS, and CloudFront you can build a scalable solution with dependable backups with ease. My goal now is to also use AWS to create an intranet for the company. I have not found much on this topic so I ma taking the time to document it here. My ideas so far:
- Use Fedora Directory Server as main LDAP solution. This can be used with EBS (Elastic Block Storage) with striped volumes to store the LDAP data. Along with having redundant multi-master replication geographically depending on where the EC2 instance is set.
- Subversion server using EBS for storage. Using EC2 for something along the lines of SVN is a good use since it speeds up performance. Along with having the backup ability of EBS and EBS Snapshots
- Twiki as documentation for intranet.
- SugarCRM for customer relationship management which will include project management and bug tracking.
- S3 for backing up users data automatically. This is helpful since it is a telecommuting company. Using a tool like JungleDisk or similar.
- The DEV and Staging environments will also be on AWS with same Amazon Machine Images (AMIs) as the production environment.
That is all I have so far I will update as the project comes along.
At my job I have recently setup an easy means of managing code as well as the whole chain of development. All without the need of giving developers direct login access to servers. Another bonus of using SVN is the ability to roll back changes in real time if needed. For our organization I have set up the following:
-Development environment consisting of a large server running OpenVZ ( I will write another entry soon on server virtualization) with images identical to the production servers. This allows the developers to have their own VPS (Virtual Private Server) to code and test on. All without fear of blowing it up. If this happens I simply type a total of 5 commands to destroy and rebuild the VPS.
-Staging environment consisting of a staging server that also hosts a stripped down data set of our production DBs. Here we have the development code that has made it to the point of being ready to be QA’d. All developers have access to Staging via SVN. They check out a working copy of the repository and work on it via their VPS or desktop. At the same time the /var/www of the Staging server itself is a checkout of the SVN repository. Using SVN hooks (namely post-commit) that once a commit is made it forces a svn update /var/www to insure that when a developer or client visit the staging site they see the changes.
-Finally there are the live web servers which have their own seperate repository where the QA person moves files from the staging working directory to the live working directory. Then does an add/commit.
All of this creates an audit trail of what has been pushed live and who/what changes were made. With our change request system each commit is refernced to its CR. So by viewing the SVN log for both staging and live you can see what is what. If a file pushed live has an issue you simple roll back to the previous version. There is no need to guess which files and if all were pushed to begin with.
I am very pleased with how this is working and the developers like it much more than our previous versioning system. For the Windows minded developers I have found Tortoise to be the easiest for them to use and understand.