Low cost software organization for open source projects: Allocation and provision.

This is a quite difficult topic to explain, as this problem is solved differently if you are in a medium sized company with a decent IT department, or you're sitting alone in your sofa and all your infrastructure is a microserver at the corridor. And I'm writing this from my sofa for tiny companies or personal projects.

To be honest, there's little point to analyse Allocation (of vms) and Provision (of software) separately, as we will find solutions that brings both aspects integrated or at least highly oriented to one another.

Allocation 

Requirements for allocation:

• It can deal with both cloud and on-premise infrastructure.
• Command line interface available. 
• Low effort in installation and maintenance.
• Low overhead, complex solution will require several boxes running the orchestration.

Non requirements, but worth to highlight that they are not.

• Autoscaling is not required.

Candidates:

• AWS ECS (Elastic Container Service) or EB (Elastic Beanstalk)
•  Obviously this doesn't fulfil the main requirement, if we used this solution, we wouldn't be able to reuse it on-premise.
• Docker
•  Docker has many facets, and among them, a remote-api call might simulate allocation of resources. Not sure if same interface works from AWS.
• Vagrant
•  Perfect tool (or at least the most suitable I know) for the job, as it offers allocation with more than a handful of backends, among them, AWS, libvirt, and docker itself. It also offers a plugins system for hooking different provisioners. 
• Manual
• It's an option, isn't it? You could allocate resources manually from both cloud and on-premise hardware.
• Terraform
• I haven't used this tool, but I think you should if you're still facing the same problem that I am. It support several frontends, but I haven't really found if libvirt is supported.

 

Provision

Requirements for provision:

• It must protect sensitive information, it can't be exposed in an open-source repository.
• Easy to manage / change / expand.
• Linux platform is the only one required.
• Not incompatible with allocation system.

Candidates:

• Chef
• I'm clearly biased towards this provisioner, as it offers free in-cloud service for small number of users (not only chef-solo), it's easily integrated to Vagrant and you end up working in a programming language.
• Secrets are managed in data bags and encrypted data bags, stored in your computer (chef-solo) or in the cloud (chef-server).
• Ruby based.
• Ansible
•  I have less experience with Ansible than with Chef, bear that in mind.
• Serverless, like chef-solo
• Yml oriented, it can be extended with Python.
• Secrets are encrypted and stored with the rest of the configuration data, somewhere in your computer.
• Puppet
• Even less experience with puppet, but for some reason, I don't know anybody that uses it.
• I can only recommend some reading if you are unfamiliar with Chef, Ansible and Puppet, just in case Puppet is better, but I cannot recommend the unknown.
• Docker
• What's docker doing here? Well, it might perfectly be consider a provisioning engine as long as you're actually automating a manual installation of software in a virtual environment.
• Secrets need, however, to be managed externally, or be sure you're pushing to your private repository.
• Docker-compos
• A even more beautiful way of relating software in our boxes, specially if, like me, you think that consolidating different microservices in one box is going to make you save money.
• Again secrets needs to be managed externally.
• Manual
• No.

 

Best options:

The one chosen by me (examples coming soon):

• Vagrant + Chef (cloud) + Docker
• Vagrant + libvirt for my microserver (allocation < 30 seconds)
• Same Vagrant + aws for the cloud (not yet explored)
• Chef installs docker, pulls the right image, and configures it to start the containers on start.
• Docker contains the software with some placeholders for environment variables containing passwords and sensitive data (therefore docker images are public).

Adopt (other options I'd use):

• Vagrant + Chef or Puppet
  
  • WebUI management of nodes, configuration and secrets
  • Push and pull modes available
• Vagrant + Ansible
• Less help, more manual

Assess (Investigate before adopting):

• Terraform +  Chef or Puppet or Ansible
  
•  Docker as platform + Chef or Puppet or Ansible

Hold (Don't bother):

• Anything + Manual
• You'll fill tired soon

Not considered for being too big or complex for small projects:

• Cloudfoundry
• Kubernetes
• Mesos