When you have many VM instances in the cloud, managing and configuring each one individually can be very time-consuming. That’s where Ansible comes in handy. It allows you to automate the setup and management of all your VMs from one place.

Why Use Ansible?

• Consistency: Ansible ensures that all your VMs are configured the same way. This helps avoid mistakes and makes sure that every instance meets your requirements.

• Efficiency: Instead of logging into each VM and making changes manually, you can write a script (called a playbook) and let Ansible do the work for you.

• Scalability: As you add more VMs, Ansible can easily handle the extra load without needing extra work from you.

• Agentless: Ansible doesn’t require any special software (agents) to be installed on your VMs. It uses standard tools like SSH to communicate with your VMs. This means you don’t have to worry about managing additional software on each VM, and it simplifies the setup process.

Walkthrough

We will try hosting two instances of our Node.js project on AWS EC2 instance and make all of our configurations from ansible including downloading dependencies to configuring SSL certificate on our servers.

Architecture Overview:

Prerequisites:

• Minimum of one target vm instance of AWS EC2 (target is the vm where we'll configure our node.js server)

• One vm instance to setup and work with ansible (optional, but doing this will make initial setup configuration easier)

• SSH into both (or many in case you have more then one target instance)

---

Get Started 🚀

1. Install Ansible on your machine/VM

sudo apt-get update

sudo apt install ansible

Check whether ansible is successfully installed or not

ansible --version

2. Enable password-less authentication to your target VM

   To manage your target VMs without entering a password each time, you need to set up password-less SSH authentication. Here’s how:

Generate a new SSH key pair (if you don't have one already) using ssh-keygen in both of your instances (ansible and target vm):

ssh-keygen -t ed25519

You can also use RSA keys with:

ssh-keygen -t rsa

Follow the prompts and press Enter to accept the default file location and leave the passphrase empty (or set one if preferred).

Use the ssh-copy-id command to copy your public key to the target VM:

ssh-copy-id -i ~/.ssh/id_ed25519.pub username@target_vm_ip

Replace username with the actual username on the target VM (which will be ubuntu in most cases, you can check with whoami) and target_vm_ip with the VM's IP address.

If this doesn't work then you can manually add your public key to authorized_keys file in ~/.ssh path of your target VM

Check if password-less authentication works

Test that you can SSH into your target VM without a password:

ssh username@target_vm_ip

3. Start writing ansible playbooks 📝

   Ansible playbook is a place where you mention every configuration that you want into your target VMs

First of all, create a inventory file on your current directory

vim inventory

Inventory as name suggest stores the IPs of your target VMs/Service

For instance

192.168.1.XX
192.168.1.XX

Now, create playbook.yml on your current directory.

A minimalistic playbook looks like this

---
- name: My first playbook
  hosts: all
  become: true
  tasks:
    - name: Ensure the latest updates are installed
      apt:
        update_cache: yes

• name: My first playbook: This is a descriptive name for the playbook. It helps you identify what the playbook does.

• hosts: all: Specifies which hosts (VMs or servers) the playbook will run on. In this case, all means it will apply to all hosts listed in your inventory file.

• become: true: This tells Ansible to execute the tasks with elevated privileges (like using sudo in Unix-based systems). It allows you to perform tasks that require administrative rights.

• tasks:: This section lists the actions you want to perform. In this example, it has a single task.

• name: Ensure the latest updates are installed: This is a descriptive name for the task. It helps you understand what the task is intended to do.

• apt:: This is a module that manages packages on Debian-based systems (like Ubuntu). Here, it’s used to update the package cache.

Checkout all modules that ansible provides here

Fast forwarding....

Here is how a minimalistic playbook to configure you VM to host your node.js server will look like

(**Note**: below configuration assumes that your project contains all the server logic in a folder named server on your root project. If not or if there is any change you can update it accordingly)

- name: Configure Doodler server on AWS EC2
  hosts: all
  become: true
  vars:
    github_repo: https://github.com/Saumya40-codes/Doodler
    app_directory: /home/ubuntu/doodler
    server_directory: "{{ app_directory }}/server"
    nodejs_port: 5000

  tasks:
    - name: Update apt cache
      apt:
        update_cache: yes

    - name: Install nodejs, npm, nginx, certbot
      apt:
        name:
          - nodejs
          - npm
          - nginx
          - certbot
          - python3-certbot-nginx
        state: present

    - name: Clone or update to latest project state of Github Repository
      git:
        repo: "{{ github_repo }}"
        dest: "{{ app_directory }}"
        version: master
        force: yes

    - name: Install server dependencies
      npm:
        path: "{{ server_directory }}"

    - name: Install PM2
      npm:
        name: pm2
        global: yes

    - name: Create .env file for the server
      copy:
        dest: "{{ server_directory }}/.env"
        content: |
          MONGODB_URI= "mongodb+srv://name:password.ckmxcot.mongodb.net/"

    - name: Start server with PM2 using npm start
      command: pm2 start npm --name doodler-server -- start
      args:
        chdir: "{{ server_directory }}"

    - name: Configure Nginx
      template:
        src: nginx.conf.j2
        dest: /etc/nginx/sites-available/default
      notify: Reload Nginx

    - name: Enable Nginx site
      file:
        src: /etc/nginx/sites-available/default
        dest: /etc/nginx/sites-enabled/default
        state: link
      notify: Reload Nginx

    - name: Obtain SSL certificate
      command: certbot --nginx -d my-server.com --non-interactive --agree-tos -m saumyab5181@gmail.com

  handlers:
    - name: Reload Nginx
      systemd:
        name: nginx
        state: reloaded

In above snippet, vars as name suggests contains the variables which can be reused at any place in this configuration file.

Nginx Configuration

Under the Configure Nginx task, we have used a template module, which copies our nginx configuration file nginx.conf.j2 to /etc/nginx/sites-available/default.

A minimalistic nginx.conf.j2 might look like below

server {
    listen 80;
    server_name {{ server_name }} www.{{ server_name }};

    location / {
        proxy_pass http://localhost:{{ nodejs_port }};
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }

    # Additional security headers
    add_header X-Frame-Options "SAMEORIGIN";
    add_header X-XSS-Protection "1; mode=block";
    add_header X-Content-Type-Options "nosniff";

    # Logging
    access_log /var/log/nginx/doodler_access.log;
    error_log /var/log/nginx/doodler_error.log;
}

Remember: the variables server name and nodejs_port comes from the vars that we had defined in our playbook.yml

Make sure to store this nginx.conf.j2 file in /templates directory of your current location.

Handlers

At the end of the snippet we have defined handlers, under this contains a task which we can run when we want throughout our configuration file. For instance, we have used notify: Reload Nginx at two of the place in our configuration.

notify: Reload Nginx is a trigger mechanism in Ansible.

Its purpose is to tell Ansible: "If this task causes any changes, remember to run the 'Reload Nginx' task later."

4. Handling Secrets Using Ansible Vault

   If you look closely to the configuration file that we made, we have hardcoded our MONGODB_URI and our myserver.com. This isn't best practices, more when you'll be having this config file open sourced on github or any other platform.

To address this, we can use ansible-vault, which acts as an encrypted .env file that can be shared publicly.

To make this, write the folllowing command:

ansible-vault create secrets.yml

This wil redirect you to a vim(probably) editor, where you can declare your secrets, for instance

MONGODB_URI: your_mongo_uri_here
server_name: your server endpoint here

Now, we can update our configuration file to use this secrets.

To load this secrets in ourplaybook.yml file, we can define on top

vars_files:
    - secrets.yml

After this whenever needed we can define our secrets like, {{ MONGODB_URI }} or {{ server_name }}

To sum up, after all this, our configuration file might look like below:

- name: Configure Doodler server on AWS EC2
  hosts: all
  become: true
  vars_files:
    - secrets.yml
  vars:
    github_repo: https://github.com/Saumya40-codes/Doodler
    app_directory: /home/ubuntu/doodler
    server_directory: "{{ app_directory }}/server"
    nodejs_port: 5000

  tasks:
    - name: Update apt cache
      apt:
        update_cache: yes

    - name: Install nodejs, npm, nginx, certbot
      apt:
        name:
          - nodejs
          - npm
          - nginx
          - certbot
          - python3-certbot-nginx
        state: present

    - name: Clone or update to latest project state of Github Repository
      git:
        repo: "{{ github_repo }}"
        dest: "{{ app_directory }}"
        version: master
        force: yes

    - name: Install server dependencies
      npm:
        path: "{{ server_directory }}"

    - name: Install PM2
      npm:
        name: pm2
        global: yes

    - name: Create .env file for the server
      copy:
        dest: "{{ server_directory }}/.env"
        content: |
          MONGODB_URI={{ MONGODB_URI }}

    - name: Start server with PM2 using npm start
      command: pm2 start npm --name doodler-server -- start
      args:
        chdir: "{{ server_directory }}"

    - name: Configure Nginx
      template:
        src: nginx.conf.j2
        dest: /etc/nginx/sites-available/default
      notify: Reload Nginx

    - name: Enable Nginx site
      file:
        src: /etc/nginx/sites-available/default
        dest: /etc/nginx/sites-enabled/default
        state: link
      notify: Reload Nginx

    - name: Obtain SSL certificate
      command: certbot --nginx -d {{ server_name }} --non-interactive --agree-tos -m saumyab5181@gmail.com

  handlers:
    - name: Reload Nginx
      systemd:
        name: nginx
        state: reloaded

5. Almost Done

   We are almost done now. You need to update your inbound rules of your target VM to ensure that there are no firewall rules or security groups in AWS EC2 blocking traffic to the specified ports. The EC2 instance should allow incoming traffic on port 80 (HTTP) and 443 (HTTPS). Additonally, you'll also need to allow traffic from the port that you are exposing, in this case port5000

6. Running Ansible Playbook

   Now that all of the necessary steps are completed, we can run our ansible playbook

ansible-playbook -i inventory playbook.yml --ask-vault-pass

It will ask you for the vault password, that you would've set during the creation with ansible-vault

If everything went right, you should be seeing something like below:

Now you can visit your server and might check any of its endpoint, for my case it looked fine, like below:

To check your pm2 logs, you can ssh into target server and type

pm2 logs

Concluding

Managing numerous VM instances in the cloud can be daunting, but tools like Ansible make the process efficient and scalable. With Ansible, you ensure consistent configurations across all your VMs, enhancing reliability and reducing manual errors. The agentless nature of Ansible simplifies setup and maintenance, making it an excellent choice for both small and large-scale environments.

In this guide, we walked through setting up a Node.js project on AWS EC2 instances using Ansible, covering essential tasks such as installing dependencies, configuring SSL certificates, and managing secrets securely with Ansible Vault. By leveraging Ansible’s powerful automation capabilities, you can focus more on developing and less on manual configurations.

As you expand your infrastructure, Ansible will continue to streamline your processes, making it easier to manage additional VMs without additional overhead. Whether you're running a single server or a complex multi-tier application, Ansible's flexibility and ease of use make it an indispensable tool in your DevOps toolkit.

What's Next?

Incorporating Terraform

• Infrastructure as Code (IaC): Terraform allows you to define and provision your cloud infrastructure using a high-level configuration language. This makes your infrastructure reproducible and version-controlled.

• Integration with Ansible: Use Terraform to provision your VMs and networking resources, then hand off to Ansible for software configuration and deployment. This creates a seamless workflow from infrastructure setup to application deployment.

Setting Up Firewall

• Security Groups: Configure security groups in AWS to control inbound and outbound traffic to your EC2 instances. Define rules to allow necessary traffic (e.g., HTTP, HTTPS, SSH) and block unwanted connections.

• Terraform for Security Groups: Use Terraform to define and apply security group rules, ensuring they are consistent and version-controlled. This can be integrated into your existing Terraform configurations for VMs.

• Network ACLs: Implement Network Access Control Lists (ACLs) for an additional layer of security. ACLs provide stateless filtering, offering fine-grained control over traffic to and from your subnets.

• Monitoring and Alerts: Set up monitoring tools (like AWS CloudWatch) and configure alerts for suspicious activity or unauthorized access attempts to enhance security and respond promptly to threats.

Continous Deployment using Github Actions

• Write configurations for github actions to run ansible configurations whenever there is push into a desired branch. (I personally tried doing this but wasn't successful enough)