Application load balancer.

Our aim to achieve the below model.

Task list:-

            Create four EC2 machines with below setup.

• Must access to port 22 and 80 in the security group.
• Install Apache and put one sample page with the name of index.html and image/video.html in “/var/www/html” for testing.

Once all EC2 machine created then its look like.

 

Check URL for all EC2 machine.

http://3.17.177.134/       Output: – This is Feenixdv-VideoServer1

http://52.15.107.7/         Output: – This is Feenixdv-VideoServer2

http://18.224.171.135/  Output: – This is Feenixdv-Imageserver1

http://3.17.56.231           Output: – This is Feenixdv-Imageserver2

Create Target Group from AWS console

Replace Healthy.html with any working URL like “index.html”.

 

Create the same for Video

Then set Target group for FeenixImageServer using add to registered.

Register both group with EC2 machine as planned.

 

 

Same for target group: FeenixVideoServer

 

Now create a load balancer

 

Then select ALB (Application Load Balancer).

 

Configure Security Groups: – select security group to allow (80 and 22)

Review and click on create button.

In the below selection, select only one group with a load balancer. We achieve ALB using rules.

 

Cross check, load balancer associated with a right group

In the load balance section now, you can see DNS name.

Try to access this DNS in the browser.

Here we can see page server from Image server one and two (one by one). As we attached FeenixImageServer so the output comes like this.

 

Now we need to configure rules in Load Balance -> listener section.

 

Add rules.

Here I make a rule, if in URL word like “*image*” then request forwarded to “FeenixImageServer” group of a server. Add same for “*video*”.

 

Finally, we have two rules.

 

 

Now test

For image:-

 

 

For video:-

 

 

 

 

Implicit and Explicit Dependencies

By studying the resource attributes used in interpolation expressions, Terraform can automatically infer when one resource depends on another. In the example above, the expression ${aws_instance.example.id} creates an implicit dependency on the aws_instance named example.

Terraform uses this dependency information to determine the correct order in which to create the different resources. In the example above, Terraform knows that the aws_instance must be created before the aws_eip.

Sometimes there are dependencies between resources that are not visible to Terraform. The depends_on argument is accepted by any resource and accepts a list of resources to create explicit dependencies for.

For example, perhaps an application we will run on our EC2 instance expects to use a specific Amazon S3 bucket, but that dependency is configured inside the application code and thus not visible to Terraform. In that case, we can use depends_on to explicitly declare the dependency:

New resource for the S3 bucket our application will use.

resource "aws_s3_bucket" "example" {
  # NOTE: S3 bucket names must be unique across _all_ AWS accounts, so
  # this name must be changed before applying this example to avoid naming
  # conflicts.
  bucket = "terraform-getting-started-guide"
  acl    = "private"
}

# Change the aws_instance we declared earlier to now include "depends_on"
resource "aws_instance" "example" {
  ami           = "ami-2757f631"
  instance_type = "t2.micro"

  # Tells Terraform that this EC2 instance must be created only after the
  # S3 bucket has been created.
  depends_on = ["aws_s3_bucket.example"]
}

 

Failure Behavior

By default, provisioners that fail will also cause the Terraform apply itself to error. The on_failure setting can be used to change this. The allowed values are:

    "continue" – Ignore the error and continue with creation or destruction.

    "fail" – Error (the default behavior). If this is a creation provisioner, taint the resource.

Example:

resource "aws_instance" "web" {
  # ...

  provisioner "local-exec" {
    command    = "echo ${self.private_ip} > file.txt"
    on_failure = "continue"
  }
}

 

Provision in Terraform

If you need to do some initial setup on your instances, then provisioners let you upload files, run shell scripts, or install and trigger other software like configuration management tools, etc.

Defining a Provisioner

provider "aws" {
 access_key = "AXXXXX"
 secret_key = "XXXXXX"
 region = "us-east-2"
}

resource "aws_instance" "Feenix_test" {
 ami = "ami-0cd3dfa4e37921605"
 instance_type = "t2.micro"
  tags {
    name = "feenix_web"
    }
  provisioner "local-exec" {
   command = "echo ${aws_instance.Feenix_test.public_ip} > public-ip.txt"
  }
 }

This adds a provisioner block within the resource block. Multiple provisioner blocks can be added to define multiple provisioning steps.
Terraform supports multiple provisioners, but for this example, we are using the local-exec provisioner.

Running Provisioners

C:\Users\bnarayan\terraform>terraform apply

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + aws_instance.Feenix_test
      id:                           <computed>
      ami:                          "ami-0cd3dfa4e37921605"
      arn:                          <computed>
.
.

aws_instance.Feenix_test: Provisioning with 'local-exec'...
aws_instance.Feenix_test (local-exec): Executing: ["cmd" "/C" "echo 3.19.30.122 > public-ip.txt"]
aws_instance.Feenix_test: Creation complete after 59s (ID: i-01b6a6e957358ecf1)

we can see IP address stored in the file.

C:\Users\bnarayan\terraform>type public-ip.txt
3.19.30.122

Terraform configuration and testing.

Infrastructure as a code

Deployment on Windows system.

• Download terraform from "https://www.terraform.io/downloads.html" and create "c:\terraform" and extract it.
  

• set PATH in environmental variable and add "c:\terraform\" in PATH.
  

C:\>mkdir terraform
C:\>cd terraform
C:\terraform>dir
04/06/2019  10:31 PM    <DIR>          .
04/06/2019  10:31 PM    <DIR>          ..
03/13/2019  12:07 AM        86,698,496 terraform.exe

Create project directory and initilize terraform plugin. To do that first create on sample tf file.

C:\Users\feenix>mkdir terraform
C:\Users\feenix>cd terraform

Create deployment configuration file with "*.tf" extansion
In this eaxmple i created "create_ec2.tf" with below infomation

provider "aws" {
 access_key = "XXXXXXXXXXXXXX"
 secret_key = "XXXXXXXXXXXXXXXXXX"
 region = "us-east-2"
}

resource "aws_instance" "Feenix_test" {
 ami = "ami-0b500ef59d8335eee"
 instance_type = "t2.micro"
  tags {
    name = "feenix_web"
    }
 }

Update access_key and secret_key.

First we need to initilize all plugin which is required.
run "terraform init" in side project deirectory(C:\Users\feenix\terraform).

 C:\Users\feenix\terraform>terraform init

Initializing provider plugins...
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "aws" (2.5.0)...

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.

* provider.aws: version = "~> 2.5"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Check all update done by "init"

C:\Users\feenix\terraform>dir
 Volume in drive C is OSDISK
 Volume Serial Number is E096-A98D

 Directory of C:\Users\feenix\terraform

04/06/2019  11:25 PM    <DIR>          .
04/06/2019  11:25 PM    <DIR>          ..
04/06/2019  10:53 PM    <DIR>          .terraform
04/06/2019  11:07 PM               285 create_ec2.tf
04/06/2019  11:11 PM               318 terraform.tfstate
04/06/2019  11:11 PM             3,902 terraform.tfstate.backup
               3 File(s)          4,505 bytes
               3 Dir(s)  55,487,930,368 bytes free

Now run "terraform plan" to check all input tf file and required plugin are ok. Its a type of dry run.

C:\Users\feenix\terraform>terraform plan
.
.
Plan: 1 to add, 0 to change, 0 to destroy.
.
.

In the output we can see one action to add.

Now if everything is ok then apply the terraform script.

C:\Users\feenix\terraform>terraform apply

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + aws_instance.Feenix_test
      id:                           <computed>
      ami:                          "ami-0b500ef59d8335eee"
      arn:                          <computed>
      associate_public_ip_address:  <computed>
.
.
Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes
 .
 .
 aws_instance.Feenix_test: Still creating... (40s elapsed)
aws_instance.Feenix_test: Creation complete after 44s (ID: i-0432c53da5e1f070a)

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

In the output its return "ID: i-0432c53da5e1f070a" which is EC2 instance ID. You can check on AWS consol.

We can show all configuration value with below command.

C:\Users\feenix\terraform>terraform show
aws_instance.Feenix_test:
  id = i-0432c53da5e1f070a
  ami = ami-0b500ef59d8335eee
  arn = arn:aws:ec2:us-east-2:827132459049:instance/i-0432c53da5e1f070a
  associate_public_ip_address = true
  availability_zone = us-east-2b
  cpu_core_count = 1

Delete or role back all changes. 

C:\Users\feenix\terraform>terraform destroy   

      

 

 

Lambda Function for Start_Stop_EC2 instance and attached to CloudWatch and scheduled to start Stop automatic on define time.

In side IAM role create Policies

 

Next, In roles “create role” and then select “Lambda” then “Permission” then select a policy from the list.

Give role name and click create

Give a name to the role.

 

Next in Lambda “Create a function”

In index.js put below code.

const AWS = require('aws-sdk');

exports.handler = (event, context, callback) => {
    const ec2 = new AWS.EC2({ region: event.instanceRegion });
    ec2.stopInstances({ InstanceIds: [event.instanceId] }).promise()
        .then(() => callback(null, `Successfully stopped ${event.instanceId}`))
        .catch(err => callback(err));

};

Create a testing case.

Test case looks like.

{
  "instanceRegion": "us-east-2",
  "instanceId": "i-004f28d6362128cc8"
}

Check test result.

Response:
"Successfully stopped i-004f28d6362128cc8"
Request ID:
"02f1a0e4-d27d-409c-ab6c-dab867ba2810"
Function Logs:
START RequestId: 02f1a0e4-d27d-409c-ab6c-dab867ba2810 Version: $LATEST
END RequestId: 02f1a0e4-d27d-409c-ab6c-dab867ba2810
REPORT RequestId: 02f1a0e4-d27d-409c-ab6c-dab867ba2810      Duration: 2248.11 ms     Billed Duration: 2300 ms          Memory Size: 128 MB    Max Memory Used: 75 MB          

On AWS console we can see instance going to stopping.

Create the same for start instance.

For start past below code

const AWS = require('aws-sdk');

exports.handler = (event, context, callback) => {
    const ec2 = new AWS.EC2({ region: event.instanceRegion });
    ec2.startInstances({ InstanceIds: [event.instanceId] }).promise()
        .then(() => callback(null, `Successfully started ${event.instanceId}`))
        .catch(err => callback(err));
};

Create the same test instance. Now you can see EC2 instance started.

For scheduling these process we under “Cloud Watch”  rules. Create a rule.

Provide a name.

Create same for the stop.

 

Recover or change lost key pair of AWS EC2 Linux instance

First, note down EBS ID of lost key pair EC2

Then in volume group section search same EBS ID.

Stop the EC2 machine which you want to recover the key.

Detach volume from the volume section

Then attach lost ec2 volume with any working EC2(you must have a key pair of working EC2).

Now login into working EC2 and mount this new EBS volume.
In this example new EBS is “/dev/xvdf2”

[root@ip-10-0-1-98 ~]# lsblk
NAME    MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda    202:0    0  10G  0 disk
+-xvda1 202:1    0   1M  0 part
+-xvda2 202:2    0  10G  0 part /
xvdf    202:80   0  10G  0 disk
+-xvdf1 202:81   0   1M  0 part
+-xvdf2 202:82   0  10G  0 part

Mount “/dev/xvdf2” to any mount point and replace “authorized_keys” of ec2-user.

[root@ip-10-0-1-98 ~]# mount -o nouuid /dev/xvdf2 /mnt/

Again attach this EBS to the original EC2 instance.
During the attachment take care device name must be  “/dev/sda1”

Then start EC2 instance and log in with a new key pair.

Sync EC2 Data In S3 Storage

Use S3 Bucket for backup Management

•     Install pip
•     Install the AWS CLI with pip
•     Add the AWS CLI Executable to Your Command Line Path
•     Installing Python on Linux
•     Install the AWS CLI on Linux Box

To install pip use the curl command to download the installation script.

    $ curl -O https://bootstrap.pypa.io/get-pip.py

Run the script with Python to download and install the latest version of pip and other required support packages.

  $ python get-pip.py --user

add in .bash_profile

export PATH=~/.local/bin:$PATH

Now you can test to verify that pip is installed correctly.

[root@ip-10-0-1-98 html]# pip2 --version
pip 19.0.3 from /root/.local/lib/python2.7/site-packages/pip (python 2.7)

Install the AWS CLI with pip
Use pip to install the AWS CLI.

$ pip3 install awscli --upgrade --user
[root@ip-10-0-1-98 html]# aws --version
aws-cli/1.16.137 Python/2.7.5 Linux/3.10.0-957.el7.x86_64 botocore/1.12.127

Now authenticate with access key and secret key

[root@ip-10-0-1-98 ~]# aws configure
AWS Access Key ID [None]: XXXXXXXXXXXXXXXXXXXX
AWS Secret Access Key [None]: XXXXXXXXXXXXXXXXXXXXXXXXXX
Default region name [None]: us-east-2
Default output format [None]: json

Install Apache and make backup of "/var/www/html" on S3 "s3://feenix-prometheus/html"

[root@ip-10-0-1-98 ~]# yum install httpd -y
[root@ip-10-0-1-98 ~]# cd /var/www/html/
[root@ip-10-0-1-98 html]# vim index.html
[root@ip-10-0-1-98 html]# aws s3 sync /var/www/html/ s3://feenix-prometheus/html
upload: ./index.html to s3://feenix-prometheus/html/index.html

Make cron job to automate this.

[root@ip-10-0-1-98 html]# crontab -l
* * * * * /root/.local/bin/aws s3 sync /var/www/html/ s3://feenix-prometheus/html

[root@ip-10-0-1-98 html]# tailf /var/log/cron
Apr  3 16:04:01 ip-10-0-1-98 CROND[4500]: (root) CMD (/root/.local/bin/aws s3 sync /var/www/html/ s3://feenix-prometheus/html)
Apr  3 16:05:01 ip-10-0-1-98 CROND[4511]: (root) CMD (/root/.local/bin/aws s3 sync /var/www/html/ s3://feenix-prometheus/html)