20 - Terraform - Provisioning Azure Infrastructure

-

 Terraform - By Asghar Ghori

  • Introduction to Infrastructure as Code
  • What is TerraForm?
  • Prepare for Terraforming
  • Build Infrastructure
  • Parametrize Infrastructure Configuration
  • Configure Remote Backend
  • Work with Modules
  • Understand and Use Meta-Arguments


Infrastructure as Code (IaC)
  • Process of managing infrastructure using code rather than manually configuring resources in a user interface
  • Define, deploy, update, and destroy infrastructure
  • Benefits:
    • Consistency across Test, Dev, QA, and Prod
    • Version controlled (can be stored in Git)
    • Repeatability
    • Validation before deployment
    • No errors
    • Fast deployments
    • Detailed logging
    • Lower cost

laC Categories
Five broad categories
  • Ad-hoc scripts
    • Step-by-step instructions in Bash, Ruby, Python, etc.)
  • Configuration management
    • Designed to run on existing servers
    • Tools: Puppet, Chef, SaltStack, and Ansible
  • Server templating
    • Create an image of the OS and use it to deploy new servers
    • Can be used for virtual machines and containers
    • Tools: Docker, Packer, Vagrant, etc.
  • Orchestration
    • Deploys, updates, load balances, auto-scales, and auto heals VMs and containers
    • Tools: Kubernetes, AWS ECS, AWS EKS, AKS, GKE, Docker, Swarm, Nomad, etc.
  • Provisioning
    • Builds any infrastructure component: networking, VM. containers, database instances, load balancers, firewall rules, etc.
    • Tools: Terraform, AWS CloudFormnation, Azure Resource Manager (ARM), Google Cloud Deployment Manager (CDM), OpenStack Heat, etc.

Considerations for Tool Selection
  • Configuration management versus provisioning
  • Mutable versus immutable
  • Declarative versus procedural
  • Master versus masterless
  • Agent versus agentless
  • Open source versus closed source
  • Cloud agnostic versus cloud native
  • Mature versus cutting-edge
  • Ability to work with other tools
  • Provisioning + configuration management
  • Provisioning + server templating
  • Provisioning + server templating + orchestration

What is TerraForm?
  • IaC tool from HashiCorp
  • Open-source. immutable, declarative
  • Code is written in HashiCorp Configuration Language (HCL)
  • Used to build, change, and manage infrastructure (VM, security group. network interface, load balancer. etc.)
  • You define your desired infrastructure in a file
  • Builds the desired infrastructure in the specified environment
  • Leverages CSP's APIs and their authentication mechanisms
  • Shows what will be created, changed. and deleted (dry run)
  • Shows dependency graph

Key Term
  • Plug-in (a.k.a. Provider): Provides a set of APIs to get jobs done (includes resource types and data sources). Available for Azure, AWS, GCP, DigitalOcean, OpenStack, etc. Three types:
    • Built-in: Part of the TF binary
    • HashiCorp-provided: The init command downloads them
    • Third-party: Must be manually installed
  • Provider block: key/value pairs to describe provider
  • Variables: key/value pairs to identify resource configuration items
  • Resource: Describes a single object (RG, VM, data disk, extension, role, NIC, NSG, Public IP, recovery services vault)
  • Resource type: resource-speciiic (azurerm_resource_group)
  • Resource name: Unique name assigned to a resource type for TF internal (“azurerm_resource_group” “vm 1 “)
  • Resource block: key/value pairs to configure a resource, includes all required configuration items at a minimum
  • Resource dependency: implicit or explicit
  • Data source: queries for the name or ID of an existing resource for consumption by a resource being created
  • Module: a collection of resources in a single directory (root module, child modules): repeatable: TF recommends no more than 1 level of depth)
  • Provisioner: allows TF to run a command/playbook (eg: after the completion of VM deployment and data disk addition) (patching, software installs, time 2one change, etc.)
  • null_resource: a special resource to run provisioners that are not directly related to a specific resource
  • Current State vs. Desired State: current actual infrastructure that TF built versus how it needs to look now

Suggested Terraform Files
  • README.md: General information
  • variables.tf:     Parametrizes values for resources
  • main.tf:            Resource definitions, dependencies, etc.
  • outputs.tf:        Exposes resource names, IP addresses, etc.
Note: TF files can be in json format. In that case, file extensions will be *.tf.json


Download Binary
RHEL image => developers.redhat.com -> 8.3.0
virtualbox.org -> 6.1
Install RHEL in VM
Azure CLI
Terraform
1) RHEL 8 image  [Download RHEL image] -     https://developers.redhat.com/products/rhel/download
2) VirtualBox    [ youtube.com/watch?v=aDolP2EJw7o]
3) VB/RHEL  [Install RHEL in VM] - https://www.youtube.com/watch?v=N3PPiwJOcuE
        4) Terraform
                https://www.terraform.io/downloads.html
Linux - 64 bit
unzip terrform_0.41xxxxxxx

    
Lab: Asghar Bhai Steps

1. After installation Centos 7
  • useradd devopsadm -G wheel
  • passwd devopsadm   or ( echo Abcd1234 | passwd --stdin devopsadm )
  • su - devopsadm
  • visudo  ** put comments # wheel ALL=(ALL)  and remove comments in NOPASSWD: ALL
  • cat /etc/yum.repos.d/local.repo    ** create repo in RHEL / No need in Centos

2. Install Python and Azure CLI
  • ** Install Python
  • $ sudo yum install gcc python3-devel libffi-devel openssl-devel -y
  • $ sudo yum install -y gcc libffi-devel python36u-devel openssl-devel"
  • $ rpm -q python3
  • python3 --version
  • sudo mkdir /usr/azure-cli
  • ** Install Azure CLI
  • $ sudo curl -L https://aka.ms/InstallAzureCli | bash
  • $ exec -l $SHELL
  • az

3. Login Azure and integrated with VM
  • $ az login
    • To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code A3TVAZQHQ to authenticate.

4. Service Principal
  • $ az ad sp create-for-rbac --role="Contributor" --scopes="/subcriptions/xxxxxxxxxxxxxxxx"
  • The output includes credentials that you must protect. Be sure that you do not include these credentials in your code or check the credentials into your source control. For more information, see https://aka.ms/azadsp-cli
            {
              "appId": "4cf0d39bxxxxxxxxxxxxx",
              "displayName": "azure-cli-2020-12-13-18-21-24",
              "name": "http://azure-cli-2020-12-13-18-21-24",
              "password": "i2mATG~xxxxxxxxxx",
              "tenant": "376aedc0-9bf4xxxxxxxxxxxxxx"
            }

  • $ az account show
          {
              "environmentName": "AzureCloud",
              "homeTenantId": "376aedc0-9bf4xxxxxxxxxxxx",
              "id": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
              "isDefault": true,
              "managedByTenants": [],
              "name": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
              "state": "Enabled",
              "tenantId": "376aedc0-9bf4xxxxxxxxxxxxxxx",
              "user": {
                "name": "xxxxxxxxxxxxxxxxxxxxxxxxx",
                "type": "user"
              }
        }


5. Update bash_profie

        $ sudo vi ~/.bash_profile  (Add all below lines in the end of bash_profile)
        export  ARM_SUBSCRIPTION_ID=“xxxxxxxxxxxxxxxxxxxxxxxxxx”
        export  ARM_TENANT_ID=“376aedc0-xxxxxxxxxxxxxxxxxxxxxxxx”
        export  ARM_CLIENT_ID=“4cf0d39b-910cxxxxxxxxxxxxxxxx”
        export  ARM_CLIENT_SECRET=“i2mATGxxxxxxxxxxxxxxxxx”
        export  TF_LOG=Trace
        export  TF_LOG_PATH=~/terraform/log/terraform.log  

        How to fill out above profile 
            export ARM_CLIENT_ID=<appID from step-4>
            export ARM_CLIENT_SECRET=<password from step-4>
            export ARM_SUBSCRIPTION_ID=$(az account show --query id | xargs)
            export ARM_TENANT_ID=$(az account show --query tenantId | xargs)

  • mkdir terraform
  • mkdir terraform/log
  • mkdir terraform/files
  • . ./.bash_profile
  • env | egrep 'ARTM|TF'

6. Download Terraforms binary
  • $ sudo wget https://releases.hashicorp.com/terraform/0.14.2/terraform_0.14.2_linux_amd64.zip
  • $ sudo mv terraform_0.14.2_freebsd_amd64.zip /usr/local/bin
  • cd /usr/local/bin
  • $ sudo unzip terraform terraform_0.14.2_linux_amd64.zip 
  • $ terraform


7. Search terraform provide block azure-cli


Lab 1 :

Resource - Data Source 

The TF code is here (main.tf):

$vi main.tf
provider "azurerm" {
  features {}
}

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 2.5.0"
    }
  }
  required_version = "~> 0.13"
}

resource "azurerm_resource_group" "rg_network" {
  name     = "rg_network"
  location = "canadacentral"
}

resource "azurerm_virtual_network" "vnet" {
  name                            = "vnet01"
  location                        = azurerm_resource_group.rg_network.location
  resource_group_name = azurerm_resource_group.rg_network.name
  address_space              = ["10.0.0.0/16"]
}

resource "azurerm_subnet" "subnet" {
  name                             = "subnet01"
  resource_group_name  = azurerm_resource_group.rg_network.name
  virtual_network_name  = azurerm_virtual_network.vnet.name
  address_prefix               = "10.0.1.0/24"
}

resource "azurerm_network_security_group" "nsg" {
  name                = "nsg01"
  location            = azurerm_resource_group.rg_network.location
  resource_group_name = azurerm_resource_group.rg_network.name

  security_rule {
    name                                   = "rule1"
    priority                                = 100
    direction                              = "Inbound"
    access                                  = "Allow"
    protocol                               = "Tcp"
    source_port_range               = "*"
    destination_port_range        = "22"
    destination_address_prefix  = "*"
    source_address_prefix         = "*"
  }
}

resource "azurerm_subnet_network_security_group_association" "subnet_association" {
  subnet_id                              = azurerm_subnet.subnet.id
  network_security_group_id = azurerm_network_security_group.nsg.id
}

[devopsadm@terraform files]$ cat nic.tf
resource "azurerm_network_interface" "w_nic" {
  name        = "windows-nic01"
  location   = azurerm_resource_group.rg_network.location
  resource_group_name = azurerm_resource_group.rg_network.name

  ip_configuration {
    name            = "windows-privateip"
    subnet_id     = azurerm_subnet.subnet.id
    private_ip_address_allocation = "Dynamic"
  }
}


Perform the following terraform commands.
  • terraform fmt
  • terraform validate
  • terraform plan
  • terraform apply
  • terraform.tfstate
  • terraform show
  • terraform state list
How to generated kyes
ssh-keygen -t rsa

___________________________________________________________________________

Lab 2 : TF codes break into proper procedure/resources :

# Infrastructure Build

$ cat main.tf
resource "azurerm_resource_group" "rg_network" {
  name     = var.rg_network
  location = var.location
}

resource "azurerm_virtual_network" "vnet" {
  name                = var.vnet
  location            = azurerm_resource_group.rg_network.location
  resource_group_name = azurerm_resource_group.rg_network.name
  address_space       = var.vnet_space
}

resource "azurerm_subnet" "subnet" {
  name                 = var.subnet
  resource_group_name  = azurerm_resource_group.rg_network.name
  virtual_network_name = azurerm_virtual_network.vnet.name
  address_prefix       = var.subnet_space
}

resource "azurerm_network_security_group" "nsg" {
  name                = var.nsg
  location            = azurerm_resource_group.rg_network.location
  resource_group_name = azurerm_resource_group.rg_network.name

  security_rule {
    name                       = "rule1"
    priority                   = 100
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "Tcp"
    source_port_range          = "*"
    destination_port_range     = "22"
    destination_address_prefix = "*"
    source_address_prefix      = "*"
  }
}

resource "azurerm_subnet_network_security_group_association" "subnet_association" {
  subnet_id                 = azurerm_subnet.subnet.id
  network_security_group_id = azurerm_network_security_group.nsg.id
}


$ cat variables.tf
provider "azurerm" {
  features {}
}

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 2.40.0"
    }
  }
  required_version = "~> 0.13"
}
variable "rg_network" {
  default     = "Terraform-Network-RG"
  type        = string
  description = "This resource group is to host all test infra."
}

variable "location" {
  default = "canadacentral"
}

variable "vnet" {
  default = "Network-CAC-VNET"
}

variable "vnet_space" {
  default = ["10.0.0.0/16"]
}

variable "subnet" {
  default = "Network-CAC-Subnet01"
}

variable "subnet_space" {
  default = "10.0.1.0/24"
}

variable "nsg" {
  default = "Network-CAC-NSG01"
}

# Virtual Linux created

$ cat vmlinux.tf
resource "azurerm_linux_virtual_machine" "vmlinux" {
  name                  = var.linux_name
  location              = var.location
  resource_group_name   = azurerm_resource_group.rg_network.name
  network_interface_ids = [azurerm_network_interface.linux_nic.id]
  computer_name         = "${var.linux_name}-nadeem"
  size                  = var.vm_size
  admin_username        = var.linux_admin_user

  admin_ssh_key {
    username   = var.linux_admin_user
    public_key = var.pub_key
  }

  os_disk {
    name                 = "${var.linux_name}-os-disk"
    caching              = var.los_disk_attr["los_disk_caching"]
    storage_account_type = var.los_disk_attr["los_storage_account_type"]
    disk_size_gb         = var.los_disk_attr["los_disk_size"]
  }

  source_image_reference {
    publisher = var.linux_publisher
    offer     = var.linux_offer
    sku       = var.linux_sku
    version   = var.linux_version
  }

}

resource "azurerm_network_interface" "linux_nic" {
  name = "${var.linux_name}-nic"

  location            = var.location
  resource_group_name = azurerm_resource_group.rg_network.name

  ip_configuration {
    name                          = "${var.linux_name}-ip"
    subnet_id                     = azurerm_subnet.subnet.id
    public_ip_address_id          = azurerm_public_ip.linux_pip.id
    private_ip_address_allocation = "dynamic"
  }
}

resource "azurerm_public_ip" "linux_pip" {
  name                = "${var.linux_name}-pip"
  location            = var.location
  resource_group_name = azurerm_resource_group.rg_network.name
  allocation_method   = "Dynamic"
}

$ cat vars-linux.tf
variable "linux_name" {
  default = "linuxvm01"
}

variable "vm_size" {
  default = "Standard_DS1_v2"
}

variable "linux_admin_user" {
  default = "devopsadm"
}

variable "pub_key" {
  default = "ssh-rsa xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx= devopsadm@terraform.example.com"
}

variable "los_disk_attr" {
  type = map(string)
  default = {
    los_storage_account_type = "Premium_LRS"
    los_disk_size            = "32"
    los_disk_caching         = "ReadWrite"
  }
}

# Variables for marketplace OS image to be installed in the VMs. Run "az vm image list --all | grep -i <ubuntu|sles|windows|rhel|centos> os.out". Review os.out and find publisher, offer, sku, and version information and set the variables.

variable "linux_publisher" {
  default = "Canonical"
}

variable "linux_offer" {
  default = "UbuntuServer"
}

variable "linux_sku" {
  default = "18.04-LTS"
}

variable "linux_version" {
  default = "18.04.202009080"
}


Lab 3

Lab 4

Lab 5



Comments

Post a Comment

Popular posts from this blog

05 - Docker - Containers

-
Image
 Docker - Containers - Container is operating system-level virtualization  ( Linux Kernal ) Allow for multiple isolated user-space instances called containers They share a single kernel Can be added or removed any times - Containers consist of a self-contained Linux file system Can be from any Linux distribution which is compatible with the host kernel Usually contain a single application such as a server - Operating  Is often used in Cloud Computing Docker Architecture Docker is available in two editions: Community Edition (CE) - open-source Enterprise Edition (EE) Docker uses a client-server architecture Client REST API  (   the primary user interface which communicates ) Over HTTP Over local Unix Socket Docker CLI docker build docker pull docker run Servers (Docker_Host) Is the Docker daemon Responsible for building, running, and distributing containers Docker daemon Container Images Registry (docker.io , DockerHub) Responsible for the storage, management, and delivery of Docker im
Read more

08 - PUPPET - Configuration Management

-
Image
 PUPPET - Configuration Management A Configuration Management Tool A framework for Systems Automation A Declarative Domain Specific Language (DSL) An Open Source software is written in Ruby Works on Linux, Unix (Solaris, AIX, *BSD), macOS, Windows |Supported Platforms) Developed by Puppet Labs Puppet Agent Required SSL certificate Default fatches after 60 minutes noop - dry run Puppet Open Source or Enterprise Puppet Langauge Declarative - Domain Specific Langauge (DSL) define STATES (No procedures) Puppet code is written in manifest ( .pp extension ) In the code, we declare resources that affect elements of the system (files, packages, services ... ) Resources are grouped into classes Classes and configuration files are organized in modules Puppet Resource Abstraction Layers Puppet Catalog Certificate Management Server Resources Facter Puppet Manifest Puppet Resources Resource Types Reference Puppet Nodes Roles Define Policy Puppet Virtual Resources Puppet Lab   Pu
Read more

06 - Docker Swarm - Container Orchestration

-
Image
 Docker Swarm - Container Orchestration Create Cluster Build & Deployment automated Rolling Deployment Update Version Docker Swarm is clustering for Docker It turns several Docker hosts into a single virtual Docker host The Swarm is controlled by Swarm Manager Each Docker code communicates with the manager It can be installed manually or by using Docker Machine Scheduler Service Discover Consul -> provide DNS Zookeeper - configuration management tools Cheat Sheet => https://lzone.de/cheat-sheet/Docker%20Swarm LAB mkdir DockerSwarm cd DockerSwarm mkdir manager vm1 vm2  *** Manager Node **** cd manager vagrant ssh sudo su - hostnamectrl set-hostname managernode.unix.in bash vi /etc/hosts 192.168.33.14 managernode.unix.in 192.168.33.15 workernode1.unix.in 192.168.33.16 workernode1.unix.in ## Install Docker C curl -fsSL get.docker.com | sh ## Add user Vagrant in Docker group sudo usermod -aG docker vagrant id vagrant ## Start Docker Service
Read more