Deploy a webserver vm using Azure CLI and bash scripts (Linux/windows)

Intro

Azure cli was another tool I had to try right after passing Az-900 azure certification, because what are certifications for if it’s not to give a little itching to get your hands a little dirty. Thus, az-cli became my third victim after oci-cli and aws-cli.
As in my previous cli tutorials, the aim is to demonstrate how the CLI tool can enrich your IaC experience when combined with powerful shell scripts. What comes along when you test your third cli tool in a row though, is how easy it is to spot differences with other cloud platforms (expect to see some in this post).

Here’s a direct link to my Github repo with all the scripts: https://github.com/brokedba/az-cli-examples
Here’s also a gif demo to see things in motion: https://bit.ly/3hvTwVu

I. Bash still rocks

With a little patience and few days debugging bash errors, my interactive scripts can now automate the provisioning of a webserver on 6 different OS (RHEL, Centos, Oracle Linux, Ubuntu, SUSE, Windows server 2016)
In total, 6 interactive BASH scripts & userdata files were used with Windows turning out to be the nastiest to deploy (I’ll explain why). The code also ensures that required network components like subnet or security group are created if missing during the launch. This will hopefully give beginners a glimpse on az-cli capabilities that make it so handy.

A big shout out to Stackoverflow community that made my life easier while bash scripting my way to pull this off.

Topology

The following figure shows the layers involved between our workstation and Azure while running the cli commands.

This image has an empty alt attribute; its file name is az-cli_instance-1.png

CLI setup and assumptions

If your cli environment is not configured yet, go check my previous post (az-cli installation) it’s very easy.

I will assume that the below elements are present/configured in your workstation:

ssh key pair to attach to your azure vm. below is the PEM based key pair I generated for this lab

$ ssh-keygen -P "" -t rsa -b 2048 -m pem -f ~/id_rsa_az
     Generating public/private rsa key pair.
 Your identification has been saved in    /home/brokedba/id_rsa_az.
 Your public key has been saved in        /home/brokedba/id_rsa_az.pub.

Default output is set to table

$  az config set defaults.output=table
+-----------------------------------------+---------+
|  Name    | Source                       | Value   | 
+----------+------------------------------+---------+ 
| output   | /home/brokedba/.azure/config | table   |
+----------+------------------------------+---------+

Note: The default region used in this lab will depend on your resource group or the configured region parameter

II. Clone the repository

Pick an area on your file system and issue the following command.
```
$ git clone https://github.com/brokedba/az-cli-examples.git
```
Below are the files you will find after cloning the directory (with embedded hyperlinks)

$ cd ./az-cli-examples
$ tree
|-- check_az_image.sh      ---> Displays most recent image details per OS type
|-- check_az_vmsize.sh     ---> Displays available vmsize per chosen vcpu number
|-- create_az_instance.sh  ---> Launch a vm (requires a vnet)   
|-- create_az_rg.sh        ---> Create a resource group    
|-- create_az_subnet.sh    ---> Create a subnet within the VNET   
|-- create_az_vnet.sh      ---> Create a VNET+subnet with its security group      
|-- cloud-init          ---> Subfolder containing bootstrap code for each OS   
    |-- centos_userdata.txt  ---> userdata script for CENTOS 7  
    |-- olinux_userdata.txt  ---> userdata script for Oracle linux 7   
    |-- rhel_userdata.txt    ---> userdata script for RHEL7
    |-- sles_userdata.txt    ---> userdata script for SUSE 15    
    |-- ubto_userdata.txt    ---> userdata script for Ubuntu    
    |-- Win_userdata.ps1     ---> PowerShell script for Windows server 2016

III. Deployment

Before starting the deployment let’s first talk about what makes azure architecture different from other cloud platforms

Network:
In Azure every subnet is a public subnet because as soon as you associate a public IP to a Vm’s VNIC, you’ll magically have internet access.Thus, the usual internet gateway is not needed here because system routes are taking care of that. Besides the CIDR range in azure is slightly larger than aws ( from /8 to /29).

ID: Azure doesn’t provide regular alpha numeric ids for its resources but a sort of path based identification (see below)

$ SUBNET ID  
/subscriptions/xx/resourceGroups/my_group/providers/Microsoft.Network/virtualNetworks/MY-VNET/subnets/My_SUBNET

Naming: Naming is
– Case insensitive (Windows baby ;)!!)
– Unique: A resource group can’t have 2 resources of the same type having the same name

Takeaways:
We can imply from the above that
1- We won’t need to handle ids but just the resource names in our scripts as they are unique within a resource group
2- Route table and internet gateway setting will be replaced by the Public IP and VNICs

CREATE A VNET/SUBNET

Below script creates both Vnet and Subnet using default or custom values for their names and CIDR block. I also added checks on the IP/CIDR format. You can click on the Script to see the content.

Network security group menu will offer to open few ports for you but will also be checked in the vm creation script

brokedba@ ./create_az_vnet.sh
  Name          Location
  ------------  ---------
  brokedba       eastus
  select the resource Group you wish to set for your resources []:
  Enter the VNET name you wish to create [CLI-VPC]: 
  selected VNET name : CLI-VCN
  selected group name: brokedba
  Enter the subnet name you wish to add [CLI-SUB]:
  Enter the VNET CIDR to assign '/8-To-/29' [192.0.0.0/8]: 192.0.0.0/8
  Enter the Subnet CIDR to assign within 192.0.0.0/8 to '/29' [192.168.0.0/16]:
  ==== Created VPC details ====
 +---------+------------+---------+--------------+---------------+--------------+
 | VNET    | Vnet_CIDR  | Subnet  |  SUB_CIDR   | Resource_group| Region       | 
 +---------+------------+---------+--------------+---------------+--------------+
 | CLI-VNET|192.0.0.0/8 | CLI-SUB |192.168.0.0/16| brokedba    | canadacentral|
 +---------+------------+---------+--------------+---------------+--------------+
 Note: make sure all bytes beyond network prefix length are always zeroed or you'll have an error

************ Security Group ! ************
1) SSH port Only             3) HTTP,RDP, and HTTPS
2) SSH, HTTP, and HTTPS
Select a security group ingress rule and press Enter: 2
*******************  Security Group detail  ******************
+---------------+---------+-------------------+----------------------------+
|    Name       | Source  |   PORT            | Type    |     Priority     |
+---------------+---------+-------------------+---------+------------------+
| Allow-WEB-IN  |Internet | ["22","80","443"] | Inbound |   100            |
+---------------+---------+-------------------+---------+------------------+

SG delete command  ==> az network vnet delete –g brokedba -n CLI-VNET
VPC delete command ==> az network nsg delete  -g brokedba -n sg_CLI-SUB_WEB 
Disassociate NSG from its Subnet => az network vnet subnet update --vnet-name CLI-VNET --name CLI-SUB –g brokedba --network-security-group ""

Note:
– If no resource group is detected in your account the script will ask you to create one using create_az_rg.sh
– Delete commands are added in each script so you could destroy/repeat without using the Web Console.

CREATE SUBNET (Optional)

You can also create a subnet for an existing vnet using create_az_subnet.sh script but it’ll only allow it if the vnet has no existing subnet (sorry, too much exception handling due to CIDR overlap in case of existing subnets)

LIST VM SIZES (Optional)

Below menu returns the available vm sizes per picked vcpu number If you to want to replace the default size defined in the create_az_instance.sh script ( $vm_size)

brokedba@ ./check_az_vmsize.sh ******* azure vm shape Selecta ! ************ list all vm sizes in eastus region depending on the CPU and Series selected. ...>> Pick vm size = CPU# 1) 1 VCPU 2) 2 VCPUs 3) 4 VCPUs 4) 8 VCPUs 5)16 VCPUs Select a number of cores and press Enter: 3 >> Vm compute Series 1) A Series (Entry-level) 3) D Series (General purpose) 2) B Series (burstable) 4) E Series (Optimized for in-memory) Select a VM series and press Enter: 3 VM VCPUS Memory_MB MaxDisks OSDisk_maxMB UserDisk_maxMB --------------- ------ -------------- ----------- ------------- ---------------- Standard_D4s_v3 4 16384 8 1047552 32768 Standard_D4_v3 4 16384 8 1047552 102400 Standard_D4d_v4 4 16384 8 1047552 153600 Standard_D4_v4 4 16384 8 1047552 0 Standard_D4ds_v4 4 16384 8 1047552 153600 Standard_D4s_v4 4 16384 8 1047552 0 Standard_D4a_v4 4 16384 8 1047552 102400 Standard_D4as_v4 4 16384 8 1047552 32768

Note: Default location is eastus.To change it, modify the variable $location at the top of the script.

LIST IMAGES (Optional)

Below menu returns the last available image per OS type. This will help you decide which OS to choose for your vm

brokedba@ ./check_image.sh 
******* AWS Image Selecta ! ************
Choose your Destiny ||{**}||

1) RHEL          3) Oracle Linux  5) Windows       7) Exit? 
2) CentOS        4) Ubuntu        6) Suse  
Select an option and press Enter: 2

Name   Publisher    Sku         Urn                              Version
-----  ----------- ------  -----------------------------------  --------------
CentOS  OpenLogic    7.7   OpenLogic:CentOS:7.7:7.7.2020111300  7.7.2020111300

CREATE INSTANCE

After running the create Vnet script, we can now launch our new vm using 6 possible OS’ to choose from.
The default vm type is “Standard_B1s” but you can give the variable another value if you want.
You’ll be asked to retry if the Vnet and Subnet names don’t match and if a subnet is missing, it’ll be created for you.

brokedba@ ./create_instance.sh 
******* Azure VM launch ! ************

********** Resource Group ***********
Name         Location
----------- -----------
 brokedba     eastus
select the resource Group you wish to set for your resources []: brokedba
Enter the name of your new Instance [Demo-Cli-Instance]: Demo-vm
-----
selected Instance name : Demo-vm
Vm size : Standard_B1s
 ********** Network ***********
         **** VNET ****
VNET      Vnet_CIDR    Resource_group    Region
--------  -----------  --------------- ----------
CLI-VNET  192.0.0.0/8  brokedba       useast
select the VNET Name for your new instance []: CLI-VNET
selected VNET name : CLI-VNET           

         **** SUBNET ***
Subnet    CIDR            ResourceGroup
--------  --------------  --------------
CLI-SUB   192.168.0.0/16  brokedba 
Select The Subnet for your new instance []: CLI-SUB
selected subnet name :  CLI-SUB
Subnet exist => Checking  the OS menu and Security group rules
...
************ Network security Security Group ! ************  
Checking the associated  NSG : sg_CLI-SUB_WEB
== Creating the instance with the below NSG.
{ "Combo_rule_Ports": "["22","80","443"]",
  "Name": "sg_CLI-SUB_WEB",
  "resourceGroup": "brokedba",
  "single_rule_Ports": "[]",
  "sub": [ /subscriptions/…/virtualNetworks/CLI-VNET/subnets/CLI-SUB" ]
}
************ Azure Image Selecta ! **************
Choose your Image ||{**}||
1) RHEL          3) Oracle Linux  5) Windows       7) Abort?
2) CentOS        4) Ubuntu        6) Suse
Select an option and press Enter: 2
Name    Publisher    Sku    Urn                                 Version
------  -----------  -----  ----------------------------------- --------------
CentOS  OpenLogic    7.7    OpenLogic:CentOS:7.7:7.7.2020111300 7.7.2020111300

===== Instance Deployment Detail ========
Selected Subnet name : CLI-SUB
Selected Instance name : Demo-vm
Selected instance Type : Standard_B1s
Selected Security Group: sg_CLI-SUB_WEB
UserName   : centos
Selected OS : CENTOS
...
Public IP creation...
Name           Location       Stat       Allocation
-------------  -------------  ---------  ------------
Demo-vm_pubip  canadacentral  Succeeded  Dynamic
Network interface creation...
Name         State      PrivateIP
-----------  ---------  -----------
Demo-vm_Nic  Succeeded  192.168.0.4

===========================================
Check the status of the new Instance
===========================================
The compute instance is being created. This will take few minutes ...
Enter the Path of your ssh key [~/id_rsa_az.pub]:
selected public key: /home/brokedba/id_rsa_az.pub 
ResourceGroup PowerState PublicIpAddress PrivateIpAddress MacAddress  Location 
------------- ---------- --------------- ---------------- ----------------- -------
brokedba      VM running  40.85.218.246     192.168.0.4   00-22-48-3B-BE-DB  useast
ssh connection to the instance ==> ssh -i /home/brokedba/id_rsa_az centos@40.85.218.246
Your website is ready at this IP :) : http://40.85.218.246

VM termination command   ==> az vm delete –g brokedba -n Demo-vm --yes
VNIC termination command ==> az network nic delete –g brokedba -n Demo-vm_Nic
IP termination command  ==> az network public-ip delete –g brokedba -n Demo-vm_Pubip
VNET termination command ==> az network vnet delete –g brokedba -n CLI-VNET
Disk termination command ==> az disk delete –g brokedba -n Demo-vm_OsDisk_1_fxxxxxxxx

Below is the vm view on your Azure Portal once the instance is provisioned.

This image has an empty alt attribute; its file name is az-cli_vm.png

CONNECTION TO THE VM AND CHECK THE PUBLIC WEBPAGE

Grab the public IP and enter the private key to connect to your vm using ssh

$ ssh -i /home/brokedba/id_rsa_az centos@40.85.218.246

Type the Public IP in your browser & check the customized Homepage using the custom-data run during bootstrap
=> http://40.85.218.246

The curious case of Windows vms

The “–custom-data” parameter which takes the cloud-init file and sends its encoded content along with the provisioning configuration in azure does it only for Linux vms.You’ve read right, when providers like aws offer full cloud-init support in their cli-tool for all available images including windows, Microsoft doesn’t for it’s own OS !!

Azure proposed cheap option
The only way to replace custom-data run when spinning windows vms is to remotely invoke commands on the vm after creation (which sucks). I had to add the below line to run my Powershell custom-data on the vm

az vm run-command invoke -g "$rg_name" -n $instance_name --command-id SetRDPPort az vm run-command invoke -g "$rg_name" -n $instance_name --command-id RunPowerShellScript --scripts @cloud-init/Win_userdata.ps1

CONCLUSION

This exercise helped to know more about azure architecture than just practicing on a cli tool and JMESPATH tricks.
Az cli is still powerful, but I have to say that I felt quite a latency all along my lab even on simple queries where nothing was created. AWS and OCI for instance are way faster in terms of responsiveness (based on my useast tests)
The fact that a custom route and internet gateway weren’t necessary a reduced my code footprint by almost 30%.
One caveat for windows provisioning is that azure doesn’t support cloud-init, but custom code can still run remotely
Feel free to fork my repository and adapt your own version of these scripts
Improvement: add the option to add a subnet to vnets with existing subnet to create_az_subnet.sh script

Thanks for reading!