Tag Archives: Virtual Machine

#Azure : Map your traditional datacenter compute with cloud VMs


Cloud has completely changed the IT architecture landscape. Since early days of IT till last decade, Architecture was an abstraction that used to play a key role at the time of transformation or new development. Once Architecture developed, it used to continue for many years with very minimal changes. Since cloud came in its inception, architecture has become a key in day to day work life of an IT professional because of its agility. If not daily then most probably weekly, you can observe some changes in the public cloud world and that needs to be taken care seriously.

In this post, I’ll try to simplify the cloud architecture for compute and will compare with traditional compute architecture. Apart from the simplification, I’ll provide you a logical design thinking approach that will make your life easy no matter what role you are playing in IT.

Let start from traditional datacenter.

If you are an experience IT professional, you must have seen or heard about these names at least once in your career.

Traditional type of Servers: Tower, Rack, and Blade servers.

A true traditional server that comes with multiple configuration options such as dual-processor or quad-processor etc.

New type of platforms: Converged and Hyper-Converged.

These new platforms are basically rack based servers that provide inbuilt advanced storage and networking capabilities by leveraging software defined data center technologies.

Virtualization: In last one decade, every organization has leveraged capabilities of virtualization that enable compute to run multiple virtual machines so that you can fully utilize your high-end servers and save cost in multiple aspects.

Now, let me explain complete compute story in public cloud such as Microsoft Azure.

When you look at compute available through cloud, you can easily determine that it is same kind of virtual machines, which we used to have in our virtualized environment. But in the cloud, the only difference is that you don’t worry about the underlying hypervisor and hardware that is being used behind the scene to provide you virtual machines.

In traditional datacenter, we use multiple racks to install different types of hardware and each rack connects with different power supply units through PDUs and these power supply units connect with main power supply unit. In many scenarios each rack deploys top-of-the-rack switches to provide network connectivity to the devices installed in the rack and in some cases one or two of the racks in the same row deploy these TOR switches. To overcome the challenge of entire datacenter failure, we use multiple datacenters in the form of high-availability and site-resiliency. When an administrator performs any maintenance activity in the traditional datacenter, he/she makes sure that the quorum is maintained while performing maintenance activity to avoid any kind of unexpected failures.

In cloud, H/W level high-availability is being provided by fault domain (unexpected) and maintenance level availability is being provided by update domain, and both features fit under one umbrella that i.e. known as availability sets. To provide high availability, Microsoft Azure uses multiple datacenters (at least two-three) in each region, and to support site resiliency Azure provides multiple region options in same geography or across multiple geographies.

I hope, now you will be able to sketch a clear picture in your mind about traditional datacenter vs cloud.

Now, let me help you with the logical design thinking approach. When you think to deploy a VM or set of VMs, follow the following steps in sequential order.

  1. Think about application and its big picture, keep end-users in your mind and their respective locations.
  2. Select the best suitable cloud region.
  3. Consider different tiering of solution.
  4. Consider security, high availability, site resiliency and load balancing requirements.
  5. Illustrate your network requirements.
  6. Illustrate your storage requirements.
  7. Illustrate your compute requirements.

Once documented all the above, create design diagram and find the approach to deploy your solution. For more details specific to Microsoft Azure compute, read the following blogpost.

#Azure : Virtual Machines

#Azure : Virtual Machine Configuration

#Azure : Virtual Machines High Availability

#Azure : Step-by-step Availability Sets

#Azure : Virtual Machines Scale Sets

#Azure : Large Virtual Machines Scale Sets

Advertisement

#Azure : Large Virtual Machines Scale Sets


In general, virtual machines scale sets provide auto scalability based on the need. With the normal scale sets, you can have a deployment of 0-100 VMs. But if you have requirements to deploy large scale set of more than 100 VMs then you can opt for large virtual machine scale sets. The basic difference between two is placement group. Placement group in virtual machine scale set is nothing but kind of availability set that maintains its own fault and update domains. This placement groups in virtual machine scale sets have been defined by a parameter “singlePlacementGroup” that has two values either “true” or “false”, if this value is set to “true” then scale set will have a single placement group and the number of virtual machine can be between 0-100, and if the value of “singlePlacementGroup” parameter has been set to “false” then scale set will have multiple placement groups and the number of virtual machine can be between 0-1000.

There is one most important consideration for large VM scale sets i.e. storage. If you choose to go with large VM scale sets then you use managed disks and don’t define your own storage accounts. In VM scale sets, if you don’t go with managed disk then you require multiple storage account i.e. 1 for every 20 VMs but in case of large scale sets you leverage managed disks that simplifies your overhead of managing multiple storage accounts.

Let see how to do it.

Login to the Azure portal and search for “scale” in the azure market place and select the “Virtual machine scale set”.

In the Virtual machine scale set panel, select “Create” to create a new Virtual machine scale set.

In the “create virtual machine scale set” fill the basics information.

Virtual machine scale set name = Enter the scale set name for your virtual machine scale set deployment.

Operating system disk image = Select the operating disk image from drop-down.

Subscription = select your subscription.

Resource group = Create a new resource group or select the existing one.

Location = Select the Azure region from drop-down.

User name = Enter the username that will be used for virtual machines.

Password = Enter the password for the user name.

Confirm password = Re-enter the password to confirm.

Scroll down and fill the required details under “Instances and Load Balancer”.

Instance count = Enter to VMs count between 0 – 100. If you enter any number more than 100 and up to 1000, all the configuration settings will be disable except instance size. As explained above, large-scale sets with more than 100 VMs use managed disk by default and deployment of these large-scale sets take place across multiple placement groups.

Instance size = Select the VM size based on your requirement.

Enable scaling beyond 100 instances = By default “No”, if you select “Yes” then rest of the settings will be disabled as described under instance count. Select “Yes” for large Virtual Machines Scale Sets.

Autoscale = By default disabled but if you enable this feature then you need to define the conditions for the auto scaling.

If Autoscale enabled, fill the required details.

Autoscale

Minimum number of VMs = Enter the minimum number of VMs that required in this scale set.

Maximum number of VMs = Enter the maximum number of VMs that required in this scale set.

Scale out

CPU threshold (%) = Enter the cpu threshold after that VM will be added.

Number of VMs to increase by = Enter the number of VMs that will added when your running VMs reach defined cpu threshold.

Scale in

CPU threshold (%) = Enter the cpu threshold after that VM will be removed.

Number of VMs to decrease by = Enter the number of VMs that will removed when your running VMs reach defined cpu threshold.

Once filled all the details, click on create to start the deployment process.

Apart from these configuration settings, you need to consider following while planning for large VM scale sets.

Limit of 1000 VMs only applicable if you use Azure Marketplace images otherwise you can scale up to 300 VMs if you use your own customize image.

When designing network for large VM scale sets using a single subnet, make sure that your subnet has enough IP addresses for all the VMs. As a best practice, reserve 20% more ip addresses than what you need to support large scale sets.

Azure Load Balancer Standard SKU work with large scale sets because of multiple placement group while scale sets with 100 VMs can leverage basic Load Balancer.

Layer 7 load balancers and application gateways don’t need any specific configuration for large scale sets.

Make sure you are defining fault domain for the VMs that should not be in same fault domain, by default VMs will be speeded across fault and update domain in a specific placement group but that doesn’t mean that two VMs (for ex: these two VMs are required all the times) will not be deployed on same hardware. Use Azure resource explorer and go to the instance view of scale set and verify the fault domain and placement group Ids for the specific VMs.

Make sure you have enough vCPU quota to support large number of VMs in a large VM scale set, otherwise request to increase your vCPU quota.

You can convert your Virtual Machine Scale Set from a single placement group (1-100 VMs) to multiple placement group (1-1000 VMs) using Azure Resource Explorer but not vice-versa. You may not get the option to upgrade to large VM scale sets if you are using old version of the Microsoft.Compute API.

#Azure : Virtual Machines Scale Sets


Microsoft Azure virtual machines scale sets are a next step in high availability and scalability of virtual machines. Virtual machine high availability can be achieved by availability sets in Microsoft Azure. Microsoft Azure virtual machines scale sets are a group of an identical compute resources deployed in multiple availability sets. It is a true scalable model of auto-scaling that can target large-scale services with big compute, large data and containerized workloads. As these virtual machine scale sets leverage multiple availability sets in the background, therefore scale operations are tacitly balanced across fault and update domains. These scale sets use five fault domains and five update domains in each availability set. Each virtual machine scale sets can host 0-1000 VMs based on platform images, and 0-300 VMs based on custom images.

To define autoscale configuration for consistent application performance, many permutation and combination can be used. Very common rules are compute, memory, and disk I/O utilization. Apart from these common rule of performance metrics, auto scaling of VMs can also depend on application response, or a fixed scheduled.

Note: Virtual Machines scale sets can also be deployed with availability zones.

Now, let me explain you how does this auto scaling works behind the scene. When a new VM added to the scale set, a VM instance Id will be provided to each VM that is unique within a scaleset. When you add or remove a virtual machine from the scale set, the existing Id doesn’t go anywhere. For Example: In a virtual machine scale set you have 10 VMs, if your 2 VMs removed from the scale set based on the configuration and need, and then after some time 5 VMs are added based on the load then new VMs will have Instance id 10, 11, 12, 13, 14 in an incremented manner and these VMs will be balanced across fault and update domains to maintain maximum availability.

Let see how to do it.

Login to the Azure portal and search for “scale” in the azure market place and select the “Virtual machine scale set”.

In the Virtual machine scale set panel, select “Create” to create a new Virtual machine scale set.

In the “create virtual machine scale set” fill the basics information.

Virtual machine scale set name = Enter the scale set name for your virtual machine scale set deployment.

Operating system disk image = Select the operating disk image from drop-down.

Subscription = select your subscription.

Resource group = Create a new resource group or select the existing one.

Location = Select the Azure region from drop-down.

User name = Enter the username that will be used for virtual machines.

Password = Enter the password for the user name.

Confirm password = Re-enter the password to confirm.

Scroll down and fill the required details under “Instances and Load Balancer”.

Instance count = Enter to VMs count between 0 – 100. If you enter any number more than 100 and up to 1000, all the configuration settings will be disabled except instance size. As large-scale sets with more than 100 VMs use managed disk by default and deployment of these large-scale sets take place across multiple placement groups.

Instance size = Select the VM size based on your requirement.

Enable scaling beyond 100 instances = By default “No”, if you select “Yes” then rest of the settings will be disabled as described under instance count.

Use managed disks = By default “Yes”.

Public IP address name = Define name of the public IP address that will be used for load balancer, which will be placed in front of the scale set.

Public IP allocation method = By default dynamic but Static can be selected.

Domain name label = Domain name label for the load balancer in front of the scale set.

Autoscale = By default disabled but if you enable this feature then you need to define the conditions for the auto scaling.

If Autoscale enabled, fill the required details.

Autoscale

Minimum number of VMs = Enter the minimum number of VMs that required in this scale set.

Maximum number of VMs = Enter the maximum number of VMs that required in this scale set.

Scale out

CPU threshold (%) = Enter the cpu threshold after that VM will be added.

Number of VMs to increase by = Enter the number of VMs that will added when your running VMs reach defined cpu threshold.

Scale in

CPU threshold (%) = Enter the cpu threshold after that VM will be removed.

Number of VMs to decrease by = Enter the number of VMs that will removed when your running VMs reach defined cpu threshold.

Once filled all the details, click on create to start the deployment process.

As you observed that in the entire process, virtual network and storage account was not asked anywhere because virtual machine scale sets take care of it behind the scene based on the configuration. Therefore, you don’t have to really worry about it.

#Azure : Virtual Machine Configuration


When you have decided to use Azure IaaS virtual machines based on your requirements, you need to look at the configuration. Resource group is required to create any resource in Microsoft Azure. Let see the configuration of virtual machine.

Login to the Microsoft Azure Portal and select “+ create a resource”. Select compute and then select OS that needs to be deployed as part of the VM. In my case, I am deploying “Windows Server 2016 Datacenter”.

In the first step, define settings as required.

Name = Name of the virtual machine, same name will be applied as a host name.

VM disk type = Either HDD or SSD

User name = Default administrator name

Password = Password for the administrative user name

Confirm password = Confirm password for the administrative user name

Subscription = Select subscription from where the VM cost will be deducted

Resource group = Either “Create new” or “Use existing”

Location = Select the location of your Azure Region

Windows license = Select “Yes” if you have windows license otherwise “No”

Finally select “OK”

In the second step, select the VM size based on your requirement. You can short the VM size by selecting disk type, vCPUs and Memory.

In the third step, configure optional features.

High Availability = If you are deploying multiple VMs in the HA mode, create an “Availability set” and define fault & update domain as needed.

Storage = If you would like to use disk managed by Microsoft, select “Yes” in “Use managed disks” option. Otherwise select “No”, if you select “No” then you need to define a storage account.

Network = Select virtual network for the VM. If you don’t have any then a new will be created for you by default but still you can define your virtual network and use the same.

Subnet = select subnet, an ip to the VM will be assigned from this subnet

Public IP address = Use public ip address, if you want to access this VM directly through the internet. Otherwise you can select “None” here.

Network security group = Use network security group, if you would like to access or deny network traffic on the VM level.

Extensions = If you need to use any extensions as part of the VM deployment then add extensions such as PowerShell DSC, Custom Script Extension etc.

Auto-shutdown = Either “On” or “Off” and define the time and time zone based on your needs.

Notification before shutdown = Either “On” or “Off”

Monitoring = Either “Enabled” or “Disabled” for boot and guest OS diagnostics. If you enable the diagnostics then you need to use a storage account. This diagnostics user account either you can create or use the existing one.

Backup = Either “Enabled” or “Disabled”. If you enable backup option then you need to define “Recovery Services vault”, Resource group (for recovery service vault) and backup policy.

Once done select OK.

In the fourth step, review all the configuration and select “Create” to start the deployment process.

Wait for couple of minutes, your VM will be available for use. If you would like to reuse the VM configuration as-is or would like to reuse the VM configuration with customization then “Download template and parameters” for future deployments.

#Azure – Base Operating System


Microsoft Azure supports multiple base operating system for VMs. There are many other supported scenarios where you get the base OS with application from the portal itself or you can use your customize image either for base OS only or base OS with application. In this blogpost, I’ll cover the list of base operating systems available for VMs.

List of supported operating systems in Microsoft Azure:

Operating Systems

Provided By

Pricing

Window Server 2016 (Datacenter, Datacenter – Sever Core, Nano Server, with Containers)

Microsoft

Free*

Windows Server 2012 R2 (Datacenter, Essentials)

Microsoft

Free*

Windows Server 2012 Datacenter

Microsoft

Free*

Windows Server 2008 R2 SP1

Microsoft

Free*

Ubuntu Server

Canonical

Free*

Red Hat Enterprise Linux 7

Red Hat

$0.06/hour

SUSE Linux Enterprise Server

SUSE

Free*

Debian Linux

Credativ

Free*

Oracle Linux 7

Oracle

BYOL

CentOS-based 7.3

Rogue Wave Software

Paid

Container Linux by CoreOS

CoreOS

Free*

Free BSD 10.3

Microsoft

Paid

Clear Linux OS

Clear Linux Project

BYOL

Open SUSE Leap 42.2

SUSE

Paid

Windows 7 Enterprise N with SP1 (x64)

Microsoft

Paid

Windows 8.1 Enterprise N (x64)

Microsoft

Paid

 

Free*: OS Price has included with VM pricing.

BYOL: Bring your own license

Paid: Additional OS cost will be added.

 

Note: The above information is true at present when I am writing this blog. List can be modified any time by Microsoft and therefore it doesn’t guarantee any accuracy for future use.