ARM Template Building Blocks and Reference Architectures

The AzureCAT team actively assists customers with the largest, most complex projects built on the Azure platform.  In addition to the super work the team does helping Azure customers the AzureCAT team have released a set of ARM templates which can be used as building blocks for ARM deployments.  The team have made these templates available on GitHub and have also released a set of reference architectures that use these ARM building blocks.

What’s really nice, is that the ARM building blocks the AzureCAT team have created, are based on the work they have done with real customers.  This means these templates are road tested and have a huge amount of customer learnings and good practices ‘baked in’ to them.

The AzureCAT team has made extensive use of nesting and arrays within their ARM templates to reduce some of the complexity inherent in ARM.

Even if you don’t use the templates directly, they are a great place to start when building your own templates and offer an interesting insight into learnings the AzureCAT team has gained over the last two years.

Current building blocks released are:

Building block Link Description
Virtual network vnet-n-subnet Used to create a virtual network with any number of subnets
Network security groups networkSecurityGroups Used to create any number of NSGs, and link them to any number of NICs and/or subnets
User defined routes userDefinedRoutes Used to create any number of UDR tables, and link them to any number of subnets
Gateway connection vpn-gateway-vpn-connection Used to create a VPN or ExpressRoute gateway and necessary connections to another network
Virtual machines multi-vm-n-nic-m-storage Used to create any number of VMs, each with any number of NICs, and any number of data disks
Load balanced workload loadBalancer-backend-n-vm Used to create a load balancer with a collection of VMs in the backend
DMZ dmz Used to create a DMZ between an Azure VNet and any other network, or the Internet

 

Advertisements

Using ARMclient to directly access Azure ARM REST API’s and list ARM Policy details

While Azure-PowerShell and the Azure xplat CLI are excellent tools, there are times when connecting directly to the Azure ARM REST API is just easier.   One of these times is when you want to find out the contents of an Azure Resource Manager Policy.  While Its easy to use PowerShell to find out the policies applied to a Resource Group.

get-azureRMpolicyassignment -Scope /subscriptions/$mysubscription/resourceGroups/$myresourcegroup

Getting the details as to the contents of an Azure Resource Manager Policy can be quite a bit trickier.  This is one situation where calling the ARM Policy provider via the ARM REST API’s directly is the easiest answer.

ARMclient is an OSS project which makes the task of connecting to Azure’s ARM REST API incredibly easy.  David Ebbo has a great blog article here on the simple steps to install ARMClient.

Once ARMclient is installed its a simple matter of logging in to the Azure tenant and then calling the provider.  In this case,  /Microsoft.authorization/policydefinitions to list the ARM Policies for a given subscription.

armclient login
armclient get "https://management.azure.com/subscriptions/<$YourSubscriptionID>/providers/Microsoft.authorization/policydefinitions?api-version=2016-04-01"

Depending on the number of ARM Policies in the subscription the output from the ARMclient can be quite large.  If you know the name of the ARM Policy you whish to see the details for you can append this to the call that you make to the API.

armclient get "https://management.azure.com/subscriptions/<$YourSubscriptionID>/providers/Microsoft.authorization/policydefinitions/tags-owner?api-version=2016-04-01"

This will display the contents of just that policy.

armclient get "https://management.azure.com/subscriptions/<$YourSubscriptionID>/providers/Microsoft.authorization/policydefinitions/tags-owner?api-version=2016-04-01"
{
  "properties": {
    "policyType": "Custom",
    "description": "Policy to set owner tags",
    "policyRule": {
      "if": {
        "allof": [
          {
            "field": "tags",
            "exists": "true"
          },
          {
            "field": "tags.owner",
            "exists": "false"
          }
        ]
      },
      "then": {
        "effect": "append",
        "details": [
          {
            "field": "tags.owner",
            "value": "Daniel"
          }
        ]
      }
    }
  },
  "id": "/subscriptions/<$YourSubscriptionID>/providers/Microsoft.Authorization/policyDefinitions/tags-owner",
  "type": "Microsoft.Authorization/policyDefinitions",
  "name": "tags-owner"
}

As you can see the output passed back by the API is exactly the same JSON format which is used when creating the ARM Policies .

There is heaps that you can do with the ARMclient, one of my favourites is to list out all the resources in an Azure subscription.

armclient get "https://management.azure.com/subscriptions/<@YourSubscriptionID>/resources?api-version=2014-04-01"

As the output is in JSON format its very easy to manipulate as required.  You can also convert the output from ARMclient to a PowerShell object using ConvertFrom-JSON in your PowerShell scripts.

The following command will list just the virtual machines in a subscription.

armclient get "https://management.azure.com/subscriptions/<$YourSubscriptionID>/providers/Microsoft.compute/virtualMachines?api-version=2016-03-30"

To get a list of all available ARM providers which you can call using the ARM REST API, you can run the following ARMclient command.

armclient get "https://management.azure.com/subscriptions/<$YourSubscriptionID>/providers?$expand=resourceTypes/aliases&api-version=2015-11-01"

This will also give you the API versions supported by the different providers.  An API version must be appended to the end of all calls to the ARM REST API.

As well as ‘get’ commands, the ARMclient can also be used to modify Azure ARM resources by using ‘post’, ‘put’ and ‘delete’ operators.  The website resources.azure.com exposes similar functionality to the ARMclient.

Azure spend now shows in subscription blade in Azure Portal

Microsoft have just released a nice little update to the Azure Portal to show your current spend on the subscription blade of the Azure portal. You used to have to go to the billing blade and be a billing administrator to see this information. The subscription blade only requires staff to have read access to the subscription to see the current and historical spend and they can’t modify the subscription in any way or see payment methods or billing address.

sub-blade-spend

The subscription blade can be found by clicking on more services at the bottom of the left hand menu and using the filter or scrolling down to the General category. Clicking the star next to subscription will add it to your quick access menu on the left hand side of the screen.

sub-blade

Microsoft have also enabled the ability to see the Azure Marketplace costs associated with each subscription. This information can be accessed under the External Service button which is visible when clicking on the subscription in the subscription blade.

sub-blade-detail

I really like the burn rate graph which is displayed on the detailed subscription blade, this is especially handy if you have an MSDN subscription as it gives you an indication if you are going to hit your cap or not ! For those that have not used an MSDN subscription you get a monthly Azure spend/cap and if you go over your cap everything gets shutdown (unless you put your credit card in) until your billing period roles over which can be embarrassing if you are in the middle of a demo :(.

Quickly switching between blades in Azure Portal

Since transitioning out of its beta status Microsoft has continued to update and develop portal.azure.com, which Microsoft staff commonly refer to as the Ibiza portal.  As more and more services which Microsoft call Resource Providers are added to the portal it can get quite confusing as to where you are and time consuming to move around between resource providers.

To make it easier to navigate, the Ibiza portal displays at the top of the page, the journey you have taken through the different blades associated with a resource.  you can back track to any blade by clicking on the name of that blade.

ibiza-journey

You can also quickly jump between resource provider blades by clicking on the chevron next to the Microsoft Azure logo at the top right of the page.  This will show you the resource providers that you have accessed during your current session.

ibiza-blade-history

You can quickly jump between resources providers by clicking on them.

ibiza-blade-history-2This makes it super easy for instance when you are testing ASR failover and want to check to see if your VM has shown up under virtual machines.  Or if you want to check Azure security center and then jump back to check on an Azure Automation job.

My final tip for navigating around the Ibiza portal is to use the minimise action to shrink blades back to a single bar.  This can be a good way clean up screen space with out having to do lots of left and right scrolling.

ibiza blade minimise.jpg

 

Azure Resource Manager Policy to add cost center and owner tags

Azure Resource Manager (ARM) allows you to enforce organisational standards through the use of custom policies.  There are a number of things that you can do with ARM policies.  These range from restricting the size of virtual machines and the location they can be deployed, through to ensuring standardised naming conventions.

ARM Policies are made up of conditions, logical operators and effects.  Policies can be applied to a subscription, resource group or resource.

{
  "if" : {
      <condition> | <logical operator>
  },
  "then" : {
      "effect" : "deny | audit | append"
  }
}

As it currently stands, if an ARM Policy is applied which requires specific ARM tags to present, deployment of resources via the Azure Portal is blocked by the ARM Policy.  The resource deployment blades in portal.azure.com do not allow the setting of tags as part of the deployment as such the ARM Policy stops the deployment.  For some customers this is not a problem as all resources are deployed using ARM templates and any template which does not have the right tags set will not be allowed to deploy. But for many customers ‘breaking’ the UI experience is really bad.

As ARM Policies have an effect of append as well as deny, a set of policies can be created to append default tags to resources as they are created, these can then be updated via the UI (or PowerShell or CLI).  This allows staff to continue using the UI for resource deployment but they will have to update the tags once the resource is provisioned.

To force all resources created in a resource group to have a default tag of ‘owner’ and ‘costcenter’ added (if not present) when the resource is being created, the following ARM Policies need to be created.

CostCenterTag.JSON

This Policy fires if tags are present for the resource but the costcenter tag is not present.  The policy appends the costcenter tag with a default value.

{
  "if": {
    "allOf": [
      {
        "field": "tags",
        "exists": "true"
      },
      {
        "field": "tags.costCenter",
        "exists": "false"
      }
    ]
  },
  "then": {
    "effect": "append",
    "details": [
      {
        "field": "tags.CostCenter",
        "value": "666"
      }
    ]
  }
}

OwnerTag.JSON

This Policy fires if tags are present for the resource but the owner tag is not present.  The policy appends the owner tag with a default value.

{
  "if": {
    "allof": [
      {
        "field": "tags",
        "exists": "true"
      },
      {
        "field": "tags.owner",
        "exists": "false"
      }
    ]
  },
  "then": {
    "effect": "append",
    "details": [
      {
        "field": "tags.owner",
        "value": "Daniel"
      }
    ]
  }
}
NoTagsPresent.JSON
This policy fires if the resource has no tags and adds the costcenter and owner tags with default values.
{
  "if": {
    "field": "tags",
    "exists": "false"
  },
  "then": {
    "effect": "append",
    "details": [
{
        "field": "tags",
        "value": {"costCenter":"666", "owner":"daniel"   }

      },
      
        ]
  }
}

All three policies need to be applied so that tags are added under the scenario that an ARM template is used that contains other tags and the scenario that a deployment is done via the UI or using an ARM template and no tags are specified .

These Policies and the PowerShell commands to deploy the policies can be downloaded from https://github.com/dbowbyes/ARM

A copy of the PowerShell script used to deploy the ARM Policies is included below.

$mysubscription = 
$myresourcegroup =


login-azurermaccount
Get-AzureRmSubscription
Select-AzureRmSubscription -SubscriptionId $mysubscription

$policy = New-AzureRmPolicyDefinition -Name tags-owner -Description "Policy to set owner tags" -Policy "tags-owner.json"
New-AzureRmPolicyAssignment -Name tags-owner -PolicyDefinition $policy -Scope /subscriptions/$mysubscription/resourceGroups/$myresourcegroup

$policy = New-AzureRmPolicyDefinition -Name tags-costcenter -Description "Policy to set costcenter" -Policy "tags-costcenter.json"
New-AzureRmPolicyAssignment -Name tags-costcenter -PolicyDefinition $policy -Scope /subscriptions/$mysubscription/resourceGroups/$myresourcegroup

$policy = New-AzureRmPolicyDefinition -Name tags-notags -Description "Policy to set costcenter" -Policy "tags-notags.json"
New-AzureRmPolicyAssignment -Name tags-notags -PolicyDefinition $policy -Scope /subscriptions/$mysubscription/resourceGroups/$myresourcegroup

The following command can be used to see what ARM Policies have been applied to a subscription or resource group

get-azureRMpolicyassignment -Scope /subscriptions/$mysubscription/resourceGroups/$myresourcegroup

More information and examples of ARM Policies can be found here.

Azure VNet Peering

VNet Peering has just been released in to public preview by Microsoft.  It allows VNets within the same region to communicate using the low latency, high bandwidth Azure backbone network .  Prior to VNet peering to allow communication between two VNet’s we had to setup VNet to VNet IKE connections which required VPN Gateways to be setup. These gateways cost money to run and inject potential throughput issues and other overheads inherent with traffic encapsulation.

Reading through the blog posts that Microsoft have released and having a play with VNet peering this evening there are a couple features that I really like:

  1. You can peer VNet’s across subscriptions and between ARM and ASM VNet’s
  2. You can control if the remote VNet can use your local gateway and if you want your VNet to use a remote VNet’s gateway
  3. All other networking features work with VNet peering, including UDR’s, Network Appliances, VPN Gateways and NSG’s
  4. VNet Peering can be setup even if VNet to VNet connections via gateways are in place. As soon as the VNet peering comes up it will be preferred over the VNet to VNet connection which allows it to be implemented with very little if any traffic impact.

VNetPeering

Microsoft have written some great blog articles on how to enable VNet peering, which can be managed via PowerShell as well as the Azure Portal.

https://azure.microsoft.com/en-us/documentation/articles/virtual-network-peering-overview/

One thing to note is that VNet peering needs to be enabled via PowerShell for each subscription that you want enable peering.  The PowerShell code below can be used to enable VNet peering for your subscription.

Login-AzureRmAccount
Get-AzureRmSubscription

Get-AzureRmSubscription -Subscriptionname "you sub name here" |Select-AzureRmSubscription

#register provider
Register-Azurermproviderfeature -FeatureName AllowVnetPeering -ProviderNamespace Microsoft.network
Register-AzureRmResourceProvider -ProviderNamespace Microsoft.Network

#show provider registration status
Get-AzureRmProviderFeature -FeatureName AllowVnetPeering -ProviderNamespace Microsoft.network

 

 


 

 

Using PowerShell to create a selfsigned cert

As an alternative to using certutil to create selfsigned certificates, the following PowerShell command can be used.  This command is part of the PKI PowerShell module which when using newer versions of PowerShell will be loaded automatically when you call the command.

New-SelfSignedCertificate -certstorelocation cert:\localmachine\my -dnsname myaddress.com

This is very handy if using PowerShell to script the deployment of AD FS for testing and you are happy to run the server with a selfsigned certificate.