ARM Template Building Blocks and Reference Architectures

The AzureCAT team actively assists customers with the largest, most complex projects built on the Azure platform.  In addition to the super work the team does helping Azure customers the AzureCAT team have released a set of ARM templates which can be used as building blocks for ARM deployments.  The team have made these templates available on GitHub and have also released a set of reference architectures that use these ARM building blocks.

What’s really nice, is that the ARM building blocks the AzureCAT team have created, are based on the work they have done with real customers.  This means these templates are road tested and have a huge amount of customer learnings and good practices ‘baked in’ to them.

The AzureCAT team has made extensive use of nesting and arrays within their ARM templates to reduce some of the complexity inherent in ARM.

Even if you don’t use the templates directly, they are a great place to start when building your own templates and offer an interesting insight into learnings the AzureCAT team has gained over the last two years.

Current building blocks released are:

Building block Link Description
Virtual network vnet-n-subnet Used to create a virtual network with any number of subnets
Network security groups networkSecurityGroups Used to create any number of NSGs, and link them to any number of NICs and/or subnets
User defined routes userDefinedRoutes Used to create any number of UDR tables, and link them to any number of subnets
Gateway connection vpn-gateway-vpn-connection Used to create a VPN or ExpressRoute gateway and necessary connections to another network
Virtual machines multi-vm-n-nic-m-storage Used to create any number of VMs, each with any number of NICs, and any number of data disks
Load balanced workload loadBalancer-backend-n-vm Used to create a load balancer with a collection of VMs in the backend
DMZ dmz Used to create a DMZ between an Azure VNet and any other network, or the Internet

 

Advertisements

Using ARMclient to directly access Azure ARM REST API’s and list ARM Policy details

While Azure-PowerShell and the Azure xplat CLI are excellent tools, there are times when connecting directly to the Azure ARM REST API is just easier.   One of these times is when you want to find out the contents of an Azure Resource Manager Policy.  While Its easy to use PowerShell to find out the policies applied to a Resource Group.

get-azureRMpolicyassignment -Scope /subscriptions/$mysubscription/resourceGroups/$myresourcegroup

Getting the details as to the contents of an Azure Resource Manager Policy can be quite a bit trickier.  This is one situation where calling the ARM Policy provider via the ARM REST API’s directly is the easiest answer.

ARMclient is an OSS project which makes the task of connecting to Azure’s ARM REST API incredibly easy.  David Ebbo has a great blog article here on the simple steps to install ARMClient.

Once ARMclient is installed its a simple matter of logging in to the Azure tenant and then calling the provider.  In this case,  /Microsoft.authorization/policydefinitions to list the ARM Policies for a given subscription.

armclient login
armclient get "https://management.azure.com/subscriptions/<$YourSubscriptionID>/providers/Microsoft.authorization/policydefinitions?api-version=2016-04-01"

Depending on the number of ARM Policies in the subscription the output from the ARMclient can be quite large.  If you know the name of the ARM Policy you whish to see the details for you can append this to the call that you make to the API.

armclient get "https://management.azure.com/subscriptions/<$YourSubscriptionID>/providers/Microsoft.authorization/policydefinitions/tags-owner?api-version=2016-04-01"

This will display the contents of just that policy.

armclient get "https://management.azure.com/subscriptions/<$YourSubscriptionID>/providers/Microsoft.authorization/policydefinitions/tags-owner?api-version=2016-04-01"
{
  "properties": {
    "policyType": "Custom",
    "description": "Policy to set owner tags",
    "policyRule": {
      "if": {
        "allof": [
          {
            "field": "tags",
            "exists": "true"
          },
          {
            "field": "tags.owner",
            "exists": "false"
          }
        ]
      },
      "then": {
        "effect": "append",
        "details": [
          {
            "field": "tags.owner",
            "value": "Daniel"
          }
        ]
      }
    }
  },
  "id": "/subscriptions/<$YourSubscriptionID>/providers/Microsoft.Authorization/policyDefinitions/tags-owner",
  "type": "Microsoft.Authorization/policyDefinitions",
  "name": "tags-owner"
}

As you can see the output passed back by the API is exactly the same JSON format which is used when creating the ARM Policies .

There is heaps that you can do with the ARMclient, one of my favourites is to list out all the resources in an Azure subscription.

armclient get "https://management.azure.com/subscriptions/<@YourSubscriptionID>/resources?api-version=2014-04-01"

As the output is in JSON format its very easy to manipulate as required.  You can also convert the output from ARMclient to a PowerShell object using ConvertFrom-JSON in your PowerShell scripts.

The following command will list just the virtual machines in a subscription.

armclient get "https://management.azure.com/subscriptions/<$YourSubscriptionID>/providers/Microsoft.compute/virtualMachines?api-version=2016-03-30"

To get a list of all available ARM providers which you can call using the ARM REST API, you can run the following ARMclient command.

armclient get "https://management.azure.com/subscriptions/<$YourSubscriptionID>/providers?$expand=resourceTypes/aliases&api-version=2015-11-01"

This will also give you the API versions supported by the different providers.  An API version must be appended to the end of all calls to the ARM REST API.

As well as ‘get’ commands, the ARMclient can also be used to modify Azure ARM resources by using ‘post’, ‘put’ and ‘delete’ operators.  The website resources.azure.com exposes similar functionality to the ARMclient.

Azure spend now shows in subscription blade in Azure Portal

Microsoft have just released a nice little update to the Azure Portal to show your current spend on the subscription blade of the Azure portal. You used to have to go to the billing blade and be a billing administrator to see this information. The subscription blade only requires staff to have read access to the subscription to see the current and historical spend and they can’t modify the subscription in any way or see payment methods or billing address.

sub-blade-spend

The subscription blade can be found by clicking on more services at the bottom of the left hand menu and using the filter or scrolling down to the General category. Clicking the star next to subscription will add it to your quick access menu on the left hand side of the screen.

sub-blade

Microsoft have also enabled the ability to see the Azure Marketplace costs associated with each subscription. This information can be accessed under the External Service button which is visible when clicking on the subscription in the subscription blade.

sub-blade-detail

I really like the burn rate graph which is displayed on the detailed subscription blade, this is especially handy if you have an MSDN subscription as it gives you an indication if you are going to hit your cap or not ! For those that have not used an MSDN subscription you get a monthly Azure spend/cap and if you go over your cap everything gets shutdown (unless you put your credit card in) until your billing period roles over which can be embarrassing if you are in the middle of a demo :(.

Quickly switching between blades in Azure Portal

Since transitioning out of its beta status Microsoft has continued to update and develop portal.azure.com, which Microsoft staff commonly refer to as the Ibiza portal.  As more and more services which Microsoft call Resource Providers are added to the portal it can get quite confusing as to where you are and time consuming to move around between resource providers.

To make it easier to navigate, the Ibiza portal displays at the top of the page, the journey you have taken through the different blades associated with a resource.  you can back track to any blade by clicking on the name of that blade.

ibiza-journey

You can also quickly jump between resource provider blades by clicking on the chevron next to the Microsoft Azure logo at the top right of the page.  This will show you the resource providers that you have accessed during your current session.

ibiza-blade-history

You can quickly jump between resources providers by clicking on them.

ibiza-blade-history-2This makes it super easy for instance when you are testing ASR failover and want to check to see if your VM has shown up under virtual machines.  Or if you want to check Azure security center and then jump back to check on an Azure Automation job.

My final tip for navigating around the Ibiza portal is to use the minimise action to shrink blades back to a single bar.  This can be a good way clean up screen space with out having to do lots of left and right scrolling.

ibiza blade minimise.jpg

 

Using Visual Studio to create and deploy Azure Resource Manager templates 

Azure Resource Manager (ARM) is Microsoft’s new way of provisioning Azure infrastructure and services.  One of the many great features of ARM is that it allows us to create JSON template files which describe the Azure services (and their relationships) we wish to deploy.   These templates can then be uploaded to Azure for deployment.
Once we have created an ARM template it is very easy to deploy the template across many  Azure subscriptions with out the potential errors that deploying the services by hand may result in.

There are many ways that ARM templates can be created but in this blog post I want to focus on using Visual Studio.  Why Visual Studio ? Well I am an IT Pro not a Programmer, so while I understand how to use the Azure portals and I get PowerShell and the Azure CLI.  I have never had any experience using Visual Studio and always thought it was a tool for Dev’s.   When I read that you could use Visual Studio to deploy ARM templates to Azure as well as create them, I was interested to find out more.

Now I don’t pretend to know how to use Visual Studio, and I am sure that I am only using a very small part of the software package, but what follows, are the steps that I have used to create an ARM template in Visual Studio and deploy the template to Azure.

First up you can download Visual Studio here

Once Visual Studio is installed select File, New, Project

Visual Studio New Project

Visual Studio Create New Project screen

The New Project screen will be displayed

Select Visual C#, Cloud, Azure Resource Group

VS-2

Visual Studio New Project popup

 

Next Visual Studio will display a list of pre made Azure Resource Manager templates, to select a blank template scroll to the bottom of the list and select Blank Template.

 

VS-3

Visual Studio Azure Template popup

 

Visual Studio will now create a new Azure Resource manager project.  On the right hand side of the screen the Solution Explorer box will show the contents of the project.  Under scripts is the PowerShell script that Visual Studio will use to deploy the template to Azure and in the templates folder you will find the ARM templates.  Double click on the DeploymentTemplate.json file to open it.

 

VS-4

Azure ARM Template

 

To add resources to the deployment template right click on the word resource as per the screenshot below and select ‘Add New Resource’

VS-5

Add New Resource to template

 

This will open the Add Resource pop up from which you can select the Azure Resources you wish to deploy.

VS-6

New Resource popup

 

Once you have added all the Azure Resources and linked them together you can test deploying your template to your Azure subscription.  To do this right click on the name of your project in the Solution Explorer and select Deploy, New Deployment.

VS-7

Deploy ARM Template to Azure

 

This will open the Deploy to Resource Group pop up and will prompt you to log in to your Azure subscription.  Clicking ‘Edit Parameters’ will bring up the Parameters dialog box which will allow you to add values for the parameters that you created when you made the ARM template.

VS-8

Subscription selection screen and parameters pop up window

 

Clicking Deploy will deploy your ARM template to the selected resource group in your Azure Subscription.

One last thing to note…..  Visual Studio uses PowerShell to deploy the ARM Template, which is all good, however with the release of Azure PowerShell 1.0 many of the Azure commands have changed.  Visual Studio creates a PowerShell deployment script that uses PowerShell 0.98 commands.  If you are using PowerShell 1.0 or greater you will need to update some of the commands in the PowerShell deployment script that gets created.  Check out the following blog post for more info

If an error is displayed stating that Switch-AzureMode is not recognised it is highly likely that you have PowerShell 1.0 installed and as such will need to follow the directions in the link above to update the deployment PowerShell script with the new 1.0 Azure commands.

Azure Security Center

This week Azure Security Center went from private preview to public preview, this new Azure service is designed to provide Azure administrators with a view of security across their Azure subscriptions.

The current public preview focuses on IaaS security, in particular VNets and virtual machines.  Azure Security Center comes with extensions that can be automatically installed in to your Azure VM’s (Windows and Ubuntu Linux with more distro’s supported in the future) which gives Azure Security center great visibility as to your security posture .

As well as reporting on the current security stance of the virtual machine the Azure Security Center also alerts if there are brute force attacks against your VM and if it is communicating with known malicious IP addresses.

Azure Security Center works with VM’s deployed using both Azure Service Manger (Classic VM’s) and Azure Resource Manager managed VM’s.

To find Azure Security Center log in to the new Azure portal http://portal.azure.com.  Using the navigation pane on the left hand side of the portal select Browse then scroll down and select Security Center.  (clicking the star will add Security Center to the left hand side navigation bar)

Azure Security Center 1

Once you have opened Security Center the first thing to do is enable the collection of information, clicking Security Policy will display your subscriptions and for each subscription you can enable the collection of security information, the storage account Security Center should store security logs for that subscription and the recommendations you wish to enable.

Enabling data collection will trigger the Security Center extension to be installed on all VM’s in that subscription.

Azure Security Center 2

Once the extensions are installed Security Center will show the security stance of your VM’s and recommend actions to remediate security issues.

Azure Security Center 3

Azure Security Center 4Some issues such as missing antimalware can be remediated from with in Azure Security Center.

Azure Security Center 6

Security Center will have more Azure services added to it over time and will be a key tool for monitoring the security of your Azure based services and infrastructure.

 

Azure AD Password Sync and Writeback – Security and Encryption 

People are often concerned regarding the risk of turning on Password Sync and Password Writeback between on premise AD and Azure AD.   This post looks to describe the password sync and password writeback processes and the encryption methods used to secure the password data in transit and at rest.

1) Password Sync to Azure AD

The password Sync agent (which is part of the Azure AD Connect tool) running on the on premises Azure AD Connect server makes an RPC call to its closest on premises DC and requests via the DC replication protocol the users password hash. The DC takes the users password hash and using an MD5 key (made up from the RPC session key and a random 128 bit salt) encrypts the password hash for transport over the wire. The DC then sends the encrypted password hash plus salt to the password sync agent over the RPC session. Note this is the same way Domain Controllers replicate password hashes.

The password sync agent then decrypts the encrypted password hash using the salt and RPC session key and immediately re-hashes the password hash to a SHA256 hashed password hash using the PBKDF2 key derivation algorithm as defined in RFC 2898.

The password sync agent then passes the hashed password hash over an SSL encrypted session to Azure AD. Azure AD then encrypts the hashed password hash using AES and stores it in its database.

2) Password Writeback

When password writeback is enabled Azure AD Connect creates a tenant specific service bus relay, protects it with a strong password and attaches to the service bus relay using TLS encryption.  Azure AD Connect also creates a public private key pair. The public key is placed in the tenant’s secret store in Azure AD and the private key stays on the on premise Azure AD Connect server.

When a password is reset in Azure AD, Azure AD encrypts the new password using the public key uploaded by Azure AD Connect and places the encrypted password on the service bus relay, Azure AD Connect picks up the encrypted password from the relay and decrypts it on the on premise Azure AD Connect server.   The Azure AD Connect server then attempts to reset the user’s password using the Active Directory DS SetPassword API.   If the password reset against the on premise AD succeeds the user is notified of the success and the resultant hashed password hash is encypted using AES and then stored in Azure AD.

If Azure AD connect is not successful in writing the password back to the on premise AD ( failing password complexity requirement for instance) or the Azure AD Connect server is down.  Azure AD will not be updated and the user attempting the password reset will be notified that the reset has not succeeded.

For further reading check out the following blogs and presentations:

http://blogs.technet.com/b/ad/archive/2014/06/28/aad-password-sync-encryption-and-and-fips-compliance.aspx

http://channel9.msdn.com/Events/TechEd/NorthAmerica/2014/DCIM-B301

  https://msdn.microsoft.com/en-us/library/azure/dn903642.aspx