Chapter 8: Introducing Blueprints

After having shown how to initially configure vRealize Automation, it is now time to deal with blueprints. Blueprints represent the most important entity within vRealize Automation – they define how to provision and manage the lifecycle of resources in vRealize Automation.

This chapter introduces the basics of blueprints and includes some design discussions. The next chapters will show how blueprints can be created and configured.

8.1 Blueprint overview

In short, a blueprint can be described as the specification of the resources that should be deployed, including the respective lifecycle information. In order to be able to fully understand blueprints, we have to address and discuss a couple of issues:

• What kinds of blueprints are available in vRealize Automation?
• Of which sub-components are blueprints comprised of?
• Which provisioning mechanisms are supported?
• Which preliminary work needs to be done to create a blueprint and deploy resources from it?
• What is the lifecycle of a deployed resource?
• What about extensibility?
• How to share blueprints?
• Can blueprints be saved as code?
• How long can deployed machines be used?
• Is there any way to reclaim resources?
• How to deal with configuration changes at runtime?
• How to monitor the deployment process?

In the following, we will deal with these questions. As soon as we have a clear picture about the fundamental functionality, we can show how the actual configuration and implementation work can be done.

New blueprint features In vRealize Automation, blueprints have been extensively overhauled and have become much more powerful and feature-rich as in previous versions. The biggest enhancement is the new Blueprint Designer that has undergone a substantial facelift. The designer now combines the creation of infrastructure and application blueprints (the functionality of the former Application Services have now been integrated into the vRealize Automation appliance). Hence, there is only one Blueprint Designer that not only allows to create simple machine blueprints, but also complex tiered applications. Another feature of the new Blueprint Designer is the tight integration of NSX components. Blueprints can now contain NSX components like load balancer, security groups or complete dedicated networks. In Fig. 1, you can see a blueprint containing two virtual machines (the equivalent of the former multi-machine blueprint). Blueprints can now be imported, exported and edited in the YAML format. This can be done either by using the CloudClient, which is now part of vRealize Automation, or by using the REST API. In addition, blueprints can now be shared between tenants.

Fig. 1: Blueprint Designer

8.1.1 Blueprint types

The concept of a blueprint has significantly changed in the new vRealize Automation version. In previous versions, there were two kind of blueprints: Blueprints and multi-machine blueprints. A normal blueprint was the specification of a physical, virtual or cloud machine, including the respective lifecycle information. Multi-machine blueprints were comprised of one or more blueprints combined with the possibility to create a dedicated network for each deployment of a multi-machine blueprint.

In vRealize Automation blueprints act more as a container element that hosts different kinds of elements to be deployed. The following resource elements can be provisioned from a blueprint:

• Single machines like a vSphere machine, an Amazon machine or a Hyper-V machine.
• Network and security components. This includes existing or on-demand networks, security groups, security tags or load-balancers
• Existing blueprints so that you can create nested blueprints.
• Software components that allow to install software components during the machine provisioning process and support the software lifecycle. This feature also helps you to update your existing instances with new software versions. Software elements must be associated with a machine component.
• Existing XaaS blueprints. You can integrate your vRealize Orchestrator workflows to be part of the deployment process. However, this only works in conjunction with at least one single-machine to be deployed.

Based on these elements, blueprints are usually categorized as follows:

• Machine blueprints: These can be either simple blueprints to provision a single machine or multi-machine blueprints that contain several different types of machine components. It is possible to add networking and security components.
• XaaS blueprint: XaaS blueprints allow you to publish Orchestrator workflows within the service catalog. XaaS blueprints help you to integrate third party elements into the service catalog. A typical example is the creation of Active Directory users in an Active Directory group.
• Application blueprints with multi-machine, XaaS, and software components: This is by far the most powerful type of blueprint. As an example, imagine a company that wants to provide a dedicated network for each business group with Active Directory integration, a database server, a load balancer and Puppet integration: First, an appropriate Active Directory Organization Unit (OU) combined with groups and attached users has to be created. Then the on-demand network for the business group can be created via NSX. After that has been accomplished, virtual machines can be deployed. At the end, the database can be installed via a software component script and the virtual machine itself can be registered with a Puppet server (the scenario itself is depicted in 2).

Fig. 2: Blueprint with XaaS, networking and software components

8.1.2 Provisioning mechanism

Besides determining what kind of resources should be deployed, we also have to know how a machine is created. vRealize Automation offers several techniques of how to provision an implemented machine. Of course, these mechanisms depend on the blueprint type. The following provisioning types are available for blueprints:

• Basic (creates a virtual machine container without operating system)
• Clone
• Linked Clone
• Flex Clone (based on NetApp Flex Clone technology)
• Linux Kickstart
• Virtual SCCM
• WIM Image
• Cloud deployment (Amazon, Open Stack)
• vCloud Director Deployment
• PXE Provisioning
• Third-party (for example Dell iDrac)

Internally, all the logic behind these provisioning workflows is stored in vRealize Automation, within the Model Manager.

8.1.3 Lifecycle

A blueprint defines all aspects of a lifecycle. Besides the question of how to provision a resource, there are other aspects such as approval, managing at runtime, retiring or archiving. The following basic lifecycle stages exist for machines to be provisioned:

Workflow stage	Description
Requested	Request for a new machine from the service catalog
Awaiting approval	Workflow is on hold until approved by an approval manager
Register machine	Machine is registered
Building machine	The action build is about to start
Machine provisioned	Machine build completed successfully
Machine activated	The requested machine is activated
Install tools	Hypervisor guest operating system tools are installed
Expired	Machine lease has expired and machine is turned off
Deactivate machine	The machine disposal process has started
Unprovision machine	The machine un-provisioning process has started
Disposing	The hypervisor is disposing the machine
Finalized	Machine has been disposed and is about to be removed from the management

Fig. 3: Machine lifecycle

Depending on the kind of blueprint, the lifecycle encompasses additional stages. For example, virtual blueprints that are provisioned by cloning can have an additional stage for applying guest specifications to a machine.

8.1.4 Extensibility

The feature set of vRealize Automation is already quite comprehensive. However, as with any standard software, probably not all the requirements of a project can be delivered out-of-the box. There is a lot of room for adaptations in a resource’s lifecycle. Many companies have their own IP address management tool, which should be integrated into the provisioning workflows. For such scenarios, blueprints have a specific extension mechanism. We have already mentioned custom properties several times. These properties can be used to customize the lifecycle of provisioned machines. For example, there is a custom property called hostname, which overrides the hostname normally set by the machine prefix. The other option for implementing extensibility is Orchestrator. We will talk about custom properties in this chapter and focus on Orchestrator later in the book.

8.1.5 Global and local blueprints

Blueprints can be defined on a business group level, per tenant or can be even shared between tenants. In order to be used, blueprints have to be published to the service catalog and users need the appropriate permissions to deploy resources from a blueprint. Tenant administrators are in charge of creating and managing global blueprints. Global blueprints can be marked as master blueprints in order to use them as templates. Business group managers can create blueprints as well – however, these blueprints are locally assigned to that business group.

8.1.6 Machine leases

While it is quite easy for end users to provision their own machines, the situation might arise at some point in time, where all the available resources for provisioning have been exhausted. In addition, there are costs arising from machines that are not used anymore. To help avoid such issues, machine leases can be assigned. This means, when setting up a blueprint, it can be defined for how long a provisioned machine will be deployed. When the lease is over, the lease has to be extended, otherwise the machine gets archived or destroyed and its resources will be released.

8.1.7 Reclamations

Even when you have machine leases configured, the remaining capacity within your datacenter might become low and no further machines can be created. From an administrator point of view, it is difficult to choose a machine for expiry. This is because in most cases administrators do not have any knowledge regarding machine use (i.e. if it is still needed). In this case, vRealize Automation helps to identify underused machines (in terms of CPU, memory, network or disk consumption). Administrators can then ask the owners of such machines whether the resources are still required. Based on the owner’s reply, the machine resources can be released or still be kept.

8.1.8 Configuration changes at runtime

While blueprints specify the hardware configuration for newly provisioned machines, changes at runtime are quite common. Such configuration changes are possible if the blueprint is configured accordingly. For example, you can initially assign 4 GB of memory, but later allow end users to have up to 8 GB of memory. At runtime, end users can use actions within the self-service catalog to apply these changes and restart machines with the new settings.

8.1.9 Infrastructure as a code

In previous versions of vRealize Automation, blueprints could only be created and configured via the graphical interface. This has changed now and it is possible to export a blueprint as code (however, this is only possible using the CloudClient or using the REST API). vRealize Automation uses YAML and JSON for blueprints, so a declarative approach for implementing the code is used. Fig 4 shows the YAML code for a simple CentOS machine blueprint.

Having blueprints as code and using this feature has a big impact. First of all, blueprints can easily be created and modified using an editor. Second, new blueprints can easily be uploaded using the CloudClient or the REST API. In addition, if you maintain a source code version system like Git, you can upload your blueprints and keep different versions of your blueprints. Compared to manually creating blueprints, the new approach to have blueprints as code, reduces the chance of human errors. Frequently, you have to create blueprints to different environments, for example you might have a QA or a DEV environment. Often a QA tester discovers a defect due to wrongly configured resources, which could lead to further delay in the test schedule. Hence, the solution to this challenge is to automate these steps by creating a script. In addition, the automation script itself can serve as documentation. Also, automated processes  are more reliable than manual configuration and it certainly reproducible. Last but not least, having your blueprints as code allows you to easily share blueprints with others.

Fig. 4: Infrastructure as code

Benefits of infrastructure as a code Infrastructure as code is one of the key drivers to achieve the agility, efficiency, and quality many companies require. The observation that many companies made, can be summarized as follows: Any mistakes regarding quality means losing business. Testing – thorough, automated testing, not only of the application but of the infrastructure – is very important. Having the ability to make modifications and push changes quickly into production increases agility. The ability to bring the complexity of the infrastructure and your application together, as source code, fundamentally improves how efficiently you can provide services to your customers.

8.1.10 Monitoring workflow executions and viewing logs

As with any software, errors can also occur in vRealize Automation and especially during provisioning. To troubleshoot these errors, vRealize Automation provides several log files, all of which are viewable from the graphical user interface. These log files serve different purposes, which are described in the following:

• Infrastructure > Monitoring > Audit Logs: Provides information about the status of virtual, Amazon and multi-machines. NSX, reclamation and reconfiguration events are also logged.
• Infrastructure > Monitoring > DEM Status: View the status of DEMs and the details of scheduled workflows.
• Infrastructure > Monitoring > Log: Default log information.
• Infrastructure > Monitoring > Workflow History: Status and history of executed DEMs and other workflows.
• Administration > Logs: Display logs for the vRealize Automation installation (only for system administrators).
• Requests: Status of current provisioning (for end users).

In addition, you can also install vRealize Log Insight as a log collection tool, which offers additional benefits:

• It acts as a centralized tool for logging.
• vRealize Log Insight offers a sophisticated query engine, including the possibility to query for certain events or filters.
• You can set up alerting based on queries.
• The Log Insight agent can also be installed within your machine templates, so that you get notified if any error occurs within your provisioned machines.

8.1.11 Providing machine templates

Before creating and setting up blueprints, there is some work to be done in advance.

Determining the deployment technique

When vRealize Automation should deploy a virtual machine based on cloning, the appropriate template has to be set up in the hypervisor before. On the other side, if you use Windows Imaging File (WIM) as your deployment technology, there is a lot of work to be done to get WIM running in advance.

Configuration techniques

Besides determining and configuring the appropriate deployment technique, there is another important issue: How to configure deployed instances. There are different approaches available for vRealize Automation. The most popular ones can be described as follows:

• Software components: As described, these are scripts that can be executed upon instance launch, configuring, start or uninstall time. This is a simple approach for configuration, but if you have a great variety of scripts and machines, it might become difficult to manage all the scripts at the enterprise level (this of course depends on how you use software components – for example, if you use a software version system like Git for your software components).
• Orchestrator workflows: vRealize Automation can also trigger an Orchestrator workflow for further configuration. The Orchestrator workflow can subsequently connect to the virtual machine and invoke scripts via the VMware tools or can use SSH to connect to a machine and execute some logic. In addition, Orchestrator can run any kind of logic, so this approach is especially useful if you have to interact with your environment first and – based on the outcome – trigger some functionality from inside of the deployed machine.
• Guest agent: This is the traditional approach for vRealize Automation to invoke scripts. However, using the plain guest agent is quite limited.
• Fully baked machine templates: If you do not want to customize your machine during provisioning, you can deploy fully baked virtual machine templates as well. This provides a faster boot-time, but on the downside, you might be limited in the functionality and have a greater set of machine templates to be maintained.
• Configuration and deployment frameworks: Using a configuration framework like Puppet, Chef or Ansible can simplify many common administrative tasks. When provisioning a machine, you can use a script invoked from a software component to handover control to your configuration framework. The most common example is granting and taking away access from deployed machines. Let’s say that Peter has been given access to multiple Linux instances within vRealize Automation using SSH. That means, Peter has a private key that enables him to log in to the machines and perform administrative tasks on the machine. But what happens if Peter leaves the company? Without having a configuration server, removing all the public keys from the deployed machines can be a very stressful task. However, using a configuration server, this task can be automated and be done with a few commands. Another advantage of using a configuration server is that configuration is idempotent. Resources that are created by the configuration server are created only once. If a user changes the configuration by himself, any modifications will be automatically reverted by the configuration tool (this helps to ensure that the integrity of the instance is not compromised by an accidental failure).

Configuring instances at boot time

In practice, finding the right approach for the machine configuration can take some time, as there are different criteria that have to be taken into consideration:

Fully baked OS	Partially baked OS	OS only
Applications and all dependencies installed	Only prerequisites installed	Only OS with latest updates
Shorter boot time	Template can be kept longer	Shortest build time
Longer build time	Trade-off between boot time and build time	Slow boot
Shorter life span

Fig. 5: Configuration approaches

• If you have fully baked machine templates, it will certainly take more time to put a machine template into production. In addition, most probably, you will have a bigger set of machine templates to be maintained, because every template might only serve a specialized purpose.
• On the other side, if you have a machine template only containing the operating system, it will take a longer time to boot the machine, as all software has to be installed and configured at start-time.
• As a trade-off, many companies tend to deploy partially configured machine templates. Those templates will have a predefined set of software installed (that only needs to be updated quite seldom) and act as a base template. Most often, new templates will only be produced if there are important security updates to be incorporated.

Fig. 5 summarizes these approaches. So, at the end of the day, you will have to decide what tools should be incorporated within the machine template. The following table (see Fig. 6) shows some of the software that is usually installed within a machine template (and hence producing some kind of a base template).

Tool	Description
VMware tools	Needed for cloning and applying guest specification. Can only be used in vSphere environments. Orchestrator uses the VMware tools to run processes within the guest machine.
Guest agent	Used for the ability to customize machines after deployment. Can also be used to trigger scripts if the machine is deployed outside of a vSphere environment and no VMware tools are available.
Software bootstrap agent	Must be installed if software components are attached to a machine template.
vRealize Log Insight agent	Publishes log information from within the virtual machine to vRealize Log Insight.
EP Ops/ Hyperic agent	Used if you want to implement application monitoring based on vRealize Operations / Hyperic.
Configuration agents	Agents for Puppet / Chef.
Third party agents	Used for scenarios like asset management, patch management or virus scan.

Fig. 6: Software deployed in a base template

8.2 Summary

In this chapter, we have explained the basics of blueprints and have triggered some design discussions. With that knowledge in mind, we can start with creating and configuring blueprints in the next chapters.