Python: How to use secrets stored in azure key vault in your Pulumi project
Software Engineers are excited to create Infrastructures just like DevOps Engineers. However, other IaC tools like Terraform and CloudFormation, relies on domain-specific Languages (DSLs) or JSON/YAML templates to create infrastructures. These IaC tools requires Software Engineers to learn these DSL’s before they can create infrastructures. Pulumi to the rescue.
What is Pulumi ? Pulumi is an Infrastructure as a Code (IaC) tool that allows developers to define and manage infrastructure using familiar programming languages. Developers can still use their existing Programming language to create infrastructures unlike Terraform and CloudFormation which requires Developers to learn new syntax. Pulumi allows you to write infrastructure code in your favorite programming language such as TypeScript, JavaScript, Python, Go and C#.
In this article you will learn how to use your secrets stored in Azure key Vault in your Pulumi project. You will be creating a Virtual Machine (VM) using Python as a programming language; you will need to store the username and password as a secret in Azure Key Vault which will be used for authentication.
What is Azure? Azure is Microsoft’s cloud computing platform that provides a broad range of services to help businesses build, develop, deploy and manage applications by utilizing Microsoft’s global network of data centers.
What is Azure key Vault ? Azure key Vault is a cloud service provided by Microsoft Azure designed to securely stores and manage sensitive information such as secrets, encryption keys and certificates. This allows you to protect your data by providing centeralized management, fine-grained access control, automated key rotation and seamless integration with other Azure services.
Below is a step-by-step guide on how to Dockerize an application in Rust and deploy this application to AWS ECR.
Below is a step-by-step guide on how to use your secrets stored (username and password) in Azure key vault while creating a Virtual Machine (VM) using python as a programming language.
Step 1: Install Pulumi
Firstly, you should ensure you have Pulumi installed in your development environment. Pulumi can be installed in Linux, macOS or Windows.
The command below can be used to install Pulumi
#How to install Pulumi on Linux
curl -fsSL https://get.pulumi.com | sh
#How to install Pulumi using Brew on macOS
brew install pulumi/tap/pulumi#How to install Pulumi on Windows
Download and run the Pulumi installer on this link https://www.pulumi.com/docs/install/ Step 2: Set Up Your Environment
Secondly, you need to setup your environment and install the python dependencies. Kindly follow this steps below in setting up your environment.
- Create an Account: Sign up for a Pulumi account here to store your stack state.
- Install Dependencies: Install python and pip on your workstation. Since we are using Python for creating infrastructure. However, if you are using other programming language, you can use this method to install the software.
Step 3: Create a New Pulumi Project
Initialize a new Pulumi project by running:
pulumi newYou will get a prompt after running the command, Pulumi will prompt you to select a template based on the cloud provider and language of your choice. For example
- GCP Go: This will be
gcp-go - Azure Python: This will be
azure-python - AWS TypeScript: This will be
aws-typescript
Step 4: Create a New Pulumi Project
After the project is created, you can now define your infrastructure using Python Programming language.
Create a new file called _main_.py
import pulumi, base64
import pulumi_azure_native as azure_native
import pulumi_azure_native.compute as compute
import pulumi_azure_native.network as network
import pulumi_azure_native.resources as resources
import pulumi_azure as azure
from pulumi import Config, export, asset, Output
from pulumi_azure_native.keyvault import get_secret_output
# Create a config object to access configuration values
config = pulumi.Config()
env = pulumi.get_stack()
appname = pulumi.get_project()
# Get the parameters
location = config.get("location")
key_vault_name = config.get("key_vault_name")
subscription_id = config.get("subscription_id")
resource_group_name = config.get("resource_group_name")
vm_publisher = config.get("vm_publisher")
vm_offer = config.get("vm_offer")
vm_sku = config.get("vm_sku")
vm_version = config.get("vm_version")
vm_name = config.get("vm_name")
resource_group = azure_native.resources.ResourceGroup("resourceGroup",
location=location,
resource_group_name=f"{appname}-{env}-rg"
)
# Build the Key Vault ID
key_vault_id = f"/subscriptions/{subscription_id}/resourceGroups/{resource_group_name}/providers/Microsoft.KeyVault/vaults/{key_vault_name}"
# Get the username and password from Azure Key Vault
admin_username = azure.keyvault.get_secret(name="adminUsername",key_vault_id=key_vault_id)
admin_password = azure.keyvault.get_secret(name="adminPassword",key_vault_id=key_vault_id)
# Check if the secrets are empty or invalid
if not admin_username or not admin_password or not git_token:
raise Exception("Failed to fetch admin credentials from Key Vault.")
# Serialize username,password and git_token
admin_username = admin_username.value
admin_password = admin_password.value
#Create virtual_network
virtual_network = azure_native.network.VirtualNetwork("virtualNetwork",
address_space={
"addressPrefixes": ["10.0.0.0/16"],
},
flow_timeout_in_minutes=10,
location=location,
resource_group_name=resource_group.name,
virtual_network_name=f"{appname}-{env}-vn")
#Create the subnet
subnet = azure_native.network.Subnet("subnet",
address_prefix="10.0.0.0/16",
resource_group_name=resource_group.name,
subnet_name=f"{appname}-{env}-sn",
virtual_network_name=virtual_network.name)
# Create a Public IP Address
public_ip = azure_native.network.PublicIPAddress(f"publicIP-{env}",
resource_group_name=resource_group.name,
location=location,
public_ip_allocation_method="Dynamic")
# Create a network interface without a public IP
network_interface = azure_native.network.NetworkInterface("networkInterface-" + env,
resource_group_name=resource_group.name,
location=location,
ip_configurations=[{
"name": "ipconfig1",
"subnet": azure_native.network.SubnetArgs(
id=subnet.id,
),
"public_ip_address": azure_native.network.PublicIPAddressArgs(
id=public_ip.id,
),
}]
)
# Create the Azure VM
vm = azure.compute.LinuxVirtualMachine("{vm_name}-{env}",
resource_group_name=resource_group.name,
location=location,
network_interface_ids=[network_interface.id],
size="Standard_B1ms",
disable_password_authentication=False,
admin_username=admin_username,
admin_password=admin_password,
os_disk=azure.compute.LinuxVirtualMachineOsDiskArgs(
storage_account_type="Standard_LRS",
caching="ReadWrite",
disk_size_gb=30,
),
source_image_reference=azure.compute.LinuxVirtualMachineSourceImageReferenceArgs(
publisher=vm_publisher,
offer=vm_offer,
sku=vm_sku,
version=vm_version,
),
custom_data=cloud_init_script_base64,
tags={"Environment": env}
)
#Export the VM name, resource_group_name and nginx_conf_url of the VM
export("resource_group_name", resource_group.name)
export("vm_name", vm.name)This piece of code below is used to build the key vault ID.
# Build the Key Vault ID
key_vault_id = f"/subscriptions/{subscription_id}/resourceGroups/{resource_group_name}/providers/Microsoft.KeyVault/vaults/{key_vault_name}"This piece of code is used to retrieve the username and password from Key Vault. where key_vault_id is the details of the name of the key vault saved in Azure key vault.
# Get the username and password from Azure Key Vault
admin_username = azure.keyvault.get_secret(name="adminUsername",key_vault_id=key_vault_id)
admin_password = azure.keyvault.get_secret(name="adminPassword",key_vault_id=key_vault_id)Step 5: Create Your secrets in Azure Key Vault
Your credentials needs to be created in Azure Key Vault and the key vault name will be added to Pulumi.dev.yaml file. Create a secret with secret name adminUsername and password with adminPassword this will contain the actual usernameand password for your Virtual Machine (VM).
Step 6: Create Pulumi.dev.yaml file
Create a new file Pulumi.dev.yaml file which will contain the configurations for creating the VM for dev environment. By having a Pulumi.dev.yaml this will inform Pulumi to create the infrastructure in dev namespace or environment.
config:
azure_key_vault_pulumi_example:location: eastus
azure_key_vault_pulumi_example:environment: dev
azure_key_vault_pulumi_example:key_vault_name: demmo
azure_key_vault_pulumi_example:subscription_id: 123-3-3-3=33
azure_key_vault_pulumi_example:resource_group_name: demo-dev-rg
azure_key_vault_pulumi_example:vm_publisher: Canonical
azure_key_vault_pulumi_example:vm_offer: 0001-com-ubuntu-server-jammy
azure_key_vault_pulumi_example:vm_sku: 22_04-lts
azure_key_vault_pulumi_example:vm_version: latest
azure_key_vault_pulumi_example:vm_name: demoVMStep 7: Authenticate with Your Cloud Provider (Azure)
You will need to authenticate with your cloud provider before deploying or managing your resource with Pulumi. For this use-case, you will be using Azure as the Cloud Provider. By running command az-login this will allow you to authenticate with Azure.
Step 8: Install the Azure CLI (If not already installed)
To be able to run command az-login you will need to make sure you have Azure CLI installed. You can install it by using the following command below.
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bashAlternatively check this link on how to install Azure CLI on different Operating System (OS).
Step 9: How to Authenticate Azure Using Azure CLI
Run the following command below to authenticate with Azure. After running the command, it will open a web browser where you can sign in with your Azure credentials. Once the authentication is successful, the Azure CLI will cache your credentials locally. Please note: if you have multiple subscriptions, you have the flexibility of selecting the one you want.
az loginStep 10: Preview Your Infrastructure
By running the command below. This will allow you to preview the changes Pulumi will make to your infrastructure.
pulumi previewStep 11: Deploy Your Infrastructure
By running the command below. This will allow you to apply the changes to deploy your infrastructure.
pulumi upPlease note: Pulumi will show you a detailed plan and as for a confirmation before proceeding to apply the changes
Conclusion
I hope you find this process of receiving and onsuming secrets from Azure Key Vault in your Pulumi Project useful and interesting. Check out the completed code on GitHub
