Read Terraform provisioned resources with terraform_remote_state data source.

Photo by Fotis Fotopoulos on Unsplash

I would like start this blog by asking question, as experienced terraform professional if you have a requirement to read the existing cloud resource with terraform code then what would have been you reply?

Well, 99% will say we need to use data source block of the resource which you want to read it your terraform code, such as: 👇

data "azurerm_storage_account" "example" {
  name                = "packerimages"
  resource_group_name = "packer-storage"
}

output "storage_account_tier" {
  value = data.azurerm_storage_account.example.account_tier
}

👆 This is correct as well but do you know that if you have a cloud infrastructure provisioned with terraform and managed by terraform remote state then you can read such cloud resources using terraform_remote_state data source?

In this blog I will sharing how to read the cloud resources with terraform_remote_state, as always we will be using azure cloud resources for our demonstration in this blog.

What is terraform remote state block?

The terraform_remote_state data source uses the latest state snapshot from a specified state backend to retrieve the root module output values from some other Terraform configuration.

You can use the terraform_remote_state data source without requiring or configuring a provider. It is always available through a built-in provider with the source address terraform.io/builtin/terraform. That provider does not include any other resources or data sources.

It means that it might possible that to create a resources with terraform you might have used terraform providers such as azurerm or any other, now to read the same provisioned resource from terraform using terraform_remote_state you do not have include or mentioned any provider names because this data source block only care about the terraform remote state location/details.

The terraform_remote_state works with different cloud providers and the configuration for terraform_remote_state requires different settings for different cloud provider.

Remote access syntax

Now let’s look at the syntax of terraform_remote_state data block for azurerm resource provider, you need to specify azure storage account container and remote state blob container address.

This requires following details:

storage_account_name = "xxxxx"    
container_name       = "xxxxx"    
environment          = "public"    
resource_group_name  = "xxxxx"    
key                  = "dev-eastus.terraform.tfstate"

When & How to use?

If your infrastructure is configured with following structure

where we are configuring Network resource first then compute and then some LOB or Lab resource, where these specific resource requires to be created in specific order such as, Network and common resources (log-analytics, keyvault, firewall etc) need to be provisioned first as these resources are something which might be shared with different LOB projects and the LOB resources to be follow as these LOB would use the network and common resources.

Step-01: NETWORK Create Network resources as such virtual network, subnet, NSG, firewall, azure polices etc in side NETWORK folder.

Step-02- Shared-Infra Create Common Log analytics, keyvault, log storage account, availability set, management vm etc inside ShareInfa folder.

Step-03- Compute Create virtual machine, Kubernetes and other compute resources.

Compute block might requires to refer to network resources and shared infra resources and to refer these dependent resources we can use the terraform_remote_state.

Let’s say shared infra is creating one storage account

This storage account now requires to expose the properties or attributes which will be requires to access by compute block, like this

Now let’s say we have ADF resource created in compute block and we are setting up ADF linked services and private endpoint which requires the storage account details which was created in shared infra, now at our compute block we can use following code block.

Terraform Resource Data source vs terraform_remote_state

If you have requirement to read an existing resource configuration in your terraform code which is not provision by your terraform code then use terraform data source block as below

data "azurerm_log_analytics_workspace" "example" {
  name                = "acctest-01"
  resource_group_name = "acctest"
}

but if the resources are provision with your terraform code block and the state of those resources are stored into state file such as azure storage account (in case of azurerm) then I would prefer to use terraform_remote_state.

with defaul data source block sometimes I have seen issues when it shows the modificaiton at the data read section even though those modification are not of our interest, such as the example below we are reading the log analytics block and its showing the changes in the data source block because of the difference it found in the tags, in case if those tags are getting modified by some external process such as azure policy, in this case every time when data source block run it shows the modification and terraform never show its up to date, where in case of data read of log analytics my only interest to read the logId and Key , I may not be interested in tag values if its modified, we dont have a life cycle ignore block which we can specify with data block, so in these kind of case I prefer to use terraform_remote_state.

as you can see the image reference below where its showing the changes when the tags were modified.

I hope you got something new to learn and use it in your terraform code.

Cheers !! 🍻🍺🍺