Mastering Lists In Terraform

Mastering Lists In Terraform

by

In today’s blog post, we will learn working with lists in terraform. In particular, defining lists and tuples, accessing list elements, terraform list functions to get the length of a list, merge two lists into one, remove duplicates, sort and reverse, split a list, compare and find the difference between two lists, checking if an item exists in a list, getting specific item from a list based on condition, and best practices. Hence without any further delay, let us get started.

Defining Lists and Tuples

In Terraform, both lists and tuples represent ordered sequences of elements. The primary difference lies in their type constraint strictness:

  • List (or list type): A collection where all elements are expected to be of the same type, though Terraform is often flexible and can implicitly convert. It is typically used for variable-length collections where the number of elements might change.
  • Tuple (or tuple type): A fixed-length, fixed-type collection where each element can have a distinct type. Tuples are less common for general resource definitions but are powerful for representing structured records.

For most common use cases, you will primarily interact with the list type.

Defining a List

You can define a list by enclosing its elements in square brackets [], separated by commas.

# main.tf

# Define a list of strings
variable "instance_names" {
  description = "A list of names for EC2 instances."
  type        = list(string)
  default     = ["web-server-01", "db-server-01", "app-server-01"]
}

# Define a list of numbers
variable "ports" {
  description = "A list of port numbers."
  type        = list(number)
  default     = [80, 443, 22]
}

# Define a list of mixed types (though less common for type-constrained variables)
output "mixed_list_example" {
  value = ["hello", 123, true, null]
  # Outcome: ["hello", 123, true, null] (type becomes list(any))
}

Explanation of the code:

  • The instance_names variable is a list of strings, providing names that can be used to create multiple instances.
  • The ports variable is a list of numbers, useful for defining security group rules.
  • The mixed_list_example output shows a list with elements of different types. When a variable’s type constraint is list(any), it allows elements of any type.

Defining a Tuple

While less frequently explicitly defined in variables, tuples are common for fixed, ordered sets of heterogeneous data.

# main.tf

# Define a tuple type for a network interface configuration
variable "network_config" {
  description = "A tuple containing IP and port for a service."
  type        = tuple([string, number]) # Tuple of string and number
  default     = ["192.168.1.100", 8080]
}

# Accessing tuple elements
output "config_ip" {
  value = var.network_config[0]
  # Outcome: "192.168.1.100"
}

output "config_port" {
  value = var.network_config[1]
  # Outcome: 8080
}

Explanation of the code:

  • The network_config variable is explicitly defined as a tuple with a string and a number. This ensures a fixed structure for the configuration.
  • Individual elements of the tuple can be accessed using zero-based indexing.

Accessing List Elements

You can retrieve specific elements from a list or tuple using their zero-based index or the element function.

# main.tf

# Accessing a list element by index
variable "my_list" {
  type    = list(string)
  default = ["apple", "banana", "cherry"]
}

output "first_fruit" {
  description = "The first element of the list."
  value       = var.my_list[0]
  # Outcome: "apple"
}

output "second_fruit" {
  description = "The second element of the list."
  value       = var.my_list[1]
  # Outcome: "banana"
}

# Using the element function (useful with count or for robustness)
output "fruit_by_element_function" {
  description = "Accessing an element using the element function."
  value       = element(var.my_list, 2)
  # Outcome: "cherry"
}

Explanation of the code:

  • Direct indexing (var.my_list[0]) is the most common way to access elements.
  • The element(list, index) function serves a similar purpose but can be more robust in certain dynamic scenarios, especially when combined with count.

Common List Functions

Terraform provides several built-in functions to manipulate lists effectively.

Getting the Length of a List

The length function returns the number of elements in a list or tuple.

# main.tf

variable "regions" {
  type    = list(string)
  default = ["us-east-1", "eu-west-1", "ap-southeast-2"]
}

output "number_of_regions" {
  description = "The number of regions in the list."
  value       = length(var.regions)
  # Outcome: 3
}

Explanation of the code:

  • The length function is useful for determining how many resources or configurations need to be created when using count with a list.

Merging Lists

The concat function joins two or more lists into a single list.

# main.tf

variable "dev_ips" {
  type    = list(string)
  default = ["10.0.1.10", "10.0.1.11"]
}

variable "prod_ips" {
  type    = list(string)
  default = ["10.0.2.10", "10.0.2.11"]
}

output "all_ips" {
  description = "Combined list of development and production IPs."
  value       = concat(var.dev_ips, var.prod_ips, ["10.0.3.1"]) # Can merge more than two
  # Outcome: ["10.0.1.10", "10.0.1.11", "10.0.2.10", "10.0.2.11", "10.0.3.1"]
}

Explanation of the code:

  • concat is valuable for combining different sets of configurations or merging default values with environment-specific overrides.

Flattening Nested Lists

The flatten function takes a list of lists and flattens it into a single list.

# main.tf

variable "nested_subnets" {
  type    = list(list(string))
  default = [
    ["subnet-a", "subnet-b"],
    ["subnet-c", "subnet-d"]
  ]
}

output "flat_subnets" {
  description = "A flattened list of subnet IDs."
  value       = flatten(var.nested_subnets)
  # Outcome: ["subnet-a", "subnet-b", "subnet-c", "subnet-d"]
}

Explanation of the code:

  • flatten is particularly useful when dealing with outputs from modules that might return nested lists, and you need a single, linear list for further processing.

Removing Duplicates

The distinct function returns a new list containing only the unique elements from the input list, preserving the original order of the first appearance.

# main.tf

variable "roles" {
  type    = list(string)
  default = ["admin", "viewer", "admin", "editor", "viewer"]
}

output "unique_roles" {
  description = "List of distinct roles."
  value       = distinct(var.roles)
  # Outcome: ["admin", "viewer", "editor"]
}

Explanation of the code:

  • distinct ensures that you are working with only unique values, preventing redundant resource creation or configuration issues.

Sorting and Reversing Lists

The sort function sorts elements alphabetically or numerically, and the reverse function reverses the order of elements in a list.

# main.tf

variable "unsorted_names" {
  type    = list(string)
  default = ["zebra", "apple", "banana"]
}

variable "unsorted_numbers" {
  type    = list(number)
  default = [5, 1, 9, 2]
}

output "sorted_names" {
  description = "Alphabetically sorted list of names."
  value       = sort(var.unsorted_names)
  # Outcome: ["apple", "banana", "zebra"]
}

output "sorted_numbers" {
  description = "Numerically sorted list of numbers."
  value       = sort(var.unsorted_numbers)
  # Outcome: [1, 2, 5, 9]
}

output "reversed_names" {
  description = "List of names in reverse order."
  value       = reverse(var.unsorted_names)
  # Outcome: ["banana", "apple", "zebra"] (original order was "zebra", "apple", "banana", reverse applies to current order)
}

Explanation of the code:

  • sort is useful for standardizing lists or when you need elements processed in a specific order.
  • reverse provides a way to simply invert the order of elements.

Slicing Lists

The slice function extracts a sub-list from a given list, similar to slicing in other programming languages.

# main.tf

variable "full_list" {
  type    = list(string)
  default = ["a", "b", "c", "d", "e"]
}

output "slice_from_start" {
  description = "Elements from index 0 up to (but not including) index 2."
  value       = slice(var.full_list, 0, 2)
  # Outcome: ["a", "b"]
}

output "slice_from_middle" {
  description = "Elements from index 1 up to (but not including) index 4."
  value       = slice(var.full_list, 1, 4)
  # Outcome: ["b", "c", "d"]
}

output "slice_to_end" {
  description = "Elements from index 2 to the end."
  value       = slice(var.full_list, 2, length(var.full_list))
  # Outcome: ["c", "d", "e"]
}

Explanation of the code:

  • slice allows you to extract specific portions of a list, defined by a starting index and an ending index (exclusive).

Set Operations

For lists that are treated as mathematical sets (where order and duplicate elements do not matter), Terraform provides functions for common set operations. These functions normalize the input lists into sets internally before performing the operation.

Union of Lists

The setunion function returns a list containing all unique elements that are present in any of the input lists.

# main.tf

variable "group_a_users" {
  type    = list(string)
  default = ["alice", "bob", "charlie"]
}

variable "group_b_users" {
  type    = list(string)
  default = ["bob", "diana", "eve"]
}

output "all_unique_users" {
  description = "Union of users from both groups."
  value       = setunion(var.group_a_users, var.group_b_users)
  # Outcome: ["alice", "bob", "charlie", "diana", "eve"] (order might vary)
}

Explanation of the code:

  • setunion is useful for combining distinct elements from multiple sources, like combining all users across different access groups.

Intersection of Lists

The setintersection function returns a list containing only the unique elements that are common to all input lists.

# main.tf

output "common_users" {
  description = "Users present in both groups."
  value       = setintersection(var.group_a_users, var.group_b_users)
  # Outcome: ["bob"] (order might vary)
}

Explanation of the code:

  • setintersection helps identify common elements between different lists, such as users with access to multiple systems.

Difference Between Lists

The setdifference function returns a list containing unique elements from the first list that are not present in any of the subsequent lists.

# main.tf

output "users_only_in_group_a" {
  description = "Users present in group A but not in group B."
  value       = setdifference(var.group_a_users, var.group_b_users)
  # Outcome: ["alice", "charlie"] (order might vary)
}

Explanation of the code:

  • setdifference is useful for finding elements unique to one list when compared to others, for example, identifying users who lost access or resources that are no longer needed.

Checking for Membership

The contains function checks if a list or set contains a specific element.

# main.tf

variable "allowed_zones" {
  type    = list(string)
  default = ["us-east-1a", "us-east-1b"]
}

output "is_zone_allowed_a" {
  description = "Checks if 'us-east-1a' is in allowed zones."
  value       = contains(var.allowed_zones, "us-east-1a")
  # Outcome: true
}

output "is_zone_allowed_c" {
  description = "Checks if 'us-east-1c' is in allowed zones."
  value       = contains(var.allowed_zones, "us-east-1c")
  # Outcome: false
}

Explanation of the code:

  • contains is a simple boolean check, very useful in conditional expressions or for validating inputs against a predefined list of allowed values.

Transforming and Filtering with For Expressions

The for expression can be used for iterating over lists, transforming elements, and filtering results. It allows you to create new lists or maps based on existing collections.

Basic Transformation

# main.tf

variable "server_names" {
  type    = list(string)
  default = ["web", "app", "db"]
}

output "formatted_server_ids" {
  description = "List of formatted server IDs."
  value       = [for name in var.server_names : "${name}-server-id"]
  # Outcome: ["web-server-id", "app-server-id", "db-server-id"]
}

Explanation of the code:

  • This for expression iterates over each name in var.server_names and creates a new string for each item, resulting in a new list of formatted IDs.

Filtering Elements

# main.tf

variable "all_ports" {
  type    = list(number)
  default = [80, 443, 22, 8080, 3306]
}

output "http_https_ports" {
  description = "List containing only HTTP and HTTPS ports."
  value       = [for port in var.all_ports : port if contains([80, 443], port)]
  # Outcome: [80, 443]
}

Explanation of the code:

  • This for expression iterates over all_ports but includes port in the new list only if it is present in the [80, 443] list, effectively filtering the original list.

Practical Use Cases

  • Creating Multiple Resources: Use count with length(var.list) and element(var.list, count.index) to create identical resources with names or other properties derived from list elements. Even better, use for_each with a set of strings or a map.
  • Dynamic Security Group Rules: Define ingress/egress rules based on lists of CIDR blocks or port ranges, using for_each on aws_security_group_rule for each list element.
  • Module Inputs/Outputs: Pass lists of IDs or names between modules to manage dependencies and organize infrastructure.

Best Practices for List Manipulation

  • Type Consistency: While Terraform is flexible, explicitly defining the type of your lists (e.g., list(string)) in variables improves readability and helps catch errors early.
  • Readability: For complex transformations, break down for expressions or chained function calls into local values for better clarity.
  • for_each vs. count: For creating multiple resources based on a list, for_each is often preferred over count when the order of elements might change or you need more robust state management (e.g., when removing an item from the middle of the list, for_each manages resource lifecycle better). However, count is still useful for simple, numerically indexed repetitions.
  • Null Values: Be mindful of null values within lists, as some functions might behave unexpectedly or require explicit handling.
  • Comments: Add clear comments for complex list manipulation logic, explaining the intent and expected outcome.

Conclusion

Mastering list and tuple manipulation in Terraform is an important skill for building dynamic and scalable infrastructure. By effectively using built-in functions like length, concat, distinct, and sort, along with the powerful for expressions for transformation and filtering, you can write more concise, reusable, and adaptable Terraform configurations.

Author

Debjeet Bhowmik

Experienced Cloud & DevOps Engineer with hands-on experience in AWS, GCP, Terraform, Ansible, ELK, Docker, Git, GitLab, Python, PowerShell, Shell, and theoretical knowledge on Azure, Kubernetes & Jenkins.
In my free time, I write blogs on ckdbtech.com

Leave a Comment