In today’s blog post, we will learn terraform ephemeral resources. Ephemeral resources are used to safely store and handle sensitive data (passwords, api tokens, secrets, hash) in terraform. Terraform is a declarative language where you declare your target infrastructure in the form of configuration files and terraform records those configuration changes in the state file.
Now what if you have some sensitive data like API keys, database passwords, or temporary tokens that you need to use in terraform. Historically, such values could persist in Terraform state files or plan outputs, raising significant security concerns. Enter Ephemeral Resources, a new feature introduced in Terraform v1.10. The ephemeral resources provides a flexible and secure way to handle sensitive or short-lived data, ensuring it never stored in your Terraform state file.
What are Ephemeral Resources?
Ephemeral resources are a new type of Terraform resource that are designed to be temporary (short-lived). Unlike standard Terraform resources that describe infrastructure that is created and managed (and whose details are stored in the state file), ephemeral resources exist only during the current Terraform operation (plan or apply). They are never persisted in the Terraform state file (terraform.tfstate) or in plan files (.tfplan).
Key characteristics of ephemeral resources:
- Temporary Existence: They are instantiated and destroyed within a single
terraform planorterraform applyrun. - No State Persistence: Their values are explicitly not stored in the Terraform state file. This is their primary security benefit.
- Provider-Specific: Ephemeral resources are implemented by individual Terraform providers to expose specific ephemeral data types (e.g., temporary credentials, access tokens).
- Special Syntax: They are declared using a new
ephemeralblock type in HCL, distinct fromresourceordatablocks.
Why are Ephemeral Resources Important for Security?
Before ephemeral resources, fetching secrets (e.g., from AWS Secrets Manager or HashiCorp Vault) using data sources would mean those secret values, however briefly, would be stored in your Terraform state file. If that state file were compromised or accidentally exposed (e.g., in a version control system without proper encryption), your secrets would be at risk.
Ephemeral resources solve this fundamental security flaw:
- Zero Persistence in State: Sensitive data fetched via an ephemeral resource is used at runtime to configure other resources and then discarded. It never resides in the state file.
- Reduced Attack Surface: The window of exposure for sensitive data is minimized to the duration of the Terraform operation.
- Enhanced Compliance: Helps meet regulatory requirements by keeping secrets out of long-term storage artifacts.
- Dynamic Credential Handling: Ideal for generating and using short-lived credentials or tokens that are revoked immediately after use.
How to Declare and Use Ephemeral Resources
Ephemeral resources are declared using the ephemeral block, similar to resource or data blocks:
ephemeral "<resource_type>" "<resource_name>" {
# provider-specific arguments
}The specific resource_type and its arguments depend entirely on the provider implementing the ephemeral resource.
Example: Fetching a Secret from AWS Secrets Manager (Ephemeral)
Let us look at a common scenario, fetching a database password from AWS Secrets Manager and providing it to a PostgreSQL provider for database setup.
Before Ephemeral Resources (using data source – NOT Recommended for Secrets):
# data_source_example.tf
# This would fetch the secret and store it in Terraform state
data "aws_secretsmanager_secret_version" "db_password" {
secret_id = "my-database-password"
}
provider "postgresql" {
host = "my.db.endpoint"
port = 5432
username = "admin"
# Secret in state!
password = data.aws_secretsmanager_secret_version.db_password.secret_string
}With Ephemeral Resources (Secure Way – Terraform v1.10+):
# ephemeral_example.tf
# 1. Declare the ephemeral resource for AWS Secrets Manager
# Note: The exact resource type depends on the provider (e.g., aws_secretsmanager_secret_version_ephemeral in some contexts, or newer provider versions may just enable ephemerality on the existing data source implicitly)
# As of current versions, many providers now offer ephemeral versions of their secret-fetching resources,
# or support 'write-only' arguments that directly consume ephemeral values.
# Let's use a common pattern where the provider has an ephemeral specific resource:
ephemeral "aws_secretsmanager_secret_version" "db_credentials" {
secret_id = "my-database-password"
}
# 2. Configure the PostgreSQL provider using the ephemeral value
# IMPORTANT: The target argument must be a "write-only" argument.
# Write-only arguments (often suffixed with _wo) are special arguments
# designed to accept ephemeral values without storing them in state.
# Check your provider's documentation for which arguments are write-only.
provider "postgresql" {
host = "my.db.endpoint"
port = 5432
# This 'password_wo' (or similar) is a fictional example for a write-only argument.
# The actual argument name depends on the PostgreSQL provider's implementation
# (e.g., it might be 'password' if the provider is updated to handle ephemerality for it).
username = "admin"
password = ephemeral.aws_secretsmanager_secret_version.db_credentials.secret_string
}
# You can also use ephemeral resources to initialize a local variable if that local variable
# itself is not used to configure a managed resource argument that gets stored in state.
locals {
# If you create a local value from an ephemeral resource, that local value also becomes ephemeral.
db_user = jsondecode(ephemeral.aws_secretsmanager_secret_version.db_credentials.secret_string)["username"]
}
# Ephemeral output example (only allowed in non-root modules)
# output "ephemeral_db_secret" {
# value = ephemeral.aws_secretsmanager_secret_version.db_credentials.secret_string
# ephemeral = true
# }What happens during terraform plan and terraform apply?
terraform plan: Terraform recognizes the ephemeral resource. Since its value is not persisted and might depend on runtime factors, it often displays(sensitive value)or(known after apply)for ephemeral values, indicating they will not be stored in the plan file.terraform apply: During the apply phase, the ephemeral resource is “opened” (e.g., Vault provides a lease, AWS Secrets Manager returns the secret). This sensitive value is used by the target resource (e.g., thepostgresqlprovider’spasswordargument). Once the operation is complete, the ephemeral resource is “closed” (e.g., the Vault lease is revoked), and the value is discarded, never written to the state file.
Common Use Cases for Ephemeral Resources
Secure Secret Injection: This is the primary driver. Fetching database credentials, API keys, or temporary tokens from secret management systems (Vault, AWS Secrets Manager, Azure Key Vault, Infisical) and using them to configure other resources or providers without state persistence.
Dynamic Credentials: Generating short-lived, one-time-use credentials (e.g., for a temporary network tunnel or an administrative action) that are automatically revoked after the Terraform run.
Provider Configuration: Providing sensitive authentication details directly to a provider block. For instance, authenticating a Kubernetes provider with a dynamically generated, short-lived service account token.
ephemeral "google_service_account_access_token" "k8s_token" {
target_service_account = "projects/my-gcp-project/serviceAccounts/my-k8s-sa@my-gcp-project.iam.gserviceaccount.com"
lifetime = "300s" # 5-minute lifetime
}
provider "kubernetes" {
host = google_container_cluster.primary.endpoint
token = ephemeral.google_service_account_access_token.k8s_token.token
cluster_ca_certificate = base64decode(google_container_cluster.primary.master_auth[0].cluster_ca_certificate)
}Write-Only Arguments: Some resources have specific “write-only” arguments (often suffixed with _wo in newer provider versions) that are designed to accept ephemeral values. This is an explicit contract between Terraform and the provider to handle sensitive data without state persistence.
# Example using a fictional random_password ephemeral resource for a write-only argument
ephemeral "random_password" "db_admin_password" {
length = 16
special = true
override_special = "!@#$%"
}
resource "aws_db_instance" "my_db" {
# ... other config ...
password_wo = ephemeral.random_password.db_admin_password.result # 'password_wo' is a write-only argument
}Ephemeral Variables and Outputs (Terraform v1.10+)
Beyond ephemeral resource blocks, the concept of ephemerality extends to variables and outputs in Terraform v1.10+:
Ephemeral Input Variables (ephemeral = true): You can mark input variables as ephemeral = true. This prevents their values from being stored in plan files for the root module, ensuring they are re-provided (or re-evaluated if from a default) during the apply phase. Useful for extremely short-lived tokens.
variable "temp_auth_token" {
type = string
sensitive = true # Still mark as sensitive for display redaction
ephemeral = true
}Ephemeral Output Variables (ephemeral = true): Output variables in non-root modules can also be marked ephemeral = true. This allows a module to expose an ephemeral value without it being stored in the calling module’s state.
# in a child module's outputs.tf
output "db_admin_secret_ephemeral" {
value = ephemeral.aws_secretsmanager_secret_version.db_credentials.secret_string
sensitive = true
ephemeral = true
}Key Considerations & Best Practices
- Provider Support is Key: Ephemeral resources and write-only arguments are provider-specific implementations. Always check your specific provider’s documentation to see which ephemeral resources are available and which arguments support ephemeral values.
- Chaining Ephemeral Values: If you pass an ephemeral value (e.g., from an
ephemeralresource or variable) to alocalvalue, thatlocalvalue also becomes ephemeral and cannot be used in contexts where persistence is required (like arguments for standard managed resources unless they are write-only). - Security Best Practices Remain: Ephemeral resources are a powerful layer of security, but they don’t replace other best practices:
- Use dedicated secret management solutions (Vault, AWS Secrets Manager, Azure Key Vault).
- Implement IAM policies to restrict access to secrets.
- Encrypt your state file if you’re not using remote backends that encrypt by default.
- Practice least privilege for Terraform execution environments.
- Debugging Ephemeral Values: Due to their non-persistent nature, debugging issues with ephemeral values can be tricky. Use
terraform planandterraform applyoutputs carefully, understanding that values will be redacted or unknown until runtime.
Conclusion
Terraform’s ephemeral resources helps you securely managing sensitive data within Infrastructure as Code. By ensuring that passwords, secrets, and credentials never persist in your state file, they drastically reduces your security attack surface and enhance compliance. While a relatively new feature (Terraform v1.10+), understanding and applying ephemeral resources, ephemeral variables, and write-only arguments will help you to build more secure and compliant infrastructure.
Author
Experienced Cloud & DevOps Engineer with hands-on experience in AWS, GCP, Terraform, Ansible, ELK, Docker, Git, GitLab, Python, PowerShell, Shell, and theoretical knowledge on Azure, Kubernetes & Jenkins.
In my free time, I write blogs on ckdbtech.com