Enforcing Tag Policies in Terraform

Every resource that reaches AWS Config already exists and is already billing. Terraform is the earlier, cheaper gate: the moment between terraform plan and terraform apply where a missing CostCenter tag is a rejected pull request instead of a drift event chasing a running resource. This page covers the pre-provisioning half of the tag-enforcement architecture — provider-level default_tags blocks, module-level tag contracts expressed as variable validation, and the harder problem those two mechanisms cannot solve: proving, at plan time, that the final merged tag set on every resource satisfies policy. That proof requires parsing the machine-readable plan, either with a policy-as-code engine like OPA/Conftest or HashiCorp Sentinel, or with a purpose-built checker against terraform show -json output. It is the shift-left complement to the continuous, post-provisioning enforcement described in Tagging Policy Enforcement with AWS Config — catching the defect before the resource exists, rather than remediating it after.

# Architecture Context & Data-Flow Position

This component sits at the validation stage of the four-stage pipeline (acquisition → normalization → allocation → persistence) defined in the parent Resource Tagging & Validation Pipelines reference, but it runs before acquisition ever has a resource to discover. Where AWS Config validates a configuration item that already exists, Terraform enforcement validates a plan — a declarative diff of what will exist if the apply proceeds. The acquisition source is not a provider API but the plan’s resource_changes array; normalization is the same canonical tag-schema comparison; and the “remediation” action is not a tag write, it is a non-zero exit code that blocks the pipeline.

Four independent mechanisms compose the enforcement surface, each with a different blast radius and a different way it fails silently if you rely on it alone:

None of these four is sufficient in isolation. default_tags and variable validation catch cheap, common mistakes early and for free; OPA/Conftest or Sentinel are the only mechanisms that can assert on the resource as Terraform will actually create it — including provider defaults, computed values, and per-resource overrides. Production pipelines run all four, in the order shown below: local guardrails first, plan-JSON policy gate last, immediately before apply.

# Core Implementation Patterns

# 1. Provider-level default_tags

The default_tags block on an AWS provider configuration merges a tag set into every resource that provider manages, without touching individual resource blocks:

provider "aws" {
  region = "us-east-1"

  default_tags {
    tags = {
      ManagedBy   = "terraform"
      CostCenter  = var.cost_center
      Environment = var.environment
      Repository  = "github.com/org/infra"
    }
  }
}

Three constraints matter in practice. First, the merge is additive but a resource-level tags argument that sets the same key to a different value wins over the provider default — a common source of accidental drift when a copy-pasted module hardcodes Environment = "dev". Second, AWS caps combined tags at 50 per resource; a provider default_tags block that grows unchecked will start silently truncating or erroring on resources that already carry many tags. Third, default_tags must be declared per provider alias — a multi-region or multi-account module with several aws provider aliases needs the block repeated in each, or the aliased resources ship untagged. default_tags reduces the surface area for missing tags; it does not prove compliance, because a module author can still override or omit a key on a specific resource. Where a default_tags value is sourced — an account-level CostCenter, a per-workspace Environment — is itself a governance decision, and belongs alongside the IAM and service-control-policy boundaries codified in Setting Up FinOps Governance Boundaries in Terraform rather than hardcoded per module.

A fourth, easy-to-miss interaction: a resource lifecycle { ignore_changes = [tags] } block, often added to stop Terraform from fighting an external auto-tagging tool, also blinds the plan to any subsequent tag drift on that resource, including a required key that a teammate accidentally strips out-of-band. If ignore_changes targets tags, scope it to the specific keys the external tool owns — ignore_changes = [tags["LastScannedAt"]] — rather than the whole map, so the plan-time gate still sees every key it is responsible for.

# 2. Module-level tag contracts with variable validation

A module’s tags input variable is the contract between the module and its callers. Enforce the contract with a validation block so a missing required key fails terraform plan immediately, before any provider call:

variable "tags" {
  description = "Resource tags. Must include the FinOps-mandated key set."
  type        = map(string)

  validation {
    condition = alltrue([
      for key in ["CostCenter", "Environment", "Owner"] : contains(keys(var.tags), key)
    ])
    error_message = "tags must include CostCenter, Environment, and Owner."
  }

  validation {
    condition     = contains(["dev", "staging", "prod", "dr"], lookup(var.tags, "Environment", ""))
    error_message = "Environment tag must be one of: dev, staging, prod, dr."
  }
}

This is the cheapest possible gate — it runs entirely inside terraform validate/plan, needs no external tooling, and gives an error message pointing at the exact module call. Its blind spot is structural: it validates the map passed into the module, not the tag set that ends up on the resource after default_tags merging, merge() calls inside the module, or per-resource ignore_changes on tags. A module can pass validation and still produce an untagged resource if its internal aws_instance block never references var.tags. Treat variable validation as a fast developer-feedback layer, not the enforcement boundary — that boundary is the plan-JSON gate below, wired into CI in Enforcing Tag Policies in Terraform Plan Checks.

# 3. Parsing terraform show -json plan output

The only artifact that reflects the actual resource Terraform will create is the plan, exported as JSON:

terraform plan -out=tfplan.binary
terraform show -json tfplan.binary > plan.json

The resulting document’s resource_changes array is what every policy engine — OPA, Sentinel, or a custom checker — actually evaluates. Each entry carries an address, a type, a change.actions list (["create"], ["update"], ["delete"], ["no-op"], or ["delete", "create"] for replacements), and change.after, the planned post-apply attribute set. Two attributes matter for tag checks: tags, the literal value from the HCL, and tags_all, the AWS provider’s computed merge of resource-level tags with the provider’s default_tags. Always evaluate tags_all when the provider supports it — checking tags alone will report false positives on every resource that relies on default_tags for a required key, because that key legitimately never appears in the resource’s own tags argument.

jq '.resource_changes[] | select(.change.actions[] == "create") | {address, type, tags: .change.after.tags_all}' plan.json

# 4. Policy structure: OPA/Conftest vs. Sentinel

Conftest runs Open Policy Agent’s Rego language directly against the plan JSON, no vendor lock-in:

package terraform.tagging

import future.keywords.in

required_tags := {"CostCenter", "Environment", "Owner", "ManagedBy"}

deny[msg] {
  change := input.resource_changes[_]
  change.change.actions[_] == "create"
  not change.type in {"random_id", "null_resource"}
  present := {k | change.change.after.tags_all[k]}
  missing := required_tags - present
  count(missing) > 0
  msg := sprintf("%s is missing required tags: %v", [change.address, missing])
}

Run it as conftest test plan.json -p policy/; a non-empty deny set exits non-zero. Sentinel achieves the same outcome but only inside Terraform Cloud/Enterprise, importing tfplan/v2 instead of raw JSON, and supports three enforcement levels — advisory (warns, never blocks), soft-mandatory (blocks but overridable by a privileged user), and hard-mandatory (blocks unconditionally). Choose Sentinel when the organization is already standardized on TFC/TFE-managed runs and needs org-wide policy sets applied without per-repo wiring; choose OPA/Conftest when pipelines run on self-hosted CI (GitHub Actions, GitLab CI, Jenkins) or need the same policy portable outside Terraform entirely, for example against Kubernetes admission review or CloudFormation. A Python checker sits between the two: no Rego or Sentinel language to learn, trivial to unit test, and easy to keep in lock-step with the same required-tag manifest the runtime AWS Config rule consumes.

# Production-Grade Python Plan-Checker Engine

The module below is a complete, self-contained plan-time tag gate. It loads terraform show -json output, filters to mutating resource_changes, asserts the required-tag contract against tags_all (falling back to tags), and exits non-zero on any violation — the exact signal a CI step needs to fail the pipeline before terraform apply runs.

#!/usr/bin/env python3
"""
Plan-time tag policy gate.

Parses the JSON output of `terraform show -json <planfile>` and asserts that
every resource_changes entry destined for create or update carries the
required tag set. Runs as a CI step immediately after `terraform plan`,
before `terraform apply` — the shift-left complement to runtime AWS Config
evaluation.
"""

from __future__ import annotations

import argparse
import json
import logging
import sys
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Dict, List, Optional, Set

# Structured JSON logging so CI log aggregators can query violations directly.
logger = logging.getLogger("tf_tag_gate")
logger.setLevel(logging.INFO)
_handler = logging.StreamHandler(sys.stdout)
_handler.setFormatter(logging.Formatter("%(message)s"))
if not logger.handlers:
    logger.addHandler(_handler)

# Actions that actually mutate real infrastructure. "no-op" and pure deletes
# leave nothing to tag and must never trip a violation.
MUTATING_ACTIONS = {"create", "update"}

# Resource types that either do not accept tags or manage them through a
# nested block Terraform reports differently (aws_autoscaling_group's `tag`
# argument is a list of {key,value,propagate_at_launch}, not a flat map).
EXEMPT_TYPES = {
    "aws_iam_policy_document",
    "aws_iam_role_policy",
    "aws_s3_bucket_policy",
    "aws_route",
    "aws_route_table_association",
    "random_id",
    "random_password",
    "null_resource",
}

# The tag contract. In production this is loaded from the same versioned
# manifest the runtime AWS Config rule consumes, so the two enforcement
# layers never drift apart on what "compliant" means.
DEFAULT_REQUIRED_TAGS = {"CostCenter", "Environment", "Owner", "ManagedBy"}


def log(event: str, **fields: Any) -> None:
    """Emit one structured JSON log line."""
    logger.info(json.dumps({"event": event, **fields}))


@dataclass
class ResourceChange:
    """The slice of a resource_changes entry the gate actually needs."""

    address: str
    resource_type: str
    provider_name: str
    actions: List[str]
    tags: Dict[str, str] = field(default_factory=dict)

    @classmethod
    def from_json(cls, entry: Dict[str, Any]) -> "ResourceChange":
        change = entry.get("change", {}) or {}
        after = change.get("after") or {}
        # tags_all is the provider's merge of resource tags + default_tags;
        # prefer it, fall back to tags for providers/resources that don't
        # expose the merged attribute.
        tags = after.get("tags_all")
        if tags is None:
            tags = after.get("tags")
        return cls(
            address=entry.get("address", ""),
            resource_type=entry.get("type", ""),
            provider_name=entry.get("provider_name", ""),
            actions=change.get("actions", []) or [],
            tags=tags or {},
        )

    @property
    def is_mutating(self) -> bool:
        return bool(MUTATING_ACTIONS.intersection(self.actions))

    @property
    def is_exempt(self) -> bool:
        return self.resource_type in EXEMPT_TYPES


@dataclass
class Violation:
    address: str
    resource_type: str
    missing_tags: List[str]

    def as_dict(self) -> Dict[str, Any]:
        return {
            "address": self.address,
            "resource_type": self.resource_type,
            "missing_tags": self.missing_tags,
        }


@dataclass
class GateResult:
    total_resources: int
    evaluated: int
    exempt: int
    violations: List[Violation] = field(default_factory=list)

    @property
    def passed(self) -> bool:
        return not self.violations


def load_resource_changes(plan_path: Path) -> List[Dict[str, Any]]:
    """Load `terraform show -json` output and return its resource_changes."""
    with plan_path.open("r", encoding="utf-8") as fh:
        plan = json.load(fh)
    return plan.get("resource_changes", [])


def evaluate_plan(
    raw_changes: List[Dict[str, Any]],
    required_tags: Set[str],
) -> GateResult:
    """Pure evaluation logic — deterministic and unit-testable in isolation."""
    result = GateResult(total_resources=len(raw_changes), evaluated=0, exempt=0)

    for entry in raw_changes:
        change = ResourceChange.from_json(entry)

        if not change.is_mutating:
            continue  # no-op / delete-only entries never need tags

        if change.is_exempt:
            result.exempt += 1
            continue

        result.evaluated += 1
        present = set(change.tags.keys())
        missing = sorted(required_tags - present)
        if missing:
            result.violations.append(
                Violation(
                    address=change.address,
                    resource_type=change.resource_type,
                    missing_tags=missing,
                )
            )

    return result


def report(result: GateResult) -> None:
    log(
        "gate_summary",
        total_resources=result.total_resources,
        evaluated=result.evaluated,
        exempt=result.exempt,
        violation_count=len(result.violations),
    )
    for v in result.violations:
        log("tag_violation", **v.as_dict())


def parse_args(argv: Optional[List[str]] = None) -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Terraform plan tag policy gate")
    parser.add_argument(
        "plan_json",
        type=Path,
        help="Path to `terraform show -json <planfile>` output",
    )
    parser.add_argument(
        "--required-tag",
        action="append",
        dest="required_tags",
        default=None,
        help="Required tag key (repeatable). Defaults to the built-in contract.",
    )
    return parser.parse_args(argv)


def main(argv: Optional[List[str]] = None) -> int:
    args = parse_args(argv)
    required = set(args.required_tags) if args.required_tags else DEFAULT_REQUIRED_TAGS

    if not args.plan_json.exists():
        log("plan_file_missing", path=str(args.plan_json))
        return 2

    raw_changes = load_resource_changes(args.plan_json)
    result = evaluate_plan(raw_changes, required)
    report(result)

    if not result.passed:
        log("gate_failed", required_tags=sorted(required))
        return 1

    log("gate_passed")
    return 0


if __name__ == "__main__":
    sys.exit(main())

Run it as python tag_gate.py plan.json directly after the terraform show -json export step; a non-zero exit fails the CI job before the apply step is reachable. The design choices that matter: tags_all is preferred over tags so default_tags-sourced keys are not false-flagged as missing; EXEMPT_TYPES is an explicit allowlist rather than an inferred heuristic, because guessing which resource types support tagging is exactly the kind of silent gap that lets a real violation through; and every violation is logged individually as structured JSON so the CI log itself is queryable without re-running the gate. The CI wiring — caching the plan artifact, running this alongside Conftest, and posting violations as PR comments — is covered in Enforcing Tag Policies in Terraform Plan Checks.

# Schema Reference Table

The checker collapses Terraform’s plan JSON fields onto the same canonical compliance model used by the runtime AWS Config evaluator, so a violation from either enforcement layer is structurally identical downstream.

# Operational Considerations

• Unknown values at plan time. When a tag value depends on a resource not yet created (for example, an id interpolated from another resource), after_unknown.tags_all reports true and the concrete value is unavailable until apply. Treat this as a defer, not a violation — failing the plan on a legitimately unresolvable value trains teams to route around the gate.
• Plan JSON size. A plan touching several thousand resources can produce a multi-hundred-megabyte JSON file; stream-parse with ijson instead of json.load once a module’s plan regularly exceeds roughly 50 MB, or the CI runner’s memory ceiling becomes the bottleneck rather than the policy logic.
• format_version drift. Terraform’s plan JSON schema is versioned independently of the Terraform CLI version; pin the format_version your gate expects and fail loudly on a mismatch rather than silently misreading a renamed field after a Terraform upgrade.
• Provider parity. tags_all is an AWS-provider convention (added in provider v4+); GCP’s google provider and Azure’s azurerm provider expose only labels/tags with no separate merged attribute, because neither has a default_tags-equivalent construct at time of writing. A multi-provider gate needs a per-provider field map, not a single hardcoded key.
• CI wiring and artifact caching. Export the plan once (terraform plan -out=tfplan.binary), convert it once, and run every policy engine — this checker, Conftest, checkov -f plan.json — against the same plan.json artifact so a rerun cannot see a plan that has drifted from what will actually be applied.
• Monitoring hooks. Emit gate_summary events to your CI observability pipeline and alert on a rising violation_count trend across PRs — a sudden spike usually means a shared module changed its tag-merging behavior, not that every author simultaneously forgot to tag resources.
• Rollout sequencing for a new required tag. Promoting a new key straight to a hard-mandatory gate breaks every open pull request simultaneously. Land the check as advisory (Sentinel) or a non-blocking CI annotation (Conftest/Python) for one release cycle, publish the violation count, then flip it to blocking once the backlog is near zero — the same staged rollout the runtime AWS Config rule uses when its own required tag set changes.
• State-only changes and imports. terraform import and moved/removed blocks can introduce resources into state without ever appearing as a create action in a subsequent plan; a gate that only inspects resource_changes will not catch a resource imported with missing tags until its next genuine update. Pair the plan-time gate with a periodic batch sweep for full coverage of imported resources.

# Troubleshooting

The gate reports violations for resources that clearly set the tag in HCL. Root cause: the checker (or the policy you wrote) reads tags instead of tags_all, and the tag in question is supplied only via the provider’s default_tags block, never in the resource’s own tags argument. Detection: the missing key is always one that appears in provider.default_tags and never in the resource block itself. Remediation: switch the field read to tags_all with a fallback to tags, exactly as ResourceChange.from_json does above.

A resource type the team knows supports tags is always flagged missing. Root cause: the provider version in use predates that resource type’s tags_all support, so only the literal tags argument is populated and it is empty because the team relies entirely on default_tags. Detection: terraform providers shows a provider version below the release that introduced default_tags propagation for that resource. Remediation: upgrade the provider, or explicitly set tags = merge(var.tags, local.extra_tags) on that resource until the upgrade lands.

The CI job passes locally but fails in the pipeline, or vice versa. Root cause: the local plan was generated against a different .tfvars file or workspace than CI uses, so Environment or another variable-derived tag differs between runs. Detection: diff the two plan.json files’ resource_changes[].change.after.tags_all for the same address. Remediation: pin the exact -var-file and workspace the gate runs against in both environments, and fail the CI step explicitly if the workspace name is unset.

Gate exits 0 even though a violation clearly exists in the plan. Root cause: the resource’s change.actions is ["no-op"] because it was accidentally excluded from the mutating-action filter, or the resource type was added to EXEMPT_TYPES by mistake during a prior incident and never removed. Detection: manually grep plan.json for the resource address and confirm its actions and type. Remediation: audit EXEMPT_TYPES on every review of the gate’s own code, and add a unit test asserting the specific resource type is not exempt.

Plan JSON parsing crashes with a KeyError or TypeError after a Terraform upgrade. Root cause: a Terraform CLI upgrade shipped a plan format_version bump that renamed or restructured a field the checker relies on. Detection: the crash trace points at entry.get("change") or a nested key access, and terraform version shows a recent bump. Remediation: pin and assert format_version at the top of load_resource_changes, and consult the Terraform JSON output format changelog before accepting the upgrade in CI.

# Frequently Asked Questions

# Related

• Resource Tagging & Validation Pipelines — the parent reference defining the acquisition→normalization→allocation→persistence model this Terraform gate implements at the pre-provisioning boundary.
• Tagging Policy Enforcement with AWS Config — the post-provisioning, continuous-evaluation counterpart that catches drift and out-of-band changes this plan-time gate cannot see.
• Enforcing Tag Policies in Terraform Plan Checks — wiring this checker and Conftest into a CI pipeline as a required status check on every pull request.
• Setting Up FinOps Governance Boundaries in Terraform — the broader Terraform governance boundaries (IAM, budgets, service-control policies) this tag gate operates alongside.
• Cross-Cloud Cost Allocation Strategies — how the compliant tags this gate enforces ultimately drive allocation and chargeback.

Up: Resource Tagging & Validation Pipelines · Home: Cloud Cost Optimization & FinOps Automation