GitHub Actions Cost-Diff Reporting

The specific bottleneck this page solves is reviewer blindness: a reviewer looking at a Terraform pull request sees resource diffs, not dollar diffs, and has no way to tell whether a change adds three t3.micro instances or one m5.4xlarge cluster without pricing it themselves. CI/CD Cost Guardrails already prices a plan and computes an aggregate monthly delta against a stored baseline — but an aggregate number (“+412 USD/mo”) tells a reviewer that something got more expensive, not which resource did it or by how much. This page builds the reporting layer that closes that gap: a script that prices two plan snapshots — one against the PR’s base commit, one against its head — diffs them resource by resource, and posts the result as a single pull-request comment that updates in place on every push instead of accumulating a new comment per commit. The hard constraint is idempotency: a workflow that fires on synchronize for every push must never post a second comment, because a ten-commit PR would otherwise bury the actual code review under nine stale cost tables.

Root Cause & Failure Modes

Three naive implementations of “post a cost comment” each fail in a way that only shows up after the workflow has been running for a few weeks:

  • Always POST a new comment. Works on the first push. On the second push the PR now has two cost comments, both slightly wrong relative to the latest commit; by push five, reviewers scroll past a wall of stale tables to find the current one. GitHub does not deduplicate comments for you — the issues/{number}/comments endpoint is append-only from the API’s point of view.
  • Diff only the totals, not the resources. An aggregate $-50/mo net delta can hide a +$300/mo new NAT gateway offset by a -$350/mo downsized database — a reviewer approving on the net number alone never sees the NAT gateway they should have questioned. The fix is a per-resource table keyed on the Terraform address, not a single scalar.
  • Match the sticky comment by author instead of content. Filtering comments by user.login == "github-actions[bot]" breaks the moment any other Action (a linter, a Dependabot summary) also posts as that same identity — the script edits the wrong comment. A hidden HTML marker embedded in the comment body (<!-- cost-diff-report:v1 -->) is the only reliable key, because GitHub strips HTML comments from the rendered markdown but preserves them verbatim in comment.body, so a substring match on the raw body is exact and immune to other bots sharing the identity.

The other constraint worth quantifying: the GET /repos/{owner}/{repo}/issues/{number}/comments endpoint returns 30 comments per page by default and up to 100 with per_page=100 — a long-lived PR with review back-and-forth can exceed that in a single page, so a lookup that only checks page one will occasionally miss the sticky comment and create a duplicate. And because the base and head snapshots come from two separate terraform plan runs against two different commits, the workflow’s total runtime roughly doubles versus pricing a single plan — worth knowing before you fire this on every synchronize event on a large monorepo.

Production Pipeline Architecture

The reporting job runs as four phases, each independently testable, and it consumes exactly the same PricedResource shape the parent CI/CD Cost Guardrails engine produces — this page adds the base-vs-head comparison and the PR-facing delivery, not a second pricing engine:

  1. Snapshot. Run terraform plan against the head commit, then again against the PR’s base commit (via a second worktree checkout), and price both plans with the same rate-map engine documented in the parent page. The result is two JSON files, base_priced.json and head_priced.json, each a flat list of {address, resource_type, monthly_usd}.
  2. Diff. Union the addresses present in either snapshot and compute a per-resource delta. A resource present only in head (a new resource_change) prices at $0 on the base side; a resource removed in head prices at $0 on the head side. Only resources whose status is not unchanged make it into the report — a PR touching 200 resources where 3 changed cost should show 3 rows, not 200.
  3. Render. Format the changed resources into a markdown table sorted by absolute delta, with the hidden sticky marker as the first line of the body. This step is pure and independently unit-testable — feed it fixed deltas, assert on the exact markdown string.
  4. Upsert. List the PR’s existing comments (paginated), search for the marker, and PATCH the existing comment if found or POST a new one if not. This is the idempotency boundary: everything upstream can be re-run freely because this step alone decides create-vs-update.

This mirrors the same shift-left, plan-time delivery pattern used for tag compliance in Enforcing Tag Policies in Terraform Plan Checks — both post a structured verdict to the PR before merge rather than scanning after the fact — but that page enforces a pass/fail gate on tags, while this one is purely informational. The pass/fail enforcement for cost specifically — failing the build when the delta breaches a budget threshold — is a separate concern covered in Blocking Terraform Plans That Exceed Cost Budgets; this page’s job is strictly to inform the reviewer, not to block the merge.

Cost-diff reporting pipeline: snapshot, diff, render, and idempotent upsert A four-stage pipeline. Stage one, snapshot, prices the head plan and the base plan into two JSON files using the guardrail engine's rate map. Stage two, diff, unions resource addresses from both snapshots and computes a per-resource delta, keeping only changed resources. Stage three, render, formats the changed resources into a markdown table with a hidden sticky marker as the first line. Stage four, upsert, lists existing pull request comments, searches for the marker, and branches: if a matching comment is found it is updated with a PATCH request, and if not found a new comment is created with a POST request. Both branches converge on the same visible comment on the pull request, so the pull request always shows exactly one cost-diff comment regardless of how many times the workflow re-runs. 1 Snapshot 2 Diff 3 Render price head plan + price base plan → two JSON snapshots union addresses, compute per-resource delta, drop unchanged markdown table + hidden sticky marker as first line of body 4 Find sticky comment GET comments, search marker found → PATCH update comment in place not found → POST create the first comment
Snapshot and diff produce the per-resource delta; render formats it once; the upsert branch is what makes re-running the workflow on every push safe.

Step-by-Step Python Implementation

The workflow runs terraform plan twice — once per commit — prices both plans with the guardrail engine’s rate map, then hands the two snapshot files to the script below.

name: cost-diff-report

on:
  pull_request:
    types: [opened, synchronize, reopened]

permissions:
  pull-requests: write
  contents: read

jobs:
  cost-diff:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - uses: hashicorp/setup-terraform@v3

      - name: Plan and price head commit
        run: |
          terraform init -input=false
          terraform plan -out=head.tfplan -input=false
          terraform show -json head.tfplan > head_plan.json
          python price_plan.py head_plan.json head_priced.json

      - name: Plan and price base commit
        run: |
          git worktree add /tmp/base $
          cd /tmp/base && terraform init -input=false
          terraform plan -out=base.tfplan -input=false
          terraform show -json base.tfplan > "$GITHUB_WORKSPACE/base_plan.json"
          cd "$GITHUB_WORKSPACE"
          python price_plan.py base_plan.json base_priced.json

      - name: Post cost-diff comment
        env:
          GITHUB_TOKEN: $
        run: |
          python cost_diff_comment.py \
            --repo "$" \
            --pr-number "$" \
            --base-snapshot base_priced.json \
            --head-snapshot head_priced.json

price_plan.py is the same resource_changesPricedResource pricing step documented in the parent guardrail page, run once per snapshot with an --out flag added. The permissions: pull-requests: write block is required at the job or workflow level — without it the default GITHUB_TOKEN on a same-repo PR only has read access, and the comment POST/PATCH calls fail with a 403.

#!/usr/bin/env python3
"""
Post or update a sticky GitHub PR comment showing the per-resource monthly
cost delta between a Terraform plan's base and head snapshots. Idempotent:
re-running against the same PR edits the existing comment instead of adding
a new one, keyed on a hidden HTML marker embedded in the comment body.
"""

from __future__ import annotations

import argparse
import json
import logging
import os
import sys
import time
from dataclasses import dataclass
from pathlib import Path
from typing import Optional

import requests

logging.basicConfig(level=logging.INFO, format="%(asctime)s | %(levelname)s | %(message)s")
logger = logging.getLogger("cost_diff_comment")

STICKY_MARKER = "<!-- cost-diff-report:v1 -->"
GITHUB_API = "https://api.github.com"
MAX_ROWS_IN_TABLE = 20


@dataclass
class PricedResource:
    address: str
    resource_type: str
    monthly_usd: float


@dataclass
class ResourceDelta:
    address: str
    resource_type: str
    base_usd: float
    head_usd: float

    @property
    def delta_usd(self) -> float:
        return round(self.head_usd - self.base_usd, 2)

    @property
    def status(self) -> str:
        if self.base_usd == 0.0 and self.head_usd != 0.0:
            return "added"
        if self.head_usd == 0.0 and self.base_usd != 0.0:
            return "removed"
        return "changed" if self.delta_usd != 0.0 else "unchanged"


def retry(times: int = 4, backoff: float = 2.0):
    """Retry a requests call on transient network errors or 429/5xx responses."""
    def decorator(fn):
        def wrapped(*args, **kwargs):
            delay = 1.0
            for attempt in range(1, times + 1):
                try:
                    resp = fn(*args, **kwargs)
                    if resp.status_code == 429 or resp.status_code >= 500:
                        raise requests.exceptions.HTTPError(f"transient {resp.status_code}", response=resp)
                    return resp
                except requests.exceptions.RequestException as exc:
                    if attempt == times:
                        raise
                    logger.warning("attempt %d/%d failed: %s; retrying in %.1fs", attempt, times, exc, delay)
                    time.sleep(delay)
                    delay *= backoff
        return wrapped
    return decorator


def load_snapshot(path: Path) -> dict[str, PricedResource]:
    with path.open("r", encoding="utf-8") as fh:
        rows = json.load(fh)
    resources = {r["address"]: PricedResource(r["address"], r["resource_type"], float(r["monthly_usd"])) for r in rows}
    logger.info("loaded %d priced resource(s) from %s", len(resources), path)
    return resources


def compute_deltas(base: dict[str, PricedResource], head: dict[str, PricedResource]) -> list[ResourceDelta]:
    """Union base/head addresses; a resource missing from one side prices at $0 there."""
    deltas = []
    for addr in sorted(set(base) | set(head)):
        rtype = (head.get(addr) or base.get(addr)).resource_type
        d = ResourceDelta(addr, rtype, base[addr].monthly_usd if addr in base else 0.0,
                           head[addr].monthly_usd if addr in head else 0.0)
        if d.status != "unchanged":
            deltas.append(d)
    deltas.sort(key=lambda d: -abs(d.delta_usd))
    return deltas


def render_comment(deltas: list[ResourceDelta], base_total: float, head_total: float) -> str:
    net = round(head_total - base_total, 2)
    lines = [
        STICKY_MARKER,
        "### Terraform cost-diff",
        "",
        f"**Net monthly change: {'+' if net >= 0 else ''}${net:,.2f}** "
        f"(base ${base_total:,.2f} → head ${head_total:,.2f})",
        "",
        "| Resource | Type | Status | Base $/mo | Head $/mo | Delta $/mo |",
        "|---|---|---|---|---|---|",
    ]
    for d in deltas[:MAX_ROWS_IN_TABLE]:
        sign = "+" if d.delta_usd >= 0 else ""
        lines.append(f"| `{d.address}` | {d.resource_type} | {d.status} | "
                      f"${d.base_usd:,.2f} | ${d.head_usd:,.2f} | {sign}${d.delta_usd:,.2f} |")
    if len(deltas) > MAX_ROWS_IN_TABLE:
        lines.append(f"\n_{len(deltas) - MAX_ROWS_IN_TABLE} additional changed resource(s) omitted._")
    return "\n".join(lines)


@retry()
def _list_comments_page(session: requests.Session, repo: str, pr_number: int, page: int) -> requests.Response:
    return session.get(f"{GITHUB_API}/repos/{repo}/issues/{pr_number}/comments",
                        params={"per_page": 100, "page": page}, timeout=15)


def find_sticky_comment_id(session: requests.Session, repo: str, pr_number: int) -> Optional[int]:
    """Paginate through every comment page looking for the hidden marker."""
    page = 1
    while True:
        resp = _list_comments_page(session, repo, pr_number, page)
        resp.raise_for_status()
        batch = resp.json()
        for comment in batch:
            if STICKY_MARKER in comment.get("body", ""):
                return comment["id"]
        if len(batch) < 100:
            return None
        page += 1


@retry()
def _post_comment(session: requests.Session, repo: str, pr_number: int, body: str) -> requests.Response:
    return session.post(f"{GITHUB_API}/repos/{repo}/issues/{pr_number}/comments", json={"body": body}, timeout=15)


@retry()
def _patch_comment(session: requests.Session, repo: str, comment_id: int, body: str) -> requests.Response:
    return session.patch(f"{GITHUB_API}/repos/{repo}/issues/comments/{comment_id}", json={"body": body}, timeout=15)


def upsert_comment(session: requests.Session, repo: str, pr_number: int, body: str) -> None:
    """Create the sticky comment on the first run; edit it in place on every later run."""
    existing_id = find_sticky_comment_id(session, repo, pr_number)
    if existing_id is None:
        resp = _post_comment(session, repo, pr_number, body)
        resp.raise_for_status()
        logger.info("created cost-diff comment id=%s", resp.json()["id"])
    else:
        _patch_comment(session, repo, existing_id, body).raise_for_status()
        logger.info("updated existing cost-diff comment id=%s", existing_id)


def parse_args(argv: Optional[list[str]] = None) -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Post a sticky Terraform cost-diff PR comment")
    parser.add_argument("--repo", required=True, help="owner/repo, e.g. org/infra")
    parser.add_argument("--pr-number", required=True, type=int)
    parser.add_argument("--base-snapshot", required=True, type=Path)
    parser.add_argument("--head-snapshot", required=True, type=Path)
    parser.add_argument("--token", default=None, help="defaults to $GITHUB_TOKEN")
    return parser.parse_args(argv)


def main() -> int:
    args = parse_args()
    token = args.token or os.environ.get("GITHUB_TOKEN")
    if not token:
        logger.error("no GitHub token supplied via --token or $GITHUB_TOKEN")
        return 2

    base = load_snapshot(args.base_snapshot)
    head = load_snapshot(args.head_snapshot)
    deltas = compute_deltas(base, head)
    base_total = round(sum(r.monthly_usd for r in base.values()), 2)
    head_total = round(sum(r.monthly_usd for r in head.values()), 2)
    logger.info("%d changed resource(s); net delta $%.2f", len(deltas), head_total - base_total)

    session = requests.Session()
    session.headers.update({
        "Authorization": f"Bearer {token}",
        "Accept": "application/vnd.github+json",
        "X-GitHub-Api-Version": "2022-11-28",
    })
    upsert_comment(session, args.repo, args.pr_number, render_comment(deltas, base_total, head_total))
    return 0


if __name__ == "__main__":
    sys.exit(main())

Verification & Testing

  • Idempotency assertion. Run main() twice against the same --repo/--pr-number and identical snapshots. The second run must call _patch_comment, never _post_comment — assert this by mocking requests.Session and counting calls to each function, not by inspecting the live PR.
  • Marker survives GitHub’s markdown rendering. Post a real comment through the script against a scratch PR and confirm via GET that STICKY_MARKER is present in comment.body even though it is invisible in the rendered comment on the PR page — HTML comments are stripped from GitHub’s rendered output but preserved in the raw API response.
  • Pagination correctness. Seed a scratch PR with more than 100 comments (a cheap loop against the same endpoint in a throwaway repo) and confirm find_sticky_comment_id still finds a marker planted on page two — a test that only checks page one will pass locally and fail in the wild on any long-lived PR.
  • Dollar-conservation check. Assert head_total - base_total computed from the two snapshots equals sum(d.delta_usd for d in deltas) — the per-resource deltas must sum to exactly the net figure shown in the comment header, or the table and the headline number will visibly disagree to a reviewer.

Common Pitfalls Checklist

  • Missing permissions: pull-requests: write. The default GITHUB_TOKEN is read-only on comments without it — fix by declaring the permission at the workflow or job level, not relying on repository-wide defaults.
  • Matching the sticky comment by bot username instead of the marker. Any other Action posting under the same identity hijacks the match — always search comment.body for STICKY_MARKER.
  • Only checking the first page of comments. A PR with heavy review traffic can exceed 100 comments — always loop until a short page confirms the end, as find_sticky_comment_id does.
  • Reporting head_total alone without the per-resource table. A single scalar hides offsetting increases and decreases — always render the changed-resource rows, not just the net delta.
  • Running the base-commit plan against a dirty working tree. git worktree add in the workflow above keeps the base plan isolated from uncommitted head-branch state; planning against git checkout in place risks picking up local changes that were never actually committed to the base ref.

Frequently Asked Questions

Why price the base commit at all instead of diffing against the stored baseline the guardrail engine already tracks?

The stored baseline in CI/CD Cost Guardrails reflects the workspace’s last successful apply, which can be stale if several PRs are open concurrently against the same workspace. Pricing the actual base commit gives an exact, PR-specific comparison — what this PR changes relative to the code it’s merging into — independent of how many other PRs are also in flight.

What happens if the comment body exceeds GitHub’s size limit?

GitHub caps comment bodies at 65,536 characters. render_comment caps the table at MAX_ROWS_IN_TABLE (20) and appends a one-line summary of how many additional changed resources were omitted, which keeps every realistic PR well under the limit while still surfacing the highest-impact changes first, since the table is sorted by absolute delta.

Does this also work for GitLab or Bitbucket pipelines?

The pricing, diffing, and rendering logic is platform-agnostic — only find_sticky_comment_id and upsert_comment are GitHub-specific. Porting to GitLab means swapping the REST calls for the Merge Request Notes API (GET/PUT on /projects/:id/merge_requests/:iid/notes), keeping the same marker-based lookup pattern.

Can this run on pull requests from forks?

Only with care. The default GITHUB_TOKEN on a pull_request event from a fork is read-only regardless of the permissions block, by GitHub’s design, to prevent a fork PR from writing to the base repository. Use pull_request_target with an explicit checkout of the PR’s head SHA, or a GitHub App installation token, exactly as the parent guardrail page recommends for its own check-run posting.

Up: CI/CD Cost Guardrails