GitHub Actions Cost-Diff Reporting
The specific bottleneck this page solves is reviewer blindness: a reviewer looking at a Terraform pull request sees resource diffs, not dollar diffs, and has no way to tell whether a change adds three t3.micro instances or one m5.4xlarge cluster without pricing it themselves. CI/CD Cost Guardrails already prices a plan and computes an aggregate monthly delta against a stored baseline — but an aggregate number (“+412 USD/mo”) tells a reviewer that something got more expensive, not which resource did it or by how much. This page builds the reporting layer that closes that gap: a script that prices two plan snapshots — one against the PR’s base commit, one against its head — diffs them resource by resource, and posts the result as a single pull-request comment that updates in place on every push instead of accumulating a new comment per commit. The hard constraint is idempotency: a workflow that fires on synchronize for every push must never post a second comment, because a ten-commit PR would otherwise bury the actual code review under nine stale cost tables.
Root Cause & Failure Modes
Three naive implementations of “post a cost comment” each fail in a way that only shows up after the workflow has been running for a few weeks:
- Always
POSTa new comment. Works on the first push. On the second push the PR now has two cost comments, both slightly wrong relative to the latest commit; by push five, reviewers scroll past a wall of stale tables to find the current one. GitHub does not deduplicate comments for you — theissues/{number}/commentsendpoint is append-only from the API’s point of view. - Diff only the totals, not the resources. An aggregate
$-50/monet delta can hide a+$300/monew NAT gateway offset by a-$350/modownsized database — a reviewer approving on the net number alone never sees the NAT gateway they should have questioned. The fix is a per-resource table keyed on the Terraformaddress, not a single scalar. - Match the sticky comment by author instead of content. Filtering
commentsbyuser.login == "github-actions[bot]"breaks the moment any other Action (a linter, a Dependabot summary) also posts as that same identity — the script edits the wrong comment. A hidden HTML marker embedded in the comment body (<!-- cost-diff-report:v1 -->) is the only reliable key, because GitHub strips HTML comments from the rendered markdown but preserves them verbatim incomment.body, so a substring match on the raw body is exact and immune to other bots sharing the identity.
The other constraint worth quantifying: the GET /repos/{owner}/{repo}/issues/{number}/comments endpoint returns 30 comments per page by default and up to 100 with per_page=100 — a long-lived PR with review back-and-forth can exceed that in a single page, so a lookup that only checks page one will occasionally miss the sticky comment and create a duplicate. And because the base and head snapshots come from two separate terraform plan runs against two different commits, the workflow’s total runtime roughly doubles versus pricing a single plan — worth knowing before you fire this on every synchronize event on a large monorepo.
Production Pipeline Architecture
The reporting job runs as four phases, each independently testable, and it consumes exactly the same PricedResource shape the parent CI/CD Cost Guardrails engine produces — this page adds the base-vs-head comparison and the PR-facing delivery, not a second pricing engine:
- Snapshot. Run
terraform planagainst the head commit, then again against the PR’s base commit (via a second worktree checkout), and price both plans with the same rate-map engine documented in the parent page. The result is two JSON files,base_priced.jsonandhead_priced.json, each a flat list of{address, resource_type, monthly_usd}. - Diff. Union the addresses present in either snapshot and compute a per-resource delta. A resource present only in
head(a newresource_change) prices at$0on the base side; a resource removed inheadprices at$0on the head side. Only resources whose status is notunchangedmake it into the report — a PR touching 200 resources where 3 changed cost should show 3 rows, not 200. - Render. Format the changed resources into a markdown table sorted by absolute delta, with the hidden sticky marker as the first line of the body. This step is pure and independently unit-testable — feed it fixed deltas, assert on the exact markdown string.
- Upsert. List the PR’s existing comments (paginated), search for the marker, and
PATCHthe existing comment if found orPOSTa new one if not. This is the idempotency boundary: everything upstream can be re-run freely because this step alone decides create-vs-update.
This mirrors the same shift-left, plan-time delivery pattern used for tag compliance in Enforcing Tag Policies in Terraform Plan Checks — both post a structured verdict to the PR before merge rather than scanning after the fact — but that page enforces a pass/fail gate on tags, while this one is purely informational. The pass/fail enforcement for cost specifically — failing the build when the delta breaches a budget threshold — is a separate concern covered in Blocking Terraform Plans That Exceed Cost Budgets; this page’s job is strictly to inform the reviewer, not to block the merge.
Step-by-Step Python Implementation
The workflow runs terraform plan twice — once per commit — prices both plans with the guardrail engine’s rate map, then hands the two snapshot files to the script below.
name: cost-diff-report
on:
pull_request:
types: [opened, synchronize, reopened]
permissions:
pull-requests: write
contents: read
jobs:
cost-diff:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: hashicorp/setup-terraform@v3
- name: Plan and price head commit
run: |
terraform init -input=false
terraform plan -out=head.tfplan -input=false
terraform show -json head.tfplan > head_plan.json
python price_plan.py head_plan.json head_priced.json
- name: Plan and price base commit
run: |
git worktree add /tmp/base $
cd /tmp/base && terraform init -input=false
terraform plan -out=base.tfplan -input=false
terraform show -json base.tfplan > "$GITHUB_WORKSPACE/base_plan.json"
cd "$GITHUB_WORKSPACE"
python price_plan.py base_plan.json base_priced.json
- name: Post cost-diff comment
env:
GITHUB_TOKEN: $
run: |
python cost_diff_comment.py \
--repo "$" \
--pr-number "$" \
--base-snapshot base_priced.json \
--head-snapshot head_priced.json
price_plan.py is the same resource_changes → PricedResource pricing step documented in the parent guardrail page, run once per snapshot with an --out flag added. The permissions: pull-requests: write block is required at the job or workflow level — without it the default GITHUB_TOKEN on a same-repo PR only has read access, and the comment POST/PATCH calls fail with a 403.
#!/usr/bin/env python3
"""
Post or update a sticky GitHub PR comment showing the per-resource monthly
cost delta between a Terraform plan's base and head snapshots. Idempotent:
re-running against the same PR edits the existing comment instead of adding
a new one, keyed on a hidden HTML marker embedded in the comment body.
"""
from __future__ import annotations
import argparse
import json
import logging
import os
import sys
import time
from dataclasses import dataclass
from pathlib import Path
from typing import Optional
import requests
logging.basicConfig(level=logging.INFO, format="%(asctime)s | %(levelname)s | %(message)s")
logger = logging.getLogger("cost_diff_comment")
STICKY_MARKER = "<!-- cost-diff-report:v1 -->"
GITHUB_API = "https://api.github.com"
MAX_ROWS_IN_TABLE = 20
@dataclass
class PricedResource:
address: str
resource_type: str
monthly_usd: float
@dataclass
class ResourceDelta:
address: str
resource_type: str
base_usd: float
head_usd: float
@property
def delta_usd(self) -> float:
return round(self.head_usd - self.base_usd, 2)
@property
def status(self) -> str:
if self.base_usd == 0.0 and self.head_usd != 0.0:
return "added"
if self.head_usd == 0.0 and self.base_usd != 0.0:
return "removed"
return "changed" if self.delta_usd != 0.0 else "unchanged"
def retry(times: int = 4, backoff: float = 2.0):
"""Retry a requests call on transient network errors or 429/5xx responses."""
def decorator(fn):
def wrapped(*args, **kwargs):
delay = 1.0
for attempt in range(1, times + 1):
try:
resp = fn(*args, **kwargs)
if resp.status_code == 429 or resp.status_code >= 500:
raise requests.exceptions.HTTPError(f"transient {resp.status_code}", response=resp)
return resp
except requests.exceptions.RequestException as exc:
if attempt == times:
raise
logger.warning("attempt %d/%d failed: %s; retrying in %.1fs", attempt, times, exc, delay)
time.sleep(delay)
delay *= backoff
return wrapped
return decorator
def load_snapshot(path: Path) -> dict[str, PricedResource]:
with path.open("r", encoding="utf-8") as fh:
rows = json.load(fh)
resources = {r["address"]: PricedResource(r["address"], r["resource_type"], float(r["monthly_usd"])) for r in rows}
logger.info("loaded %d priced resource(s) from %s", len(resources), path)
return resources
def compute_deltas(base: dict[str, PricedResource], head: dict[str, PricedResource]) -> list[ResourceDelta]:
"""Union base/head addresses; a resource missing from one side prices at $0 there."""
deltas = []
for addr in sorted(set(base) | set(head)):
rtype = (head.get(addr) or base.get(addr)).resource_type
d = ResourceDelta(addr, rtype, base[addr].monthly_usd if addr in base else 0.0,
head[addr].monthly_usd if addr in head else 0.0)
if d.status != "unchanged":
deltas.append(d)
deltas.sort(key=lambda d: -abs(d.delta_usd))
return deltas
def render_comment(deltas: list[ResourceDelta], base_total: float, head_total: float) -> str:
net = round(head_total - base_total, 2)
lines = [
STICKY_MARKER,
"### Terraform cost-diff",
"",
f"**Net monthly change: {'+' if net >= 0 else ''}${net:,.2f}** "
f"(base ${base_total:,.2f} → head ${head_total:,.2f})",
"",
"| Resource | Type | Status | Base $/mo | Head $/mo | Delta $/mo |",
"|---|---|---|---|---|---|",
]
for d in deltas[:MAX_ROWS_IN_TABLE]:
sign = "+" if d.delta_usd >= 0 else ""
lines.append(f"| `{d.address}` | {d.resource_type} | {d.status} | "
f"${d.base_usd:,.2f} | ${d.head_usd:,.2f} | {sign}${d.delta_usd:,.2f} |")
if len(deltas) > MAX_ROWS_IN_TABLE:
lines.append(f"\n_{len(deltas) - MAX_ROWS_IN_TABLE} additional changed resource(s) omitted._")
return "\n".join(lines)
@retry()
def _list_comments_page(session: requests.Session, repo: str, pr_number: int, page: int) -> requests.Response:
return session.get(f"{GITHUB_API}/repos/{repo}/issues/{pr_number}/comments",
params={"per_page": 100, "page": page}, timeout=15)
def find_sticky_comment_id(session: requests.Session, repo: str, pr_number: int) -> Optional[int]:
"""Paginate through every comment page looking for the hidden marker."""
page = 1
while True:
resp = _list_comments_page(session, repo, pr_number, page)
resp.raise_for_status()
batch = resp.json()
for comment in batch:
if STICKY_MARKER in comment.get("body", ""):
return comment["id"]
if len(batch) < 100:
return None
page += 1
@retry()
def _post_comment(session: requests.Session, repo: str, pr_number: int, body: str) -> requests.Response:
return session.post(f"{GITHUB_API}/repos/{repo}/issues/{pr_number}/comments", json={"body": body}, timeout=15)
@retry()
def _patch_comment(session: requests.Session, repo: str, comment_id: int, body: str) -> requests.Response:
return session.patch(f"{GITHUB_API}/repos/{repo}/issues/comments/{comment_id}", json={"body": body}, timeout=15)
def upsert_comment(session: requests.Session, repo: str, pr_number: int, body: str) -> None:
"""Create the sticky comment on the first run; edit it in place on every later run."""
existing_id = find_sticky_comment_id(session, repo, pr_number)
if existing_id is None:
resp = _post_comment(session, repo, pr_number, body)
resp.raise_for_status()
logger.info("created cost-diff comment id=%s", resp.json()["id"])
else:
_patch_comment(session, repo, existing_id, body).raise_for_status()
logger.info("updated existing cost-diff comment id=%s", existing_id)
def parse_args(argv: Optional[list[str]] = None) -> argparse.Namespace:
parser = argparse.ArgumentParser(description="Post a sticky Terraform cost-diff PR comment")
parser.add_argument("--repo", required=True, help="owner/repo, e.g. org/infra")
parser.add_argument("--pr-number", required=True, type=int)
parser.add_argument("--base-snapshot", required=True, type=Path)
parser.add_argument("--head-snapshot", required=True, type=Path)
parser.add_argument("--token", default=None, help="defaults to $GITHUB_TOKEN")
return parser.parse_args(argv)
def main() -> int:
args = parse_args()
token = args.token or os.environ.get("GITHUB_TOKEN")
if not token:
logger.error("no GitHub token supplied via --token or $GITHUB_TOKEN")
return 2
base = load_snapshot(args.base_snapshot)
head = load_snapshot(args.head_snapshot)
deltas = compute_deltas(base, head)
base_total = round(sum(r.monthly_usd for r in base.values()), 2)
head_total = round(sum(r.monthly_usd for r in head.values()), 2)
logger.info("%d changed resource(s); net delta $%.2f", len(deltas), head_total - base_total)
session = requests.Session()
session.headers.update({
"Authorization": f"Bearer {token}",
"Accept": "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28",
})
upsert_comment(session, args.repo, args.pr_number, render_comment(deltas, base_total, head_total))
return 0
if __name__ == "__main__":
sys.exit(main())
Verification & Testing
- Idempotency assertion. Run
main()twice against the same--repo/--pr-numberand identical snapshots. The second run must call_patch_comment, never_post_comment— assert this by mockingrequests.Sessionand counting calls to each function, not by inspecting the live PR. - Marker survives GitHub’s markdown rendering. Post a real comment through the script against a scratch PR and confirm via
GETthatSTICKY_MARKERis present incomment.bodyeven though it is invisible in the rendered comment on the PR page — HTML comments are stripped from GitHub’s rendered output but preserved in the raw API response. - Pagination correctness. Seed a scratch PR with more than 100 comments (a cheap loop against the same endpoint in a throwaway repo) and confirm
find_sticky_comment_idstill finds a marker planted on page two — a test that only checks page one will pass locally and fail in the wild on any long-lived PR. - Dollar-conservation check. Assert
head_total - base_totalcomputed from the two snapshots equalssum(d.delta_usd for d in deltas)— the per-resource deltas must sum to exactly the net figure shown in the comment header, or the table and the headline number will visibly disagree to a reviewer.
Common Pitfalls Checklist
- Missing
permissions: pull-requests: write. The defaultGITHUB_TOKENis read-only on comments without it — fix by declaring the permission at the workflow or job level, not relying on repository-wide defaults. - Matching the sticky comment by bot username instead of the marker. Any other Action posting under the same identity hijacks the match — always search
comment.bodyforSTICKY_MARKER. - Only checking the first page of comments. A PR with heavy review traffic can exceed 100 comments — always loop until a short page confirms the end, as
find_sticky_comment_iddoes. - Reporting
head_totalalone without the per-resource table. A single scalar hides offsetting increases and decreases — always render the changed-resource rows, not just the net delta. - Running the base-commit plan against a dirty working tree.
git worktree addin the workflow above keeps the base plan isolated from uncommitted head-branch state; planning againstgit checkoutin place risks picking up local changes that were never actually committed to the base ref.
Frequently Asked Questions
Why price the base commit at all instead of diffing against the stored baseline the guardrail engine already tracks?
The stored baseline in CI/CD Cost Guardrails reflects the workspace’s last successful apply, which can be stale if several PRs are open concurrently against the same workspace. Pricing the actual base commit gives an exact, PR-specific comparison — what this PR changes relative to the code it’s merging into — independent of how many other PRs are also in flight.
What happens if the comment body exceeds GitHub’s size limit?
GitHub caps comment bodies at 65,536 characters. render_comment caps the table at MAX_ROWS_IN_TABLE (20) and appends a one-line summary of how many additional changed resources were omitted, which keeps every realistic PR well under the limit while still surfacing the highest-impact changes first, since the table is sorted by absolute delta.
Does this also work for GitLab or Bitbucket pipelines?
The pricing, diffing, and rendering logic is platform-agnostic — only find_sticky_comment_id and upsert_comment are GitHub-specific. Porting to GitLab means swapping the REST calls for the Merge Request Notes API (GET/PUT on /projects/:id/merge_requests/:iid/notes), keeping the same marker-based lookup pattern.
Can this run on pull requests from forks?
Only with care. The default GITHUB_TOKEN on a pull_request event from a fork is read-only regardless of the permissions block, by GitHub’s design, to prevent a fork PR from writing to the base repository. Use pull_request_target with an explicit checkout of the PR’s head SHA, or a GitHub App installation token, exactly as the parent guardrail page recommends for its own check-run posting.
Related
- CI/CD Cost Guardrails — the parent engine whose plan-pricing and rate-map logic this reporting script reuses for both snapshots.
- Blocking Terraform Plans That Exceed Cost Budgets — the sibling page that turns the same priced delta into a merge-blocking policy verdict instead of an informational comment.
- Enforcing Tag Policies in Terraform Plan Checks — the same plan-time, PR-facing delivery pattern applied to tag compliance instead of cost.
- Cloud Cost Anomaly Detection & Budget Automation — the broader reference this preventive, plan-time reporting stage belongs to, alongside the reactive anomaly-detection and budget-alert stages.