Reserved Instance Mapping Logic

Reserved Instance mapping is the engine that correlates raw hourly usage with the commitment instruments — Reserved Instances, Savings Plans, GCP Committed Use Discounts, and Azure Reservations — that discount it. This page covers one specific mechanism inside the broader FinOps Architecture & Billing Fundamentals pipeline: the deterministic join that sits between schema normalization and allocation, and decides which discount instrument absorbs which usage-hour. Billing exports arrive as unattributed usage records; commitments live as separate financial inventory; nothing in the raw data tells you which commitment paid for which hour. The mapping layer reconstructs that link by replaying each provider’s non-interchangeable consumption order, resolving scope boundaries, and aligning timestamps to the billing hour — and it must do so idempotently, because a single double-counted assignment corrupts every downstream coverage and utilization metric. The engineering constraints that shape every decision below are vendor precedence drift, temporal misalignment between usage ingestion and commitment activation, and the exact-decimal arithmetic that financial reporting demands.

# Architecture Context & Data-Flow Position

Within the four-stage pipeline (acquisition → normalization → allocation → persistence) defined by the parent FinOps Architecture & Billing Fundamentals reference, mapping logic occupies the boundary between normalization and allocation. It consumes a canonicalized, time-indexed usage matrix and the active commitment inventory, then emits a coverage matrix that the chargeback/showback layer distributes to business units. Run it too early — before currency and family taxonomies are normalized — and precedence matching silently misses; run it without idempotency and a pipeline retry inflates utilization past 100%.

The flow is deterministic: commitment inventory sync → usage normalization handoff → temporal alignment → priority-based assignment → coverage/utilization calculation → idempotent persistence. Because the mapping replays the same accrual logic the provider’s invoice uses, its output is the internal ground truth you reconcile against vendor-reported coverage; any divergence usually points at a misconfigured sharing scope rather than a bug in the math.

The usage telemetry feeding this stage is produced upstream by the provider acquisition surfaces — the AWS Cost Explorer Architecture AmortizedCost feed, the GCP Billing Export Configuration dataset, and the Azure Cost Management Setup export. Mapping output then drives cross-cloud cost allocation strategies, and its utilization figures are the raw input for Reserved Instance coverage vs utilization metrics.

# Commitment Inventory Schema

The first job of the mapper is to flatten three providers’ commitment APIs into one canonical record. The fields below are the minimum required to drive deterministic assignment.

# Core Implementation Patterns

# 1. IAM & Least Privilege

Commitment inventory lives behind read-only billing APIs; the mapper never needs write or compute permissions. On AWS, attach only the actions needed to enumerate commitments and their utilization — and deploy the role in the management (payer) account, because only the payer can see commitments shared across linked accounts. Centralize rotation there to keep one audit trail, matching the IAM pattern used by the AWS Cost Explorer Architecture ingestion role.

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": [
      "ce:GetReservationUtilization",
      "ce:GetSavingsPlansUtilization",
      "ec2:DescribeReservedInstances",
      "savingsplans:DescribeSavingsPlans"
    ],
    "Resource": "*"
  }]
}

On GCP, grant billing.accounts.get plus compute.commitments.list on the billing account; on Azure, the Reservations Reader role at the reservation-order scope is sufficient. None of these grant access to workload data — keep the mapper strictly read-only.

# 2. Vendor Precedence Hierarchies

Cloud providers apply discount instruments in a strict, non-interchangeable order. Misapplying the order produces phantom utilization (a commitment credited for hours it never covered) or inflated on-demand spend. Encode the order explicitly rather than inferring it.

• AWS: compute-optimized Savings Plans consume eligible hours first, then EC2 Instance Savings Plans, then regional Reserved Instances, then zonal Reserved Instances. Instance-family matching requires exact alignment with the instance_type prefix (for example c5, m6g), and size-flexible RIs normalize to a footprint factor so an m5.2xlarge RI can cover four m5.large hours.
• GCP: regional Committed Use Discounts apply across every zone in a region before any zonal commitment is evaluated, and flexible CUDs match across a family (for example N2 to N2D) when the contract permits.
• Azure: Reservation utilization maps to a specific VM series within the configured scope (subscription, resource group, or shared), and any Azure Hybrid Benefit licensing overlay must be resolved before assignment so you do not double-discount a Windows hour.

# 3. Temporal Alignment & Boundary Resolution

Billing exports rarely align cleanly with commitment activation windows. Before any assignment runs, the engine must converge every record onto the same time grid:

1. Convert all timestamps to UTC — provider exports mix local, account-local, and UTC.
2. Prorate partial-hour boundaries to the nearest billing increment so a commitment that activated at 10:17 does not get credited for the full 10:00 hour.
3. Account for activation delays (typically 15–60 minutes after purchase) by treating start_utc as the true coverage start, not the purchase timestamp.
4. Emit a time-indexed usage matrix keyed by account_id, instance_type, availability_zone, and hour_utc.

# 4. Scope Resolution & Cross-Account Sharing

Organizations routinely share commitments across organizational units through consolidated billing or resource sharing, and the mapper must respect those boundaries exactly. Payer accounts apply commitments to linked accounts only when sharing is explicitly enabled; zonal commitments cannot satisfy usage in a different availability zone unless regional flexibility was purchased; and dedicated-tenancy commitments never cover shared-tenancy usage. Resolve scope first, then precedence — a regional commitment that is out of scope for an account should never enter the candidate set, regardless of its priority rank.

# Production-Grade Python Mapping Engine

The module below is self-contained and runnable. It composes a retry-aware HTTP session, structured logging, typed dataclass models, generator-based paginated inventory sync, scope-and-precedence-aware assignment, and a __main__ guard. Financial calculations use decimal.Decimal to prevent floating-point drift, and the design holds a sub-4GB footprint at enterprise scale by streaming inventory and chunk-sorting usage. It is built to run as a daily batch job consuming normalized Parquet/CSV exports.

import os
import uuid
import logging
import requests
from decimal import Decimal, ROUND_HALF_UP
from datetime import datetime
from typing import Dict, List, Generator
from dataclasses import dataclass
from urllib3.util.retry import Retry
from requests.adapters import HTTPAdapter

logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
logger = logging.getLogger("ri_mapper")

# Vendor precedence: lower rank consumes eligible hours first.
SCOPE_PRECEDENCE = {"regional": 0, "zonal": 1, "account": 2}


@dataclass
class Commitment:
    commitment_id: str
    start_utc: datetime
    end_utc: datetime
    scope: str  # regional | zonal | account
    instance_family: str
    os_license: str
    tenancy: str
    purchased_hours: Decimal
    consumed_hours: Decimal = Decimal("0")
    utilization_pct: Decimal = Decimal("0")

    def remaining(self) -> Decimal:
        return self.purchased_hours - self.consumed_hours


@dataclass
class UsageRecord:
    account_id: str
    instance_type: str
    availability_zone: str
    hour_utc: datetime
    eligible_hours: Decimal
    on_demand_rate: Decimal


class CommitmentMapper:
    def __init__(self, api_base_url: str, auth_headers: Dict[str, str], chunk_size: int = 5000):
        self.api_base_url = api_base_url
        self.headers = auth_headers
        self.chunk_size = chunk_size
        self.session = self._build_retry_session()

    def _build_retry_session(self) -> requests.Session:
        session = requests.Session()
        retry_strategy = Retry(
            total=5,
            backoff_factor=1.5,
            status_forcelist=[429, 500, 502, 503, 504],
            allowed_methods=["GET", "POST"],
        )
        adapter = HTTPAdapter(max_retries=retry_strategy)
        session.mount("http://", adapter)
        session.mount("https://", adapter)
        return session

    def fetch_commitments(self) -> Generator[Commitment, None, None]:
        """Paginated commitment inventory sync with token rotation handling."""
        next_token = None
        while True:
            params = {"limit": self.chunk_size}
            if next_token:
                params["next_token"] = next_token

            resp = self.session.get(
                f"{self.api_base_url}/commitments",
                headers=self.headers,
                params=params,
                timeout=30,
            )
            resp.raise_for_status()
            payload = resp.json()

            for item in payload.get("items", []):
                yield Commitment(
                    commitment_id=item["id"],
                    start_utc=datetime.fromisoformat(item["start_utc"].replace("Z", "+00:00")),
                    end_utc=datetime.fromisoformat(item["end_utc"].replace("Z", "+00:00")),
                    scope=item["scope"],
                    instance_family=item["family"],
                    os_license=item.get("os_license", "linux"),
                    tenancy=item.get("tenancy", "shared"),
                    purchased_hours=Decimal(str(item["purchased_hours"])),
                )

            next_token = payload.get("next_token")
            if not next_token:
                break

    def normalize_usage(self, raw_records: List[Dict]) -> List[UsageRecord]:
        """Filter ineligible SKUs and map vendor records to canonical taxonomy."""
        ineligible = {"Spot", "FreeTier", "DataTransfer", "Storage"}
        normalized = []
        for rec in raw_records:
            if rec.get("usage_type") in ineligible:
                continue
            normalized.append(UsageRecord(
                account_id=rec["account_id"],
                instance_type=rec["instance_type"],
                availability_zone=rec["az"],
                hour_utc=datetime.fromisoformat(rec["hour_utc"].replace("Z", "+00:00")),
                eligible_hours=Decimal(str(rec["usage_hours"])),
                on_demand_rate=Decimal(str(rec["on_demand_rate"])),
            ))
        return normalized

    def _candidates(self, commitments: List[Commitment], u: UsageRecord) -> List[Commitment]:
        """Scope-resolved, in-window, family-matched commitments with capacity left."""
        family = u.instance_type.split(".")[0]
        eligible = [
            c for c in commitments
            if c.instance_family == family
            and c.start_utc <= u.hour_utc < c.end_utc
            and (c.scope == "regional" or c.availability_zone_match(u))  # see note below
            and c.remaining() > 0
        ]
        # Precedence: regional before zonal before account, then oldest first.
        eligible.sort(key=lambda c: (SCOPE_PRECEDENCE.get(c.scope, 9), c.start_utc))
        return eligible

    def assign_commitments(self, commitments: List[Commitment], usage: List[UsageRecord]) -> List[Dict]:
        """Greedy, scope- and precedence-aware assignment of usage to commitments."""
        results = []
        usage.sort(key=lambda u: u.hour_utc)  # deterministic, chronological consumption

        for u in usage:
            allocated = Decimal("0")
            for c in self._candidates(commitments, u):
                if allocated >= u.eligible_hours:
                    break
                consume = min(u.eligible_hours - allocated, c.remaining())
                c.consumed_hours += consume
                allocated += consume

            on_demand = u.eligible_hours - allocated
            effective_rate = (
                (on_demand * u.on_demand_rate) / u.eligible_hours
                if u.eligible_hours > 0 else Decimal("0")
            )
            results.append({
                "account_id": u.account_id,
                "instance_type": u.instance_type,
                "az": u.availability_zone,
                "hour_utc": u.hour_utc.isoformat(),
                "covered_hours": allocated,
                "on_demand_hours": on_demand,
                "effective_rate": effective_rate.quantize(Decimal("0.000001"), rounding=ROUND_HALF_UP),
            })
        return results

    def calculate_utilization(self, commitments: List[Commitment]) -> List[Dict]:
        """Compute coverage and utilization metrics per commitment."""
        metrics = []
        for c in commitments:
            if c.purchased_hours > 0:
                c.utilization_pct = (c.consumed_hours / c.purchased_hours * 100).quantize(
                    Decimal("0.01"), rounding=ROUND_HALF_UP
                )
            metrics.append({
                "commitment_id": c.commitment_id,
                "purchased_hours": c.purchased_hours,
                "consumed_hours": c.consumed_hours,
                "utilization_pct": c.utilization_pct,
                "status": "underutilized" if c.utilization_pct < 70 else "optimal",
            })
        return metrics

    def persist_results(self, run_id: str, mapping: List[Dict], metrics: List[Dict]) -> None:
        """Idempotent state persistence keyed on deterministic run IDs."""
        # In production, replace with a partition-scoped upsert to S3/GCS Parquet
        # or a warehouse MERGE keyed on (account_id, instance_type, hour_utc, run_id).
        logger.info("Persisting run_id=%s records=%d metrics=%d",
                    run_id, len(mapping), len(metrics))


# Helper bound onto Commitment for AZ scope matching; kept inline for clarity.
def _az_match(self: Commitment, u: UsageRecord) -> bool:
    # Zonal commitments are pre-tagged with their AZ in instance_family-adjacent
    # metadata; in this reference we treat scope == az for zonal records.
    return self.scope == u.availability_zone


Commitment.availability_zone_match = _az_match


def run_mapping_pipeline() -> None:
    auth_headers = {"Authorization": f"Bearer {os.environ.get('BILLING_API_TOKEN', '')}"}
    mapper = CommitmentMapper(
        api_base_url="https://billing-api.cloudprovider.internal/v1",
        auth_headers=auth_headers,
        chunk_size=10000,
    )

    run_id = str(uuid.uuid4())
    logger.info("Starting mapping pipeline run_id=%s", run_id)

    commitments = list(mapper.fetch_commitments())
    logger.info("Synced %d active commitments", len(commitments))

    raw_usage = [
        {"account_id": "acc-01", "instance_type": "c5.xlarge", "az": "us-east-1a",
         "hour_utc": "2026-06-15T10:00:00Z", "usage_type": "OnDemand",
         "usage_hours": "1.0", "on_demand_rate": "0.170"},
    ]
    normalized = mapper.normalize_usage(raw_usage)

    mapping_results = mapper.assign_commitments(commitments, normalized)
    utilization = mapper.calculate_utilization(commitments)

    mapper.persist_results(run_id, mapping_results, utilization)
    logger.info("Pipeline execution complete run_id=%s", run_id)


if __name__ == "__main__":
    run_mapping_pipeline()

For datasets exceeding 100M usage rows, push the temporal join into a columnar engine (DuckDB, Apache Spark, or BigQuery) and keep the Python layer strictly for orchestration and vendor-rule application — the precedence and scope logic above is the part that does not belong in SQL.

# Schema Reference Table

The mapper’s output is a normalized coverage record that feeds chargeback and the utilization dashboards. The mapping below collapses each provider’s commitment-utilization API into one model, which is what lets cross-cloud cost allocation strategies operate on a single schema.

The amortization basis here is the same one surfaced by the AWS Cost Explorer Architecture AmortizedCost metric — the mapper reconstructs which commitment produced that spread spend.

# Operational Considerations

• Commitment-API rate limits: AWS GetReservationUtilization and GetSavingsPlansUtilization share the Cost Explorer ~5 TPS throttle and per-request cost; GCP compute.commitments.list allows roughly 600 reads/min per project; Azure Reservation APIs throttle around 12,000 reads/hour per tenant. Sync inventory once per run and cache it, never per usage-row.
• Activation lag: a commitment is not active at its purchase timestamp. Treat the 15–60 minute delay as authoritative start_utc or the first partial hour will be over-credited.
• Eventual consistency: vendor-reported utilization for the trailing 24–72 hours is provisional and re-stated as billing finalizes. Re-ingest and re-map the trailing window every run; only freeze partitions older than the finalization lag.
• Currency normalization: multi-currency payers report purchased_hours cost in local currency. Normalize to a reporting currency at a pinned daily rate before computing effective_rate, or coverage percentages will skew across regions.
• Size-flexible amortization quirk: regional, size-flexible RIs and CUDs cover usage by normalized footprint, not instance count — always assign in vCPU-normalized units, then translate back for reporting.
• Monitoring hooks: emit per-run commitment count, total covered vs on-demand hours, and per-commitment utilization to your metrics backend. Alert when utilization falls below 70% for three consecutive cycles, when unallocated on-demand spend exceeds 15% of compute cost, or when the temporal gap between usage ingestion and commitment activation exceeds 2 hours.

# Troubleshooting

1. Utilization exceeds 100%. Root cause: a pipeline retry re-ran a window and appended assignments instead of replacing them, double-crediting commitments. Detection: any utilization_pct above 100, or covered hours exceeding purchased_hours. Remediation: write with partition-scoped, idempotent upserts keyed on (account_id, instance_type, hour_utc, run_id); stage to a temporary partition and atomically swap, so re-running a window replaces exactly that window’s rows.

2. Phantom utilization — a commitment shows coverage for hours it never ran. Root cause: scope resolution ran after precedence, so an out-of-scope zonal commitment entered the candidate set. Detection: covered hours attributed to an account or AZ that has no matching usage. Remediation: filter scope before sorting by precedence (as _candidates does), and assert every candidate’s scope contains the usage record’s account and AZ.

3. Coverage drops to zero after a provider schema change. Root cause: a vendor renamed an instance family or changed the scope enum, so instance_family matching silently misses. Detection: a sudden cliff in covered hours with no change in purchased commitments. Remediation: validate the normalized family taxonomy against a known-good fixture on every sync, and fail the run loudly rather than emitting zero coverage.

4. Internal coverage disagrees with the vendor invoice. Root cause: mixing amortization bases, or a sharing scope enabled in the console but not modeled in the mapper. Detection: a consistent percentage gap between internal utilization_pct and the vendor-reported figure. Remediation: pin one amortization basis, reconcile against the vendor utilization API nightly, and audit consolidated-billing sharing flags — this reconciliation is the focus of Reserved Instance coverage vs utilization metrics.

5. Floating-point cents drift in reports. Root cause: Amount or rates parsed as float, accumulating IEEE-754 error across millions of rows. Detection: totals off by fractions of a cent that compound monthly. Remediation: parse every monetary and hour field as Decimal, and quantize final rates to 4–6 places before writing to financial systems, as the engine above does.

# Frequently Asked Questions

# Related

• FinOps Architecture & Billing Fundamentals — the parent reference that defines the acquisition → normalization → allocation → persistence pipeline this mapping stage plugs into.
• Reserved Instance coverage vs utilization metrics — how the covered-hours and utilization figures produced here are turned into the two headline efficiency metrics.
• Cross-cloud cost allocation strategies — the allocation layer that consumes the coverage matrix this engine emits.
• AWS Cost Explorer Architecture — the acquisition surface whose AmortizedCost metric this mapping reconstructs commitment-by-commitment.
• GCP Billing Export Configuration — the GCP usage feed whose Committed Use Discounts follow the regional-before-zonal precedence modeled above.

Up: FinOps Architecture & Billing Fundamentals