Seasonal Decomposition for Cloud Cost Baselines

The specific bottleneck this page solves is weekly seasonality masquerading as noise. Most cloud cost series have a strong 7-day cycle — nightly batch jobs, business-hours autoscaling, weekend traffic troughs — and a flat rolling baseline treats that cycle as random variance around a single mean. The result is predictable and expensive: every Monday’s real but recurring uptick reads as a 3-sigma spike, on-call gets paged on a schedule instead of an incident, and the team’s next move is either muting the detector (missing real anomalies) or hand-tuning a per-weekday threshold that breaks the moment the batch schedule shifts. STL decomposition — Seasonal-Trend decomposition using Loess — fixes this at the source by splitting the series into trend, seasonal, and residual components and scoring the residual, which no longer carries the weekly shape. This page is the seasonal-aware detector that sits alongside the z-score and MAD baselines covered in Building Cost Anomaly Detection Models — use it whenever a service’s baseline has a stable, repeating weekly pattern rather than reaching for it as a universal default.

# Root Cause & Failure Modes

A rolling mean and standard deviation assume the series is stationary around a single level. Weekly-seasonal cost series are not: compute spend on a service that runs a Sunday-night batch reconciliation job might swing 40-60% between its weekday floor and its Sunday peak, every single week, indefinitely. Three consequences follow from applying a flat baseline to that shape:

• Systematic false positives on the same weekday. The rolling mean sits between the weekday floor and the weekend peak, so the peak day is always several standard deviations above center — not because anything changed, but because the mean never represents any single day well. Anomaly logs from a naive z-score detector on a seasonal service cluster hard on one or two weekdays, week after week.
• A widened baseline that hides real spikes. Once a team notices the false positives, the common fix is to widen the rolling window or raise the threshold until Mondays stop firing. That same widening makes the detector blind to a genuine anomaly of similar magnitude to the seasonal swing — the exact class of incident (a runaway job, a misconfigured autoscaler) most likely to look “Monday-sized.”
• Holiday and deploy weeks break the seasonal assumption outright. A US holiday week collapses the weekly pattern (no business-hours ramp), and a deploy week can introduce a one-time step change that a seasonal model mistakes for part of the recurring cycle if it isn’t explicitly excluded. Feeding either into an STL fit without guarding for it pollutes the seasonal component for weeks afterward, since statsmodels’ STL estimates the seasonal shape from the whole series, not just the current cycle.

The fix has two parts: decompose the series into trend + seasonal + residual with an explicit period, and run anomaly detection on the residual — which is what’s left after both the slow drift and the repeating weekly shape are subtracted out — using the rolling z-score technique from Detecting Cost Spikes with a Rolling Z-Score, applied to residuals instead of raw cost.

# Production Pipeline Architecture

This detector runs as a four-phase pipeline layered on top of the same normalized daily series the parent engine consumes, and it assumes at least two full seasonal cycles of finalized history before it fits anything.

1. Period selection. For daily cloud cost data the period is 7 — one calendar week — because the dominant driver of the cycle (weekday vs. weekend usage, weekly batch schedules) repeats every seven observations. statsmodels.tsa.seasonal.STL takes this as the period argument directly; there is no need to infer it via autocorrelation for billing data, since the business calendar already fixes it.
2. Calendar exclusion. Before fitting, mask out holiday days and the first few days after a known deploy or usage-pattern change, replacing them with an interpolated value rather than the raw observation. STL has no concept of “this day doesn’t count” — an unmasked holiday gets baked into both the trend and seasonal estimate and distorts every subsequent residual until it ages out of the fit window.
3. STL fit. Decompose the masked series into trend, seasonal, and resid components. The seasonal component captures the repeating weekly shape; the trend captures slow drift (organic growth, a rightsizing project taking effect); the residual is everything else — which is exactly what an anomaly detector should be scoring.
4. Residual scoring. Apply a rolling z-score to the residual component using the same mechanics as the flat-baseline detector, just on a series that no longer has a weekly shape to confuse it. A residual crossing the threshold means the day deviated from both its trend-adjusted level and its normal weekly pattern — a much higher-precision signal than scoring raw cost.

This is a deliberately narrower tool than the multi-method engine in Building Cost Anomaly Detection Models: it costs more to fit (STL is iterative, not a single vectorized rolling operation) and needs more history to warm up, so it earns its place only on services where a plain z-score or MAD baseline is provably fooled by seasonality — verify that first with the weekday-clustering check in Detecting Cost Spikes with a Rolling Z-Score before switching a service over. The deseasonalized baseline this pipeline produces is also useful on its own — as a cleaner input to the daily rollups described in Time-Series Aggregation for Daily Cloud Cost Tracking — since a trend line with the weekly noise stripped out is a better “is spend actually growing” signal than raw daily cost.

# Step-by-Step Python Implementation

The module below fits STL on a daily cost series with holiday masking, scores the residual with a rolling z-score, and returns both a deseasonalized baseline (trend + seasonal) and a list of flagged dates. It is idempotent — re-running over the same masked input reproduces the same components and flags — and logs every decision that affects the fit.

import logging
from dataclasses import dataclass, field
from datetime import date

import numpy as np
import pandas as pd
from statsmodels.tsa.seasonal import STL

logging.basicConfig(level=logging.INFO, format="%(asctime)s | %(levelname)s | %(message)s")
logger = logging.getLogger("finops.anomaly.seasonal")

SEASONAL_PERIOD = 7          # daily data, weekly cycle
MIN_CYCLES_REQUIRED = 4      # STL needs several periods to estimate a stable seasonal shape
RESIDUAL_ZSCORE_THRESHOLD = 3.0
RESIDUAL_WINDOW = 21         # rolling window for scoring the residual, in days


@dataclass
class DecompositionResult:
    service_id: str
    trend: pd.Series
    seasonal: pd.Series
    resid: pd.Series
    baseline: pd.Series          # trend + seasonal: the deseasonalized expected cost
    flagged_dates: list[date] = field(default_factory=list)


def mask_calendar_exceptions(s: pd.Series, holiday_dates: set[pd.Timestamp],
                              deploy_dates: set[pd.Timestamp], deploy_settle_days: int = 3) -> pd.Series:
    """Replace holiday days and the settle window after a deploy with a
    linear interpolation, so they never distort the STL fit.

    Unmasked exceptions get baked into both trend and seasonal estimates
    and keep distorting residuals for as long as they stay in the window.
    """
    masked = s.copy()
    exception_days: set[pd.Timestamp] = set(holiday_dates)
    for d in deploy_dates:
        exception_days |= {d + pd.Timedelta(days=i) for i in range(deploy_settle_days)}

    hit = [d for d in exception_days if d in masked.index]
    if hit:
        masked.loc[hit] = np.nan
        masked = masked.interpolate(method="linear", limit_direction="both")
        logger.info("masked %d calendar-exception day(s) before STL fit", len(hit))
    return masked


def fit_stl_baseline(s: pd.Series, service_id: str) -> DecompositionResult | None:
    """Decompose a daily cost series into trend + weekly seasonal + residual.

    Returns None rather than a degraded fit when the series doesn't yet
    span enough seasonal cycles — a partial fit produces an unstable
    seasonal component that generates its own false positives.
    """
    min_len = SEASONAL_PERIOD * MIN_CYCLES_REQUIRED
    if len(s) < min_len:
        logger.info("%s: %d points < %d required (%d cycles of period %d) — skipping",
                     service_id, len(s), min_len, MIN_CYCLES_REQUIRED, SEASONAL_PERIOD)
        return None

    stl = STL(s, period=SEASONAL_PERIOD, robust=True)  # robust=True downweights outliers during the fit itself
    fit = stl.fit()
    baseline = fit.trend + fit.seasonal
    return DecompositionResult(
        service_id=service_id,
        trend=fit.trend,
        seasonal=fit.seasonal,
        resid=fit.resid,
        baseline=baseline,
    )


def score_residual(result: DecompositionResult) -> DecompositionResult:
    """Rolling z-score the residual component and flag crossings.

    Scoring the residual instead of raw cost is the entire point: the
    residual no longer carries the weekly shape, so a Monday that looks
    exactly like every other Monday scores near zero.
    """
    resid = result.resid
    mean = resid.rolling(RESIDUAL_WINDOW, min_periods=RESIDUAL_WINDOW).mean()
    std = resid.rolling(RESIDUAL_WINDOW, min_periods=RESIDUAL_WINDOW).std(ddof=0)
    z = (resid - mean) / std.replace(0, np.nan)

    flagged = z[z.abs() >= RESIDUAL_ZSCORE_THRESHOLD].index.tolist()
    result.flagged_dates = [d.date() for d in flagged]
    if flagged:
        logger.warning("%s: %d residual anomaly day(s) flagged: %s",
                        result.service_id, len(flagged), result.flagged_dates)
    return result


def _synthetic_demo_series() -> pd.Series:
    """56 days of weekly-seasonal cost with one injected non-seasonal spike."""
    rng = np.random.default_rng(seed=11)
    dates = pd.date_range("2026-05-01", periods=56, freq="D")
    weekday_floor = 400
    weekend_peak = 650  # Sunday batch reconciliation job
    seasonal_shape = np.array([weekday_floor] * 6 + [weekend_peak])
    seasonal = np.tile(seasonal_shape, 8)
    trend = np.linspace(0, 60, 56)  # slow organic growth
    noise = rng.normal(0, 12, size=56)
    costs = seasonal + trend + noise
    costs[40] += 500  # a genuine anomaly on a Thursday, not on the seasonal peak day
    return pd.Series(costs, index=dates, name="net_cost")


def main() -> None:
    series = _synthetic_demo_series()
    masked = mask_calendar_exceptions(series, holiday_dates=set(), deploy_dates=set())
    result = fit_stl_baseline(masked, service_id="compute-engine")
    if result is None:
        return
    result = score_residual(result)
    logger.info("deseasonalized baseline (last 5 days):\n%s", result.baseline.tail(5))
    logger.info("flagged dates: %s", result.flagged_dates)


if __name__ == "__main__":
    main()

# Verification & Testing

Confirming the decomposition is doing its job means checking that the seasonal component actually absorbs the weekly shape and that the residual is what’s left, not a proxy for the same problem:

• Seasonal-shape stability. Split the fit into two halves of the series and compare fit.seasonal for a repeated weekday (e.g. all Sundays) between them. They should be close in magnitude — if the seasonal estimate swings wildly between halves, the series doesn’t have a stable weekly pattern and STL is the wrong tool for it.
• Weekday-clustering check on the residual. Group result.resid by day-of-week and assert the per-weekday mean is near zero across all seven days. A residual that still has a weekday-dependent mean means the seasonal component under-fit the cycle — usually because period was set wrong or a calendar exception was left unmasked.
• Injected-spike recovery. As in the demo main(), inject a known non-seasonal spike on an otherwise-normal day and assert score_residual flags exactly that date. This is the regression test that catches a silently broken fit before it reaches production.
• Reconstruction check. Assert (result.trend + result.seasonal + result.resid).equals(masked) within floating-point tolerance — STL’s decomposition is additive by construction, and a mismatch means the components were mutated out of sync somewhere downstream.

# Common Pitfalls Checklist

• Fitting STL with fewer than two full seasonal cycles. The seasonal component is unstable and effectively fits noise — enforce MIN_CYCLES_REQUIRED (4 here, 2 is the bare statistical minimum) and return None rather than a shaky fit.
• Leaving holidays and deploy weeks unmasked. A single unmasked holiday depresses that week’s seasonal estimate and keeps distorting residuals until it ages out of the window — mask and interpolate before fitting, never after.
• Scoring raw cost after fitting STL “for visibility.” If the anomaly threshold is applied to the original series instead of fit.resid, the seasonal shape is still there and you’re back to the original false-positive problem — score the residual, not the input.
• Using a fixed period for irregular cycles. period=7 is correct for weekday/weekend patterns; a service with a monthly billing-cycle-driven cost shape needs a different period (or a second seasonal component), and forcing 7 onto it produces a meaningless decomposition.
• Re-fitting STL on every new day in a tight loop. STL is iterative and noticeably more expensive than a rolling z-score — batch the fit (e.g. nightly per service) rather than calling it inline per incoming data point.

# Frequently Asked Questions

# Related

• Building Cost Anomaly Detection Models — the parent engine comparing rolling z-score, EWMA, MAD, and STL, and where to decide whether a service needs seasonal handling at all.
• Detecting Cost Spikes with a Rolling Z-Score — the flat-baseline sibling method this page’s residual scoring borrows its z-score mechanics from, and the weekday-clustering check that tells you when to switch to STL.
• Time-Series Aggregation for Daily Cloud Cost Tracking — the upstream daily rollup that produces the series this decomposition consumes, and a consumer of its deseasonalized baseline.
• Cloud Cost Anomaly Detection & Budget Automation — the discipline this scoring stage belongs to, including how flagged residuals feed alert routing.
• Budget Alert Automation with Webhooks — the downstream stage that turns a flagged residual date into a routed notification.

Up: Building Cost Anomaly Detection Models