GITCO: Gated Inference-Time Context Optimization in TSFMs
Original reporting by arXiv (cs.AI)
Time Series Foundation Models (TSFMs) represent a significant leap in forecasting capabilities, adept at processing vast datasets to predict future trends. Yet, even these sophisticated models harbor a subtle vulnerability: "context poisoning." This occurs when structurally anomalous data segments, or "patches," within a time series disproportionately capture the model's attention, silently degrading the accuracy of its zero-shot forecasts. The challenge has been how to mitigate this without the costly and complex process of retraining these massive models.
Optimizing without retraining
A new framework, Gated Inference-Time Context Optimization (GITCO), offers an elegant solution. Instead of altering the fundamental weights of a pre-trained TSFM, GITCO intervenes at the point of inference, dynamically optimizing the input context. This lightweight, three-component system—comprising a Gate, Router, and Critic—works to selectively identify and suppress these harmful, attention-grabbing patches *before* they can skew predictions. The innovation lies in its ability to enhance model performance without a single parameter update, preserving the integrity of the foundational model while boosting its real-world utility.
Evaluated on the robust TimesFM 2.5 across 53 diverse datasets, GITCO demonstrated an impressive average 1.95% reduction in Mean Absolute Scaled Error (MASE), achieving nearly 90% of the theoretical maximum improvement. This work also introduces the concept of "context sensitivity profiles," a novel way to characterize TSFMs based on how their accuracy responds to targeted context interventions, offering deeper insights into their behavior and paving the way for more robust forecasting systems.
The introduction of GITCO marks a significant advance in the practical application of Time Series Foundation Models. By effectively mitigating the challenge of "context poisoning," where anomalous data patches silently degrade forecast quality, GITCO offers a robust, inference-time solution that requires no model retraining. Its demonstrated ability to improve forecast accuracy by nearly 2% MASE on a prominent TSFM like TimesFM 2.5, capturing a substantial portion of the theoretical upper bound, underscores its immediate utility. This lightweight framework — comprising a Gate, Router, and Critic — enhances model reliability and trustworthiness without the computational overhead of weight updates, making TSFMs more dependable for critical forecasting tasks.