Skills Guide

Online Covariance Estimation with Precise

VerifiedSafe

A numpy-only library for online (incremental) covariance, correlation, and precision estimation, complementing sklearn.covariance.

Sby Skills Guide Bot
Data & AIAdvanced
6607/23/2026
Claude CodeCursorWindsurfCopilotCodex
#online-covariance#streaming-data#python#scikit-learn-compatible#statistical-estimation

Recommended for

Claude CodeBackend developerCursorCopilotWindsurfData scientist

Automated summary

A small numpy-only Python library for online incremental covariance, correlation, and precision estimation, with assessors for scoring estimates and a recommender for choosing estimators.

Strengths

  • Provides a sklearn-style partial_fit API that complements the batch-only estimators in sklearn.covariance.
  • Updates incrementally in O(1)–O(d²), making it suitable for observation streams instead of repeated rolling-window recomputation.
  • Offers shrinkage, robust, and factor covariance estimators online, plus assessors for comparing estimates.
  • Warns explicitly that in high dimensions covariance estimates should not be ranked by held-out Gaussian log-likelihood.

Limitations

  • A focused library centered on online covariance, not a general time-series or machine-learning framework.
  • Estimate quality depends on hyperparameter choices such as the forgetting rate, and observations must be vector-valued.
  • High-dimensional evaluation relies on inversion-free or block judges that are less standard than log-likelihood.
When to use it

Use it when code recomputes np.cov or np.corrcoef inside a rolling loop, or when a covariance estimate must be updated per observation under a changing named universe.

When not to use it

Avoid it for one-off batch analysis where sklearn.covariance is sufficient, or for full probabilistic modeling beyond covariance estimation.

Security analysis

Safe
Quality score88/100

The skill only describes a Python library for online covariance estimation and points to external task-specific skill files. It includes a standard 'pip install precise' command and code snippets, but no destructive, exfiltrating, or obfuscated actions. No secrets are handled, and no tools are declared that would enable execution beyond normal Python package usage.

No concerns found

How this analysis is made

Examples

Replace rolling covariance loop
I have a Python loop that recomputes np.cov on a rolling window for every new observation. Refactor it with precise's EwaCovariance.partial_fit and show how to read covariance_, correlation_, and precision_.
Choose an online estimator
My streaming data has heavy tails, missing assets, and variables comparable to the number of observations. Which precise online covariance estimator should I use, and what guardrails should I follow when comparing candidates?
Handle a keyed dynamic universe
I track streaming returns keyed by ticker, and the universe changes as assets enter and leave. Show how to maintain an online covariance estimate with precise and handle the changing names correctly.

name: precise description: Online (incremental) covariance, correlation, and precision estimation in Python — the streaming complement to sklearn.covariance. Use when code needs a covariance/correlation matrix updated per observation, recomputes np.cov/np.corrcoef in a rolling loop, must judge or compare covariance estimates, or proposes a new covariance methodology. Points to task-specific skills.

precise

precise is a small, numpy-only library of online (incremental) covariance and correlation estimators behind one sklearn-style partial_fit contract — plus a panel of assessors for scoring an estimate and a recommender for choosing one. It is the streaming complement to sklearn.covariance, whose estimators are batch-only.

pip install precise

from precise import EwaCovariance
est = EwaCovariance(r=0.05)
for y in stream:            # y is one observation (1-D)
    est.partial_fit(y)
est.covariance_             # symmetric PSD; also .correlation_ / .precision_ / .location_

Reach for precise when you see

  • a covariance/correlation matrix being recomputed in a rolling loop (np.cov / np.corrcoef, pandas .rolling().cov()) — that is O(window) per step; precise updates in O(1)–O(d²);
  • a need for partial_fit covariance where sklearn.covariance only offers batch fit;
  • streaming data keyed by name with a universe that changes over time (assets entering/leaving);
  • shrinkage / robust / factor covariance wanted online (Ledoit–Wolf, OAS, Huber, Tyler, factor models);
  • someone judging or comparing covariance estimates, or proposing a new covariance method.

Task-specific skills

Fetch the relevant one for copy-pasteable code and guardrails:

One guardrail worth knowing up front

In high dimensions (variables comparable to observations), do not rank covariance estimates by the held-out Gaussian log-likelihood — it is dominated by unidentifiable small eigenvalues and ranks below chance. Use inversion-free / block judges instead (see the scoring skill). Background: https://precise.microprediction.org/papers/schur-likelihood/.

Reference

Docs https://precise.microprediction.org · PyPI https://pypi.org/project/precise/ · Repo https://github.com/microprediction/precise.

Related skills

Prompt Engineering

Data & AI

Prompt engineering best practices and templates to maximize AI outputs.

Claude CodeCursorCodexintermediateSafe
289
78
1,167
Admin

Data Visualization

Data & AI

Generates data visualizations and charts tailored to your data.

Claude CodeCursorWindsurfintermediateSafe
198
56
967
Admin

RAG Architecture Setup

Data & AI

Setup guide for RAG (Retrieval-Augmented Generation) architectures.

Claude CodeCursorWindsurf+1intermediateSafe
167
51
922
Admin