Our review

Validates topological data analysis results against known benchmarks and internal consistency checks, including persistence diagram sanity, null model behavior, and Wasserstein-based significance.

Strengths

Provides a practical checklist covering persistence diagram sanity, total persistence scaling, and null-model consistency.
Includes concrete expected values for the trajectory_tda / P01 dataset, enabling quick regression checks.
Combines internal consistency checks with known benchmark comparisons, reducing false positives in TDA conclusions.
Addresses specific statistical pitfalls like null-null distances and p-value stability.

Limitations

Requires prior knowledge of TDA concepts and the specific trajectory_tda pipeline to interpret failures.
Focuses on one dataset family (P01) and may not generalize to other TDA analyses without adapting benchmarks.
Checklist does not automatically fix issues; it only identifies potential inconsistencies.

When to use it

Use when you need to verify the correctness and reproducibility of topological data analysis results before reporting them.

When not to use it

Do not use for non-topological analyses or when no known benchmarks or null models are available.

Examples

Validate a persistent homology result file

/validate-topology trajectory_tda results/trajectory_tda_integration/04_nulls_wasserstein.json

Check persistence diagram sanity

Run validation on the trajectory TDA null Wasserstein results and report any consistency failures.

Verify P01 benchmark alignment

/validate-topology trajectory_tda results/trajectory_tda_integration/04_nulls_wasserstein.json

/validate-topology — Validate Topological Results

Check mathematical correctness of TDA results against known benchmarks and internal consistency checks.

Usage

/validate-topology [domain] [result-file]

Example: /validate-topology trajectory_tda results/trajectory_tda_integration/04_nulls_wasserstein.json

Validation checklist

Persistence diagram sanity

[ ] All birth values ≥ 0
[ ] All death values > birth (finite features)
[ ] H₀ has exactly one infinite feature (the final connected component)
[ ] Feature counts are plausible for landmark count L: H₀ has L-1 finite features; H₁ count varies

Total persistence scaling

[ ] Total persistence scales approximately linearly with L (±15% across a 2× range)
[ ] Maximum persistence is stable across L values (should vary < 5%)

Null model consistency

[ ] Label/cohort shuffle p-values are non-significant (negative control)
[ ] Markov-2 null generates more total persistence than Markov-1 (higher-order Markov → more structured surrogates)
[ ] Null distribution standard deviations are plausible (not near-zero, not huge)

Wasserstein-specific

[ ] W(obs↔null) and W(null↔null) are of comparable magnitude (within ~3×)
[ ] p-value = proportion of null-null distances ≥ mean(obs-null distances)
[ ] 500 null-null pairs is sufficient for stable p-value at 3 decimal places

Cross-era replication

[ ] BHPS-era order-shuffle H₀ p-value ≈ USoc order-shuffle direction (both significant or both not)
[ ] BHPS-era Markov-1 direction ≈ USoc (both non-significant under total persistence)

Known benchmarks (trajectory_tda / P01)

| Test | Expected | Source | |---|---|---| | USoc order-shuffle H₀ (total persistence, L=5000) | p < 0.005 | P01 v5 Table 2 | | USoc Markov-1 H₀ (total persistence) | p = 1.000 | P01 v5 Table 2 | | USoc Markov-1 H₀ (Wasserstein, L=2000) | p = 0.002 | P01 v5 Table 2b | | BHPS order-shuffle H₀ | p = 0.000 | P01 v5 §4.7 | | BHPS Markov-1 H₀ | p = 1.000 | P01 v5 §4.7 | | GMM bootstrap ARI | 0.646 ± 0.086 | P01 v5 §3.5 |

Validate Topological Results

Recommended for