Supernova classifications look like data problems. They are usually code problems. The same photometry, run through two pipelines, can produce different verdicts on whether an object is a peculiar superluminous event or an ordinary Type Ia with noisy photometry. What separates the two is rarely the telescope. It is the smoothing kernel, the redshift prior, the error propagation nobody implemented. Archived code decides which results hold up.

The Supernova That Refused to Fade

A preprint once described a supernova that stayed bright far longer than any known template predicted. The light curve was smooth, almost too smooth, and the classification followed: a new class of superluminous event. Within weeks, two groups reran the same public photometry through their own pipelines. One recovered the original classification. The other did not. The difference traced to how each pipeline handled gaps in the observing cadence.

One group interpolated across missing nights with a broad smoothing kernel. The other fitted only the nights that existed. The smoothed version produced a light curve that refused to fade on schedule. The unsmoothed version looked like a normal supernova with sparse coverage. Neither pipeline was wrong in a computational sense. They answered slightly different questions.

This is the quiet failure mode in supernova science. The data release is public, documented, and citable. The code that turned those magnitudes into a classification is often neither. A reader who downloads the photometry and applies a reasonable default will not necessarily reproduce the published verdict.

A related piece on this site, The Redshift Drift Test Measures Expansion Without Assuming a Model, makes a similar point about assumptions that travel quietly inside analysis choices.

How Code Becomes the Hidden Variable

Supernova classification relies on template-fitting algorithms. Each template carries a redshift prior, a stretch parameter, and a color law. Small changes in those priors shift classifications near the boundary between types. A supernova that sits close to the dividing line can move from one class to another without any new data arriving.

Most papers cite the data release and the survey. They rarely cite the code version. An archived notebook, when one exists, often reveals preprocessing steps that never made it into the methods section: a sigma-clipping threshold, a rest-frame correction applied twice, a filter response curve interpolated from a table. None of these are misconduct. They are the ordinary residue of doing science in a scripting language.

The consequence is that a classification can be reproducible in principle and irreproducible in practice. The data are there. The recipe is not. A reader who wants to check the result has to reconstruct the recipe from a methods paragraph that was never meant to be a specification.

Consider the choice of a smoothing kernel. A common approach is a Gaussian kernel with a fixed width in days, chosen to match the typical cadence of the survey. If the cadence varies, a fixed width may over-smooth regions with dense sampling and under-smooth regions with sparse sampling. A more robust approach is to use a kernel that adapts to the local density of observations. The difference between these choices can shift the inferred peak time by a day or more, which in turn shifts the stretch parameter and the classification. None of this appears in a typical methods paragraph, which might say only that the light curve was smoothed.

Another hidden variable is the treatment of upper limits. Many surveys report non-detections. How a pipeline incorporates those limits into a fit can change the result. Some pipelines ignore them entirely. Others treat them as zero-flux measurements with large errors. A few implement a proper likelihood that accounts for censored data. The choice can move a marginal classification from one class to another. Again, the methods section often omits this detail.

The Case of the Missing Error Bars

Error propagation is often skipped in supernova pipelines. When a pipeline smooths photometry before fitting, it reduces the apparent scatter. If the fitting routine then treats the smoothed points as if they carried the original uncertainties, the errors are understated. That can inflate the outlier rate or, depending on the analysis, suppress it. The mechanism is subtle and easy to miss.

In practice, this means that a headline number, such as the fraction of objects that fail a quality cut, can depend on a preprocessing step that seems cosmetic. The number is quoted in abstracts and reviews. The choice that produced it lives in a function call. When the pipeline is archived, the mechanism becomes visible. Without it, a discrepancy between two analyses can look like a difference in sample selection or even a difference in the underlying data.

This is not a hypothetical concern. Groups that have rerun published pipelines with archived code have found that small changes in smoothing or error handling can shift outlier rates by several percentage points. The exact figures vary by sample and by survey, but the pattern is consistent: preprocessing choices propagate into final results.

What Consensus Actually Requires

Consensus in supernova cosmology does not require everyone to use the same pipeline. It requires independent replication with independent code. When two groups analyze the same data with separately written pipelines and reach the same classification, the result is robust to implementation details. When they reach different classifications, the disagreement is informative.

Blind analysis challenges have become more common in cosmology. Analysts hide the cosmological parameters until the pipeline is frozen, then unblind once. The method works, but it is expensive and slow. It also does not guarantee reproducibility if the code is never shared.

Journals increasingly request code and environment files alongside data. Enforcement remains uneven. Some collaborations deposit full container images. Others provide a GitHub link that may or may not resolve in five years. A related piece on this site, Reproducibility Audits and Journal Sanctions Diverge on Retraction Reach, notes that audit practices and journal responses do not move in step.

Lessons for Computational Astronomy

Version control is not a reproducibility strategy. A Git repository with a hundred commits preserves history but not environment. The code may not run on a different machine, with a different NumPy version, or with a different BLAS library. The commit hash tells you what changed. It does not tell you what the author intended.

Containerization preserves dependencies but not intent. A Docker image can freeze the exact software stack, but if the smoothing kernel was chosen by trial and error, the image will not explain why. Intent lives in documentation, and documentation is the step most likely to be skipped under deadline.

Peer review must include code execution, not just reading. A reviewer who reads a methods section can spot a missing citation. A reviewer who runs the pipeline can spot a redshift prior applied in the wrong frame. Few journals currently fund that kind of review.

Training in software carpentry remains optional in most astronomy graduate programs. A student can complete a PhD in supernova cosmology without ever learning to write a unit test for a fitting routine.

Practical Steps for Reproducible Supernova Work

Archive code with a DOI and an environment lockfile. A DOI makes the code citable; the lockfile makes it runnable. Deposit both in a repository that commits to long-term preservation, not a personal page.

Run your own pipeline on a blinded subset before unblinding the full sample. If your classification changes when you swap smoothing choices, you have learned something about the object and something about your method.

Document every smoothing and filtering choice in the methods section, including the ones you tried and rejected. A reader who knows which kernels you tested can judge how sensitive the result is.

Share intermediate data products, not just final catalogs. The photometry after preprocessing, the fitted parameters before selection cuts, the object-level flags. These let other groups diagnose disagreements without rerunning everything from raw frames.