Future development
metasignal's stdpy measures and MATLAB-to-Python translation are complete — see
analysis/rahnev_comparison/
for the full validation against the original MATLAB pipeline. The items below track known gaps and
planned directions. Contributions on any of these are welcome — see
Contributing.
| Area | Status | Details |
|---|---|---|
Robust HMeta-d, variational inference, Gaussian mixture, bivariate hierarchical, and state-space models (sdtbayes/{robust,variational,mixture,multivariate,statespace}.py) |
Removed | These five approaches always raised NotImplementedError/RuntimeError — each was permanently blocked by an upstream brmspy/rpy2 limitation (lost R S3 classes, un-convertible mixed-block stanvar() lists, or a VI API mismatch) with no cmdstanpy-backed replacement. Removed rather than shipped as importable dead API; revisit if brmspy/rpy2/cmdstanpy close the underlying gaps. |
brmspy-backed HMeta-d (sdtbayes/full_metad.py) |
Migrated to cmdstanpy | fit_full_metad / fit_full_metad_comparison now delegate to the Stan (cmdstanpy) backend via fit_meta_formula — the original brmspy stanvar() injection path was removed because brmspy/rpy2 cannot round-trip a list of mixed-block stanvar() objects. Worth revisiting if brmspy/rpy2 fixes the underlying conversion. |
| First PyPI release | Not yet published | CHANGELOG.md and pyproject.toml are at 2025.1.0, and a wheel/sdist build exists locally, but there's no tagged GitHub release and no version on PyPI yet. nox -s release-to-pypi is ready to go. |
MetaSignal MATLAB engine wrapper (src/metasignal/wrapper.py) |
Deprecated | Emits a DeprecationWarning on instantiation ("will be removed in a future release") and only wraps one method (compute_all_measures) via a live MATLAB engine + licence. metasignal.stdpy is the maintained pure-Python replacement. Removing wrapper.py (and the metasignal[matlab] extra) is a planned cleanup once users have had time to migrate. |
Additional sdtbayes diagnostics/plots |
Open | sdtbayes/diagnostics.py covers posterior summaries and basic convergence checks; richer diagnostic plots (trace, rank, pair plots via ArviZ) are not yet wrapped as one-call helpers. |
Broader OS/Python coverage for sdtbayes |
Open | cmdstanpy/CmdStan installation is the main friction point for new contributors; a documented Docker-based dev path for sdtbayes work would lower the barrier (the repo has a Dockerfile and docker-image.yml workflow already — worth confirming it covers the Stan toolchain). |
Alternative SDT model family (metasignal.sdtr, new subpackage) |
Proposed | Bring in the model family from Macho (2020), SDT-Models in R — Gaussian SDT with per-signal decision bounds, Gaussian mixture models (MIX.PD, MIX.2), the dual-process model (DPSDT), the double high-threshold model (HT.n), the bivariate Gaussian model allowing violations of decisional separability (SDT.2D, SDT2D.MIX.2), forced-choice/ranking models with recollection-and-guessing and position-bias components (SDT.Rank, HTSDT.Rank, HTSDT.Bias.Rank, MIX.Rank, MIX.Bias.Rank), and probabilistic-IRF SDT (SDT-IRF-Gauss). None of these overlap with the 26 measures already in stdpy or the hierarchical models in sdtbayes. Plan: start with a thin R wrapper (analogous in spirit to the deprecated wrapper.py MATLAB bridge, but for R — likely via rpy2) around Macho's SDT-*.R source files so the models are usable from Python immediately, then port each model natively to Python incrementally, following the same MATLAB→stdpy translation precedent established in stdpy. Note the sdtbayes/brmspy experience above: R-object round-tripping through rpy2-based bridges has hit real limitations (lost S3 classes) — the wrapper should stay to simple, flat return types (vectors/data frames) to sidestep that class of bug. |
Relative psychometric function (metasignal.rpf, new subpackage) |
Proposed | Bring in the RPF framework from Peters, Graham Castaneda, Odegaard, Morales, Rajananda, Denison & Maniscalco (2025), The relative psychometric function: a general analysis framework for relating psychological processes, Journal of Vision, 25(8):1790. RPF characterizes the nonlinear relationship between a "base" psychometric function (e.g. Type-1 performance, d') and a "relative" one (e.g. confidence, meta-d') across conditions, generalizing beyond single-point M-ratio comparisons. Reference implementation: CNClaboratory/RPF (MATLAB, depends on Palamedes + a bundled meta-d' toolbox; entry points RPF_fit_F, RPF_get_R, RPF_plot). Plan: implement as a new pure-Python subpackage, metasignal.rpf, reusing stdpy's existing fit_meta_d_mle/compute_all_measures machinery for the underlying d'/meta-d' fits rather than wrapping the MATLAB toolbox, since the RPF layer itself (curve fitting + plotting across conditions) is fairly self-contained. |
Metacognitive-bias measure from Fleming lab (metasignal.itmc or new module) |
Proposed | Steven Fleming's group has proposed a new measure of metacognitive bias: OSF preprint. Currently a preprint; revisit for inclusion once it is peer-reviewed and published. |
Reference-value validation for itmc (src/metasignal/itmc/) |
Validated | Cross-checked backend='statconfr' against the real R statConfR::estimateMetaI() on 12 shared synthetic participants — see analysis/itmc_comparison/. Deterministic core math matches R exactly (r=1.0000, all 5 measures); Monte Carlo bias-corrected values agree well aside from a known ratio-measure instability at very low d′ (present in both implementations, not a Python bug). Also directly reproduces Dayan (2023)'s own hand-worked meta-I example from the paper text (Table 1) exactly. Still open: validating meta_Ir1/meta_Ir1_acc/RMI against Dayan's Figure 1 table, which needs real Shekhar & Rahnev (2021) per-trial data (a synthetic reconstruction without genuine stimulus-category variability isn't valid for the d′-dependent normalizers). |
If you want to pick up one of these, open an issue on the tracker referencing this page so work isn't duplicated, and see the GitHub Project board (linked from the repo) for anything already in progress.