Pooling strangers to answer a question about you

A personal trial has an uncomfortable amount in common with a very small study. Sixty-two nights sounds like a lot of data until you remember it is one person, the observations are correlated with each other, and everything that varies in a life varies alongside the thing you are testing.

The obvious fix is to borrow from other people. The obvious problem with the obvious fix is that borrowing too hard replaces you with the average, which is what made population trials unsatisfying in the first place.

Partial pooling

Hierarchical models occupy the middle. Each person's treatment effect is modelled as drawn from a population-level distribution rather than estimated in isolation or assumed identical to everyone else's. The consequence is adaptive: where your own data is weak or noisy, the estimate shrinks toward the population; where your data is strong, it stays close to what your data says.

This is a well-established approach for aggregating series of n-of-1 trials, not something invented for consumer health. Zucker, Ruthazer and Schmid set out the methodological considerations in 2010, working through models fitted to n-of-1 trials of amitriptyline and a combination therapy [1]. They found the best-fitting specification combined a random intercept — capturing each patient's own baseline response — with a fixed treatment effect.

Where it breaks

The same paper is direct about the failure mode, and it is worth quoting the shape of it: Bayesian hierarchical models improved precision, but were highly sensitive to the priors placed on within-patient variance.

That sensitivity is not a technicality. The within-patient variance prior governs how much the model believes your night-to-night noise explains, and therefore how hard it pulls your estimate toward the group. Choose it carelessly and you can manufacture a confident personal result that is mostly an echo of the population, or an unreasonably wide one that ignores relevant information. The prior is doing real work, and pretending otherwise is how a hierarchical model becomes a machine for producing confident nonsense.

The practical response is unglamorous: state the priors, check whether the posterior moves when they change, and report it when it does. A result that survives only under one convenient prior is not a result.

The other trade-off

Zucker and colleagues also frame a design tension that does not go away. Effort spent increasing precision within each patient — more cycles, longer periods — competes with effort spent recruiting enough patients to explain population heterogeneity through covariates. You can know a few people well or many people roughly, and the right balance depends on whether the question is about an individual or about who responds.

Because our question is the individual one, we weight toward the first, and we treat the cohort as a prior rather than as an answer.

Reporting

N-of-1 trials have a reporting standard: CENT 2015, the CONSORT extension for n-of-1 trials, which provides additional guidance for 14 of the 25 items on the CONSORT 2010 checklist [2]. It exists because n-of-1 designs have been reported badly often enough that readers could not judge validity or reproduce what worked. We hold our own write-ups to it, which is why our results carry period structure, randomisation detail, and washout alongside the effect size.