Speaker

Jonathan C. Moyer, PhD
Statistician, NIH Office of Disease Prevention

Moderator: Andrea J. Cook, PhD
Senior Biostatistics Investigator, Kaiser Permanente Washington Health Research Institute

Slides

Keywords

Individually randomized group treatments; Intervention; Randomization; Clinical trials

Key Points

• In individually randomized trials (IRTs), individuals are randomized to either control or intervention arms. In Group- or cluster-randomized trials (GRTs), pre-existing groups are randomized to either control or intervention arms. Both trial types are common, but it’s important to note that observations are correlated before and after randomization.

• In individually randomized group treatments (IRGTs), individuals are randomized to either control or intervention arms, similar to individually randomized trials, but there might be post randomization grouping or clustering in one or both conditions. In IRGT trials, individuals are randomly assigned to arms, but treatment is delivered in groups or through shared intervention agents.

• Participants who are connected by group membership or share the same intervention agent will likely have correlated outcomes, which are often quantified using the interclass correlation (ICC), which reflects the extra variation attributable to group or shared agent. Failing to account for ICC is shown to inflate type I error rates in the context of GRTs. Similar type I error rate inflation is possible with IGRTs, but the potential impact of this correlation is acknowledged less frequently.

• In this trial, the researchers were interested in three main data structures. The first is the fully nested structure in which agents are present in both arms and each agent interacts with participants in only one arm. The second is the partially nested structure in which agents are only present in one arm. The third is the crossed structure in which the same agents interact with participants in both arms.

• The researchers also considered multiple membership structures. In a single membership, each participants interacts with one agent. In a multiple membership structure, a participant may interact with more than one intervention agent. Finally, in a single agent structure, there’s only one agent present per arm in the fully nested case or in the trial as a whole for the partially nested or crossed structures.

• In multiple membership structures, random effects for agents are weighted, which reflects the proportion of treatment a participant receives from an agent. Expressions for ICC are more complicated with multiple membership structures, since the value of each ICC depends on the agent weights for ICCs found for pairs of agents.

• In this trial, the researchers looked at five data generation mechanisms: fully nested, partially nested, crossed, crossed-interaction, and crossed-imbalanced. The results found type I error rates for multiple membership, single membership, single agent, and alternative analyses for each mechanism, in addition to power for each.

• The results of this analysis suggest that crossed designs protect the type I error rate, allow flexibility in analytic models, and provide good power with sufficient sample size. However, there is a risk of contamination with crossed designs. For nested models, the analytic model should match the expected structure of the data, and naïve models should not be used. Since power in small studies is less adequate, a power analysis with realistic and data-based estimates is key.

• Some limitations concluded from this research include that researchers only looked at continuous outcomes, rather than binary outcomes. Additionally, the number of participants for this analysis remained consistent, so it would be worthwhile to conduct future studies with variation in the number of participants per agent.

Discussion Themes

-It seems that any trial that involves an intervention that delivers the intervention through agents, people, or in a group formation you discussed that would be a large fraction of trials in public health and medicine. Are most of these trials being done incorrectly? Studies should probably plan how they are desired to be planned. For example, maybe you would expect to have one agent interact with the participants in a group. Then, as you are analyzing it, maybe as a secondary analysis, keep a record of who the participants are interacting with and see how much of an impact it has made. When trials are being planned, we assume there will be some loss to follow up. We account for that in our sample size calculations. If you think there’s a potential for, say multiple membership to arise, you might think of what exactly that mechanism is and the power calculations you do to account for that. I don’t think there are actually very many closed form formulas for the multi-membership case. I know if one recent resource for cross classification, but you could use simulation methods to analyze the setting that you think matches what is possible.  

-The lowest ICC used in this simulation study started at 0.05, which is pretty large. Why not use smaller ICCs? Generally speaking, the greater the ICC, the more closely the participants are interacting. In the case of individually randomized trials where the people are interacting with the same agent, for instance, the same surgeon or acupuncturist, the results are likely the be more strongly correlated in that setting. The smaller ICCs generally correspond to larger kind of structures like communities or neighborhoods. These ICCs were the ones that were more representative of what we saw in individually randomized group treatment trials as opposed to GRTs.

Tags

#pctGR, @Collaboratory1