-
Notifications
You must be signed in to change notification settings - Fork 27
R Wishlist
Several groups have been working on statistical code and/or more user friendly software for network meta-analysis. Notably there is the generalized linear models code from Bristol and Leicester (the DSU models), model generation work from Groningen, separate model generation work from Brown, and plotting and post-processing code from Ioannina. In addition, Groningen, Leicester, and Brown are all working on graphical user interfaces (GUIs) of some sort.
All parties agree that it would be beneficial to minimize the amount of work that is being duplicated. While there are some overlaps in the GUI functionality (e.g. visualizations), the goals and requirements are clearly different. Therefore the logical focus is on developing a shared back end that can perform the analyses and produce the appropriate summaries and plots.
Why does a shared back end for analysis code make sense? How does this fit with each of the groups' requirements? These questions are answered below.
-
The Brown GUI (Open Meta-Analyst) aims to provide a broadly applicable user interface for meta-analysis by leveraging existing R packages. Developing the analysis code itself is not a focus of this project. Hence, it would make sense to use existing code, specifically an R package.
-
Chris Schmid has developed modular R code to generate models for multinomial network meta-analysis with (potentially) missing data. His ambition was to extend this code to include different likelihoods, but he does not have time to do the work himself. Hence, someone else could take his code and extend it.
-
Gert van Valkenhoef has developed software to generate network meta-analysis models. Similar to Chris' code, it has been set up in a modular fashion and should be easy to extend to other likelihoods, but time constraints have prevented this. This especially because dual implementations in Java an R/BUGS were needed for all likelihoods. The plans for the ADDIS software architecture have changed so that only an implementation in R is necessary. The R package is currently available on CRAN (gemtc).
-
Leicester has a grant from J&J to develop a GUI for network meta-analysis. They have implemented several existing and/or innovative visualizations in R, and are now ready to move on to the user interface work. They were planning to rely mostly on existing BUGS code provided in the DSUs for the analyses. Thus, while the grant calls for a user interface to be developed specifically for J&J, back end functionality can be shared.
-
The shared platform is the R statistical software and may include packages readily available from the R package network (CRAN). Each of the groups already depends on R, so this is an easy choice to make.
-
The code should be able to "stand on its own" as an R package with a programmable (command line) interface. Then, it can be distributed through CRAN, which means that it will "just work" whenever R does. This will also increase the impact of the software, through broader use and potentially a publication in J Stat Soft.
-
Due to distribution constraints for both Open Meta-Analyst and the Leicester software, which need to ship all the dependencies with the software, external dependencies should be kept to a minimum.
-
A common format for the data needs to be agreed upon so that the different GUIs can talk to the R package.
-
Model estimation will be based on BUGS/JAGS because existing analysis code is all available for these packages, and they work well. Moreover, there is a high level of trust in BUGS/JAGS and being able to output the BUGS/JAGS code that has been generated will increase the trust in the R package.
-
Code generation must work for WinBUGS and OpenBUGS and JAGS for several reasons. First, JAGS appears to be more actively developed and faster. Second, JAGS is cross-platform and so can easily be distributed for both Windows and Mac (and Linux). Third, WinBUGS is more universally known and trusted, and some stakeholders (e.g. NICE) may demand an analysis in BUGS. Fourth, the differences between BUGS and JAGS are slight, and the cost of supporting both is minimal.
This section lists items where writing the software should be fairly straightforward.
-
All likelihoods and link functions described in the NICE DSUs
-
A "pluggable" model where one could write an R function (or several functions) that describe the likelihood would be useful.
-
This is work in progress. Implemented:
o normal/identity (since 0.1)
o binom/logit (since 0.1)
o binom/cloglog (since 0.3)
o poisson/log (since 0.4)
-
-
Models for the assessment of inconsistency (node-splitting, design-by-treatment interaction, independent mean effects, ...)
-
Regression models (e.g. dose-response, population characteristics, duration of treatment, ...)
- Again, a "pluggable" model where different structures for the linear model can be easily added would be beneficial, but perhaps more difficult to realize than for the likelihoods.
-
Automated and manual convergence checking
-
Automatically generated starting values for several chains
-
Stopping criteria for the simulation (Brooks-Gelman-Rubin, MCMC error, ...)
-
Ability to extend the simulation on user request
-
Diagnostic summaries and plots (directly available in CODA)
-
-
Prior selection
-
Automated vague default priors
-
Empirically based informative priors (e.g. Rebecca Turner's paper)
-
User defined priors
-
-
Summaries and plots for the assessment of heterogeneity and inconsistency
- Gert is working on a new plot idea (who knows whether it will ever be finished...)
-
Summaries and plots for the interpretation of the consistency model
-
Sze Huey has developed some new ones and implemented some existing ones
-
There is also publicly available code from Ioannina (Georgia Salanti et al.)
-
Forest plots and rank probabilities available in GeMTC R package
-
-
Summaries and plots for the assessment of model fit
-
DIC, residual deviance, ...
-
Deviance and leverage plots, ...
-
-
Access to the generated code for each of the models, as well as the raw samples. For example to be used in a cost-effectiveness analysis.
-
Facilitated sensitivity analysis - e.g. "1 click" to remove a study etc
-
Calculate and graphically present absolute effects (for specified baselines)
-
Dichotomous outcomes: transform between scales using a specified baseline (e.g. LOR -> LRR using assumed LO). See this Gist: https://gist.github.com/gertvv/4978453
This section lists items where there may be additional challenges.
-
Heterogeneous variance models
-
More exotic likelihoods (like Chris' multinomial missing data mixtures)
-
Individual Patient Data (IPD) models
-
Mixed IPD and summary data models
-
Models with duplicated arms: e.g. in a review where we define treatment with drug X anywhere between 15 and 20 mg/day as treatment B, and we have a study that has both X 15 mg/day and X 20 mg/day. Since our premise is that 15 mg/day and 20 mg/day are essentially the same, we would like to include both arms. This raises some issues:
-
In a random effects model, the normal code would translate to
delta[i,2:3] ~ dmnorm(c(dAB, dAB), ...)but this implies a withinstudy* random effect, which is not appropriate for the normal interpretation of the random effect. Do we need an additional level to deal with this? I.e.delta[i, 2:3] <- c(delta.t[i,2], delta.t[i,2])anddelta.t[i,2] ~ dnorm(dAB, ...). -
A lot of the R code currently assumes that (study, treatment) uniquely identifies a row in the data set, this assumption fails when there are duplicated arms. This implies some rework.
-
How can we present duplicated arms within a trial in a forest plot? Even for a pair-wise analysis?
-
How is trial decomposition supposed to work with duplicated arms?
-
-
...
-
(2013-07-31, 0.4) Fixed effect and random effects models
-
(2013-07-16, 0.3) Mixed models (e.g. arm-based and contrast-based data)
-
(2013-02-13, 0.2) "For example, the GeMTC dependency on Java is not acceptable." Dependency removed, however inconsistency models currently unavailable.
-
(2013-02-18, 0.2) "Note: it might be beneficial to set up a test suite to automatically check that all code works on all of these pieces of software." This has been set up for gemtc_0.2, seven MTCs are now validated against "known correct" posterior summaries (using all three samplers: rjags, BRugs, R2WinBUGS).