terra::predict sometimes fails with nnet::multinom models, with unclear reasons why

[connor-duffin]

I am using a multinomial logistic regression to estimate proportions of land-use across the globe. After estimating the model with nnet::multinom (which trains fine and suffers no convergence issues), I then use the trained model to predict the probabilities of land use across the globe, using terra::predict, using the following:

p <- terra::predict(terra::droplevels(newdata), lu_model, type = "probs", factors = NULL)

where newdata is our proprietary data used to predict across the globe, and lu_model is the trained nnet::multinom model. This behaves inconsistently, with occasional failures, within the same R session, with an error message saying:

# <subscriptOutOfBoundsError in `[<-`(`*tmp*`, keep, , value = matrix(.C(VR_nntest, as.integer(ntr),
#    as.double(x), tclass = double(ntr * nout), as.double(object$wts))$tclass,
#    ntr, nout)): subscript out of bounds>

Which suggests that this is some sort of internal problem with the output matrix. I have been able to rectify this through getting the probabilities separately, e.g.:

p1 <- terra::predict(terra::droplevels(newdata), lu_model, type = "probs", factors = NULL, index = 1)
p2 <- terra::predict(terra::droplevels(newdata), lu_model, type = "probs", factors = NULL, index = 2)
p3 <- terra::predict(terra::droplevels(newdata), lu_model, type = "probs", factors = NULL, index = 3)

Leading me to again believe this is a problem with the matrix internals, with this strange inconsistent error.

Apologies for the lack of data/reprex, as mentioned we are working on a proprietary dataset.