How to save results of using torch.compile with tensorrt backend?

[ved-genmo]

I have successfully compiled a model, but I'm wondering how to cache it, so I can avoid the long compilation process in the future.
E.g, right now I'm using:

model = torch.compile(model, fullgraph=True, backend="tensorrt")
batch = {}
batch['fake_input'] = torch.randn(1, 15, 37, 192, 336).to(device=device, dtype=torch.float32)
with torch.cuda.amp.autocast(dtype=torch.float16):
    result = model(batch['fake_input'])

But, ideally, I'd save the result of torch.compile to disk, so I can use it in the future.

[narendasan]

torch.compile is not serializable, torch.export + torch_tensorrt.compile (or more precisely torch_tensorrt.dynamo.compile) will give you an artifact which can be saved to disk (https://pytorch.org/TensorRT/user_guide/saving_models.html). Since you are using fullgraph=True, this should be a drop-in change.

In the future we are adding support for caching engines to speed up torch.compile in cases where you cannot use torch.export