Understanding Trainer.initialize: batch size x number of inputs?

[hmf]

I am going through the book, and in chapter 4.5.4 I see the code:

   Trainer trainer = model.newTrainer(config);

    trainer.initialize(new Shape(batchSize, 2));

I cannot understand those parameters. Initially I though that the shape would be for the input (batch-size, number of parameters). However looking at the source code I see we can input several of these shapes.

Can anyone direct me to an example that shows the use of several shapes and explain the when and why?

TIA

[zachgk]

When you call trainer.initialize(shape), the shape you pass into it is the input shape that your model accepts. It then uses the input shape into the main block to compute the input shape into all children blocks and parameters to initialize all of them as well. So, the mapping from input shape to number of parameters is part of the block class; you don't have to worry about it.

If your model input is a batch of RGB images of size 28x28, you would give it trainer.initialize(new Shape(batchSize, 3, 28, 28)). You could also give it trainer.initialize(new Shape(1, 3, 28, 28)) because the batch size doesn't affect parameter initialization. Or, if it is a model that accepts two images, then it might be trainer.initialize(new Shape(1, 3, 28, 28), new Shape(1, 3, 28, 28)) or trainer.initialize(new Shape(1, 2, 3, 28, 28)) depending on how the model was written.

[hmf]

@zachgk

Thanks for the information.

You could also give it ... because the batch size doesn't affect parameter initialization.

Ok, this one was confusing me. I think I will stick to the first form, even if it inconsequential.

When you say "parameter initialization" are you referring to initializing the model weights or something else?

As for your last example, if you have any simple example handy, let me know.

Thank you.

[zachgk]

Yeah, it is just weight initialization. If you are looking for a complete simple example, you can find one near the middle of our ~10min beginner tutorial: http://docs.djl.ai/jupyter/tutorial/02_train_your_first_model.html#step-5-initialize-training

[hmf]

Thank you. I have been through those. I was referring specifically to the use of multiple Shape in initialize. Wanted to know how to use multiple inputs. But no problem. When I get to that point I will be back with more questions 8-)