Copying Weights using NetTrain in Mathematica

Question

I would like to implement the PPO algorithm - Reinforcement Learning - in Mathematica. And for that, I need to be able to copy network weights from say subnetwork NN1 to subnetwork NN2 as an update and as a part of the training process.
Is doing this possible in Mathematica? maybe my question can be split into two parts:

how can I copy weights?

how can I do that as a part of the training process, i.e. using NetTrain?

Alexey Golyshev · Answer

RNNs have ports "State" (all) and "CellState" (LSTM only).
NetGraph[
 {
  BasicRecurrentLayer[8],
  BasicRecurrentLayer[8]
  },
 {
  NetPort[1, "State"] -> NetPort[2, "State"]
  }
 ]

With ConvolutionLayer and LinearLayer it's more complicated. They have no ports other than "Input" and "Output". We can copy and paste weights. But I do not know how to do this in NetTrain without breaking the training loop.
conv = ConvolutionLayer[1, {3, 3}, "Input" -> {1, Automatic, Automatic}] // NetInitialize;
w = NetExtract[conv, "Weights"];
b = NetExtract[conv, "Biases"];
convNew = ConvolutionLayer[1, {3, 3}, "Weights" -> w, "Biases" -> b]

Copying Weights using NetTrain in Mathematica

One Answer

Add your own answers!

Ask a Question