TransWikia.com

Backpropagating across convolution layers with fixed kernels in Tensorflow

Data Science Asked by Zachary Miller on February 18, 2021

Background: I am currently working with volumetric data where I feed one 2D slice from the volume into a denoising CNN with learnable kernel weights and calculate a slice-wise denoising loss, and then pass the remaining 2D slices from the volume into a fixed version of this denoising CNN with fixed weights (essentially I am copying the learnable denoising CNN in this fixed version during training using tf.nn.conv2d where a kernel can be fixed). I then concatenate these denoised slices into a volume, and use this volume to calculate a volume-wise loss. These losses are then summed into a final loss where gradients are then taken.

Problem: I want both gradients from the slice wise and volume-wise losses to propagate back to the learnable kernel weights in the denoising CNN. It appears that only the slice-wise loss propagates back to the learnable kernel weights, and not the volume-wise loss. In other words it appears adding the fixed version of the denoising CNN effectively disconnects the graph.

My Attempted Workaround: Sharing Weights -I pass all the slices through the learnable CNN using a for loop, but still only use one slice to learn the loss. I then concateante the denoised slices into the volume, and calculate a data-consistency loss. This fixes the backpropagation problem, but appears to be really memory intensive. It seems like even with sharing weights in a for loop in tensorflow, for N iterations, I get seperate N convolutions rather than passing N pieces of data into one convolution.

Question 1: With respect to sharing weights workaround,Is there a way to use for loops in tensorflow where instead of unrolling to N convolutions, I can pass N pieces of data into one convolution

Question 2: With respect to the fixed convolution approach, Is there a way to effectively reconnect the graph and backpropagate across fixed convolutions to the learnable layers?

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP