Data Science Asked by deepsnow on February 5, 2021
I’m currently investigating the paper FCNN: Fourier Convolutional Neural
Networks. The main contribution of the paper is that CNN training is entirely shifted to the Fourier domain without loss of effectiveness. The proposed architecture looks as follows:
The authors state that the implementation was done in keras, however, it is not publicly available. I know I can define a Fourier transformation in the following way:
model.add(layers.Lambda(lambda v: tf.real(tf.spectral.rfft(v))))
But this is not a Fourier Convolution, right? How should I go from here?
An FFT-based convolution can be broken up into 3 parts: an FFT of the input images and the filters, a bunch of element-wise products followed by a sum across input channels, and then an IFFT of the outputs (Source).
Or as it is written in the paper:
So, for a Fourier Convolution Layer you need to:
input_fft = tf.spectral.rfft2d(input)
Take each kernel and transform it to the Fourier domain:
weights_fft = tf.spectral.rfft2d(layer.get_weights())
Note: The Fourier domain "images" for the input and the kernels need to be of the same size.
Perform element-wise multiplication between the input's Fourier transform and Fourier transform of each of the kernels:
conv_fft = keras.layers.Multiply(input_fft, weights_fft)
layer_output = tf.spectral.irfft2d(conv_fft)
Note: I used pseudo-code, it will probably needs some tuning for it to actually to work.
Answered by Mark.F on February 5, 2021
Get help from others!
Recent Questions
Recent Answers
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP