"concat" mode can only merge layers with matching output shapes except for the concat axis

Question

I have a function I am trying to debug which is yielding the following error message:

ValueError: “concat” mode can only merge layers with matching output
shapes except for the concat axis. Layer shapes: [(None, 128, 80,
256), (None, 64, 80, 80)]

I’m running a kernel from a Kaggle competition called Dstl Satellite Imagery Feature Detection (kernel available here)

This is the function where I am having issues merging a list of tensors into a single tensor:

 def get_unet():
    inputs = Input((8, ISZ, ISZ))
    conv1 = Convolution2D(32, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(inputs)
    conv1 = Convolution2D(32, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(conv1)
    pool1 = MaxPooling2D(pool_size=(2, 2), dim_ordering="th")(conv1)

    conv2 = Convolution2D(64, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(pool1)
    conv2 = Convolution2D(64, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(conv2)
    pool2 = MaxPooling2D(pool_size=(2, 2), dim_ordering="th")(conv2)

    conv3 = Convolution2D(128, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(pool2)
    conv3 = Convolution2D(128, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(conv3)
    pool3 = MaxPooling2D(pool_size=(2, 2), dim_ordering="th")(conv3)

    conv4 = Convolution2D(256, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(pool3)
    conv4 = Convolution2D(256, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(conv4)
    pool4 = MaxPooling2D(pool_size=(2, 2), dim_ordering="th")(conv4)

    conv5 = Convolution2D(512, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(pool4)
    conv5 = Convolution2D(512, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(conv5)

    up6 = merge([UpSampling2D(size=(2, 2), dim_ordering="th")(conv5), conv4], mode='concat', concat_axis=1)
    conv6 = Convolution2D(256, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(up6)
    conv6 = Convolution2D(256, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(conv6)

    up7 = merge([UpSampling2D(size=(2, 2), dim_ordering="th")(conv6), conv3], mode='concat', concat_axis=1)
    conv7 = Convolution2D(128, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(up7)
    conv7 = Convolution2D(128, 3, 3, activation='relu', border_mode='same')(conv7)

    up8 = merge([UpSampling2D(size=(2, 2), dim_ordering="th")(conv7), conv2], mode='concat', concat_axis=1)
    conv8 = Convolution2D(64, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(up8)
    conv8 = Convolution2D(64, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(conv8)

    up9 = merge([UpSampling2D(size=(2, 2), dim_ordering="th")(conv8), conv1], mode='concat', concat_axis=1)
    conv9 = Convolution2D(32, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(up9)
    conv9 = Convolution2D(32, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(conv9)

    conv10 = Convolution2D(N_Cls, 1, 1, activation='sigmoid', dim_ordering="th")(conv9)

    model = Model(input=inputs, output=conv10)
    model.compile(optimizer=Adam(), loss='binary_crossentropy', metrics=[jaccard_coef, jaccard_coef_int, 'accuracy'])
    return model

I’m running keras with a TensorFlow backend. My thoughts are that there are some compatibility issues with versions of software (i.e. the original code is over a year old). Or, perhaps I need to reshape the data somehow.

What might be causing this error?

This is the full error:

Traceback (most recent call last):

  File "<ipython-input-1-e8f13915ac9b>", line 1, in <module>
    runfile('/Users/aaron/temp/tmp/kaggle_dstl_v3.py', wdir='/Users/aaron/temp/tmp')

  File "/Users/aaron/anaconda3/envs/kaggle-dstl-env/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 705, in runfile
    execfile(filename, namespace)

  File "/Users/aaron/anaconda3/envs/kaggle-dstl-env/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 102, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "/Users/aaron/temp/tmp/kaggle_dstl_v3.py", line 513, in <module>
    model = train_net()

  File "/Users/aaron/temp/tmp/kaggle_dstl_v3.py", line 416, in train_net
    model = get_unet()

  File "/Users/aaron/temp/tmp/kaggle_dstl_v3.py", line 294, in get_unet
    up8 = merge([UpSampling2D(size=(2, 2), dim_ordering="th")(conv7), conv2], mode='concat', concat_axis=1)

  File "/Users/aaron/anaconda3/envs/kaggle-dstl-env/lib/python3.6/site-packages/keras/legacy/layers.py", line 458, in merge
    name=name)

  File "/Users/aaron/anaconda3/envs/kaggle-dstl-env/lib/python3.6/site-packages/keras/legacy/layers.py", line 111, in __init__
    node_indices, tensor_indices)

  File "/Users/aaron/anaconda3/envs/kaggle-dstl-env/lib/python3.6/site-packages/keras/legacy/layers.py", line 191, in _arguments_validation
    'Layer shapes: %s' % (input_shapes))

ValueError: "concat" mode can only merge layers with matching output shapes except for the concat axis. Layer shapes: [(None, 128, 80, 256), (None, 64, 80, 80)]

convolutional neural network deep learning kaggle keras tensorflow

Djib2011 · Accepted Answer

First of all, you are correct that your code is old as some functions being used are deprecated (e.g. Convolution2D is now Conv2D see here).

However, the error clearly states that you are trying to concatenate two tensors that their dimensions do not match. When concatenating two tensors along a specific axis, all other dimensions except the one being concatenated must be the same. In your case, you are trying to concatenate along axis=1, but the last dimension is different (256 for the first tensor and 80 for the second).

I would recommend basing your U-Net code on a newer implementation (link1, link2, link3).

These implementations replace your

merge([UpSampling2D(size=(2, 2), dim_ordering="th")(conv5), conv4], mode='concat', concat_axis=1)

layers with

up_conv5 = UpSampling2D(size=(2, 2), data_format="channels_last")(conv5)
ch, cw = get_crop_shape(conv4, up_conv5)
crop_conv4 = Cropping2D(cropping=(ch,cw), data_format="channels_last")(conv4)
up6   = concatenate([up_conv5, crop_conv4], axis=concat_axis)`

which also crops the height and the width of the previous layer to perform the concatenation correctly (the shape of the crop is determined through an auxiliary function).

"concat" mode can only merge layers with matching output shapes except for the concat axis

One Answer

Add your own answers!

Ask a Question