Why does my model sometimes not learn well from same data?

I have a dataset of 2 classes, both containing 2K images. I have split that into 1500 images for training and 500 images for validation.

This is a simple structure for testing purposes, and each image is classified depending on the colour of a particular pixel. Either Green or Red.

I have run this model many times and I find that sometimes the models gets low loss/ high accuracy within a few epochs, but other times it gets stuck at accuracy 50%.

The datasets are exactly the same each time with only difference coming from model.fit “shuffle” option.

I tested the LR Range first:

and I “cycle” the learning rate through an appropriate range.

model = keras.Sequential([
        keras.layers.Dense(112, activation=tf.nn.relu, input_shape=(224, 224, 3)),
        keras.layers.Dropout(0.4),
        keras.layers.Flatten(),
        keras.layers.Dense(3, activation=tf.nn.softmax)
    ])


LRS = CyclicLR(base_lr=0.000005, max_lr=0.0003, step_size=200.)

model.fit(train_images, train_labels, shuffle=True, epochs=10,
              callbacks=[checkpoint,
                         LRS],
              validation_data = (test_images, test_labels)
              )

Why does the model sometimes NOT get a good fit?

EDIT 1

Re Serali’s suggestion:

myOpt = keras.optimizers.Adam(lr=0.001,decay=0.01)
model.compile(optimizer=myOpt, loss='categorical_crossentropy',  metrics=['accuracy'])

reduce_lr = keras.callbacks.ReduceLROnPlateau(monitor='val_loss', factor=0.2,verbose=1,patience=5, min_lr=0.00001)