Binary classification using images and an external dataset

Question

I currently have a project in which I must create a binary classifier to detect defective products. I have image data which has already been labeled (each part has been labeled as a pass or fail), as well as an external dataset which has specific measurements for each image (it contains parameters which could not be accurately captured by a camera, such as temperature, length of a particular piece, etc.). I have achieved decent accuracy by applying basic classification algorithms such as logistic regression and support vector machines (in scikit-learn) to only this external dataset, but I would like to incorporate the image data to improve accuracy. I have read about training a CNN on the images, but I do not know how to incorporate the external dataset to train the model (in fact, I don't even know if this is possible). Is there any way to use both the image data as well as the external dataset to classify these images? Any help is much appreciated.

Djib2011 · Answer

Yes you can most certainly combine the two ways of information in CNNs.
Let's take a look at a CNN:

Roughly speaking, CNNs consist of two parts:

one that has convolutional layers and optinally pooling ones
and one that has fully connected layers.

The first is used for extracting features from the image, while the second is used for using these extracted features to classify the image. The last part essentially is like a regular fully-connected Neural Network.
If you want to add more information to the CNN, which is not an image, you can add this to the first fully connected layer of the network. This way this information bypasses the feature extraction step (which is relevant only for imaging data) and is used only for classification.

Binary classification using images and an external dataset

One Answer

Add your own answers!

Ask a Question