Accuracy assessment (confusion matrix) of image classification from Landsat imagery in SAGA GIS

I have classified Landsat images in two different classes, namely pervious and impervious, in SAGA GIS, using training sample in polygon shapes. This training sample has only one column of attribute table comprising of class value of each polygon (value = 1 for impervious and 2 for impervious). The total of polygon shapes is 65.

I created a new test sample for evaluating the accuracy of the classification result. The test sample also has attribute value with one column specifying the class value of each polygon. The total of polygon shapes is 45.

I tried to do the evaluation using Confusion Matrix (Polygons/Grid) module from SAGA GIS.
I specified the grid as the result of the classification and the polygons are from the test samples.
The result of the confusion matrix is shown in this figure:

I am struggling to understand this confusion matrix.
There are around 46000 features are assessed in the evaluation shown in the table.
I am wondering what are these features?
What method I should use to evaluate my classification? Should I just manually check my test samples instead whether it s correctly classified or not?

PS: This is the grid system of the classified Landsat image, and the classification is done using random forest classifier.