The caption command in an algorithm block gives this error: Missing number, treated as zero

Quite a contrived error. It happens that

SetKwData{M}{m}

not only defines M, but also an internal macro called @M which is already preempted by the LaTeX kernel. This redefinition happens with no check for @M already being defined.

In the LaTeX kernel, @M stands for 10000 and is used in several places for the value of a penalty that prohibits line or page breaks.

Use

SetKwData{MM}{m}

and MM in the body of the algorithm; or choose an altogether different name.

If you feel bold, you can fix the wrong strategy of algorithm2e; also SetKw suffers from the same problem.

documentclass{article}
usepackage{algorithm2e}

makeatletter
%%% fix the wrong code in algorithm2e
renewcommand{SetKwData}[2]{%
  algocf@newcommand{algocf@kwdata@#1}[1]{DataSty{#2(}ArgSty{##1}DataSty{)}}%
  algocf@newcommand{#1}{%
    @ifnextcharbgroup{csname algocf@kwdata@#1endcsname}{DataSty{#2}xspace}}%
  }%
renewcommand{SetKw}[2]{%
        algocf@newcommand{algocf@kw@#1}[1]{KwSty{#2} ArgSty{##1}}
        algocf@newcommand{#1}{@ifnextcharbgroup{csname algocf@kw@#1endcsname}{KwSty{#2}xspace}}%
}%
makeatother

begin{document}

begin{algorithm}[H]
SetKwInput{Var}{Variables}
SetKwInput{Alg}{Algorithm}
SetKwData{D}{d}SetKwData{Eps}{eps}SetKwData{Min}{minPts}
SetKwData{M}{m}SetKwData{N}{n}SetKwData{Dist}{dist}
SetKwData{Indices}{indices}SetKwData{Classn}{class_no}
SetKwData{Nei}{neighbors}SetKwData{Neic}{neighbor_count}
SetKwData{Core}{core_neig}SetKwData{Class}{class}
SetKwData{Point}{point}

SetKwFunction{Distance}{distance}
SetKwFunction{Find}{find}
SetKwFunction{Count}{count}
SetKwFunction{Check}{check_core_neighbour}

 KwIn{
 D: the dataset
 Eps: the neighbourhood distance 
 Min: the minimum number of points
 }
 KwOut{Discovered outliers and clusters }
 Var{
 M, N: row and column values of D matrix, respectively
 Dist: distance vector
 Indices: indices that distance of points is lower than Eps
 Classn: indicates the clusters - default 1}
BlankLine

Alg{
    import the dataset into D
    For{$i leftarrow 1$ KwTo M}{
        Dist $leftarrow Distance(i, D)$
        Nei$leftarrow$Find(Dist$leq$Eps)
        Neic$leftarrow$Count(Nei)
        Core$leftarrow$Check(Nei)
    
        uIf{(Neic$geq$Min)}{
            Class($i$)$leftarrow$Classn
            While{more points near $i$}{
                Class(Point)=Classn
            }
            Classn$leftarrow$Classn $+1$
        }
        uElseIf{(Neic$<$Min$&&$Core$==$true)}{
            Class($i$)$leftarrow 0$tcc*[r]{border point}
        }
        uElseIf{(Neic$<$Min)}{
            Class($i$)$leftarrow -1$tcc*[r]{outlier point}
        }
    }
    KwRet{Class}
}
BlankLine
caption{DBSCAN protectcite{ccelik2011anomaly}}
label{alg:db}
end{algorithm} 

end{document}

Using a more complex prefix instead of the simple @ will avoid such problems.