Maximize the margin formula in support vector machines algorithm

Question

I was recently reading about support vector machines and how they work and I stumbled on an article and came across Maximize the distance margin.

Can anyone tell me what do we have to minimize here? I wasn't able to understand this part I pasted below. What is $w$ and $m$ here in the formulas given below?

Maximize the margin

For the sake of simplicity, we will skip the derivation of the formula for calculating the margin, m which is
$$m = frac{2}{|vec{w}|}$$

The only variable in this formula is w, which is indirectly proportional to m, hence to maximize the margin we will have to minimize ||w||. This leads to the following optimization problem:

$$min_{(vec{w}, b)} frac{|vec{w}|^2}{2}$$

subject to $$y_i(vec{w}cdotvec{x}+b) ge 1, forall i = 1, ldots, n$$

The above is the case when our data is linearly separable. There are many cases where the data can not be perfectly classified through linear separation. In such cases, Support Vector Machine looks for the hyperplane that maximizes the margin and minimizes the misclassifications.

Link to the article : Hackerearth

Siong Thye Goh · Accepted Answer

The separating hyperplane that we are looking for is $w^Tx+b=0$. That is $w$ is the coefficient or the slope of the separating hyperplane and $m$ is the margin.

By scaling, for the positive class, we want the closest data point (support vectors) to satisfies $w^Tx+b=1$ and for the negative data class, we want the closest data point to satisfies $w^Tx+b=-1$.

Recall from geometry class, (a $3$ dimensional case is discussed here) that the distance from a point $y$ from a plane $w^Tx+b=0$ is $$frac{|w^Ty+b|}{|w|}.$$

Hence as a result, the distance of the closest point to the hyperplane would be $frac{|pm1|}{|w|}=frac{1}{|w|}$ and hence, the margin, that is the closest distance between the two classes would be $frac2{|w|}$.

We want to maximize the margin, $frac2{|w|}$ which is equivalent to minimize $frac{|w|}2$ which is equivalent to minimize $frac{|w|^2}2$ subject to the constraints that we classify the points correctly.

Maximize the margin formula in support vector machines algorithm

One Answer

Add your own answers!

Ask a Question