TransWikia.com

Calculate Hessian of a "weird" function

Mathematics Asked by Ben Schneider on November 12, 2021

let be $L $ an invertible matrix, $b in mathbb{R}^{n}$ and $mu in mathbb{R}$

and $J: mathbb{R}^{n} rightarrow mathbb{R}$ defined as $J(x)=|L x|_{2}^{2}+mu x^{t} x-b^{t} x$

How do I calculate its hessian matrix?
I cant operate as a usual function and calculate its gradient and also i dont know what to do with that norm

Progress:

$|L x|_{2}^{2} =sum_{i=1}^{n}sum_{j=1}^{n} (l_{i,j}cdot x_{j})^2$

$mu x^{t}cdot x =sum_{i=1}^{n} mu cdot x_{i}^2$

$-b^{t} x = – sum_{i=1}^{n} b_{i} cdot x_{i}$

so $J(x)=\sum_{i=1}^{n}sum_{j=1}^{n} (l_{i,j}cdot x_{j})^2+sum_{i=1}^{n} mu cdot x_{i}^2 – sum_{i=1}^{n} b_{i} cdot x_{i} $

so the gradient is

$nabla J(x)=(sum_{i=1}^{n} l_{i,1} + 2 cdot mu cdot x_{1} – b_{1}, …, sum_{i=1}^{n} l_{i,n} + 2 cdot mu cdot x_{n} – b_{n} ) $

and the Hessian is the diagonal matrix of $2 cdot mu$

Dunno if it is right and I also think there has to be an easier way of doing it, like operating with the vectors and matrix instead of deriving each variable on their own.

FINAL EDIT:

As some comments sugested(thank you all) i can write the function as :
$J(x)=x^tL^tLx + mu x^tx -b^tx$

So:

$ nabla J(x)= 2(L^tL+Imu)x-b$

and

$HJ(x)=2LL^t+2 I mu$

2 Answers

Since the Hessian is the matrix of second partial derivatives $frac{partial^2}{partial x_i partial x_j}$, the linear term will vanish. You can rearrange the second order term to:

$$(Lx)^t(Lx) + mu x^tx = x^t(L^tL + mu I)x.$$

Each second derivative will then select one of the coefficients of this second-degree homogeneous polynomial. Can you see your way to the end of the computation from here?


As for thinking of this in terms of matrices, think first of the derivative. Every linear function can be written as $p_1(x) = v^Tx$ for some vector $v$. The gradient of a smooth function $f$ at a point is defined as this vector for the best linear approximation to $f$ at that point.

Every homogeneous second-degree polynomial $p$ can be written as $$ p(x) = x^THx $$ for some symmetric matrix $H$. The Hessian of a smooth function $f$ at a point is defined as this matrix for the best second-order approximation to $f$ at that point.

So now you see the root of the strategy to isolate the second order term.

Answered by Neal on November 12, 2021

Hint When $h to 0$, we have begin{equation} J(x + h) = J(x) + text{linear terms} + |L h|_2^2 + mu h^t h end{equation}

Answered by Gribouillis on November 12, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP