MathOverflow Asked on November 3, 2021
In a lot of computational math, operations research, such as algorithm design for optimization problems and the like, authors like to use $$langle cdot, cdot rangle$$ as opposed to $$(cdot)^T (cdot)$$
Even when the space is clearly Euclidean and the operation is clearly the dot product. What is the benefit or advantage for doing so? Is it so that the notations generalize nicely to other spaces?
Update: Thank you for all the great answers! Will take a while to process…
Lots of great answers so far, but I'll add another (hopefully at least good) answer: the notation $v^T u$ makes it somewhat difficult to speak of collections of bilinear pairings depending on a parameter. Typical examples:
These are all difficult to express using the transpose notation. The closest you can get is to write, for instance $v^T A_i u$ where $A_i$ is a family matrices, but particularly when one is speaking of continuously varying families of inner products you run into all sorts of difficult issues with coordinate systems, and it becomes very difficult to keep things straight.
Answered by Paul Siegel on November 3, 2021
I consider the distinction quite important. There are two separate operations that look superficially like each other but are in fact different.
First, the abstract description. If $V$ is an abstract vector space and $V^*$ is its dual, then there is the natural evaluation operation of $v in V$ and $theta in V^*$, which is commonly written as $$ langletheta,vrangle = langle v,thetarangle $$ No inner product is needed here. If you choose a basis $(e_1, dots, e_n)$ of $V$ and use the corresponding dual basis $(eta^1, dots, eta^n)$ of $V^*$ and write $v = v^ie_i$ and $theta = theta_ieta^i$, then $$ langletheta,vrangle = theta_iv^i. $$ The distinction between up and down indices indicates whether the object is a vector or a dual vector ($1$-form).
If $V$ has an inner product and $(e_1, e_n)$ is an orthonormal basis, then given two vectors $v = v^ie_i, w = w^ie_i in V$, then $$ vcdot w = v^iw^i $$ Notice that here both indices are up. There is a similar formula for the dot product of two dual vectors. Here, the formula only works if the basis is orthonormal.
How does this look in terms of row and column vectors? My personal convention, a common one, is the following:
ASIDE: I gotta say that having such precisely defined conventions is crucial to my ability to do nontrivial calculations with vectors and tensors. When I was a graduate student, my PhD advisor, Phillip Griffiths, once asked me, "Have you developed your own notation yet?" I also have to acknowledge that my notation is either exactly or based closely on Robert Bryant's notation.
Answered by Deane Yang on November 3, 2021
Maybe it's worth mentioning that the computer language APL has a "generalized" inner product where you can use any two functions of two arguments (i.e., "dyadic functions" in APL terms) to form an inner product. Thus, for example, ordinary inner product is written as "A+.xB", which can apply to two arrays A, B of any dimension whatsoever (vectors, matrices, three-dimensional arrays, etc.), provided that the last dimension of A matches the first dimension of B.
Thus, for example, A^.=B represents string matching of A against B, Ax.*B evaluates a number given its prime divisors A and prime factorization exponents B, etc.
The authors of APL, Iverson and Falkoff, cared intensely about notation and tried to find the most general interpretation of every new item they added to the language.
Answered by Jeffrey Shallit on November 3, 2021
Mathematical notation in a given mathematical field $X$ is basically a correspondence $$ mathrm{Notation}: { hbox{well-formed expressions}} to { hbox{abstract objects in } X }$$ between mathematical expressions (or statements) on the written page (or blackboard, electronic document, etc.) and the mathematical objects (or concepts and ideas) in the heads of ourselves, our collaborators, and our audience. A good notation should make this correspondence $mathrm{Notation}$ (and its inverse) as close to a (natural) isomorphism as possible. Thus, for instance, the following properties are desirable (though not mandatory):
To evaluate these sorts of qualities, one has to look at the entire field $X$ as a whole; the quality of notation cannot be evaluated in a purely pointwise fashion by inspecting the notation $mathrm{Notation}^{-1}(C)$ used for a single mathematical concept $C$ in $X$. In particular, it is perfectly permissible to have many different notations $mathrm{Notation}_1^{-1}(C), mathrm{Notation}_2^{-1}(C), dots$ for a single concept $C$, each designed for use in a different field $X_1, X_2, dots$ of mathematics. (In some cases, such as with the metrics of quality in desiderata 1 and 7, it is not even enough to look at the entire notational system $mathrm{Notation}$; one must also consider its relationship with the other notational systems $widetilde{mathrm{Notation}}$ that are currently in popular use in the mathematical community, in order to assess the suitability of use of that notational system.)
Returning to the specific example of expressing the concept $C$ of a scalar quantity $c$ being equal to the inner product of two vectors $u, v$ in a standard vector space ${bf R}^n$, there are not just two notations commonly used to capture $C$, but in fact over a dozen (including several mentioned in other answers):
It is not a coincidence that there is a lot of overlap and similarity between all these notational systems; again, see desiderata 1 and 7.
Each of these notations is tailored to a different mathematical domain of application. For instance:
And so on and so forth. There is no unique "best" choice of notation to use for this concept; it depends on the intended context and application domain. For instance, matrix notation would be unsuitable if one does not want the reader to accidentally confuse the scalar product $u^T v$ with the rank one operator $vu^T$, Hilbert space notation would be unsuitable if one frequently wished to perform coordinatewise operations (e.g., Hadamard product) on the vectors and matrices/linear transformations used in the analysis, and so forth.
(See also Section 2 of Thurston's "Proof and progress in mathematics", in which the notion of derivative is deconstructed in a fashion somewhat similar to the way the notion of inner product is here.)
ADDED LATER: One should also distinguish between the "one-time costs" of a notation (e.g., the difficulty of learning the notation and avoiding standard pitfalls with that notation, or the amount of mathematical argument needed to verify that the notation is well-defined and compatible with other existing notations), with the "recurring costs" that are incurred with each use of the notation. The desiderata listed above are primarily concerned with lowering the "recurring costs", but the "one-time costs" are also a significant consideration if one is only using the mathematics from the given field $X$ on a casual basis rather than a full-time one. In particular, it can make sense to offer "simplified" notational systems to casual users of, say, linear algebra even if there are more "natural" notational systems (scoring more highly on the desiderata listed above) that become more desirable to switch to if one intends to use linear algebra heavily on a regular basis.
Answered by Terry Tao on November 3, 2021
One advantage of $langle cdot, cdot rangle$ is that you don't have to worry about changes in basis.
Suppose we have a coordinate system $alpha$ in which our (real) inner product space is explicitly Euclidean, and an alternative coordinate system $beta$. A vector $v$ is expressed in the coordinates systems as, respectively, the column vectors $[v]_alpha$ and $[v]_beta$. Let $P$ denote the change of basis matrix
$$ [v]_beta = P [v]_alpha $$
The inner product, which in coordinate system $alpha$ is $langle v, vrangle = [v]_{alpha}^T [v]_{alpha}$ is certainly not in general $[v]_beta^T[v]_beta$ in the second coordinate system. (It is only so if $P$ is orthogonal.)
That said: given any Hilbert space $V$, by Riesz-representation there exists an (anti-)isomorphism from $V$ to its dual space $V^*$. You can certainly choose to call this mapping $v mapsto v^*$ (in Riemannian geometry contexts this is more usually denoted using the musical isomorphism notation $flat$ and $sharp$) and I don't think in this case there are reasons to prefer one to another. But a major caveat if you do things this way is that unless you are working in an orthonormal basis, you cannot associate $v mapsto v^*$ to the "conjugate transpose" operation on matrices.
Answered by Willie Wong on November 3, 2021
The family $F$ of (real) quadratic polynomials is a vector space isomorphic to the vector space $mathbb{R}^3.$ One way to make $F$ an inner product space is to define $langle f, g rangle =int_a^bf(t)g(t),dt$ for some fixed interval $[a,b].$ Instead of quadratic polynomials one might consider all polynomials or all bounded integrable functions. One could also define the inner product as $langle f, g rangle =int_a^bf(t)g(t)mu(t)dt$ for some weight function $mu.$ There isn’t a natural role for transposes here.
Answered by Aaron Meyerowitz on November 3, 2021
One huge advantage, to my mind, of the bracket notation is that it admits 'blanks'. So one can specify the notation for an inner product as $langle , rangle$, and given $langle , rangle : V times V rightarrow K$, one can define elements of the dual space $V^star$ by $langle u , - rangle$ and $langle -, v rangle$. (In the complex case one of these is only conjugate linear.)
More subjective I know, but on notational grounds I far prefer to write $langle Au, v rangle = langle u, A^dagger v rangle$ for the adjoint map than $(Au)^t v = u^t (A^tv)$. The former also emphasises that the construction is basis independent. It generalises far better to Hilbert spaces and other spaces with a non-degenerate bilinear form (not necessarily an inner product).
I'll also note that physicists, and more recently anyone working in quantum computing, have taken the 'bra-ket' formulation to the extreme, and use it to present quite intricate eigenvector calculations in a succinct way. For example, here is the Hadamard transform in bra-ket notation:
$$ frac{| 0 rangle + |1 rangle}{sqrt{2}} langle 0 | + frac{| 0 rangle - |1rangle}{sqrt{2}} langle 1 |. $$
To get the general Hadamard transform on $n$ qubits, just taken the $n$th tensor power: this is compatible with the various implicit identifications of vectors and elements of the dual space.
Finally, may I issue a plea for everyone to use $langle u ,v rangle$, with the LaTeX langle
and rangle
rather than the barbaric $<u,v>$.
Answered by Mark Wildon on November 3, 2021
Inner product is defined axiomatically, as a function from $Vtimes Vto k$, where $k$ is a field and $V$ is a $k$-vector space, satisfying the three well-known axioms. The usual notation is $(x,y)$. So when you want to say anything about an arbitrary inner product, you use this notation (or some similar one). $(x,y)=x^*y$ is just one example of an inner product on the space $mathbb C^n$. There are other examples on the same space, $(x,y)=x^*Ay$ where $A$ is an arbitrary Hermitian positive definite matrix, and there are dot products on other vector spaces.
Answered by Alexandre Eremenko on November 3, 2021
I do not see a compelling argument for $langle cdot, cdot rangle$ over $(cdot)^T(cdot)$, or, better $(cdot)^*(cdot)$, so that the star operator can be generalized to other more complicated settings (complex vectors, Hilbert spaces with a dual operation).
Let me summarize the arguments in the comments:
An advantage of the $u^*v$ notation, in my view, it that it makes clear that some properties are just a consequence of associativity. Consider for instance the orthogonal projection on the orthogonal space to $u$
$$Pv = (I-uu^*)v = v - u(u^*v).$$
If one writes it as $v - langle v,u rangle u$ (especially by putting the scalar on the left as is customary), it is less clear that it is equivalent to applying the linear operator $I-uu^*$ to the vector $v$. Also, the notation generalizes nicely to repeated projections $$ (I-u_1u_1^* - u_2u_2^*)v = (I - begin{bmatrix}u_1 & u_2end{bmatrix}begin{bmatrix}u_1^* \ u_2^*end{bmatrix})v = (I - UU^*)v. $$
A disadvantage, of course, is working with spaces of matrices, where transposes already have another meaning; for instance, working with the trace scalar product $langle A,B rangle := operatorname{Tr}(A^TB)$ one really needs the $langle A,B rangle$ notation.
Answered by Federico Poloni on November 3, 2021
This is to expand on my comment in response to Federico Poloni:
$langle u,vrangle $ is explicitly a number, whereas $u^Tv$ is a 1 by 1 matrix :).
While it is true that there is a canonical isomorphism between the two, how do you write the expansion of $u$ in an orthonormal base ${v_i}$? Something like $$ u=sum_i u^Tv_i v_i $$ feels uncomfortable as if you view everything as matrices, the dimensions do not allow for multiplication. So, I would at least feel a need to insert parentheses, $$ u=sum_i (u^Tv_i) v_i, $$ to indicate that the canonical isomorphism is applied. But that is still vague-ish while already cancelling any typographical advantages of $u^Tv$.
(I do also share the sentiment that the basis-dependent language is inferior and should be avoided when possible.)
Answered by Kostya_I on November 3, 2021
Get help from others!
Recent Answers
Recent Questions
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP