Expectation value of commutator $langle p|[hat{x},hat{p}]|prangle$

Well, as we were taught in kindergarten, it is wrong to assume that $infty - infty = 0$.

begin{align} plangle pverthat{x}vert prangle - plangle pverthat{x}vert prangle &= ppartial_pdelta (p-p)-ppartial_pdelta (p-p) &= pdelta' (0)-pdelta'(0) &= infty - infty end{align}

But, importantly, your second calculation is also mistaken. Notice that $langle pvert qrangle=delta(p-q)$ and thus, $langle pvert ihbarvert prangle=ihbardelta (0)neq ihbar$. Notice that this is perfectly expected because the identity operator is $delta (p-q)$ not $delta_{pq}$.

---

All of this "craziness" arises because the eigenstates of the position and the momentum operators are not normalizable. They are not in the Hilbert space, they only provide a useful basis for the states in the Hilbert space. A famous argument that closely relates to your first calculation is that $[hat{x},hat{p}]$ cannot be $ihbar$ because we know that the trace of a commutator must vanish. This argument does not work because we cannot define trace for infinite-dimensional Hilbert spaces, or, as the learned like to say, the position and the momentum operators are not "trace-class". See Trace of a commutator is zero - but what about the commutator of $x$ and $p$?.