Drastic drop in Somers' D ? Why?

Say you have a set of pairs $(x_i,y_i)$, Somers D is linked to Kendall's tau, which is defined by :

$$tau=frac{N_C-N_D}{n(n-1)/2}$$

Where $N_C$ is the number of concordant pairs (ranks of both elements agree, $x_i>x_j$ and $y_i>y_j$ or $x_i<x_j$ and $y_i<y_j$) and $N_D$ the umber of pairs that are not. First it is possible that there might be a problem with the special case where $x_i=x_j$ or $y_i=y_j$, of which you have a lot of examples. This case might be handled differently depending on your implementation. Overall it quantifies the number of difference between the number of concordant and discordant pairs. By construction Kendall's tau is symetric in X and Y. (And this is probably what you wanted to use.)

Somers D, treat the variables differently. Somers D of $Y$ with respect to $X$ is defined as : $$D(Y,X)=tau(X,Y)/tau(X,X)$$

That is it quantifies the difference between the number of concordant and discordant pairs, divided by the number of pairs with $X$ values in the pair being unequal. You can see that it is not symmetric. It explain that there is a difference between your cases.

As to why there is such a difference : the main explaination would be a bout what is your "reference" in one case, if you set B as a reference, it is about 2 weak people (among 4) that A believe are strong, if you set A as the reference, the disagreement seems about 2 (among 38) strong people that B believe are weak.(2/4 is 50%, 2/38 is near 95%, not sure where the discrepency come from exactly)