How to change the joining code?

Consider using ArrayFlatten.

A={{1,2},{2,3}};
B={{2,1},{3,3}};
DD={{4,5},{6,37}};
EE={{12,11},{13,14}};
I2=IdentityMatrix[2];

The result is composed below.

ArrayFlatten[{{KroneckerProduct[I2,A],
    KroneckerProduct[Transpose[B],I2]},  
  {KroneckerProduct[I2 ,DD],
    KroneckerProduct[Transpose[EE],I2]}}]

and evaluates to :

{{1,2,0,0,2,0,3,0},
 {2,3,0,0,0,2,0,3},
 {0,0,1,2,1,0,3,0},
 {0,0,2,3,0,1,0,3},.    
 {4,5,0,0,12,0,13,0},
 {6,37,0,0,0,12,0,13},
 {0,0,4,5,11,0,14,0},
 {0,0,6,37,0,11,0,14}}

Hope this helps.