Is there a word for one side in a pair?

Until we find a more complicated esoteric word that matches your requirements, try "complement" for a while.

Bifurcate:

— vb 1. to fork or divide into two parts or branches

Branch:

12. to divide into separate parts or subdivisions; diverge: The main road branches off to the left. 13. to expand or extend, as business activities: The bank has plans to branch throughout the state. verb (used with object) 14. to divide into branches or sections.

Moiety:

Moiety Moi"e*ty (moi"[-e]*t[y^]), n.; pl. {Moieties} (moi"[-e]*t[i^]z). [F. moiti['e], L. medietas, fr. medius middle, half. See {Mid}, a., and cf. {Mediate}, {Mediety}.] 1. One of two equal parts; a half; as, a moiety of an estate, of goods, or of profits; the moiety of a jury, or of a nation. --Shak. [1913 Webster]

Given that a pair is often referred to by the use of the Greek derived "dyad" you could use "monad" (also derived from the Greek) for a single part of the pair (although monad doesn't specifically describe part of a pair as much as it does a single unit of any kind).

http://www.thefreedictionary.com/dyad

http://www.thefreedictionary.com/monad

I'm going with "counterpart", as suggested to me off-site. Thanks for the feedback though!

This is one of those occasions when one realizes the unique abilities of each language. English with most number of words among the world languages, does not have a word for each member of a pair. We, hence, use the word "pair" (as in "where is the other pair?") when we mean a member of the pair. In Persian we do not have this problem as there is a word for members of the dyad. The pair or couple is "Joft" (جفت) and each member of that pair is a "lengeh" (لنگه.) Persian also have a word for a mismatched pair: "lengeh be lengeh!"

when

Cugel uses "part" to describe their response, https://english.stackexchange.com/a/164938/227352, so why not just use "part" to describe part of a pair? Everyone knows a pair is only comprised of two objects, so it should be obvious that there could be only two parts.

Odd (Oxford Online Dictionary, def 3): As in the odd glove. Half of a pair.

Partner, as in "howdy partner," based on the latin partitio, "partition"

How about mate, which implies pairing to one other thing, etc, or to a group?

3 : one of a pair: such as

a : either member of a couple and especially a married couple

b : either member of a breeding pair of animals

c : either of two matched objects

https://www.merriam-webster.com/dictionary/mate