Covariance between sample mean of two simple random samples

Suppose that we have two simple random samples without replacement $A$ and $B$ from a population $P$ of size $N$ such that $B = P-A$ where $n$ is the size of $A$ and $m$ is the size of $B$.

I want to find the covariance between $bar x_A, bar x_B$

I have done the following:

$Cov(bar x_A, bar x_B) = Cov(frac{1}{n} sum_{i=1}^n X_i, frac{1}{m} sum_{j=1}^m X_j) =frac{1}{nm} sum_{i=1}^n sum_{j=1}^m Cov(X_i,X_j) = frac{1}{nm} sum_{i=1}^N sum_{j=1}^N Cov(X_i I_i ,X_j I_j)$

where $I_i, I_j$ are the indicator functions

$ = frac{1}{nm} sum_{i=1}^N sum_{j=1}^N X_i X_j Cov(I_i ,I_j)$

And this is where I do not know how to proceed.