Performance analysis of ML detection in MIMO systems with co-channel interference

In this letter, we analyse a receiver technique for MIMO systems with flat fading channel, in which the Co-Channel Interference (CCI) along with white Gaussian noise is modelled as correlated Gaussian noise. This correlated Gaussian noise is whitened and then Maximum likelihood detection (MLD) is performed over the desired user's streams. An upper bound on the bit error rate (BER) is derived to understand the performance of this approach. Using the upper bound, we show that the covariance matrix of the interference has to be rank deficient for the BER to reduce with SNR. Monte Carlo simulations are performed to verify the diversity order predicted by the upper bound. © 2011 IEEE.