Distributed Source Coding

Introduction    DSC using Convolutional Codes    DSC Using Turbo Codes     Other DSC Work

(Last modified November 2001. For more recent work, refer to the Wyner-Ziv Coding of Video webpage)

Introduction

    Consider a communication system with two correlated signals, X and Y (see Figure 1).  Assume that X and Y come from two separate sources that cannot communicate with each other, that is, the signals are encoded independently or are “distributed”.  The receiver, on the other hand, can see both encoded signals and can perform joint decoding.  A sensor system composed of low-complexity spatially separated sensor nodes, sending correlated information to a central processing receiver, is an example of such system. What is the minimum encoding rate required such that X and Y can still be recovered perfectly?

    If a joint encoder and a joint decoder are used, the minimum combined rate for probability of decoding error to approach zero is simply the joint entropy H(X,Y).  Surprisingly, as proven by Slepian and Wolf in 1973 [1], a combined rate of H(X,Y) is sufficient even if the correlated signals are encoded separately.  According to the Slepian-Wolf coding theorem, the achievable rate region for distributed sources X and Y is given by 
 
        
        
        
 
    The Slepian-Wolf theorem is an encouraging conceptual basis for distributed source coding but until today the theoretical limits have not yet been achieved, nor have been closely approached, by practical applications.  In [6], Pradhan and Ramchandran demonstrate the intuition behind Slepian-Wolf coding by providing a solution for X and Y separated by a maximum Hamming distance.  Pradhan and Ramchandran also developed the DISCUS (Distributed Source Coding Using Syndromes) method [6][7].  In DISCUS, instead of sending the codeword representing X, the syndrome of the codeword coset is sent and the receiver decodes by choosing the codeword in the given coset that is closest to Y.  DISCUS, with trellis and lattice encoding, was used in their study to encode correlated Gaussian sources.  However, it was not shown that the techniques clearly approached the information-theoretic bound.  The objective of this project is to find practical coding techniques that can perform close to the Slepian-Wolf achievable rate region and would be effective for different types of X and Y correlation.