# Communication Theory

The basis of Information Technology and Communications (IT&C) is the processing and transmission of information. Information theory places well-defined limits on what is possible in terms of our everyday use of information, while communication theory defines principles upon which practical communication systems are designed. In popular terms, information theory tells us how well we can theoretically do and communication theory tells us how we can practically do it.

Communication and information theory are the theories of modern digital communication systems, where "digital" means that we are transmitting information as symbols (or numbers) from a finite alphabet (or limited set of numbers). Although physical signals are continuous waveforms in time, the principles of communication theory, allows us to consider the continuous waveforms we are transmitting and receiving over a noisy and interfering communication channel (a telephone cable or the radio waves propagation of a mobile phone antenna) as a digital system, randomly perturbing the information that we are transmitting.

The translation of physically transmitted and received electrical signals into an equivalent digital system, transmitting and receiving digital numbers represents the very heart of communication theory. This, in turn, allows for advanced signal processing techniques to be applied in transmitters and receivers, leading to the design of increasingly more efficient digital communication systems, closer and closer to fundamental limits.

## A Historical Perspective of Digital Communications

The telegraph |

It is remarkable that the earliest form of electrical communication,
namely telegraphy developed by Samuel Morse in 1837, was a digital
communication system. Although Morse was responsible for the
development of the first electrical digital communication system, the
beginnings of what we now regard as modern digital communications stem
from the work of Nyquist in 1924. His studies led him to conclude that
for binary data transmission (transmitting one of two numbers, 0 or 1)
over a noiseless channel of bandwidth *W* Hertz, the maximum
pulse rate is 2*W* pulses per second.

N. Wiener |

Hartley extended this work in 1928 to non-binary data transmission, while Kolmogorov and Wiener independently in 1939 and 1942, respectively, solved the problem of optimally estimating a signal in the presence of additive noise. In 1948 Claude Shannon established the mathematical foundation for information transmission and derived fundamental limits for digital communication systems. His work can arguably be considered as the true beginning of the information age.

V. Kotelnikov |

Another important contribution to the field of digital communication is the work of Kotelnikov in 1947, who provided a coherent analysis and consequently a principle for optimal design of such systems. His work was later extended by Wozencraft and Jacobs in 1965, leading to the principles used to design the communication systems of today.

The work of Hamming in 1950 on error control coding to combat detrimental effects of channel noise completes the classic contributions to modern digital communication systems.

C. Berrou |

A. Viterbi |

Of more modern contributions, the Viterbi decoding algorithm for trellis codes, proposed by Andrew Viterbi in 1967 is now found in almost all wireless communication systems. Efficient error control decoding makes mobile communication systems what they are today. The latest significant leap forward for improvements of communications systems was in 1993 with the discovery of the "turbo principle" by Berrou and Glavieux. The special turbo codes developed based on these principles can be efficiently decoded using a very powerful iterative signal processing approach. The resulting coding system performs very close to fundamental limits for a range of different channels. In practical terms, this leads to the most efficient use of bandwidth and power, which is very important for portable wireless devices.

## Modern Digital Communication Systems

A modern communication system is traditionally modelled as shown in the figure below. As illustrated in the figure, the current paradigm for digital communications systems is to separate the various functions of the system. For example, source coding and channel coding is done separately where source coding removes inherent source redundancy, while channel coding adds control redundancy to combat interference introduced over the channel. Based on this design paradigm, the signal processing required for different functionalities in the system are designed separately and applied sequentially in a concatenated fashion.

The figure below shows a typical system diagram of a modern digital communications system. You can click on the various components to learn about their function.

## Some Key Research Challenges

As mentioned, the current paradigm for digital communications networks is to separate the various functions of the network. However information theory indicates that a more coordinated system design may be more efficient. This means that joint design of multiple functionalities across multiple layers in the communication protocol stack, with corresponding joint receiver processing may lead to more efficient communications networks. The problem is that joint processing in many cases leads to a prohibiting level of processing complexity. A key challenge is to find processing strategies that approach optimal performance, but has an implementable level of complexity. The iterative paradigm applied in turbo coding may be a way forward.

Another key challenge is introduced by the development of turbo coding. Now it is possible to communicate close to fundamental limits, leading to very power-efficient communication. In popular terms, we can now in principle build decoders that can communicate at signal levels which are virtually below the inherent noise level in the receiver electronics. Communicating at such low signal-to-noise levels makes it a very difficult challenge to synchronize and estimate channel parameters necessary for the signal processing required prior to decoding.

Specific projects may include:

- Joint cross-layer communication protocol design;
- Efficient communication protocols for ad-hoc networks;
- Efficient system design for high speed data transmission over highly mobile channels;
- Synchronization and channel estimation at very low signal-to-noise ratio;
- Optimal receiver structures for unknown channels;
- Efficient system design for wireless packet data transmission systems with random multiple access.

## More Information

- InformationTheory.net site promoting Information Theory in Australia
- IEEE Information Theory Society
- IEEE Communications Society
- The history of communications
- The Samuel F.B. Morse Historic Site
- Harry Nyquist biography
- In memory of A. N. Kolmogorov
- The history of Norbert Wiener
- A Mathematical Theory of Communication
- The history of R. W. Hamming
- The Error Control Code Site, by Robert Morelos-Zaragoza. A page with programs implementing many different coding schemes as well as many links to other coding sites.
- An Introduction to the Analysis of Iterative Coding Systems, by Thomas Richardson and Rudiger Urbanke.
- Turbo Code Site at Ecole Nationale Superieure des Telecommunications de Bretagne (birthplace of turbo codes).

## Australian Communications Theory Researchers

Note: You can search for ACoRN Members using the Member Search facility