SIMG-714 INFORMATION THEORY FOR IMAGING SYSTEMS

I. Course:

1.1. Four (4) credit hours

1.2. Four (4) lecture hours per week

1.3. Prerequisites: M.S. Imaging Science core; an introduction to probability or consent of instructor.

II. Course Catalog Description:

The course develops a basic understanding of the efficient representation of information for storage and transmission. Classical concepts of information theory are developed and applied to image compression, storage and transmission. The intent is to develop a foundation for the efficient handling of image-based information in imaging systems.

III. Course Objective:

3.1. Understanding of information measures

3.2. Understanding of source coding techniques based on symbol sequences

3.3. Understanding of storage and communication channel models

3.4. Understanding of transformation, quantization and coding for compression

3.5. Understanding of error control techniques for storage and communications

IV Course Outline:

4.1. Efficient representation of discrete symbol sequences. Instantaneous codes for IID sources. Kraft’s inequality, McMillan’s inequality and Huffman codes.
    1. The classical measure of information and its relationship to entropy. The relationship of entropy to efficient source coding.
    2. Mutual information and channel capacity
    3. Limits on information transmission rate. Shannon’s channel coding theorem discussed via the law of large numbers.
    4. Basic error control techniques using linear block codes.

4.6. Waveform and image coding by use of linear prediction to reduce sample redundancy.

4.7. Image transformations for the reduction of redundancy and match to perceptual factors. Emphasis on DCT.

4.8. Examples of transform-based lossy compression (JPEG).

V Instructional Techniques:
    1. Lectures
    2. Modeling using computer tools

VI. Evaluation:

6.1 Exams

6.2. Computer assignments

6.3. Homework assignments

6.4. Class participation

VII. Bibliography:

7.1. Journal papers

7.2. Instructor’s notes