ECE 563 Information Theory Course Overview
Get all relevant information about the ECE 563 Information Theory course taught by Instructor Olgica Milenkovic and TA Rebecca Golm at the University of Illinois. Learn about course requirements, format, exams, and textbook. Remember, no cheating is allowed, and email policy guidelines are in place for better communication.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
ECE 563 INFORMATION THEORY Instructor: Olgica Milenkovic TA: Rebecca Golm LOGISTICS HISTORY AND INTRODUCTORY MATERIAL
ALL RELEVANT COURSE INFORMATION IS AVAILABLE AT THE WEBSITE: https://courses.engr.illinois.edu/ece563/fa2022/ COURSE REQUIREMENT: ECE 313 OR EQUIVALENT (BASIC PROBABILITY THEORY)
ALL RELEVANT COURSE INFORMATION IS AVAILABLE AT THE WEBSITE: https://courses.engr.illinois.edu/ece563/fa2022/ COURSE REQUIREMENT: ECE 313 OR EQUIVALENT (BASIC PROBABILITY THEORY) COURSE FORMAT: ROUGHLY BIWEEKLY HOMEWORK, ONE MIDTERM, ONE FINAL, PROJECT PRESENTATIONS
ALL RELEVANT COURSE INFORMATION IS AVAILABLE AT THE WEBSITE: https://courses.engr.illinois.edu/ece563/fa2022/ COURSE REQUIREMENT: ECE 313 OR EQUIVALENT (BASIC PROBABILITY THEORY) COURSE FORMAT: ROUGHLY BIWEEKLY HOMEWORK, ONE MIDTERM, ONE FINAL, PROJECT PRESENTATIONS MIDTERM SOMETIME MID OCTOBER (WILL ASK FOR FEEDBACK) FINAL EXAM TIME DETERMINED BY SCHOOL FINAL PROJECT PRESENTATION 2-3 DAYS AFTER CLASSES END (WILL ASK FOR FEEDBACK)
ALL RELEVANT COURSE INFORMATION IS AVAILABLE AT THE WEBSITE: https://courses.engr.illinois.edu/ece563/fa2022/ PLEASE, NO CHEATING! RULES ARE STRICT AND I HAVE TO FILE A FAIR REPORT. THIS STAYS ON YOUR RECORD AND WILL BE REPORTED TO YOUR ADVISOR. IT IS ALSO UNFAIR AND INAPPROPRIATE BEHAVIOUR THAT SHOULD NOT BE TOLERATED.
ALL RELEVANT COURSE INFORMATION IS AVAILABLE AT THE WEBSITE: https://courses.engr.illinois.edu/ece563/fa2022/ PLEASE, NO CHEATING! RULES ARE STRICT AND I HAVE TO FILE A FAIR REPORT. THIS STAYS ON YOUR RECORD AND WILL BE REPORTED TO YOUR ADVISOR. IT IS ALSO UNFAIR AND INAPPROPRIATE BEHAVIOUR THAT SHOULD NOT BE TOLERATED. EMAIL POLICY: WE HAVE PLENTY OF OFFICE HOURS AND I AM AVAILABLE AFTER THE LECTURE FOR QUESTIONS. PLEASE DO NOT SEND EMAILS 2-3 PAGES LONG ASKING FOR EXPLANATIONS. THESE ARE HARD TO READ AND EASY TO MISUNDERSTAND. ALSO, BOTH INSTRUCTOR AND TA ARE VERY BUSY AND HAVE TO HAVE FREE TIME FOR RESEARCH, SERVICE, DEPARTMENTAL WORK ETC.
ALL RELEVANT COURSE INFORMATION IS AVAILABLE AT THE WEBSITE: https://courses.engr.illinois.edu/ece563/fa2022/ COURSE TEXTBOOK: COVER AND THOMAS, Information Theory, AND MANY ONLINE RESOURCES
ALL RELEVANT COURSE INFORMATION IS AVAILABLE AT THE WEBSITE: https://courses.engr.illinois.edu/ece563/fa2022/ COURSE TEXTBOOK: COVER AND THOMAS, Information Theory, AND MANY ONLINE RESOURCES Information theory has been used with spectacular success in communication theory, for storage systems of all kinds, for quantization and compression, error control coding, in biology and physics. It has very close ties with statistics, complexity theory, machine learning, quantum, statistical and black hole physics, molecular computing etc. But one approach does not work for all areas and modifications are required.
ALL RELEVANT COURSE INFORMATION IS AVAILABLE AT THE WEBSITE: https://courses.engr.illinois.edu/ece563/fa2022/ TOPICS TO BE COVERED: GENERAL INTRODUCTORY MATERIAL WHAT IS INFORMATION WHERE IS IT USED CLASSICAL INFORMATION THEORY INFORMATION MEASURES AND AXIOMATIC APPROACHES TO DEFINING ENTROPY SHANNON, RENYI AND OTHER TYPES OF ENTROPY MUTUAL INFORMATION, ENTROPY RATES FANO S INEQUALITY DATA PROCESSING INEQUALITY METHOD OF TYPES ASYMPTOTIC EQUIPARTITION PROPERTY
ALL RELEVANT COURSE INFORMATION IS AVAILABLE AT THE WEBSITE: https://courses.engr.illinois.edu/ece563/fa2022/ TOPICS TO BE COVERED: SOURCE CODING LOSSLESS COMPRESSION KRAFT S INEQUALITY, HUFFMAN, ADAPTIVE HUFFMAN, ARITHMETIC CODING THE LEMPEL-ZIV ALGORITHM THE BURROWS-WHEELER TRANSFORM RATE-DISTORTION THEORY QUANTIZATION IF TIME PERMITS, COMPRESSION VIA NN GENOMIC DATA COMPRESSION
ALL RELEVANT COURSE INFORMATION IS AVAILABLE AT THE WEBSITE: https://courses.engr.illinois.edu/ece563/fa2022/ TOPICS TO BE COVERED: CHANNEL CODING CHANNEL CAPACITY THE CHANNEL CODING THEOREM CAPACITY OF THE GAUSSIAN NOISE CHANNEL HAMMING CODES CODES ON GRAPHS (LOW-DENSITY PARITY-CHECK CODES) POLAR CODES (MARTINGALES)
ALL RELEVANT COURSE INFORMATION IS AVAILABLE AT THE WEBSITE: https://courses.engr.illinois.edu/ece563/fa2022/ TOPICS TO BE COVERED: JOINT SOURCE-CHANNEL CODING JOINT SOURCE-CHANNEL CODING THEOREM SLEPIAN-WOLF CODING
ALL RELEVANT COURSE INFORMATION IS AVAILABLE AT THE WEBSITE: https://courses.engr.illinois.edu/ece563/fa2022/ TOPICS TO BE COVERED: CONSTRAINED CODING CAPACITY OF CONSTRAINED CODING SYSTEMS CAPACITY OF GRAPHS (PENTAGON) RUNLENGTH LIMITED CODES IF TIME PERMITS, SPECTRAL NULL CODES
ALL RELEVANT COURSE INFORMATION IS AVAILABLE AT THE WEBSITE: https://courses.engr.illinois.edu/ece563/fa2022/ TOPICS TO BE COVERED: INFORMATION THEORY AND STATISTICS KULBACK-LEIBLER DIVERGENCE LARGE DEVIATION THEORY ELEMENTS OF ESTIMATION THEORY, FISHER INFORMATION MINIMUM DESCRIPTION LENGTH PRINCIPLE
ALL RELEVANT COURSE INFORMATION IS AVAILABLE AT THE WEBSITE: https://courses.engr.illinois.edu/ece563/fa2022/ TOPICS TO BE COVERED: SPECIAL TOPICS KOLMOGOROV COMPLEXITY INFORMATION THEORY APPROACHES BEHIND MOLECULAR STORAGE BASICS OF QUANTUM INFORMATION THEORY INFORMATION THEORY AND STATISTICAL PHYSICS OTHER TOPICS UPON REQUEST
ALL RELEVANT COURSE INFORMATION IS AVAILABLE AT THE WEBSITE: https://courses.engr.illinois.edu/ece563/fa2022/ https://plus.maths.org/content/information-baby-steps https://plus.maths.org/content/information-surprise
ALL RELEVANT COURSE INFORMATION IS AVAILABLE AT THE WEBSITE: https://courses.engr.illinois.edu/ece563/fa2022/ https://www.britannica.com/science/information-theory
