Future Video Coding Technology and Applications

Future video coding led by Prof. Dr.-Ing. Jörn Ostermann focuses on the constant need for improved compression, catering to evolving infrastructure and device demands. It envisions advancements in functionalities like high dynamic range and augmented reality to support emerging markets in existing and new sectors. The relevant bodies, including ISO/IEC and ITU-T, collaborate on standards such as MPEG-4 AVC and HEVC to enhance video coding capabilities. The exploration of Joint Exploration Model (JEM) aims to improve compression beyond HEVC through experimental open-source software.

• Video Coding
• Future Technology
• Compression
• Emerging Markets
• Standards

cochrane_m Follow

Uploaded on Oct 06, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. Future Video Coding Prof. Dr.-Ing. J rn Ostermann Institut f r Informationsverarbeitung

2. Video Coding Technology Improvements Infrastructure Device Constant need for improved compression More devices, more pixels, more frames, more user-generated content 50% rate reduction in 10 years Constant need for additional functionality High dynamic range, wide color gamut, head mounted displays, augmented reality, virtual reality Prof. Dr-Ing- J rn Ostermann ostermann@tnt.uni-hannover.de 2

3. Video Coding New usages: Panorama mono Prof. Dr-Ing- J rn Ostermann ostermann@tnt.uni-hannover.de 3

4. Video Coding New usages: Panorama stereo Prof. Dr-Ing- J rn Ostermann ostermann@tnt.uni-hannover.de 4

5. Use Cases for Future Video Codec Existing Markets Terrestrial, and Satellite broadcasting, cable services, managed IPTV services, surveillance New Markets Over-the-top (OTT) services, User generated content, video conferencing, gaming Emerging Markets Augmented and Virtual reality, multicast streaming/broadcast; media centric internet of things Prof. Dr-Ing- J rn Ostermann ostermann@tnt.uni-hannover.de 5

6. Requirements on Future Video Codec Source material 8k, 0Hz, 120 Hz, WCG, HDR, 10bit Rectangular, panorama, projected / viewports Natural, synthetic Monoscopic, multiview Network interface Streaming, error resilience, scalability, low delay Emerging Markets Augmented and Virtual reality, multicast streaming/broadcast; media centric internet of things Prof. Dr-Ing- J rn Ostermann ostermann@tnt.uni-hannover.de 6

7. Relevant Bodies ISO/IEC JTC1/SC29/WG11 Motion Picture Experts Group (MPEG) ITU-T Study Group 16 with Q6/16 (VCEG) Common work between MPEG and VCEG MPEG-4 AVC (ITU-T H.264, ISO/IEC 14496-10) HEVC (ITU-T H.265, ISO/IEC 23008-2) MV-HEVC, 3D-HEVC as extensions of HEVC Joint Video Experts Team (JVET) Future Video Coding Prof. Dr-Ing- J rn Ostermann ostermann@tnt.uni-hannover.de 7

8. JEM where we are Experimental open source software Joint Exploration Model (JEM) Intended to investigate potential for better compression beyond HEVC Open source code available for download https://jvet.hhi.fraunhofer.de/ Documentation openly and freely available for download: http://phenix.it-sudparis.eu/jvet/ HEVC software plus additional or improved compression tools General architecture similar to HEVC Hybrid closed-loop, high level syntax, reference picture management, etc. Improved/new compression tools Prof. Dr-Ing- J rn Ostermann ostermann@tnt.uni-hannover.de 8

9. Overall: Hybrid Coding / Closed Loop Input Video Signal General Control General Control Data Transform, Scaling & Quantization - Quantized Transform Coefficients Split into CTUs Scaling & Inverse Transform Coded Bitstream Header Formatting & CABAC Intra Prediction Data Intra-Picture Estimation Filter Control Filter Control Data Intra-Picture Prediction Deblocking & SAO Filters Motion Data Motion Compensation Intra/Inter Selection Output Video Signal Decoded Picture Buffer Motion Estimation Prof. Dr-Ing- J rn Ostermann ostermann@tnt.uni-hannover.de 9 Source: Prof. Dr.-Ing. Jens Ohm

10. JEM Results compared to HEVC (HM) Measured average rate reduction for same PSNR (Bjontegaard delta method) Average encoder and decoder run time Random Access Over HM-16.16 (sequential, gcc6.2) U V -33.1% -35.2% -45.2% -38.2% -36.9% -31.7% -31.2% -33.6% -30.0% -31.5% -35.4% -33.9% Y EncT 10.7x 7.8x 8.8x 11.3x 11.5x 9.9x DecT 5.6x 6x 6.3x 7.8x 11.7x 7.2x Class A1 Class A2 Class B Class C Class D Overall -27.2% -36.3% -27.0% -26.3% -26.0% -28.5% Prof. Dr-Ing- J rn Ostermann ostermann@tnt.uni-hannover.de 10 Source: Prof. Dr-Ing. Jens Ohm

11. JEM HEVC AV1 Basketball Drive (Random Access) 40 39 38 37 PSNR [dB] 36 35 34 33 32 0 2000 4000 6000 8000 Bit rate [kbps] 10000 12000 14000 16000 18000 AV1 JEM HM Prof. Dr-Ing- J rn Ostermann ostermann@tnt.uni-hannover.de 11

12. Call for Evidence results Joint Call for Evidence on Video Compression with Capability beyond HEVC (April 2017) Test cases with eight 4K/SDR, five HD/SDR, five HD/HDR, and five 360-degree video sequences 4 rate points each Results of Joint Call for Evidence on Video Compression with Capability beyond HEVC (July 2017) Evaluates JEM anchors and other proposals against HEVC Main 10 Expert viewing tests with double-stimulus approach, allows grading on an MOS (mean opinion score) scale Prof. Dr-Ing- J rn Ostermann ostermann@tnt.uni-hannover.de 12 Source: Prof. Dr.-Ing. Jens Ohm

13. Call for Evidence results Test cases SDR / UHD 4K, MOS over rate [kbps 5 5 HM JEM PA PB HM JEM PA PB 4 4 3 3 2 2 1 1 0 1000 2000 3000 4000 5000 6000 7000 8000 0 2000 4000 6000 8000 10000 12000 "Daylight Road" "Campfire Party" Source: JVET-G1004 Prof. Dr-Ing- J rn Ostermann ostermann@tnt.uni-hannover.de 13 Source: Prof. Dr.-Ing. Jens Ohm

14. Call for Evidence results Test cases HDR / HD, MOS over rate [kbps] 5 5 HM JEM PC PD HM JEM PC PD 4 4 3 3 2 2 1 1 0 500 1000 1500 2000 2500 0 500 1000 1500 2000 2500 3000 3500 "Hurdles" "Cosmos" Source: JVET-G1004 Prof. Dr-Ing- J rn Ostermann ostermann@tnt.uni-hannover.de 14 Source: Prof. Dr.-Ing. Jens Ohm

15. Call for Evidence results Test cases 360 Video, dynamic viewports, MOS over rate [kbps] 5 5 HM JEM PE PF PG HM JEM PE PF PG 4 4 3 3 2 2 1 1 0 1000 2000 3000 4000 5000 6000 7000 8000 0 500 1000 1500 2000 2500 3000 3500 Source: JVET-G1004 "Kite Flite" "Harbour" Prof. Dr-Ing- J rn Ostermann ostermann@tnt.uni-hannover.de 15 Source: Prof. Dr.-Ing. Jens Ohm

16. Next Steps Joint Call for Proposals on Video Compression with Capability beyond HEVC Test cases with 4K/SDR, HD/SDR, HD/HDR, 360-degree video 4 rate points each HM anchors and JEM anchors Formal subjective assessment in labs Timeline: CfP October 2017 Submissions by February 2018, subjective testing March/April Evaluation of proposals: April 2018 Next generation standard by end of 2020 Prof. Dr-Ing- J rn Ostermann ostermann@tnt.uni-hannover.de 16 Source: Prof. Dr.-Ing. Jens Ohm

17. Conclusions Future Video Coding will focus on: Rectangular 10 bit WCG HDR UHD video Future Video Coding will enable: Support for panoramic video Edge-based rendering for interactivity, games Future Video coding will deliver in 2020: Coding efficiency for UHD at 50% of the rate of HEVC for the same quality UHD at typically less than 4MBit/s MPEG continues to deliver compression technology beyond the state of the art Prof. Dr-Ing- J rn Ostermann ostermann@tnt.uni-hannover.de 17