AI-Enhanced Video Coding: Advancements and Results

Explore the latest developments in AI-enhanced video coding, including MPAI-EVC Evidence Project, MPEG5-EVC, deep learning enhancements, reference schema, quantization parameter results, super resolution techniques, test combined results, and performance analysis on various sequences. Witness the future potential of AI in video coding.

• Video coding
• AI enhancement
• MPAI-EVC
• Deep learning
• Super resolution

faar Follow

Uploaded on Oct 04, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. AI-Enhanced Video Coding Roberto Iacoviello, 2023 March 31 T10:40 &20:40 1 4-Oct-24

2. Video data volume increases rapidly MPAI- EVC Evidence Project Mainstream video coding standards The MPAI Enhanced Video Coding mission 2 4-Oct-24

3. Starting point is MPEG5-EVC Essential Video Coding Royalty-free Boost EVC with AI-tools 3 4-Oct-24

4. Traditional block-based hybrid framework 4 4-Oct-24

5. Deep learning Intra enhancement Context is sent to an autoencoder The autoencoder returns an enhanced Intra prediction Autoencoder output replaces the DC predictor inconditionally Context Predictor The bitstream can be decoded 5 4-Oct-24

6. Reference schema O3 D0 D1 Predictor ENC DEC ABS D2 Context Predictor 6 4-Oct-24

7. Results with Quantisation Parameter QP [22-42] 7 4-Oct-24

8. Super resolution Starting point: Densely Residual Laplacian Super Resolution 8 4-Oct-24

9. Test combined results 4K-resolution SR-enhanced 4K-resolution SR-enhanced 4K-resolution bicubic-enhanced vs. vs. DRLN-refined SR DRLN-pretrained SR Bicubic SR HD-resolution NN-Intra enhanced HD-resolution NN- Intra enhanced HD-resolution native anchor 9 4-Oct-24

10. Combined results Intra+SR NNIntra+DRLN pre-trained NNIntra+DRLN refined Sequences Name BD-Rate [%] BD-PSNR [dB] BD-Rate [%] BD-PSNR [dB] Campfire -3.36 0.11 -11.36 0.39 CatRobot -4.51 0.15 -18.36 0.64 DaylightRoad 2 -8.92 0.21 -22.51 0.54 FoodMarket 4 -5.46 0.22 -9.46 0.39 ParkRunning 3 -0.75 0.03 -26.41 1.08 Tango 2 -4.16 0.12 -11.33 0.33 AVG -4.53 0.14 -16.57 0.56 10 4-Oct-24

11. Next combined tests Test Test Test Test refined Vs. trained from scratch Test random patches Vs. importance sampling Test patches Vs full frame 11 4-Oct-24

12. In-loop Starting point: A Deep Learning Approach for Multi-Frame In-Loop Filter of HEVC Image extraction is performed before EVC filtering A neural network is used for image enhancement The enhanced image is integrated into the EVC pipeline Partitioning information is retrieved from the EVC process BD-rate calculation is currently in progress 12 4-Oct-24

13. Inter Prediction Starting point: Affine Transformation-Based Deep Frame Prediction by Hyomin Choi , Member, IEEE, and Ivan V. Baji , Senior Member, IEEE We are able to run this software under Ubuntu pytorch environment The next step is to integrate the software with the EVC project 13 4-Oct-24

14. Combined results Intra+SR+Inloop Future Plans and Perspectives Integration of Inter enhancement into EVC Next tool 14 4-Oct-24

15. Join MPAI Share the fun Build the future! We look forward to your participation in this exciting project! https://mpai.community/ 15 4-Oct-24