Skip to content

WebRTC HEVC RFC 7798 RTP Payload Format Implementation - #15494

Merged
webkit-commit-queue merged 1 commit into
WebKit:mainfrom
xingri:eng/WebRTC-HEVC-RFC-7798-RTP-Payload-Format-Implementation
Sep 6, 2023
Merged

WebRTC HEVC RFC 7798 RTP Payload Format Implementation#15494
webkit-commit-queue merged 1 commit into
WebKit:mainfrom
xingri:eng/WebRTC-HEVC-RFC-7798-RTP-Payload-Format-Implementation

Conversation

@xingri

@xingri xingri commented Jul 3, 2023

Copy link
Copy Markdown
Contributor

93eb48d

WebRTC HEVC RFC 7798 RTP Payload Format Implementation
https://bugs.webkit.org/show_bug.cgi?id=258794

Reviewed by Youenn Fablet.

The current WebRTC HEVC is using generic packetization instead of RFC 7789 Packetization.
So this is about implementing the RFC 7798 Packetization.

Fix HEVC depacketizer issues. (#185)
Enalbing low latency mode for RTC (#169)
Enable HEVC support. (#165)
Fix out-of-bounds write in H265VpsSpsPpsTracker (#163)
Apply fix bitstream logic to RtpVideoStreamReceiver2 (#142)
Add missing CODEC_H265 switch case (#136)
Add HEVC support for iOS/Android (#68)
H265 packetization_mode setting fix (#53)
Add H.265 QP parsing logic (#47)

This patch is extracted from following Open WebRTC Toolkit code changes:
<open-webrtc-toolkit/owt-deps-webrtc#185>
<open-webrtc-toolkit/owt-deps-webrtc#169>
<open-webrtc-toolkit/owt-deps-webrtc#165>
<open-webrtc-toolkit/owt-deps-webrtc#163>
<open-webrtc-toolkit/owt-deps-webrtc#142>
<open-webrtc-toolkit/owt-deps-webrtc#136>
<open-webrtc-toolkit/owt-deps-webrtc#68>
<open-webrtc-toolkit/owt-deps-webrtc#53>
<open-webrtc-toolkit/owt-deps-webrtc#47>

co-authoured by:
taste1981 <jianlin.qiu@intel.com>
jianjunz <jianjun.zhu@intel.com>
Cyril Lashkevich <notorca@gmail.com>
Piasy <xz4215@gmail.com>
ShiJinCheng <874042641@qq.com>
Andreas Unterhuber <andreas.unterhuber@keepinmind.info>
dong-heun <63987238+dong-heun@users.noreply.github.com>

* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.xcconfig:
* Source/ThirdParty/libwebrtc/Source/webrtc/api/video/video_codec_type.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/api/video_codecs/video_codec.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/api/video_codecs/video_codec.h:
(webrtc::VideoCodecH265::operator!= const):
* Source/ThirdParty/libwebrtc/Source/webrtc/api/video_codecs/video_decoder_software_fallback_wrapper.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/api/video_codecs/video_encoder.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/api/video_codecs/video_encoder.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/call/rtp_payload_params.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/common_video/h265/h265_common.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/common_video/h265/h265_common.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/common_video/h265/h265_pps_parser.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/common_video/h265/h265_pps_parser.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/common_video/h265/h265_sps_parser.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/common_video/h265/h265_sps_parser.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/common_video/h265/h265_vps_parser.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/common_video/h265/h265_vps_parser.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/logging/rtc_event_log/rtc_event_log_parser.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/media/base/media_constants.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/media/base/media_constants.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/media/engine/internal_decoder_factory.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/rtp_rtcp/source/create_video_rtp_depacketizer.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/rtp_rtcp/source/rtp_format.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/rtp_rtcp/source/rtp_format_h265.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/rtp_rtcp/source/rtp_format_h265.h:
(webrtc::RtpPacketizerH265::PacketUnit::PacketUnit):
(webrtc::VideoRtpDepacketizerH265::~VideoRtpDepacketizerH265): Deleted.
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/rtp_rtcp/source/rtp_sender_video.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/rtp_rtcp/source/rtp_video_header.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/video_coding/codecs/h265/include/h265_globals.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/video_coding/encoded_frame.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/video_coding/h265_vps_sps_pps_tracker.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/video_coding/h265_vps_sps_pps_tracker.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/video_coding/include/video_codec_interface.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/video_coding/packet_buffer.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/rtc_base/experiments/min_video_bitrate_experiment.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/nalu_rewriter.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/nalu_rewriter.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/test/scenario/video_stream.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/test/video_codec_settings.h:
(webrtc::test::CodecSettings):
* Source/ThirdParty/libwebrtc/Source/webrtc/video/config/video_encoder_config.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/video/config/video_encoder_config.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/video/encoder_overshoot_detector.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/video/rtp_video_stream_receiver2.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/video/rtp_video_stream_receiver2.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/video/send_statistics_proxy.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/video/video_stream_encoder.cc:
* Source/ThirdParty/libwebrtc/libwebrtc.xcodeproj/project.pbxproj:

Canonical link: https://commits.webkit.org/267677@main

6c3a4cf

Misc iOS, tvOS & watchOS macOS Linux Windows
✅ 🧪 style ✅ 🛠 ios ✅ 🛠 mac ✅ 🛠 wpe ✅ 🛠 wincairo
✅ 🛠 ios-sim ✅ 🛠 mac-AS-debug ✅ 🧪 wpe-wk2
✅ 🧪 webkitperl ✅ 🧪 ios-wk2 ✅ 🧪 api-mac ✅ 🛠 gtk
✅ 🧪 ios-wk2-wpt ✅ 🧪 mac-wk1 ✅ 🧪 gtk-wk2
✅ 🧪 api-ios ✅ 🧪 mac-wk2 ✅ 🧪 api-gtk
✅ 🛠 tv ✅ 🧪 mac-AS-debug-wk2
✅ 🛠 tv-sim
✅ 🛠 🧪 merge ✅ 🛠 watch
✅ 🛠 watch-sim

@webkit-early-warning-system

webkit-early-warning-system commented Jul 3, 2023

Copy link
Copy Markdown
Collaborator

@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Jul 3, 2023
@Ahmad-S792 Ahmad-S792 added the WebRTC For bugs in WebRTC label Jul 3, 2023
@xingri
xingri force-pushed the eng/WebRTC-HEVC-RFC-7798-RTP-Payload-Format-Implementation branch from bbfa1ab to ac7d27f Compare July 3, 2023 20:16
@webkit-early-warning-system

webkit-early-warning-system commented Jul 3, 2023

Copy link
Copy Markdown
Collaborator

@Ahmad-S792 Ahmad-S792 removed the merging-blocked Applied to prevent a change from being merged label Jul 3, 2023
@xingri
xingri force-pushed the eng/WebRTC-HEVC-RFC-7798-RTP-Payload-Format-Implementation branch from ac7d27f to 88e68c2 Compare July 6, 2023 22:16
@webkit-early-warning-system

webkit-early-warning-system commented Jul 6, 2023

Copy link
Copy Markdown
Collaborator

@aproskuryakov
aproskuryakov requested a review from youennf July 8, 2023 05:21
Comment thread Source/ThirdParty/libwebrtc/Source/webrtc/api/video_codecs/video_codec.h Outdated

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is negative Key-Frame interval possible ?

Comment thread Source/ThirdParty/libwebrtc/Source/webrtc/api/video_codecs/video_encoder.cc Outdated

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for initializing bool types, it is better to use "true" or "false" instead of "0".

Comment thread Source/ThirdParty/libwebrtc/Source/webrtc/common_video/h265/h265_pps_parser.cc Outdated
Comment thread Source/ThirdParty/libwebrtc/Source/webrtc/api/video_codecs/video_encoder.cc Outdated
@webkit-early-warning-system

Copy link
Copy Markdown
Collaborator

Starting EWS tests for 4782a1e. Live statuses available at the PR page, #15494

@webkit-early-warning-system

webkit-early-warning-system commented Jul 21, 2023

Copy link
Copy Markdown
Collaborator

@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Jul 21, 2023
@Ahmad-S792 Ahmad-S792 removed the merging-blocked Applied to prevent a change from being merged label Jul 21, 2023
@webkit-ews-buildbot webkit-ews-buildbot added the merging-blocked Applied to prevent a change from being merged label Jul 22, 2023
@xingri
xingri force-pushed the eng/WebRTC-HEVC-RFC-7798-RTP-Payload-Format-Implementation branch from 4782a1e to 6c3a4cf Compare July 23, 2023 23:27
@webkit-early-warning-system

webkit-early-warning-system commented Jul 23, 2023

Copy link
Copy Markdown
Collaborator

@Ahmad-S792 Ahmad-S792 removed the merging-blocked Applied to prevent a change from being merged label Jul 24, 2023
@MenloDorian MenloDorian added the merge-queue Applied to send a pull request to merge-queue label Sep 6, 2023
https://bugs.webkit.org/show_bug.cgi?id=258794

Reviewed by Youenn Fablet.

The current WebRTC HEVC is using generic packetization instead of RFC 7789 Packetization.
So this is about implementing the RFC 7798 Packetization.

Fix HEVC depacketizer issues. (WebKit#185)
Enalbing low latency mode for RTC (WebKit#169)
Enable HEVC support. (WebKit#165)
Fix out-of-bounds write in H265VpsSpsPpsTracker (WebKit#163)
Apply fix bitstream logic to RtpVideoStreamReceiver2 (WebKit#142)
Add missing CODEC_H265 switch case (WebKit#136)
Add HEVC support for iOS/Android (WebKit#68)
H265 packetization_mode setting fix (WebKit#53)
Add H.265 QP parsing logic (WebKit#47)

This patch is extracted from following Open WebRTC Toolkit code changes:
<open-webrtc-toolkit/owt-deps-webrtc#185>
<open-webrtc-toolkit/owt-deps-webrtc#169>
<open-webrtc-toolkit/owt-deps-webrtc#165>
<open-webrtc-toolkit/owt-deps-webrtc#163>
<open-webrtc-toolkit/owt-deps-webrtc#142>
<open-webrtc-toolkit/owt-deps-webrtc#136>
<open-webrtc-toolkit/owt-deps-webrtc#68>
<open-webrtc-toolkit/owt-deps-webrtc#53>
<open-webrtc-toolkit/owt-deps-webrtc#47>

co-authoured by:
taste1981 <jianlin.qiu@intel.com>
jianjunz <jianjun.zhu@intel.com>
Cyril Lashkevich <notorca@gmail.com>
Piasy <xz4215@gmail.com>
ShiJinCheng <874042641@qq.com>
Andreas Unterhuber <andreas.unterhuber@keepinmind.info>
dong-heun <63987238+dong-heun@users.noreply.github.com>

* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.xcconfig:
* Source/ThirdParty/libwebrtc/Source/webrtc/api/video/video_codec_type.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/api/video_codecs/video_codec.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/api/video_codecs/video_codec.h:
(webrtc::VideoCodecH265::operator!= const):
* Source/ThirdParty/libwebrtc/Source/webrtc/api/video_codecs/video_decoder_software_fallback_wrapper.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/api/video_codecs/video_encoder.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/api/video_codecs/video_encoder.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/call/rtp_payload_params.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/common_video/h265/h265_common.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/common_video/h265/h265_common.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/common_video/h265/h265_pps_parser.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/common_video/h265/h265_pps_parser.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/common_video/h265/h265_sps_parser.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/common_video/h265/h265_sps_parser.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/common_video/h265/h265_vps_parser.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/common_video/h265/h265_vps_parser.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/logging/rtc_event_log/rtc_event_log_parser.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/media/base/media_constants.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/media/base/media_constants.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/media/engine/internal_decoder_factory.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/rtp_rtcp/source/create_video_rtp_depacketizer.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/rtp_rtcp/source/rtp_format.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/rtp_rtcp/source/rtp_format_h265.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/rtp_rtcp/source/rtp_format_h265.h:
(webrtc::RtpPacketizerH265::PacketUnit::PacketUnit):
(webrtc::VideoRtpDepacketizerH265::~VideoRtpDepacketizerH265): Deleted.
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/rtp_rtcp/source/rtp_sender_video.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/rtp_rtcp/source/rtp_video_header.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/video_coding/codecs/h265/include/h265_globals.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/video_coding/encoded_frame.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/video_coding/h265_vps_sps_pps_tracker.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/video_coding/h265_vps_sps_pps_tracker.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/video_coding/include/video_codec_interface.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/modules/video_coding/packet_buffer.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/rtc_base/experiments/min_video_bitrate_experiment.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/nalu_rewriter.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/nalu_rewriter.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/test/scenario/video_stream.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/test/video_codec_settings.h:
(webrtc::test::CodecSettings):
* Source/ThirdParty/libwebrtc/Source/webrtc/video/config/video_encoder_config.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/video/config/video_encoder_config.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/video/encoder_overshoot_detector.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/video/rtp_video_stream_receiver2.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/video/rtp_video_stream_receiver2.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/video/send_statistics_proxy.cc:
* Source/ThirdParty/libwebrtc/Source/webrtc/video/video_stream_encoder.cc:
* Source/ThirdParty/libwebrtc/libwebrtc.xcodeproj/project.pbxproj:

Canonical link: https://commits.webkit.org/267677@main
@webkit-commit-queue
webkit-commit-queue force-pushed the eng/WebRTC-HEVC-RFC-7798-RTP-Payload-Format-Implementation branch from 6c3a4cf to 93eb48d Compare September 6, 2023 16:48
@webkit-commit-queue

Copy link
Copy Markdown
Collaborator

Committed 267677@main (93eb48d): https://commits.webkit.org/267677@main

Reviewed commits have been landed. Closing PR #15494 and removing active labels.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

WebRTC For bugs in WebRTC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants