| // Copyright 2018-2021 The Khronos Group Inc. |
| // |
| // SPDX-License-Identifier: CC-BY-4.0 |
| |
| [[video-encode-operations]] |
| == Video Encode Operations |
| |
| Before the application can start recording Vulkan command buffers for the |
| Video Encode Operations, it must: do the following, beforehand: |
| |
| . Ensure that the implementation can encode the Video Content by querying |
| the supported codec operations and profiles using |
| flink:vkGetPhysicalDeviceQueueFamilyProperties2. |
| . By using flink:vkGetPhysicalDeviceVideoFormatPropertiesKHR and providing |
| one or more video profiles, choose the Vulkan formats supported by the |
| implementation. |
| The formats for <<input-encode-picture,input>> and |
| <<reference-picture,reference>> pictures must: be queried and chosen |
| separately. |
| Refer to the section on |
| <<supported-video-input-output-dpb-formats-enumeration,enumeration of |
| supported video formats>>. |
| . Before creating an image to be used as a video picture resource, obtain |
| the supported image creation parameters by querying with |
| flink:vkGetPhysicalDeviceFormatProperties2 and |
| flink:vkGetPhysicalDeviceImageFormatProperties2 using one of the |
| reported formats and adding slink:VkVideoProfilesKHR to the pname:pNext |
| chain of slink:VkFormatProperties2. |
| When querying the parameters with |
| flink:vkGetPhysicalDeviceImageFormatProperties2 for images targeting |
| <<input-encode-picture,input>> and <<reference-picture,reference (DPB)>> |
| pictures, the slink:VkPhysicalDeviceImageFormatInfo2::pname:usage field |
| should contain ename:VK_IMAGE_USAGE_VIDEO_ENCODE_SRC_BIT_KHR and |
| ename:VK_IMAGE_USAGE_VIDEO_ENCODE_DPB_BIT_KHR, respectively. |
| . Create none, some, or all of the required <<VkImage,images>> for the |
| <<input-encode-picture,input>> and <<reference-picture,reference>> |
| pictures. |
| More Video Picture Resources can: be created at some later point if |
| needed while processing the content to be encoded. |
| Also, if the size of the picture to be encoded is expected to change, |
| the images can: be created based on the maximum expected content size. |
| . Create the <<video-session,video session>> to be used for video encode |
| operations. |
| Before creating the Encode Video Session, the encode capabilities |
| should: be queried with flink:vkGetPhysicalDeviceVideoCapabilitiesKHR to |
| obtain the limits of the parameters allowed by the implementation for a |
| particular codec profile. |
| . Bind memory resources with the encode video session by calling |
| flink:vkBindVideoSessionMemoryKHR. |
| The video session cannot: be used until memory resources are allocated |
| and bound to it. |
| In order to determine the required memory sizes and heap types of the |
| device memory allocations, flink:vkGetVideoSessionMemoryRequirementsKHR |
| should: be called. |
| . Create one or more <<video-session-parameters-introduction,Session |
| Parameter objects>> for use across command buffer recording operations, |
| if required by the codec extension in use. |
| These objects must: be created against a <<video-session,video session>> |
| with the parameters required by the codec. |
| Each <<video-session-parameters-introduction,Session Parameter object>> |
| created is a child object of the associated <<video-session, Session |
| object>> and cannot: be bound in the command buffer with any other |
| <<video-session,Session Object>>. |
| |
| |
| The recording of Video Encode Commands against a Vulkan Command Buffer |
| consists of the following sequence: |
| |
| . flink:vkCmdBeginVideoCodingKHR starts the recording of one or more Video |
| Encode operations in the command buffer. |
| For each Video Encode Command operation, a Video Session must: be bound |
| to the command buffer within this command. |
| This command establishes a Vulkan Video Encode Context that consists of |
| the bound Video Session Object, Session Parameters Object, and the |
| required Video Picture Resources. |
| The established Video Encode Context is in effect until the |
| flink:vkCmdEndVideoCodingKHR command is recorded. |
| If more Video Encode operations are to be required after the |
| flink:vkCmdEndVideoCodingKHR command, another Video Encode Context can: |
| be started with the flink:vkCmdBeginVideoCodingKHR command. |
| . flink:vkCmdEncodeVideoKHR specifies one or more frames to be encoded. |
| The slink:VkVideoEncodeInfoKHR parameters, and the codec extension |
| structures chained to this, specify the details of the encode operation. |
| . flink:vkCmdControlVideoCodingKHR records operations against the encoded |
| data, encoding device, or the Video Session state. |
| . flink:vkCmdEndVideoCodingKHR signals the end of the recording of the |
| Vulkan Video Encode Context, as established by |
| flink:vkCmdBeginVideoCodingKHR. |
| |
| In addition to the above, the following commands can: be recorded between |
| flink:vkCmdBeginVideoCodingKHR and flink:vkCmdEndVideoCodingKHR: |
| |
| * Query operations |
| * Global Memory Barriers |
| * Buffer Memory Barriers |
| * Image Memory Barriers (these must: be used to transition the Video |
| Picture Resources to the proper |
| ename:VK_IMAGE_LAYOUT_VIDEO_ENCODE_SRC_KHR and |
| ename:VK_IMAGE_LAYOUT_VIDEO_ENCODE_DPB_KHR layouts). |
| * Pipeline Barriers |
| * Events |
| * Timestamps |
| * Device Groups (device mask) |
| |
| The following Video Encode related commands must: be recorded *outside* the |
| Vulkan Video Encode Context established with the |
| flink:vkCmdBeginVideoCodingKHR and flink:vkCmdEndVideoCodingKHR commands: |
| |
| * Sparse Memory Binding |
| * Copy Commands |
| * Clear Commands |
| |
| |
| === Video Encode Vulkan Command Buffer Commands |
| |
| [open,refpage='vkCmdEncodeVideoKHR',desc='Encode operation for bitstream generation',type='protos'] |
| -- |
| To launch an encode operation that results in bitstream generation, call: |
| |
| include::{generated}/api/protos/vkCmdEncodeVideoKHR.txt[] |
| |
| * pname:commandBuffer is the command buffer to be filled with this |
| function for encoding to generate a bitstream. |
| * pname:pEncodeInfo is a pointer to a slink:VkVideoEncodeInfoKHR |
| structure. |
| |
| include::{generated}/validity/protos/vkCmdEncodeVideoKHR.txt[] |
| -- |
| |
| [open,refpage='VkVideoEncodeInfoKHR',desc='Structure to chain codec-specific structures to',type='structs'] |
| -- |
| The slink:VkVideoEncodeInfoKHR structure is defined as: |
| |
| include::{generated}/api/structs/VkVideoEncodeInfoKHR.txt[] |
| * pname:sType is the type of this structure. |
| * pname:pNext is a pointer to a structure extending this structure. |
| A codec-specific extension structure must: be chained to specify what |
| bitstream unit to generate with this encode operation. |
| * pname:flags is a bitmask of elink:VkVideoEncodeFlagBitsKHR specifying |
| encode flags, and is reserved for future versions of this specification. |
| * pname:qualityLevel is the coding quality level of the encoding. |
| It is defined by the codec-specific extensions. |
| * pname:codedExtent is the coded size of the encode operations. |
| * pname:dstBitstreamBuffer is the buffer where the encoded bitstream |
| output will be produced. |
| * pname:dstBitstreamBufferOffset is the offset in the |
| pname:dstBitstreamBuffer where the encoded bitstream output will start. |
| pname:dstBitstreamBufferOffset's value must: be aligned to |
| slink:VkVideoCapabilitiesKHR::pname:minBitstreamBufferOffsetAlignment, |
| as reported by the implementation. |
| * pname:dstBitstreamBufferMaxRange is the maximum size of the |
| pname:dstBitstreamBuffer that can be used while the encoded bitstream |
| output is produced. |
| pname:dstBitstreamBufferMaxRange's value must: be aligned to |
| slink:VkVideoCapabilitiesKHR::pname:minBitstreamBufferSizeAlignment, as |
| reported by the implementation. |
| * pname:srcPictureResource is the Picture Resource of the |
| <<input-encode-picture,Input Picture>> to be encoded by the operation. |
| * pname:pSetupReferenceSlot is a pointer to a |
| slink:VkVideoReferenceSlotKHR structure used for generating a |
| reconstructed reference slot and Picture Resource. |
| pname:pSetupReferenceSlot->slotIndex specifies the slot index number to |
| use as a target for producing the Reconstructed (DPB) data. |
| pname:pSetupReferenceSlot must: be one of the entries provided in |
| slink:VkVideoBeginCodingInfoKHR via the pname:pReferenceSlots within the |
| flink:vkCmdBeginVideoCodingKHR command that established the Vulkan Video |
| Encode Context for this command. |
| * pname:referenceSlotCount is the number of Reconstructed Reference |
| Pictures that will be used when this encoding operation is executing. |
| * pname:pReferenceSlots is `NULL` or a pointer to an array of |
| slink:VkVideoReferenceSlotKHR structures that will be used when this |
| encoding operation is executing. |
| Each entry in pname:pReferenceSlots must: be one of the entries provided |
| in slink:VkVideoBeginCodingInfoKHR via the pname:pReferenceSlots within |
| the flink:vkCmdBeginVideoCodingKHR command that established the Vulkan |
| Video Encode Context for this command. |
| |
| Multiple flink:vkCmdEncodeVideoKHR commands may: be recorded within a Vulkan |
| Video Encode Context. |
| The execution of each flink:vkCmdEncodeVideoKHR command will result in |
| generating codec-specific bitstream units. |
| These bitstream units are generated consecutively into the bitstream buffer |
| specified in pname:dstBitstreamBuffer of slink:VkVideoEncodeInfoKHR within |
| the flink:vkCmdBeginVideoCodingKHR command. |
| The produced bitstream is the sum of all these bitstream units, including |
| any padding between the bitstream units. |
| Any bitstream padding must: be filled with data compliant to the codec |
| standard so as not to cause any syntax errors during decoding of the |
| bitstream units with the padding included. |
| The range of the bitstream buffer written can: be queried via |
| <<queries-video-encode-bitstream-buffer-range, video encode bitstream buffer |
| range queries>>. |
| |
| include::{generated}/validity/structs/VkVideoEncodeInfoKHR.txt[] |
| -- |
| |
| [open,refpage='VkVideoEncodeFlagBitsKHR',desc='Video Encode Command Flags',type='enums'] |
| -- |
| The flink:vkCmdEncodeVideoKHR flags are defined with the following |
| enumeration: |
| |
| include::{generated}/api/enums/VkVideoEncodeFlagBitsKHR.txt[] |
| |
| * ename:VK_VIDEO_ENCODE_RESERVED_0_BIT_KHR The current version of the |
| specification has reserved this value for future use. |
| -- |
| |
| [open,refpage='VkVideoEncodeFlagsKHR',desc='Bitmask specifying the video encode flink:vkCmdEncodeVideoKHR flags',type='flags'] |
| -- |
| include::{generated}/api/flags/VkVideoEncodeFlagsKHR.txt[] |
| |
| tlink:VkVideoEncodeFlagsKHR is a bitmask type for setting a mask of zero or |
| more elink:VkVideoEncodeFlagBitsKHR. |
| -- |
| |
| [open,refpage='VkVideoEncodeRateControlInfoKHR',desc='Structure to set encode rate control parameters',type='structs'] |
| -- |
| The slink:VkVideoEncodeRateControlInfoKHR structure is defined as: |
| |
| include::{generated}/api/structs/VkVideoEncodeRateControlInfoKHR.txt[] |
| * pname:sType is the type of this structure. |
| * pname:pNext is `NULL` or a pointer to a structure extending this |
| structure. |
| * pname:flags is a bitmask of |
| elink:VkVideoEncodeRateControlModeFlagBitsKHR specifying encode rate |
| control flags. |
| * pname:rateControlMode is a elink:VkVideoEncodeRateControlModeFlagBitsKHR |
| value specifying the encode rate control mode. |
| * pname:averageBitrate is the average bitrate in bits/second. |
| Valid when rate control is not |
| ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_NONE_BIT_KHR. |
| * pname:peakToAverageBitrateRatio is the peak bitrate to average bitrate |
| in percentage. |
| Valid when rate control is |
| ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_VBR_BIT_KHR. |
| * pname:frameRateNumerator is the numerator of the frame rate. |
| Valid when rate control is not |
| ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_NONE_BIT_KHR. |
| * pname:frameRateDenominator is the denominator of the frame rate. |
| Valid when rate control is not |
| ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_NONE_BIT_KHR. |
| * pname:virtualBufferSizeInMs is the leaky bucket model virtual buffer |
| size in milliseconds, with respect to peak bitrate. |
| Valid when rate control is not |
| ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_NONE_BIT_KHR. |
| For example, virtual buffer size is (pname:virtualBufferSizeInMs * |
| pname:peakToAverageBitrateRatio * pname:averageBitrate / 100000). |
| |
| A codec-specific extension structure for further rate control parameter |
| settings may: be chained to slink:VkVideoEncodeRateControlInfoKHR. |
| |
| include::{generated}/validity/structs/VkVideoEncodeRateControlInfoKHR.txt[] |
| -- |
| |
| [open,refpage='VkVideoEncodeRateControlModeFlagBitsKHR',desc='Video encode rate control modes',type='enums'] |
| -- |
| The rate control modes are defined with the following enums: |
| |
| include::{generated}/api/enums/VkVideoEncodeRateControlModeFlagBitsKHR.txt[] |
| |
| * ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_NONE_BIT_KHR for disabling rate |
| control. |
| * ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_CBR_BIT_KHR for constant bitrate |
| rate control mode. |
| * ename:VK_VIDEO_ENCODE_RATE_CONTROL_MODE_VBR_BIT_KHR for variable bitrate |
| rate control mode. |
| -- |
| |
| [open,refpage='VkVideoEncodeRateControlModeFlagsKHR',desc='Bitmask specifying the video encode flink:vkCmdControlVideoCodingKHR flags',type='flags'] |
| -- |
| include::{generated}/api/flags/VkVideoEncodeRateControlModeFlagsKHR.txt[] |
| |
| tlink:VkVideoEncodeRateControlModeFlagsKHR is a bitmask type for setting a |
| mask of zero or more elink:VkVideoEncodeRateControlModeFlagBitsKHR. |
| -- |
| |
| [open,refpage='VkVideoEncodeRateControlFlagBitsKHR',desc='Video Encode Rate Control Flags',type='enums'] |
| -- |
| The flink:vkCmdControlVideoCodingKHR flags are defined with the following |
| enumeration: |
| |
| include::{generated}/api/enums/VkVideoEncodeRateControlFlagBitsKHR.txt[] |
| |
| * ename:VK_VIDEO_ENCODE_RESERVED_0_BIT_KHR The current version of the |
| specification has reserved this value for future use. |
| -- |
| |
| [open,refpage='VkVideoEncodeRateControlFlagsKHR',desc='Bitmask specifying the video encode flink:vkCmdControlVideoCodingKHR flags',type='flags'] |
| -- |
| include::{generated}/api/flags/VkVideoEncodeRateControlFlagsKHR.txt[] |
| |
| tlink:VkVideoEncodeRateControlFlagsKHR is a bitmask type for setting a mask |
| of zero or more elink:VkVideoEncodeRateControlFlagBitsKHR. |
| -- |