| // Copyright 2016-2021 The Khronos Group, Inc. |
| // |
| // SPDX-License-Identifier: CC-BY-4.0 |
| |
| include::{generated}/meta/{refprefix}VK_KHR_external_memory.txt[] |
| |
| === Other Extension Metadata |
| |
| *Last Modified Date*:: |
| 2016-10-20 |
| *IP Status*:: |
| No known IP claims. |
| *Interactions and External Dependencies*:: |
| - Interacts with `apiext:VK_KHR_dedicated_allocation`. |
| - Interacts with `apiext:VK_NV_dedicated_allocation`. |
| - Promoted to Vulkan 1.1 Core |
| *Contributors*:: |
| - Jason Ekstrand, Intel |
| - Ian Elliot, Google |
| - Jesse Hall, Google |
| - Tobias Hector, Imagination Technologies |
| - James Jones, NVIDIA |
| - Jeff Juliano, NVIDIA |
| - Matthew Netsch, Qualcomm Technologies, Inc. |
| - Daniel Rakos, AMD |
| - Carsten Rohde, NVIDIA |
| - Ray Smith, ARM |
| - Chad Versace, Google |
| |
| === Description |
| |
| An application may wish to reference device memory in multiple Vulkan |
| logical devices or instances, in multiple processes, and/or in multiple |
| APIs. |
| This extension enables an application to export non-Vulkan handles from |
| Vulkan memory objects such that the underlying resources can be referenced |
| outside the scope of the Vulkan logical device that created them. |
| |
| === Promotion to Vulkan 1.1 |
| |
| All functionality in this extension is included in core Vulkan 1.1, with the |
| KHR suffix omitted. |
| The original type, enum and command names are still available as aliases of |
| the core functionality. |
| |
| include::{generated}/interfaces/VK_KHR_external_memory.txt[] |
| |
| === Issues |
| |
| 1) How do applications correlate two physical devices across process or |
| Vulkan instance boundaries? |
| |
| *RESOLVED*: New device ID fields have been introduced by |
| `apiext:VK_KHR_external_memory_capabilities`. |
| These fields, combined with the existing |
| slink:VkPhysicalDeviceProperties::pname:driverVersion field can be used to |
| identify compatible devices across processes, drivers, and APIs. |
| slink:VkPhysicalDeviceProperties::pname:pipelineCacheUUID is not sufficient |
| for this purpose because despite its description in the specification, it |
| need only identify a unique pipeline cache format in practice. |
| Multiple devices may be able to use the same pipeline cache data, and hence |
| it would be desirable for all of them to have the same pipeline cache UUID. |
| However, only the same concrete physical device can be used when sharing |
| memory, so an actual unique device ID was introduced. |
| Further, the pipeline cache UUID was specific to Vulkan, but correlation |
| with other, non-extensible APIs is required to enable interoperation with |
| those APIs. |
| |
| 2) If memory objects are shared between processes and APIs, is this |
| considered aliasing according to the rules outlined in the |
| <<resources-memory-aliasing,Memory Aliasing>> section? |
| |
| *RESOLVED*: Yes. |
| Applications must take care to obey all restrictions imposed on aliased |
| resources when using memory across multiple Vulkan instances or other APIs. |
| |
| 3) Are new image layouts or metadata required to specify image layouts and |
| layout transitions compatible with non-Vulkan APIs, or with other instances |
| of the same Vulkan driver? |
| |
| *RESOLVED*: Separate instances of the same Vulkan driver running on the same |
| GPU should have identical internal layout semantics, so applications have |
| the tools they need to ensure views of images are consistent between the two |
| instances. |
| Other APIs will fall into two categories: Those that are Vulkan- compatible, |
| and those that are Vulkan-incompatible. |
| Vulkan-incompatible APIs will require the image to be in the GENERAL layout |
| whenever they are accessing them. |
| |
| Note this does not attempt to address cross-device transitions, nor |
| transitions to engines on the same device which are not visible within the |
| Vulkan API. |
| Both of these are beyond the scope of this extension. |
| |
| 4) Is a new barrier flag or operation of some type needed to prepare |
| external memory for handoff to another Vulkan instance or API and/or receive |
| it from another instance or API? |
| |
| *RESOLVED*: Yes. |
| Some implementations need to perform additional cache management when |
| transitioning memory between address spaces and other APIs, instances, or |
| processes which may operate in a separate address space. |
| Options for defining this transition include: |
| |
| * A new structure that can be added to the pname:pNext list in |
| slink:VkMemoryBarrier, slink:VkBufferMemoryBarrier, and |
| slink:VkImageMemoryBarrier. |
| * A new bit in tlink:VkAccessFlags that can be set to indicate an |
| "`external`" access. |
| * A new bit in tlink:VkDependencyFlags |
| * A new special queue family that represents an "`external`" queue. |
| |
| A new structure has the advantage that the type of external transition can |
| be described in as much detail as necessary. |
| However, there is not currently a known need for anything beyond |
| differentiating between external and internal accesses, so this is likely an |
| over-engineered solution. |
| The access flag bit has the advantage that it can be applied at buffer, |
| image, or global granularity, and semantically it maps pretty well to the |
| operation being described. |
| Additionally, the API already includes ename:VK_ACCESS_MEMORY_READ_BIT and |
| ename:VK_ACCESS_MEMORY_WRITE_BIT which appear to be intended for this |
| purpose. |
| However, there is no obvious pipeline stage that would correspond to an |
| external access, and therefore no clear way to use |
| ename:VK_ACCESS_MEMORY_READ_BIT or ename:VK_ACCESS_MEMORY_WRITE_BIT. |
| tlink:VkDependencyFlags and tlink:VkPipelineStageFlags operate at command |
| granularity rather than image or buffer granularity, which would make an |
| entire pipeline barrier an internal->external or external->internal barrier. |
| This may not be a problem in practice, but seems like the wrong scope. |
| Another downside of tlink:VkDependencyFlags is that it lacks inherent |
| directionality: there are no ptext:src and ptext:dst variants of it in the |
| barrier or dependency description semantics, so two bits might need to be |
| added to describe both internal->external and external->internal |
| transitions. |
| Transitioning a resource to a special queue family corresponds well with the |
| operation of transitioning to a separate Vulkan instance, in that both |
| operations ideally include scheduling a barrier on both sides of the |
| transition: Both the releasing and the acquiring queue or process. |
| Using a special queue family requires adding an additional reserved queue |
| family index. |
| Re-using ename:VK_QUEUE_FAMILY_IGNORED would have left it unclear how to |
| transition a concurrent usage resource from one process to another, since |
| the semantics would have likely been equivalent to the currently-ignored |
| transition of |
| ename:VK_QUEUE_FAMILY_IGNORED{nbsp}->{nbsp}ename:VK_QUEUE_FAMILY_IGNORED. |
| Fortunately, creating a new reserved queue family index is not invasive. |
| |
| Based on the above analysis, the approach of transitioning to a special |
| "`external`" queue family was chosen. |
| |
| 5) Do internal driver memory arrangements and/or other internal driver image |
| properties need to be exported and imported when sharing images across |
| processes or APIs. |
| |
| *RESOLVED*: Some vendors claim this is necessary on their implementations, |
| but it was determined that the security risks of allowing opaque metadata to |
| be passed from applications to the driver were too high. |
| Therefore, implementations which require metadata will need to associate it |
| with the objects represented by the external handles, and rely on the |
| dedicated allocation mechanism to associate the exported and imported memory |
| objects with a single image or buffer. |
| |
| 6) Most prior interoperation and cross-process sharing APIs have been based |
| on image-level sharing. |
| Should Vulkan sharing be based on memory-object sharing or image sharing? |
| |
| *RESOLVED*: These extensions have assumed memory-level sharing is the |
| correct granularity. |
| Vulkan is a lower-level API than most prior APIs, and as such attempts to |
| closely align with to the underlying primitives of the hardware and |
| system-level drivers it abstracts. |
| In general, the resource that holds the backing store for both images and |
| buffers of various types is memory. |
| Images and buffers are merely metadata containing brief descriptions of the |
| layout of bits within that memory. |
| |
| Because memory object-based sharing is aligned with the overall Vulkan API |
| design, it enables the full range of Vulkan capabilities with external |
| objects. |
| External memory can be used as backing for sparse images, for example, |
| whereas such usage would be awkward at best with a sharing mechanism based |
| on higher-level primitives such as images. |
| Further, aligning the mechanism with the API in this way provides some hope |
| of trivial compatibility with future API enhancements. |
| If new objects backed by memory objects are added to the API, they too can |
| be used across processes with minimal additions to the base external memory |
| APIs. |
| |
| Earlier APIs implemented interop at a higher level, and this necessitated |
| entirely separate sharing APIs for images and buffers. |
| To co-exist and interoperate with those APIs, the Vulkan external sharing |
| mechanism must accommodate their model. |
| However, if it can be agreed that memory-based sharing is the more desirable |
| and forward-looking design, legacy interoperation constraints can be |
| considered another reason to favor memory-based sharing: while native and |
| legacy driver primitives that may be used to implement sharing may not be as |
| low-level as the API here suggests, raw memory is still the least common |
| denominator among the types. |
| Image-based sharing can be cleanly derived from a set of base memory- object |
| sharing APIs with minimal effort, whereas image-based sharing does not |
| generalize well to buffer or raw-memory sharing. |
| Therefore, following the general Vulkan design principle of minimalism, it |
| is better to expose interopability with image-based native and external |
| primitives via the memory sharing API, and place sufficient limits on their |
| usage to ensure they can be used only as backing for equivalent Vulkan |
| images. |
| This provides a consistent API for applications regardless of which platform |
| or external API they are targeting, which makes development of multi-API and |
| multi-platform applications simpler. |
| |
| 7) Should Vulkan define a common external handle type and provide Vulkan |
| functions to facilitate cross-process sharing of such handles rather than |
| relying on native handles to define the external objects? |
| |
| *RESOLVED*: No. |
| Cross-process sharing of resources is best left to native platforms. |
| There are myriad security and extensibility issues with such a mechanism, |
| and attempting to re-solve all those issues within Vulkan does not align |
| with Vulkan's purpose as a graphics API. |
| If desired, such a mechanism could be built as a layer or helper library on |
| top of the opaque native handle defined in this family of extensions. |
| |
| 8) Must implementations provide additional guarantees about state implicitly |
| included in memory objects for those memory objects that may be exported? |
| |
| *RESOLVED*: Implementations must ensure that sharing memory objects does not |
| transfer any information between the exporting and importing instances and |
| APIs other than that required to share the data contained in the memory |
| objects explicitly shared. |
| As specific examples, data from previously freed memory objects that used |
| the same underlying physical memory, and data from memory obects using |
| adjacent physical memory must not be visible to applications importing an |
| exported memory object. |
| |
| 9) Must implementations validate external handles the application provides |
| as inputs to memory import operations? |
| |
| *RESOLVED*: Implementations must return an error to the application if the |
| provided memory handle cannot be used to complete the requested import |
| operation. |
| However, implementations need not validate handles are of the exact type |
| specified by the application. |
| |
| === Version History |
| |
| * Revision 1, 2016-10-20 (James Jones) |
| - Initial version |