| // Copyright 2018 The Fuchsia Authors. All rights reserved. |
| // Use of this source code is governed by a BSD-style license that can be |
| // found in the LICENSE file. |
| @available(added=7) |
| library fuchsia.mediacodec; |
| |
| using fuchsia.media; |
| using zx; |
| |
| // CreateDecoder_Params |
| // |
| // Input parameters for creating a decoder (audio or video). |
| // |
| |
| /// Whether buffers need to be secure. If not specified, the default is OFF. |
| /// |
| /// This enum may have additional values added later; code handling this type |
| /// should be written with this in mind. For example, in C++, having a |
| /// "default" case in any switch statement on this type will avoid compilation |
| /// warnings/errors when a new value is added. |
| // |
| // Later we can add DYNAMIC, as needed. |
| type SecureMemoryMode = strict enum : uint32 { |
| // Normal memory. This is the default if a field of this type is not set. |
| OFF = 0; |
| // Secure memory. |
| ON = 1; |
| }; |
| |
| type CreateDecoder_Params = table { |
| /// Input mime type for a decoder. |
| /// |
| /// The recognized mime types for now: |
| /// video/h264 |
| /// video/vp9 |
| /// audio/aac |
| /// input_details.oob_bytes must be an AudioSpecificConfig() as defined |
| /// by AAC spec. |
| /// audio/sbc |
| /// input_details.oob_bytes must be Codec Specific Information Elements |
| /// for SBC as defined by the A2DP spec. |
| 1: input_details fuchsia.media.FormatDetails; |
| |
| // The settings below nail down more details. |
| |
| /// This must be true in order for the client to be permitted to put a |
| /// timestamp on an input packet, which is in turn required to get any |
| /// timestamps on any output packets. |
| /// |
| /// It is always legal to provide separate Access Units (henceforth AUs) to a |
| /// decoder, but this boolean must be true for a decoder to accept and |
| /// propagate timestamp values. |
| /// |
| /// This must be true when creating a video encoder, or the CodecFactory |
| /// channel will close. |
| /// |
| /// If not set, interpreted as false. |
| 2: promise_separate_access_units_on_input bool; |
| |
| // "require" fields: |
| // |
| // Specifying any of these "require" fields can result in failure to get a |
| // Codec if there's no suitable codec. None of these correspond to any |
| // required features of a codec server. |
| // |
| // TODO(dustingreen): implement filtering codecs based on these fields. |
| |
| /// Require that the selected codec be capable of accepting input where |
| /// AUs are not separated into separate packets. |
| /// |
| /// This does not imply that the decoder can find the start of the first AU; |
| /// for that see require_can_find_start. This does not imply that the decoder |
| /// can re-sync on its own if the stream data is damaged; for that see |
| /// require_can_re_sync. |
| /// |
| /// If both promise_separate_access_units_on_input and |
| /// require_can_stream_bytes_input are true, the CodecFactory channel will |
| /// close. |
| /// |
| /// If this is false, the client must feed separate AUs on the fuchsia.ui.input. This |
| /// must be false for a video encoder, and if true the CodecFactory channel |
| /// will close. |
| /// |
| /// Unless a client demands a decoder capable of taking concatenated AUs |
| /// (require_can_stream_bytes_input true), the client must feed a decoder |
| /// separate AUs. This means the client cannot have parts of two separate AUs |
| /// in the same packet, unless require_can_stream_bytes_input is true. |
| /// |
| /// If not set, interpreted as false. |
| 3: require_can_stream_bytes_input bool; |
| |
| /// A decoder is allowed to be capable of streaming bytes but not capable of |
| /// searching for the start of the first usable AU. To require both, set both |
| /// require_can_stream_bytes_input and require_can_find_start. Setting |
| /// require_can_find_start without require_can_stream_bytes_input is invalid. |
| /// |
| /// With require_can_stream_bytes_input true but require_can_find_start false, |
| /// the client must start the first packet with the start of an AU, but can |
| /// send a stream of bytes after that. |
| /// |
| /// If not set, interpreted as false. |
| 4: require_can_find_start bool; |
| |
| /// On problematic input data, all decoders are expected to at least be able to |
| /// close the channel rather than getting stuck in a failed and/or broken |
| /// state. |
| /// |
| /// A decoder returned from a request with require_can_re_sync is potentially |
| /// able to handle damaged input without closing the Codec channel. Such a |
| /// Codec is encouraged, but not required, to also satisfy requirements of |
| /// require_report_all_detected_errors. |
| /// |
| /// If not set, interpreted as false. |
| 5: require_can_re_sync bool; |
| |
| /// Sometimes a client would rather fail an overall use of a decoder than fail |
| /// to notice data corruption. For such scenarios, the client can specify |
| /// require_report_all_detected_errors. For any codec returned from a |
| /// request with require_report_all_detected_errors set, on detection of |
| /// any input data corruption the codec will report in one or more of these |
| /// ways: |
| /// * closing the Codec channel |
| /// * OnStreamFailed() |
| /// * error_detected_before |
| /// * error_detected_during |
| /// |
| /// If false, a codec may silently skip past corrupted input data. |
| /// |
| /// No decoder can detect all corruption, because some corruption can look like |
| /// valid stream data. This requirement is only to request a codec that |
| /// is written to attempt to detect _and report_ input stream corruption. |
| /// |
| /// This flag is not intended to be 100% bulletproof. If a client needs robust |
| /// assurance that _all_ detectable stream corruption is _always_ detected, |
| /// this flag is not enough of a guarantee to achieve that. Since some stream |
| /// corruption is inherently non-detectable in any case, such a client should |
| /// consider using stronger techniques upstream to ensure that corruption can |
| /// be detected with the needed probability very close to 1. |
| /// |
| /// This flag being true doesn't imply anything about whether the codec will |
| /// discard damaged data vs. producing corresponding damaged output. Only that |
| /// the codec will set error_detected_* bools to true when appropriate. |
| /// |
| /// Regardless of this setting, not all timestamp_ish values provided on input |
| /// are guaranteed to show up on output. |
| /// |
| /// If not set, interpreted as false. |
| 6: require_report_all_detected_errors bool; |
| |
| /// If true, require that the returned codec is HW-accelerated. See also |
| /// `require_sw`. |
| /// |
| /// If not set, interpreted as false. |
| 7: require_hw bool; |
| |
| /// permit_lack_of_split_header_handling |
| /// |
| /// This field is a temporary field that will be going away. |
| /// |
| /// TODO(dustingreen): Remove this field once we're down to zero codecs with |
| /// problems handling split headers. |
| /// |
| /// By default, a Codec instance is required to handle "split headers", meaning |
| /// that a client is allowed to deliver parts of an AU one byte at a time, |
| /// including parts near the beginning of the AU, and the codec is required to |
| /// tolerate and handle that properly. However, unfortunately not all codecs |
| /// properly support split headers. If a client is willing to permit such a |
| /// codec to be used, the client can set this to true. Clients are not |
| /// encouraged to set this, but setting it may be necessary to find a codec for |
| /// some formats _for now_. If a client sets this to true, the client should |
| /// deliver data of each AU with many contiguous non-split bytes from the start |
| /// of each AU. The client is not strictly required to deliver one AU at a |
| /// time, only to ensure that either all the AU bytes are in a single packet or |
| /// that many bytes at the start of each AU are in a single packet. |
| /// |
| /// The specification for how a client should use this and how a client should |
| /// behave if setting this to true is intentionally vague, because lack of |
| /// support for header splitting is not ideal, and is expected to be |
| /// temporary, and all codecs should handle split headers in the long run. |
| /// The main intent of this field is to avoid giving an innocent client using |
| /// default value of false here a codec that can't properly handle split |
| /// headers. This is not an attempt at a mechanism to fully work around a |
| /// codec that doesn't handle split headers. |
| /// |
| /// If not set, interpreted as false. |
| // |
| // TODO(dustingreen): In the near term, wire this up so that SoftAAC2.cpp |
| // used for ADTS is not selected when this field is false, even if there is |
| // no other suitable codec. In the long term, fix or work around the header |
| // handling behavior of SoftAAC2 when used in ADTS mode (and any other |
| // similar issues in other codecs) and remove this field. |
| 8: permit_lack_of_split_header_handling bool; |
| |
| /// If set to ON, the decoder must support secure buffers on output, and |
| /// must reject non-secure buffers on output. |
| /// |
| /// If set to OFF or not set, the created decoder will reject secure buffers |
| /// on output by closing the StreamProcessor channel. |
| /// |
| /// If secure_input_mode ON, secure_output_mode must also be ON. |
| 9: secure_output_mode SecureMemoryMode; |
| |
| /// If set to ON, the decoder must support secure buffers on input and must |
| /// reject non-secure buffers on input. |
| /// |
| /// If set to OFF or not set, the created decoder will reject secure buffers |
| /// on input by closing the StreamProcessor channel. |
| /// |
| /// If secure_input_mode ON, secure_output_mode must also be ON. |
| 10: secure_input_mode SecureMemoryMode; |
| |
| /// If true, require that the returned codec is entirely SW-based, not |
| /// HW-accelerated (other than possibly using vector CPU instructions). |
| /// This can be useful for testing purposes or other special scenarios, but |
| /// is not recommended for performance-sensitive scenarios. Also, some |
| /// builds may lack a SW-based decoder for some formats. See also |
| /// `require_hw`. |
| /// |
| /// If not set, interpreted as false. |
| 11: require_sw bool; |
| }; |
| |
| /// Parameters used to request an encoder. |
| type CreateEncoder_Params = table { |
| /// The format of the uncompressed input data. |
| /// |
| /// This field should be a raw mime_type (e.g. 'video/raw') and uncompressed |
| /// format details for the encoder to use when reading buffers. |
| /// |
| /// To be elibigible an encoder must support the input format. |
| 1: input_details fuchsia.media.FormatDetails; |
| |
| /// If true, require that the returned codec is HW-accelerated. |
| /// |
| /// If not set, interpreted as false. |
| 2: require_hw bool; |
| }; |
| |
| type CodecType = strict enum { |
| DECODER = 0; |
| ENCODER = 1; |
| }; |
| |
| /// Constrain the mime_type to a size that won't cause problems. |
| const CODEC_FACTORY_MAX_MIME_TYPE_LENGTH uint32 = 256; |
| |
| type CodecDescription = struct { |
| /// Decoder or encoder. |
| codec_type CodecType; |
| /// The mime type of the compressed format. For decoders this is the mime |
| /// type of the input. For encoders, this is the mime type of the output. |
| mime_type string:CODEC_FACTORY_MAX_MIME_TYPE_LENGTH; |
| |
| // TODO(dustingreen): All these fields should be optional. |
| // |
| // TODO(dustingreen): Re-evaluate this for encoders. |
| // |
| /// For each of these fields, the default is the most-capable setting, but |
| /// if a codec doesn't support the most-capable behavior, then the codec |
| /// must override the default. |
| @allow_deprecated_struct_defaults |
| can_stream_bytes_input bool = true; |
| @allow_deprecated_struct_defaults |
| can_find_start bool = true; |
| @allow_deprecated_struct_defaults |
| can_re_sync bool = true; |
| @allow_deprecated_struct_defaults |
| will_report_all_detected_errors bool = true; |
| @allow_deprecated_struct_defaults |
| is_hw bool = true; |
| @allow_deprecated_struct_defaults |
| split_header_handling bool = true; |
| }; |
| |
| /// More than this many calls to AttachLifetimeTracking() |
| /// without a Create{X}() call will result in CodecFactory channel closing from |
| /// the server end. |
| const CODEC_FACTORY_LIFETIME_TRACKING_EVENTPAIR_PER_CREATE_MAX uint32 = 16; |
| |
| /// Constrain the length of the codec list. |
| const CODEC_FACTORY_CODEC_LIST_SIZE_MAX uint32 = 256; |
| |
| /// CodecFactory |
| /// |
| /// The purpose of the media::CodecFactory interface is to create |
| /// media::StreamProcessor instances for decoders and encoders. |
| /// |
| /// The interface methods don't attempt to homogenize all codec types, |
| /// preferring to have a separate dedicated message for decoders. |
| @discoverable |
| protocol CodecFactory { |
| /// Driver-based local CodecFactory(s) will send this once shortly after the |
| /// main CodecFactory connects to the driver-local CodecFactory. |
| /// |
| /// For now, the main CodecFactory will not send this. |
| /// |
| /// A SW-based local CodecFactory(s) will not send this event. |
| /// |
| /// Each codec in the list must be separately-described, for clean aggregation. |
| -> OnCodecList(struct { |
| codecs vector<CodecDescription>:CODEC_FACTORY_CODEC_LIST_SIZE_MAX; |
| }); |
| |
| // Requests: |
| |
| /// CreateDecoder: |
| /// |
| /// decoder_params - See CreateDecoder_Params comments for required |
| /// and optional parameters for creating a decoder. |
| /// |
| /// decoder - a Codec.NewRequest() which will hopefully be connected to |
| /// a Codec server, or the Codec channel will get closed if no suitable |
| /// codec can be found. We don't return any additional Codec-specific |
| /// status here because finding the Codec is allowed to be fully async, so |
| /// we don't necessarily yet know on return from this method which Codec |
| /// will be selected, if any. |
| /// |
| /// Rough sequence to create a decoder: |
| /// |
| /// factory = ConnectToEnvironmentService(CodecFactory); |
| /// CreateDecoder_Params params; |
| /// [fill out params] |
| /// CreateDecoder(params, decoder_request); |
| /// |
| /// See use_media_decoder code for more detail. |
| CreateDecoder(resource struct { |
| decoder_params CreateDecoder_Params; |
| decoder server_end:fuchsia.media.StreamProcessor; |
| }); |
| |
| /// CreateEncoder: |
| /// |
| /// encoder_params - See CreateEncoder_Params comments for required |
| /// and optional parameters for creating a decoder. |
| /// |
| /// encoder - a Codec.NewRequest() which will hopefully be connected to |
| /// a Codec server, or the Codec channel will get closed if no suitable |
| /// codec can be found. We don't return any additional Codec-specific |
| /// status here because finding the Codec is allowed to be fully async, so |
| /// we don't necessarily yet know on return from this method which Codec |
| /// will be selected, if any. |
| CreateEncoder(resource struct { |
| encoder_params CreateEncoder_Params; |
| encoder server_end:fuchsia.media.StreamProcessor; |
| }); |
| |
| /// AttachLifetimeTracking: |
| /// |
| /// Attach an eventpair endpoint to the next Create{X}(), so that the |
| /// codec_end will be closed when the number of buffers allocated reaches |
| /// 'buffers_remaining'. Multiple eventpair endpoints can be attached per |
| /// create, with an enforced limit of |
| /// CODEC_FACTORY_LIFETIME_TRACKING_EVENTPAIR_PER_CREATE_MAX. |
| /// |
| /// The lifetime signalled by this event is intended to track all resources |
| /// used by the codec, including sysmem-allocated buffers created internally |
| /// by the codec. The sysmem buffer collections visible to the client, for |
| /// input and output buffers, are not included in the resources tracked |
| /// here, because those can be tracked separately via |
| /// fuchsia.sysmem.BufferCollection.AttachLifetimeTracking(). It is |
| /// permitted to send a duplicate of codec_end to both this |
| /// AttachLifetimeTracking() and also to |
| /// fuchsia.sysmem.BufferCollection.AttachLifetimeTracking(). |
| /// The ZX_EVENTPAIR_PEER_CLOSED will happen when both/all lifetimes are |
| /// fully over. This conveniently avoids needing multiple separate async |
| /// waits by the client. |
| /// |
| /// In the case of server process crashes, or failure of a codec to plumb |
| /// codec_end to sysmem, ZX_EVENTPAIR_PEER_CLOSED signalled on the peer of |
| /// codec_end may occur shortly before all resources are freed. |
| /// |
| /// A maximum of CODEC_FACTORY_LIFETIME_TRACKING_EVENTPAIR_PER_CREATE_MAX |
| /// calls to AttachLifetimeTracking() are allowed |
| /// before any Create{X}(). There is no way to cancel an attach short of |
| /// closing the CodecFactory channel. Closing the client end of the |
| /// eventpair doesn't subtract from the number of pending attach(es). For |
| /// this reason, it can be good to only send attach message(s) immediately |
| /// before the relevant Create{X}(), when it's known by the client that both |
| /// the attach message(s) and the Create{X}() messages will be sent. |
| /// |
| /// Closing the client's end doesn't result in any action by the server. |
| /// If the server listens to events from the client end at all, it is for |
| /// debug logging only. |
| /// |
| /// The server intentionally doesn't "trust" any bits signalled by the |
| /// client. This mechanism intentionally uses only ZX_EVENTPAIR_PEER_CLOSED |
| /// which can't be triggered early, and is only triggered when all handles |
| /// to codec_end are closed. No meaning is associated with any of the other |
| /// signal bits, and clients should functionally ignore any other signal |
| /// bits on either end of the eventpair or its peer. |
| /// |
| /// The codec_end may lack ZX_RIGHT_SIGNAL or ZX_RIGHT_SIGNAL_PEER, but must |
| /// have ZX_RIGHT_DUPLICATE (and must have ZX_RIGHT_TRANSFER to transfer |
| /// without causing CodecFactory channel failure). |
| AttachLifetimeTracking(resource struct { |
| codec_end zx.handle:EVENTPAIR; |
| }); |
| }; |