Initially, GstVideoDecoder calls @start when the decoder element
is activated, which allows the subclass to perform any global setup.
GstVideoDecoder calls @set_format to inform the subclass of caps
describing input video data that it is about to receive, including
possibly configuration data.
While unlikely, it might be called more than once, if changing input
parameters require reconfiguration.
Incoming data buffers are processed as needed, described in Data
Processing below.
GstVideoDecoder calls @stop at end of all processing.
The base class gathers input data, and optionally allows subclass
to parse this into subsequently manageable chunks, typically
corresponding to and referred to as 'frames'.
Each input frame is provided in turn to the subclass' @handle_frame
callback.
When the subclass enables the subframe mode with gstvideo.video_decoder.VideoDecoder.setSubframeMode,
the base class will provide to the subclass the same input frame with
different input buffers to the subclass @handle_frame
callback. During this call, the subclass needs to take
ownership of the input_buffer as @GstVideoCodecFrame.input_buffer
will have been changed before the next subframe buffer is received.
The subclass will call gstvideo.video_decoder.VideoDecoder.haveLastSubframe
when a new input frame can be created by the base class.
Every subframe will share the same @GstVideoCodecFrame.output_buffer
to write the decoding result. The subclass is responsible to protect
its access.
If codec processing results in decoded data, the subclass should call
@gst_video_decoder_finish_frame to have decoded data pushed
downstream. In subframe mode
the subclass should call @gst_video_decoder_finish_subframe until the
last subframe where it should call @gst_video_decoder_finish_frame.
The subclass can detect the last subframe using GST_VIDEO_BUFFER_FLAG_MARKER
on buffers or using its own logic to collect the subframes.
In case of decoding failure, the subclass must call
@gst_video_decoder_drop_frame or @gst_video_decoder_drop_subframe,
to allow the base class to do timestamp and offset tracking, and possibly
to requeue the frame for a later attempt in the case of reverse playback.
When the pipeline is seeked or otherwise flushed, the subclass is
informed via a call to its @reset callback, with the hard parameter
set to true. This indicates the subclass should drop any internal data
queues and timestamps and prepare for a fresh set of buffers to arrive
for parsing and decoding.
End Of Stream
At end-of-stream, the subclass @parse function may be called some final
times with the at_eos parameter set to true, indicating that the element
should not expect any more data to be arriving, and it should parse and
remaining frames and call gstvideo.video_decoder.VideoDecoder.haveFrame if possible.
The subclass is responsible for providing pad template caps for
source and sink pads. The pads need to be named "sink" and "src". It also
needs to provide information about the output caps, when they are known.
This may be when the base class calls the subclass' @set_format function,
though it might be during decoding, before calling
@gst_video_decoder_finish_frame. This is done via
@gst_video_decoder_set_output_state
The subclass is also responsible for providing (presentation) timestamps
(likely based on corresponding input ones). If that is not applicable
or possible, the base class provides limited framerate based interpolation.
Similarly, the base class provides some limited (legacy) seeking support
if specifically requested by the subclass, as full-fledged support
should rather be left to upstream demuxer, parser or alike. This simple
approach caters for seeking and duration reporting using estimated input
bitrates. To enable it, a subclass should call
@gst_video_decoder_set_estimate_rate to enable handling of incoming
byte-streams.
The base class provides some support for reverse playback, in particular
in case incoming data is not packetized or upstream does not provide
fragments on keyframe boundaries. However, the subclass should then be
prepared for the parsing and frame processing stage to occur separately
(in normal forward processing, the latter immediately follows the former),
The subclass also needs to ensure the parsing stage properly marks
keyframes, unless it knows the upstream elements will do so properly for
incoming data.
The bare minimum that a functional subclass needs to implement is:
Provide pad templates
Inform the base class of output caps via
@gst_video_decoder_set_output_state
Parse input data, if it is not considered packetized from upstream
Data will be provided to @parse which should invoke
@gst_video_decoder_add_to_frame and @gst_video_decoder_have_frame to
separate the data belonging to each video frame.
Accept data in @handle_frame and provide decoded results to
@gst_video_decoder_finish_frame, or call @gst_video_decoder_drop_frame.
This base class is for video decoders turning encoded data into raw video frames.
The GstVideoDecoder base class and derived subclasses should cooperate as follows:
Configuration
Data processing
Shutdown phase
Additional Notes
The subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named "sink" and "src". It also needs to provide information about the output caps, when they are known. This may be when the base class calls the subclass' @set_format function, though it might be during decoding, before calling @gst_video_decoder_finish_frame. This is done via @gst_video_decoder_set_output_state
The subclass is also responsible for providing (presentation) timestamps (likely based on corresponding input ones). If that is not applicable or possible, the base class provides limited framerate based interpolation.
Similarly, the base class provides some limited (legacy) seeking support if specifically requested by the subclass, as full-fledged support should rather be left to upstream demuxer, parser or alike. This simple approach caters for seeking and duration reporting using estimated input bitrates. To enable it, a subclass should call @gst_video_decoder_set_estimate_rate to enable handling of incoming byte-streams.
The base class provides some support for reverse playback, in particular in case incoming data is not packetized or upstream does not provide fragments on keyframe boundaries. However, the subclass should then be prepared for the parsing and frame processing stage to occur separately (in normal forward processing, the latter immediately follows the former), The subclass also needs to ensure the parsing stage properly marks keyframes, unless it knows the upstream elements will do so properly for incoming data.
The bare minimum that a functional subclass needs to implement is: