AudioEncoder

This base class is for audio encoders turning raw audio samples into encoded audio data.

GstAudioEncoder and subclass should cooperate as follows.

Configuration

  • Initially, GstAudioEncoder calls @start when the encoder element is activated, which allows subclass to perform any global setup.
  • GstAudioEncoder calls @set_format to inform subclass of the format of input audio data that it is about to receive. Subclass should setup for encoding and configure various base class parameters appropriately, notably those directing desired input data handling. While unlikely, it might be called more than once, if changing input parameters require reconfiguration.
  • GstAudioEncoder calls @stop at end of all processing.

As of configuration stage, and throughout processing, GstAudioEncoder maintains various parameters that provide required context, e.g. describing the format of input audio data. Conversely, subclass can and should configure these context parameters to inform base class of its expectation w.r.t. buffer handling.

Data processing

  • Base class gathers input sample data (as directed by the context's frame_samples and frame_max) and provides this to subclass' @handle_frame.
  • If codec processing results in encoded data, subclass should call gstaudio.audio_encoder.AudioEncoder.finishFrame to have encoded data pushed downstream. Alternatively, it might also call gstaudio.audio_encoder.AudioEncoder.finishFrame (with a NULL buffer and some number of dropped samples) to indicate dropped (non-encoded) samples.
  • Just prior to actually pushing a buffer downstream, it is passed to @pre_push.
  • During the parsing process GstAudioEncoderClass will handle both srcpad and sinkpad events. Sink events will be passed to subclass if @event callback has been provided.

Shutdown phase

  • GstAudioEncoder class calls @stop to inform the subclass that data parsing will be stopped.

Subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named "sink" and "src". It also needs to set the fixed caps on srcpad, when the format is ensured. This is typically when base class calls subclass' @set_format function, though it might be delayed until calling @gst_audio_encoder_finish_frame.

In summary, above process should have subclass concentrating on codec data processing while leaving other matters to base class, such as most notably timestamp handling. While it may exert more control in this area (see e.g. @pre_push), it is very much not recommended.

In particular, base class will either favor tracking upstream timestamps (at the possible expense of jitter) or aim to arrange for a perfect stream of output timestamps, depending on #GstAudioEncoder:perfect-timestamp. However, in the latter case, the input may not be so perfect or ideal, which is handled as follows. An input timestamp is compared with the expected timestamp as dictated by input sample stream and if the deviation is less than #GstAudioEncoder:tolerance, the deviation is discarded. Otherwise, it is considered a discontuinity and subsequent output timestamp is resynced to the new position after performing configured discontinuity processing. In the non-perfect-timestamp case, an upstream variation exceeding tolerance only leads to marking DISCONT on subsequent outgoing (while timestamps are adjusted to upstream regardless of variation). While DISCONT is also marked in the perfect-timestamp case, this one optionally (see #GstAudioEncoder:hard-resync) performs some additional steps, such as clipping of (early) input samples or draining all currently remaining input data, depending on the direction of the discontuinity.

If perfect timestamps are arranged, it is also possible to request baseclass (usually set by subclass) to provide additional buffer metadata (in OFFSET and OFFSET_END) fields according to granule defined semantics currently needed by oggmux. Specifically, OFFSET is set to granulepos (= sample count including buffer) and OFFSET_END to corresponding timestamp (as determined by same sample count and sample rate).

Things that subclass need to take care of:

  • Provide pad templates
  • Set source pad caps when appropriate
  • Inform base class of buffer processing needs using context's frame_samples and frame_bytes.
  • Set user-configurable properties to sane defaults for format and implementing codec at hand, e.g. those controlling timestamp behaviour and discontinuity processing.
  • Accept data in @handle_frame and provide encoded results to gstaudio.audio_encoder.AudioEncoder.finishFrame.

Members

Functions

allocateOutputBuffer
gst.buffer.Buffer allocateOutputBuffer(size_t size)

Helper function that allocates a buffer to hold an encoded audio frame for enc's current output format.

finishFrame
gst.types.FlowReturn finishFrame(gst.buffer.Buffer buffer, int samples)

Collects encoded data and pushes encoded data downstream. Source pad caps must be set when this is called.

getAllocator
void getAllocator(gst.allocator.Allocator allocator, gst.allocation_params.AllocationParams params)

Lets #GstAudioEncoder sub-classes to know the memory allocator used by the base class and its params.

getAudioInfo
gstaudio.audio_info.AudioInfo getAudioInfo()
getDrainable
bool getDrainable()

Queries encoder drain handling.

getFrameMax
int getFrameMax()
getFrameSamplesMax
int getFrameSamplesMax()
getFrameSamplesMin
int getFrameSamplesMin()
getHardMin
bool getHardMin()

Queries encoder hard minimum handling.

getHardResync
bool getHardResync()
getLatency
void getLatency(gst.types.ClockTime min, gst.types.ClockTime max)

Sets the variables pointed to by min and max to the currently configured latency.

getLookahead
int getLookahead()
getMarkGranule
bool getMarkGranule()

Queries if the encoder will handle granule marking.

getPerfectTimestamp
bool getPerfectTimestamp()

Queries encoder perfect timestamp behaviour.

getTolerance
gst.types.ClockTime getTolerance()

Queries current audio jitter tolerance threshold.

mergeTags
void mergeTags(gst.tag_list.TagList tags, gst.types.TagMergeMode mode)

Sets the audio encoder tags and how they should be merged with any upstream stream tags. This will override any tags previously-set with gstaudio.audio_encoder.AudioEncoder.mergeTags.

negotiate
bool negotiate()

Negotiate with downstream elements to currently configured #GstCaps. Unmark GST_PAD_FLAG_NEED_RECONFIGURE in any case. But mark it again if negotiate fails.

proxyGetcaps
gst.caps.Caps proxyGetcaps(gst.caps.Caps caps, gst.caps.Caps filter)

Returns caps that express caps (or sink template caps if caps == NULL) restricted to channel/rate combinations supported by downstream elements (e.g. muxers).

setAllocationCaps
void setAllocationCaps(gst.caps.Caps allocationCaps)

Sets a caps in allocation query which are different from the set pad's caps. Use this function before calling gstaudio.audio_encoder.AudioEncoder.negotiate. Setting to null the allocation query will use the caps from the pad.

setDrainable
void setDrainable(bool enabled)

Configures encoder drain handling. If drainable, subclass might be handed a NULL buffer to have it return any leftover encoded data. Otherwise, it is not considered so capable and will only ever be passed real data.

setFrameMax
void setFrameMax(int num)

Sets max number of frames accepted at once (assumed minimally 1). Requires frame_samples_min and frame_samples_max to be the equal.

setFrameSamplesMax
void setFrameSamplesMax(int num)

Sets number of samples (per channel) subclass needs to be handed, at most or will be handed all available if 0.

setFrameSamplesMin
void setFrameSamplesMin(int num)

Sets number of samples (per channel) subclass needs to be handed, at least or will be handed all available if 0.

setHardMin
void setHardMin(bool enabled)

Configures encoder hard minimum handling. If enabled, subclass will never be handed less samples than it configured, which otherwise might occur near end-of-data handling. Instead, the leftover samples will simply be discarded.

setHardResync
void setHardResync(bool enabled)
setLatency
void setLatency(gst.types.ClockTime min, gst.types.ClockTime max)

Sets encoder latency. If the provided values changed from previously provided ones, this will also post a LATENCY message on the bus so the pipeline can reconfigure its global latency.

setLookahead
void setLookahead(int num)

Sets encoder lookahead (in units of input rate samples)

setMarkGranule
void setMarkGranule(bool enabled)

Enable or disable encoder granule handling.

setOutputFormat
bool setOutputFormat(gst.caps.Caps caps)

Configure output caps on the srcpad of enc.

setPerfectTimestamp
void setPerfectTimestamp(bool enabled)

Enable or disable encoder perfect output timestamp preference.

setTolerance
void setTolerance(gst.types.ClockTime tolerance)

Configures encoder audio jitter tolerance threshold.

Mixed In Members

From mixin PresetT!()

deletePreset
bool deletePreset(string name)

Delete the given preset.

getMeta
bool getMeta(string name, string tag, string value)

Gets the value for an existing meta data tag. Meta data tag names can be something like e.g. "comment". Returned values need to be released when done.

getPresetNames
string[] getPresetNames()

Get a copy of preset names as a null terminated string array.

getPropertyNames
string[] getPropertyNames()

Get a the names of the GObject properties that can be used for presets.

isEditable
bool isEditable()

Check if one can add new presets, change existing ones and remove presets.

loadPreset
bool loadPreset(string name)

Load the given preset.

renamePreset
bool renamePreset(string oldName, string newName)

Renames a preset. If there is already a preset by the new_name it will be overwritten.

savePreset
bool savePreset(string name)

Save the current object settings as a preset under the given name. If there is already a preset by this name it will be overwritten.

setMeta
bool setMeta(string name, string tag, string value)

Sets a new value for an existing meta data item or adds a new item. Meta data tag names can be something like e.g. "comment". Supplying null for the value will unset an existing value.

Inherited Members

From Element

makeFromUri
gst.element.Element makeFromUri(gst.types.URIType type, string uri, string elementname)

Creates an element for handling the given URI.

register
bool register(gst.plugin.Plugin plugin, string name, uint rank, gobject.types.GType type)

Create a new elementfactory capable of instantiating objects of the type and add the factory to plugin.

stateChangeReturnGetName
string stateChangeReturnGetName(gst.types.StateChangeReturn stateRet)

Gets a string representing the given state change result.

stateGetName
string stateGetName(gst.types.State state)

Gets a string representing the given state.

typeSetSkipDocumentation
void typeSetSkipDocumentation(gobject.types.GType type)

Marks type as "documentation should be skipped". Can be useful for dynamically registered element to be excluded from plugin documentation system.

abortState
void abortState()

Abort the state change of the element. This function is used by elements that do asynchronous state changes and find out something is wrong.

addPad
bool addPad(gst.pad.Pad pad)

Adds a pad (link point) to element. pad's parent will be set to element; see gst.object.ObjectGst.setParent for refcounting information.

addPropertyDeepNotifyWatch
gulong addPropertyDeepNotifyWatch(string propertyName, bool includeValue)
addPropertyNotifyWatch
gulong addPropertyNotifyWatch(string propertyName, bool includeValue)
callAsync
void callAsync(gst.types.ElementCallAsyncFunc func)

Calls func from another thread and passes user_data to it. This is to be used for cases when a state change has to be performed from a streaming thread, directly via gst.element.Element.setState or indirectly e.g. via SEEK events.

changeState
gst.types.StateChangeReturn changeState(gst.types.StateChange transition)

Perform transition on element.

continueState
gst.types.StateChangeReturn continueState(gst.types.StateChangeReturn ret)

Commit the state change of the element and proceed to the next pending state if any. This function is used by elements that do asynchronous state changes. The core will normally call this method automatically when an element returned gst.types.StateChangeReturn.Success from the state change function.

createAllPads
void createAllPads()

Creates a pad for each pad template that is always available. This function is only useful during object initialization of subclasses of #GstElement.

decorateStreamId
string decorateStreamId(string streamId)

Creates a stream-id for element by combining the upstream information with the stream_id.

foreachPad
bool foreachPad(gst.types.ElementForeachPadFunc func)

Call func with user_data for each of element's pads. func will be called exactly once for each pad that exists at the time of this call, unless one of the calls to func returns false in which case we will stop iterating pads and return early. If new pads are added or pads are removed while pads are being iterated, this will not be taken into account until next time this function is used.

foreachSinkPad
bool foreachSinkPad(gst.types.ElementForeachPadFunc func)

Call func with user_data for each of element's sink pads. func will be called exactly once for each sink pad that exists at the time of this call, unless one of the calls to func returns false in which case we will stop iterating pads and return early. If new sink pads are added or sink pads are removed while the sink pads are being iterated, this will not be taken into account until next time this function is used.

foreachSrcPad
bool foreachSrcPad(gst.types.ElementForeachPadFunc func)

Call func with user_data for each of element's source pads. func will be called exactly once for each source pad that exists at the time of this call, unless one of the calls to func returns false in which case we will stop iterating pads and return early. If new source pads are added or source pads are removed while the source pads are being iterated, this will not be taken into account until next time this function is used.

getBaseTime
gst.types.ClockTime getBaseTime()

Returns the base time of the element. The base time is the absolute time of the clock when this element was last put to PLAYING. Subtracting the base time from the clock time gives the running time of the element.

getBus
gst.bus.Bus getBus()

Returns the bus of the element. Note that only a #GstPipeline will provide a bus for the application.

getClock
gst.clock.Clock getClock()

Gets the currently configured clock of the element. This is the clock as was last set with gst.element.Element.setClock.

getCompatiblePad
gst.pad.Pad getCompatiblePad(gst.pad.Pad pad, gst.caps.Caps caps)

Looks for an unlinked pad to which the given pad can link. It is not guaranteed that linking the pads will work, though it should work in most cases.

getCompatiblePadTemplate
gst.pad_template.PadTemplate getCompatiblePadTemplate(gst.pad_template.PadTemplate compattempl)

Retrieves a pad template from element that is compatible with compattempl. Pads from compatible templates can be linked together.

getContext
gst.context.Context getContext(string contextType)

Gets the context with context_type set on the element or NULL.

getContextUnlocked
gst.context.Context getContextUnlocked(string contextType)

Gets the context with context_type set on the element or NULL.

getContexts
gst.context.Context[] getContexts()

Gets the contexts set on the element.

getCurrentClockTime
gst.types.ClockTime getCurrentClockTime()

Returns the current clock time of the element, as in, the time of the element's clock, or GST_CLOCK_TIME_NONE if there is no clock.

getCurrentRunningTime
gst.types.ClockTime getCurrentRunningTime()

Returns the running time of the element. The running time is the element's clock time minus its base time. Will return GST_CLOCK_TIME_NONE if the element has no clock, or if its base time has not been set.

getFactory
gst.element_factory.ElementFactory getFactory()

Retrieves the factory that was used to create this element.

getMetadata
string getMetadata(string key)

Get metadata with key in klass.

getPadTemplate
gst.pad_template.PadTemplate getPadTemplate(string name)

Retrieves a padtemplate from element with the given name.

getPadTemplateList
gst.pad_template.PadTemplate[] getPadTemplateList()

Retrieves a list of the pad templates associated with element. The list must not be modified by the calling code.

getRequestPad
gst.pad.Pad getRequestPad(string name)

The name of this function is confusing to people learning GStreamer. gst.element.Element.requestPadSimple aims at making it more explicit it is a simplified gst.element.Element.requestPad.

getStartTime
gst.types.ClockTime getStartTime()

Returns the start time of the element. The start time is the running time of the clock when this element was last put to PAUSED.

getState
gst.types.StateChangeReturn getState(gst.types.State state, gst.types.State pending, gst.types.ClockTime timeout)

Gets the state of the element.

getStaticPad
gst.pad.Pad getStaticPad(string name)

Retrieves a pad from element by name. This version only retrieves already-existing (i.e. 'static') pads.

isLockedState
bool isLockedState()

Checks if the state of an element is locked. If the state of an element is locked, state changes of the parent don't affect the element. This way you can leave currently unused elements inside bins. Just lock their state before changing the state from #GST_STATE_NULL.

iteratePads
gst.iterator.Iterator iteratePads()

Retrieves an iterator of element's pads. The iterator should be freed after usage. Also more specialized iterators exists such as gst.element.Element.iterateSrcPads or gst.element.Element.iterateSinkPads.

iterateSinkPads
gst.iterator.Iterator iterateSinkPads()

Retrieves an iterator of element's sink pads.

iterateSrcPads
gst.iterator.Iterator iterateSrcPads()

Retrieves an iterator of element's source pads.

link
bool link(gst.element.Element dest)

Links src to dest. The link must be from source to destination; the other direction will not be tried. The function looks for existing pads that aren't linked yet. It will request new pads if necessary. Such pads need to be released manually when unlinking. If multiple links are possible, only one is established.

linkFiltered
bool linkFiltered(gst.element.Element dest, gst.caps.Caps filter)

Links src to dest using the given caps as filtercaps. The link must be from source to destination; the other direction will not be tried. The function looks for existing pads that aren't linked yet. It will request new pads if necessary. If multiple links are possible, only one is established.

linkPads
bool linkPads(string srcpadname, gst.element.Element dest, string destpadname)

Links the two named pads of the source and destination elements. Side effect is that if one of the pads has no parent, it becomes a child of the parent of the other element. If they have different parents, the link fails.

linkPadsFiltered
bool linkPadsFiltered(string srcpadname, gst.element.Element dest, string destpadname, gst.caps.Caps filter)

Links the two named pads of the source and destination elements. Side effect is that if one of the pads has no parent, it becomes a child of the parent of the other element. If they have different parents, the link fails. If caps is not null, makes sure that the caps of the link is a subset of caps.

linkPadsFull
bool linkPadsFull(string srcpadname, gst.element.Element dest, string destpadname, gst.types.PadLinkCheck flags)

Links the two named pads of the source and destination elements. Side effect is that if one of the pads has no parent, it becomes a child of the parent of the other element. If they have different parents, the link fails.

lostState
void lostState()

Brings the element to the lost state. The current state of the element is copied to the pending state so that any call to gst.element.Element.getState will return gst.types.StateChangeReturn.Async.

messageFull
void messageFull(gst.types.MessageType type, glib.types.Quark domain, int code, string text, string debug_, string file, string function_, int line)

Post an error, warning or info message on the bus from inside an element.

messageFullWithDetails
void messageFullWithDetails(gst.types.MessageType type, glib.types.Quark domain, int code, string text, string debug_, string file, string function_, int line, gst.structure.Structure structure)

Post an error, warning or info message on the bus from inside an element.

noMorePads
void noMorePads()

Use this function to signal that the element does not expect any more pads to show up in the current pipeline. This function should be called whenever pads have been added by the element itself. Elements with #GST_PAD_SOMETIMES pad templates use this in combination with autopluggers to figure out that the element is done initializing its pads.

postMessage
bool postMessage(gst.message.Message message)

Post a message on the element's #GstBus. This function takes ownership of the message; if you want to access the message after this call, you should add an additional reference before calling.

provideClock
gst.clock.Clock provideClock()

Get the clock provided by the given element. > An element is only required to provide a clock in the PAUSED > state. Some elements can provide a clock in other states.

query
bool query(gst.query.Query query)

Performs a query on the given element.

queryConvert
bool queryConvert(gst.types.Format srcFormat, long srcVal, gst.types.Format destFormat, long destVal)

Queries an element to convert src_val in src_format to dest_format.

queryDuration
bool queryDuration(gst.types.Format format, long duration)

Queries an element (usually top-level pipeline or playbin element) for the total stream duration in nanoseconds. This query will only work once the pipeline is prerolled (i.e. reached PAUSED or PLAYING state). The application will receive an ASYNC_DONE message on the pipeline bus when that is the case.

queryPosition
bool queryPosition(gst.types.Format format, long cur)

Queries an element (usually top-level pipeline or playbin element) for the stream position in nanoseconds. This will be a value between 0 and the stream duration (if the stream duration is known). This query will usually only work once the pipeline is prerolled (i.e. reached PAUSED or PLAYING state). The application will receive an ASYNC_DONE message on the pipeline bus when that is the case.

releaseRequestPad
void releaseRequestPad(gst.pad.Pad pad)

Makes the element free the previously requested pad as obtained with gst.element.Element.requestPad.

removePad
bool removePad(gst.pad.Pad pad)

Removes pad from element. pad will be destroyed if it has not been referenced elsewhere using gst.object.ObjectGst.unparent.

removePropertyNotifyWatch
void removePropertyNotifyWatch(gulong watchId)
requestPad
gst.pad.Pad requestPad(gst.pad_template.PadTemplate templ, string name, gst.caps.Caps caps)

Retrieves a request pad from the element according to the provided template. Pad templates can be looked up using gst.element_factory.ElementFactory.getStaticPadTemplates.

requestPadSimple
gst.pad.Pad requestPadSimple(string name)

Retrieves a pad from the element by name (e.g. "src_\d"). This version only retrieves request pads. The pad should be released with gst.element.Element.releaseRequestPad.

seek
bool seek(double rate, gst.types.Format format, gst.types.SeekFlags flags, gst.types.SeekType startType, long start, gst.types.SeekType stopType, long stop)

Sends a seek event to an element. See gst.event.Event.newSeek for the details of the parameters. The seek event is sent to the element using gst.element.Element.sendEvent.

seekSimple
bool seekSimple(gst.types.Format format, gst.types.SeekFlags seekFlags, long seekPos)

Simple API to perform a seek on the given element, meaning it just seeks to the given position relative to the start of the stream. For more complex operations like segment seeks (e.g. for looping) or changing the playback rate or seeking relative to the last configured playback segment you should use gst.element.Element.seek.

sendEvent
bool sendEvent(gst.event.Event event)

Sends an event to an element. If the element doesn't implement an event handler, the event will be pushed on a random linked sink pad for downstream events or a random linked source pad for upstream events.

setBaseTime
void setBaseTime(gst.types.ClockTime time)

Set the base time of an element. See gst.element.Element.getBaseTime.

setBus
void setBus(gst.bus.Bus bus)

Sets the bus of the element. Increases the refcount on the bus. For internal use only, unless you're testing elements.

setClock
bool setClock(gst.clock.Clock clock)

Sets the clock for the element. This function increases the refcount on the clock. Any previously set clock on the object is unreffed.

setContext
void setContext(gst.context.Context context)

Sets the context of the element. Increases the refcount of the context.

setLockedState
bool setLockedState(bool lockedState)

Locks the state of an element, so state changes of the parent don't affect this element anymore.

setStartTime
void setStartTime(gst.types.ClockTime time)

Set the start time of an element. The start time of the element is the running time of the element when it last went to the PAUSED state. In READY or after a flushing seek, it is set to 0.

setState
gst.types.StateChangeReturn setState(gst.types.State state)

Sets the state of the element. This function will try to set the requested state by going through all the intermediary states and calling the class's state change function for each.

syncStateWithParent
bool syncStateWithParent()

Tries to change the state of the element to the same as its parent. If this function returns false, the state of element is undefined.

unlink
void unlink(gst.element.Element dest)

Unlinks all source pads of the source element with all sink pads of the sink element to which they are linked.

unlinkPads
void unlinkPads(string srcpadname, gst.element.Element dest, string destpadname)

Unlinks the two named pads of the source and destination elements.

connectNoMorePads
ulong connectNoMorePads(T callback, Flag!"After" after)

Connect to NoMorePads signal.

connectPadAdded
ulong connectPadAdded(T callback, Flag!"After" after)

Connect to PadAdded signal.

connectPadRemoved
ulong connectPadRemoved(T callback, Flag!"After" after)

Connect to PadRemoved signal.

From Preset

getAppDir
string getAppDir()

Gets the directory for application specific presets if set by the application.

setAppDir
bool setAppDir(string appDir)

Sets an extra directory as an absolute path that should be considered when looking for presets. Any presets in the application dir will shadow the system presets.

deletePreset
bool deletePreset(string name)

Delete the given preset.

getMeta
bool getMeta(string name, string tag, string value)

Gets the value for an existing meta data tag. Meta data tag names can be something like e.g. "comment". Returned values need to be released when done.

getPresetNames
string[] getPresetNames()

Get a copy of preset names as a null terminated string array.

getPropertyNames
string[] getPropertyNames()

Get a the names of the GObject properties that can be used for presets.

isEditable
bool isEditable()

Check if one can add new presets, change existing ones and remove presets.

loadPreset
bool loadPreset(string name)

Load the given preset.

renamePreset
bool renamePreset(string oldName, string newName)

Renames a preset. If there is already a preset by the new_name it will be overwritten.

savePreset
bool savePreset(string name)

Save the current object settings as a preset under the given name. If there is already a preset by this name it will be overwritten.

setMeta
bool setMeta(string name, string tag, string value)

Sets a new value for an existing meta data item or adds a new item. Meta data tag names can be something like e.g. "comment". Supplying null for the value will unset an existing value.