opus_multistream(3)
π opus_multistream β the Opus Multistream API, bestie edition π
okay so this is opus_multistream(3), and it lives in the Opus manual, section 3 (that's the library-functions section, iykyk). grab a strawberry drink π and let's get into it, because this one is lowkey a whole vibe.
β¨ NAME β¨
opus_multistream β the Opus Multistream API.
- the multistream API lets individual Opus streams get combined into a single packet, which is how we get support for up to 255 channels. yes. two hundred and fifty-five. iconic.
π¦ SYNOPSIS π¦
πΈ Typedefs πΈ
typedef struct OpusMSEncoder OpusMSEncoderβ Opus multistream encoder state. bestie, this is the encoder's whole personality.typedef struct OpusMSDecoder OpusMSDecoderβ Opus multistream decoder state. and this one is the decoder's.
π Multistream encoder functions π
opus_int32 opus_multistream_encoder_get_size (int streams, int coupled_streams)β Gets the size of anOpusMSEncoderstructure.opus_int32 opus_multistream_surround_encoder_get_size (int channels, int mapping_family)OpusMSEncoder * opus_multistream_encoder_create (opus_int32 Fs, int channels, int streams, int coupled_streams, const unsigned char *mapping, int application, int *error)β Allocates and initializes a multistream encoder state. β¨OpusMSEncoder * opus_multistream_surround_encoder_create (opus_int32 Fs, int channels, int mapping_family, int *streams, int *coupled_streams, unsigned char *mapping, int application, int *error)int opus_multistream_encoder_init (OpusMSEncoder *st, opus_int32 Fs, int channels, int streams, int coupled_streams, const unsigned char *mapping, int application)β Initialize a previously allocated multistream encoder state.int opus_multistream_surround_encoder_init (OpusMSEncoder *st, opus_int32 Fs, int channels, int mapping_family, int *streams, int *coupled_streams, unsigned char *mapping, int application)int opus_multistream_encode (OpusMSEncoder *st, const opus_int16 *pcm, int frame_size, unsigned char *data, opus_int32 max_data_bytes)β Encodes a multistream Opus frame.int opus_multistream_encode24 (OpusMSEncoder *st, const opus_int32 *pcm, int frame_size, unsigned char *data, opus_int32 max_data_bytes)β Encodes a multistream Opus frame.int opus_multistream_encode_float (OpusMSEncoder *st, const float *pcm, int frame_size, unsigned char *data, opus_int32 max_data_bytes)β Encodes a multistream Opus frame from floating point input.void opus_multistream_encoder_destroy (OpusMSEncoder *st)β Frees anOpusMSEncoderallocated byopus_multistream_encoder_create().int opus_multistream_encoder_ctl (OpusMSEncoder *st, int request,...)β Perform a CTL function on a multistream Opus encoder.
π Multistream decoder functions π
opus_int32 opus_multistream_decoder_get_size (int streams, int coupled_streams)β Gets the size of anOpusMSDecoderstructure.OpusMSDecoder * opus_multistream_decoder_create (opus_int32 Fs, int channels, int streams, int coupled_streams, const unsigned char *mapping, int *error)β Allocates and initializes a multistream decoder state.int opus_multistream_decoder_init (OpusMSDecoder *st, opus_int32 Fs, int channels, int streams, int coupled_streams, const unsigned char *mapping)β Initialize a previously allocated decoder state object.int opus_multistream_decode (OpusMSDecoder *st, const unsigned char *data, opus_int32 len, opus_int16 *pcm, int frame_size, int decode_fec)β Decode a multistream Opus packet.int opus_multistream_decode24 (OpusMSDecoder *st, const unsigned char *data, opus_int32 len, opus_int32 *pcm, int frame_size, int decode_fec)β Decode a multistream Opus packet.int opus_multistream_decode_float (OpusMSDecoder *st, const unsigned char *data, opus_int32 len, float *pcm, int frame_size, int decode_fec)β Decode a multistream Opus packet with floating point output.int opus_multistream_decoder_ctl (OpusMSDecoder *st, int request,...)β Perform a CTL function on a multistream Opus decoder.void opus_multistream_decoder_destroy (OpusMSDecoder *st)β Frees anOpusMSDecoderallocated byopus_multistream_decoder_create().
π Detailed Description π
okay, real talk time. the multistream API allows individual Opus streams to be combined into a single packet, and that's what gives you support for up to 255 channels. periodt.
here's the tea tho: unlike an elementary Opus stream, the encoder and decoder must negotiate the channel configuration before the decoder can successfully interpret the data in the packets produced by the encoder. some basic info β like packet duration β can be computed without any special negotiation, so that part's chill.
the format for multistream Opus packets is defined in RFC 7845, and it's based on the self-delimited Opus framing described in Appendix B of RFC 6716. and bestie, plot twist: normal Opus packets are just a degenerate case of multistream Opus packets π
, and you can encode or decode them with the multistream API by setting streams to 1 when initializing the encoder or decoder.
multistream Opus streams can contain up to 255 elementary Opus streams. these may be either "uncoupled" or "coupled", which basically means the decoder is configured to decode them to either 1 or 2 channels, respectively. and the streams are ordered so that all coupled streams appear at the beginning. no skipping the line, babes.
a mapping table defines which decoded channel i should be used for each input/output (I/O) channel j. this table is typically provided as an unsigned char array. let i = mapping[j] be the index for I/O channel j. so:
- if
i < 2*coupled_streams, then I/O channeljis encoded as the left channel of stream(i/2)ifiis even, or as the right channel of stream(i/2)ifiis odd. - otherwise, I/O channel
jis encoded as mono in stream(i - coupled_streams)β unless it has the special value255, in which case it is omitted from the encoding entirely (the decoder will reproduce it as silence π).
each value i must either be the special value 255 or be less than streams + coupled_streams. that's the rule, don't fight it.
the output channels specified by the encoder should use the Vorbis channel ordering. a decoder may wish to apply an additional permutation to the mapping the encoder used to achieve a different output channel order (e.g. for outputting in WAV order).
each multistream packet contains an Opus packet for each stream, and all of the Opus packets in a single multistream packet must have the same duration. therefore the duration of a multistream packet can be extracted from the TOC sequence of the first stream, which is located at the beginning of the packet, just like an elementary Opus stream:
int nb_samples;
int nb_frames;
nb_frames = opus_packet_get_nb_frames(data, len);
if (nb_frames < 1)
return nb_frames;
nb_samples = opus_packet_get_samples_per_frame(data, 48000) * nb_frames;
the general encoding and decoding process proceeds exactly the same as in the normal Opus Encoder and Opus Decoder APIs. go read their documentation for an overview of how to use the corresponding multistream functions. π«Ά
π§ Typedef Documentation π§
β typedef struct OpusMSDecoder OpusMSDecoder β
Opus multistream decoder state. this contains the complete state of a multistream Opus decoder. it is position independent and can be freely copied. no cap.
See also
opus_multistream_decoder_createopus_multistream_decoder_init
π typedef struct OpusMSEncoder OpusMSEncoder π
Opus multistream encoder state. this contains the complete state of a multistream Opus encoder. it is position independent and can be freely copied. same energy, different direction.
See also
opus_multistream_encoder_createopus_multistream_encoder_init
π₯ Function Documentation π₯
π« int opus_multistream_decode (OpusMSDecoder * st, const unsigned char * data, opus_int32 len, opus_int16 * pcm, int frame_size, int decode_fec) π«
Decode a multistream Opus packet.
Parameters
- st
OpusMSDecoder*: Multistream decoder state. - data
const unsigned char*: Input payload. Use aNULLpointer to indicate packet loss. - len
opus_int32: Number of bytes in payload. - pcm
opus_int16*: Output signal, with interleaved samples. This must contain room forframe_size*channelssamples. - frame_size
int: The number of samples per channel of available space inpcm. If this is less than the maximum packet duration (120 ms; 5760 for 48kHz), this function will not be capable of decoding some packets. In the case of PLC (data==NULL) or FEC (decode_fec=1), thenframe_sizeneeds to be exactly the duration of audio that is missing, otherwise the decoder will not be in the optimal state to decode the next incoming packet. For the PLC and FEC cases,frame_sizemust be a multiple of 2.5 ms. - decode_fec
int: Flag (0or1) to request that any in-band forward error correction data be decoded. If no such data is available, the frame is decoded as if it were lost.
Returns
- Number of samples decoded on success or a negative error code (see Error codes) on failure.
bestie pls do not skip the frame_size rules, the decoder will be in its flop era if you get it wrong π
π« int opus_multistream_decode24 (OpusMSDecoder * st, const unsigned char * data, opus_int32 len, opus_int32 * pcm, int frame_size, int decode_fec) π«
Decode a multistream Opus packet.
Parameters
- st
OpusMSDecoder*: Multistream decoder state. - data
const unsigned char*: Input payload. Use aNULLpointer to indicate packet loss. - len
opus_int32: Number of bytes in payload. - pcm
opus_int32*: Output signal, with interleaved samples representing (or slightly exceeding) 24-bit values. This must contain room forframe_size*channelssamples. - frame_size
int: The number of samples per channel of available space inpcm. If this is less than the maximum packet duration (120 ms; 5760 for 48kHz), this function will not be capable of decoding some packets. In the case of PLC (data==NULL) or FEC (decode_fec=1), thenframe_sizeneeds to be exactly the duration of audio that is missing, otherwise the decoder will not be in the optimal state to decode the next incoming packet. For the PLC and FEC cases,frame_sizemust be a multiple of 2.5 ms. - decode_fec
int: Flag (0or1) to request that any in-band forward error correction data be decoded. If no such data is available, the frame is decoded as if it were lost.
Returns
- Number of samples decoded on success or a negative error code (see Error codes) on failure.
π« int opus_multistream_decode_float (OpusMSDecoder * st, const unsigned char * data, opus_int32 len, float * pcm, int frame_size, int decode_fec) π«
Decode a multistream Opus packet with floating point output.
Parameters
st(OpusMSDecoder*) β the multistream decoder state. That's the whole vibe container πdata(const unsigned char*) β the input payload. Pass aNULLpointer here to say "bestie this packet is GONE" (i.e. packet loss π).len(opus_int32) β how many bytes are chilling in that payload.pcm(opus_int16*) β the output signal, samples interleaved. It must have room forframe_size*channelssamples, no skimping, fr fr.frame_size(int) β the number of samples per channel of available space inpcm. If this is less than the maximum packet duration (120 ms, which is5760at 48kHz), then this function will not be capable of decoding some packets, and that's just the rules π€. In the case of PLC (data==NULL) or FEC (decode_fec=1), thenframe_sizeneeds to be exactly the duration of audio that is missing, otherwise the decoder will not be in the optimal state to decode the next incoming packet. And for the PLC and FEC cases specifically,frame_sizemust be a multiple of2.5 ms. Slay, but precisely.decode_fec(int) β a flag (0or1) requesting that any in-band forward error correction data be decoded. If no such data is available, the frame is decoded as if it were lost. Ngl that's just how it goes.
Returns
Number of samples decoded on success, or a negative error code (see Error codes) on failure. β¨
β¨ opus_multistream_decoder_create() β¨
OpusMSDecoder * opus_multistream_decoder_create (opus_int32 Fs, int
channels, int streams, int coupled_streams, const unsigned char *
mapping, int * error)
Allocates and initializes a multistream decoder state. Call opus_multistream_decoder_destroy() to release this object when you're done with it β don't just ghost it, bestie π«Ά.
Parameters
Fs(opus_int32) β sampling rate to decode at (in Hz). Must be one of8000,12000,16000,24000, or48000. That's the whole menu, no substitutions.channels(int) β number of channels to output. Must be at most255. It may be different from the number of coded channels (streams + coupled_streams). Flexible, iconic πstreams(int) β the total number of streams coded in the input. Must be no more than255.coupled_streams(int) β number of streams to decode as coupled (2 channel) streams. Must be no larger than the total number of streams. Additionally, the total number of coded channels (streams + coupled_streams) must be no more than255.mapping(const unsigned char[channels]) β mapping from coded channels to output channels, as described in Opus Multistream API.error(int *) β returnsOPUS_OKon success, or an error code (see Error codes) on failure.
π¦ opus_multistream_decoder_ctl() π¦
int opus_multistream_decoder_ctl (OpusMSDecoder * st, int request, ...)
Performs a CTL function on a multistream Opus decoder. Generally the request and subsequent arguments are generated by a convenience macro β let the macro cook π
Parameters
st(OpusMSDecoder*) β multistream decoder state.requestβ this and all remaining parameters should be replaced by one of the convenience macros in Generic CTLs, Decoder related CTLs, or Multistream specific encoder and decoder CTLs.
See also
- Generic CTLs
- Decoder related CTLs
- Multistream specific encoder and decoder CTLs
πΈ opus_multistream_decoder_destroy() πΈ
void opus_multistream_decoder_destroy (OpusMSDecoder * st)
Frees an OpusMSDecoder allocated by opus_multistream_decoder_create(). Cleanup era, we love a responsible main character π§Ήβ¨
Parameters
st(OpusMSDecoder) β multistream decoder state to be freed.
β opus_multistream_decoder_get_size() β
opus_int32 opus_multistream_decoder_get_size (int streams, int
coupled_streams)
Gets the size of an OpusMSDecoder structure. Size check first, always π
Parameters
streams(int) β the total number of streams coded in the input. Must be no more than255.coupled_streams(int) β number streams to decode as coupled (2 channel) streams. Must be no larger than the total number of streams. Additionally, the total number of coded channels (streams + coupled_streams) must be no more than255.
Returns
The size in bytes on success, or a negative error code (see Error codes) on error.
π opus_multistream_decoder_init() π
int opus_multistream_decoder_init (OpusMSDecoder * st, opus_int32 Fs, int
channels, int streams, int coupled_streams, const unsigned char *
mapping)
Initializes a previously allocated decoder state object. The memory pointed to by st must be at least the size returned by opus_multistream_encoder_get_size(). This is intended for applications which use their own allocator instead of malloc β DIY energy, respect π«‘. To reset a previously initialized state, use the OPUS_RESET_STATE CTL.
See also
opus_multistream_decoder_createopus_multistream_deocder_get_size
Parameters
st(OpusMSEncoder*) β multistream encoder state to initialize.Fs(opus_int32) β sampling rate to decode at (in Hz). Must be one of8000,12000,16000,24000, or48000.channels(int) β number of channels to output. Must be at most255. It may be different from the number of coded channels (streams + coupled_streams).streams(int) β the total number of streams coded in the input. Must be no more than255.coupled_streams(int) β number of streams to decode as coupled (2 channel) streams. Must be no larger than the total number of streams. Additionally, the total number of coded channels (streams + coupled_streams) must be no more than255.mapping(const unsigned char[channels]) β mapping from coded channels to output channels, as described in Opus Multistream API.
Returns
OPUS_OK on success, or an error code (see Error codes) on failure. β¨
π₯ opus_multistream_encode() π₯
int opus_multistream_encode (OpusMSEncoder * st, const opus_int16 * pcm, int
frame_size, unsigned char * data, opus_int32 max_data_bytes)
Encodes a multistream Opus frame. Let's get it encoded, bestie π
Parameters
st(OpusMSEncoder*) β multistream encoder state.pcm(const opus_int16*) β the input signal as interleaved samples. This must containframe_size*channelssamples.frame_size(int) β number of samples per channel in the input signal. This must be an Opus frame size for the encoder's sampling rate. For example, at 48 kHz the permitted values are120,240,480,960,1920, and2880. Passing in a duration of less than 10 ms (480samples at 48 kHz) will prevent the encoder from using the LPC or hybrid modes β so like, short frames = limited mode options, noted πdata(unsigned char*) β output payload. This must contain storage for at leastmax_data_bytes.max_data_bytes(opus_int32) β size of the allocated memory for the output payload. This may be used to impose an upper limit on the instant bitrate, but should not be used as the only bitrate control. UseOPUS_SET_BITRATEto control the bitrate. (Real ones useOPUS_SET_BITRATE, no cap.)
Returns
The length of the encoded packet (in bytes) on success or a negative error code (see Error codes) on failure.
π opus_multistream_encode24() π
int opus_multistream_encode24 (OpusMSEncoder * st, const opus_int32 * pcm,
int frame_size, unsigned char * data, opus_int32 max_data_bytes)
Encodes a multistream Opus frame. (24-bit flavor, extra fancy β¨)
Parameters
st(OpusMSEncoder*) β multistream encoder state.pcm(const opus_int32*) β the input signal as interleaved samples representing (or slightly exceeding) 24-bit values. This must containframe_size*channelssamples.frame_size(int) β number of samples per channel in the input signal. This must be an Opus frame size for the encoder's sampling rate. For example, at 48 kHz the permitted values are120,240,480,960,1920, and2880. Passing in a duration of less than 10 ms (480samples at 48 kHz) will prevent the encoder from using the LPC or hybrid modes.data(unsigned char*) β output payload. This must contain storage for at leastmax_data_bytes.max_data_bytes(opus_int32) β size of the allocated memory for the output payload. This may be used to impose an upper limit on the instant bitrate, but should not be used as the only bitrate control. UseOPUS_SET_BITRATEto control the bitrate.
Returns
The length of the encoded packet (in bytes) on success or a negative error code (see Error codes) on failure.
π opus_multistream_encode_float() π
int opus_multistream_encode_float (OpusMSEncoder * st, const float * pcm,
int frame_size, unsigned char * data, opus_int32 max_data_bytes)
Encodes a multistream Opus frame from floating point input. Floats, the glamorous option π«§
Parameters
st(OpusMSEncoder*) β multistream encoder state.pcm(const float*) β the input signal as interleaved samples with a normal range of+/-1.0. Samples with a range beyond+/-1.0are supported but will be clipped by decoders using the integer API and should only be used if it is known that the far end supports extended dynamic range. This must containframe_size*channelssamples. > So yeah, going past+/-1.0is giving risky, only do it if the other side can handle it πframe_size(int) β number of samples per channel in the input signal. This must be an Opus frame size for the encoder's sampling rate. For example, at 48 kHz the permitted values are120,240,480,960,1920, and2880. Passing in a duration of less than 10 ms (480samples at 48 kHz) will prevent the encoder from using the LPC or hybrid modes.data(unsigned char*) β output payload. This must contain storage for at leastmax_data_bytes.max_data_bytes(opus_int32) β size of the allocated memory for the output payload. This may be used to impose an upper limit on the instant bitrate, but should not be used as the only bitrate control. UseOPUS_SET_BITRATEto control the bitrate.
Returns
The length of the encoded packet (in bytes) on success or a negative error code (see Error codes) on failure.
π opus_multistream_encoder_create() π
OpusMSEncoder * opus_multistream_encoder_create (opus_int32 Fs, int
channels, int streams, int coupled_streams, const unsigned char *
mapping, int application, int * error)
Allocates and initializes a multistream encoder state. Call opus_multistream_encoder_destroy() to release this object when finished β again, cleanup is self care π§
Parameters
Fs(opus_int32) β sampling rate of the input signal (in Hz). Must be one of8000,12000,16000,24000, or48000.channels(int) β number of channels in the input signal. Must be at most255. It may be greater than the number of coded channels (streams + coupled_streams).streams(int) β the total number of streams to encode from the input. Must be no more than the number of channels.coupled_streams(int) β number of coupled (2 channel) streams to encode. Must be no larger than the total number of streams. Additionally, the total number of encoded channels (streams + coupled_streams) must be no more than the number of input channels.mapping(const unsigned char[channels]) β mapping from encoded channels to input channels, as described in Opus Multistream API. As an extra constraint, the multistream encoder does not allow encoding coupled streams for which one channel is unused, since this is never a good idea. Periodt.application(int) β the target encoder application. Must be one of the following:OPUS_APPLICATION_VOIPβ Process signal for improved speech intelligibility.OPUS_APPLICATION_AUDIO
OPUS_APPLICATION_AUDIOβ favor faithfulness to the original input. bestie this is the "don't mess with my music" setting, it keeps the vibe of the input intact πOPUS_APPLICATION_RESTRICTED_LOWDELAYβ configure the minimum possible coding delay by disabling certain modes of operation. lowkey it trades options away for speed β¨error*int **: returnsOPUS_OKon success, or an error code (see Error codes) on failure. π«Ά
int opus_multistream_encoder_ctl (OpusMSEncoder * st, int request, ...)
Perform a CTL function on a multistream Opus encoder. Generally the request and subsequent arguments are generated by a convenience macro β so like, you don't hand-write these, the macro does it for you, iykyk π
Parameters πΈ
st*OpusMSEncoder**: multistream encoder state.requestβ this and all remaining parameters should be replaced by one of the convenience macros in Generic CTLs, Encoder related CTLs, or Multistream specific encoder and decoder CTLs.
See also π¦
- Generic CTLs
- Encoder related CTLs
- Multistream specific encoder and decoder CTLs
void opus_multistream_encoder_destroy (OpusMSEncoder * st)
Frees an OpusMSEncoder allocated by opus_multistream_encoder_create(). it's giving cleanup era, we love a tidy queen π§
Parameters π
st*OpusMSEncoder**: multistream encoder state to be freed.
opus_int32 opus_multistream_encoder_get_size (int streams, int coupled_streams)
Gets the size of an OpusMSEncoder structure. size check, slay π
Parameters β¨
streamsint: the total number of streams to encode from the input. this must be no more than 255.coupled_streamsint: number of coupled (2 channel) streams to encode. this must be no larger than the total number of streams. additionally, the total number of encoded channels (streams + coupled_streams) must be no more than 255.
Returns π
The size in bytes on success, or a negative error code (see Error codes) on error.
int opus_multistream_encoder_init (OpusMSEncoder * st, opus_int32 Fs, int channels, int streams, int coupled_streams, const unsigned char * mapping, int application)
Initialize a previously allocated multistream encoder state. the memory pointed to by st must be at least the size returned by opus_multistream_encoder_get_size(). this is intended for applications which use their own allocator instead of malloc. to reset a previously initialized state, use the OPUS_RESET_STATE CTL. β¨
See also π«Ά
opus_multistream_encoder_createopus_multistream_encoder_get_size
Parameters π
st*OpusMSEncoder**: multistream encoder state to initialize.Fsopus_int32: sampling rate of the input signal (in Hz). this must be one of8000,12000,16000,24000, or48000.channelsint: number of channels in the input signal. this must be at most255. it may be greater than the number of coded channels (streams + coupled_streams).streamsint: the total number of streams to encode from the input. this must be no more than the number of channels.coupled_streamsint: number of coupled (2 channel) streams to encode. this must be no larger than the total number of streams. additionally, the total number of encoded channels (streams + coupled_streams) must be no more than the number of input channels.mappingconst unsigned char[channels]: mapping from encoded channels to input channels, as described in Opus Multistream API. as an extra constraint, the multistream encoder does not allow encoding coupled streams for which one channel is unused since this is never a good idea. bestie, don't do it. periodt πapplicationint: the target encoder application. this must be one of the following:OPUS_APPLICATION_VOIPβ process signal for improved speech intelligibility.OPUS_APPLICATION_AUDIOβ favor faithfulness to the original input.OPUS_APPLICATION_RESTRICTED_LOWDELAYβ configure the minimum possible coding delay by disabling certain modes of operation.
Returns β
OPUS_OK on success, or an error code (see Error codes) on failure.
and then these three just vibe here, signatures and all, no extra lore attached:
OpusMSEncoder * opus_multistream_surround_encoder_create (opus_int32 Fs, int
channels, int mapping_family, int * streams, int * coupled_streams,
unsigned char * mapping, int application, int * error)
opus_int32 opus_multistream_surround_encoder_get_size (int channels, int
mapping_family)
int opus_multistream_surround_encoder_init (OpusMSEncoder * st, opus_int32
Fs, int channels, int mapping_family, int * streams, int *
coupled_streams, unsigned char * mapping, int application)
π Author π
Generated automatically by Doxygen for Opus from the source code. (the bots ate, fr fr π€π )
okay bestie that's a wrap on the whole page π this was Version 1.6.1, dated Thu Jan 29 2026 14:35:32, and you just read opus_multistream(3). go forth and multistream, you absolute icon πβ¨π¦
π¦ where this page comes from
- ubuntu resolute: libopus-doc 1.6.1-1 Β· BSD-3-Clause
- ubuntu stonking: libopus-doc 1.6.1-1 Β· BSD-3-Clause
- π girlified by deepseek-ai/DeepSeek-V4.1-Flash on : a language model's rewrite of the original, under its terms