Get started with GStreamer

This is about GStreamer on Linux.

Installing

https://gstreamer.freedesktop.org/documentation/installing/on-linux.html

Playing video the simple way

To play a video from command line, we can use gst-launch-1.0.

gst-launch-1.0 is a tool that builds and runs basic GStreamer pipelines.

In its simplest form, a PIPELINE-DESCRIPTION is a list of elements separated by exclamation marks (!). Properties may be appended to elements in the form property=value.

Please note that gst-launch-1.0 is primarily a debugging tool. You should not build applications on top of it. For applications, use the gst_parse_launch() function of the GStreamer API as an easy way to construct pipelines from pipeline descriptions.

A pipeline consists of elements and links. Elements can be put into bins of different sorts. Elements, links, and bins can be specified in a pipeline description in any order.

Please read this to understand more about GStreamer pipeline.

Let’s start with the most easy way to play video by using playbin.

Playbin provides a stand-alone everything-in-one abstraction for an audio and/or video player.

Learn more about playbin.

For example, I have a video located at /home/user/Videos/trailer_1080p_h264_mp3.avi (video can be downloaded from http://linode.boundarydevices.com/videos/trailer_1080p_h264_mp3.avi):

$ gst-launch-1.0 playbin uri=file:///home/user/Videos/trailer_1080p_h264_mp3.avi
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Redistribute latency...
Redistribute latency...
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstPulseSinkClock
Got EOS from element "playbin0".
Execution ended after 0:00:33.057856101
Setting pipeline to NULL ...
Freeing pipeline ...

Try other video sink:

$ gst-launch-1.0 playbin uri=file:///home/user/Videos/trailer_1080p_h264_mp3.avi \
                         video-sink=glimagesink

As we know, playbin select elements automatically. Let’s see which elements are really used in the above pipeline.
Note: These element selections depend on hardware, driver, graphic environment, …, so they may be different in your system.

One simple and intuitive way to check this is using pipeline graphs.

These are .dot files, readable with free programs like GraphViz, that describe the topology of your pipeline, along with the caps negotiated in each link.

To obtain .dot files, simply set the GST_DEBUG_DUMP_DOT_DIR environment variable to point to the folder where you want the files to be placed. gst-launch-1.0 will create a .dot file at each state change, so you can see the evolution of the caps negotiation.

$ export GST_DEBUG_DUMP_DOT_DIR=/tmp/pipe_graph
$ mkdir -p $GST_DEBUG_DUMP_DOT_DIR
$ gst-launch-1.0 playbin uri=file:///home/user/Videos/trailer_1080p_h264_mp3.avi

There will be several .dot files created, let’s check one of them. You can convert .dot file to image by command dot from package graphviz.

$ ls $GST_DEBUG_DUMP_DOT_DIR
0.00.00.013971195-gst-launch.NULL_READY.dot    0.00.00.209734072-gst-launch.PAUSED_PLAYING.dot  0.00.33.329271430-gst-launch.PAUSED_READY.dot
0.00.00.205720496-gst-launch.READY_PAUSED.dot  0.00.33.266401422-gst-launch.PLAYING_PAUSED.dot

$ dot -Tpng $GST_DEBUG_DUMP_DOT_DIR/0.00.33.266401422-gst-launch.PLAYING_PAUSED.dot > /tmp/pipeline.png

Custom pipeline

We can play a video easily with playbin, but what if we want to specify each element by our own? Here is another example for playing video:

$ gst-launch-1.0 filesrc location=/home/user/Videos/trailer_1080p_h264_mp3.avi \
                 ! avidemux ! h264parse \
                 ! avdec_h264 ! videoconvert \
                 ! xvimagesink

Break down:

filesrc: Read from arbitrary point in a file. Path of video file is set through its property location.
avidemux: Demultiplex an avi file into audio and video.
h264parse: Parses H.264 streams.
avdec_h264: libav h264 decoder.
xvimagesink: A Xv based videosink.

GStreamer Plugin/Element info can be checked by command gst-inspect-1.0. For example:

$ gst-inspect-1.0 filesrc
Factory Details:
  Rank                     primary (256)
  Long-name                File Source
  Klass                    Source/File
  Description              Read from arbitrary point in a file
  Author                   Erik Walthinsen <[email protected]>

Plugin Details:
  Name                     coreelements
  Description              GStreamer core elements
  Filename                 /usr/lib64/gstreamer-1.0/libgstcoreelements.so
  Version                  1.18.4
  License                  LGPL
  Source module            gstreamer
  Source release date      2021-03-15
  Binary package           Fedora GStreamer package
  Origin URL               http://download.fedoraproject.org

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseSrc
                         +----GstFileSrc

Implemented Interfaces:
  GstURIHandler

Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      ANY

Element has no clocking capabilities.

URI handling capabilities:
  Element can act as source.
  Supported URI protocols:
    file

Pads:
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
  blocksize           : Size in bytes to read per buffer (-1 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096 
  do-timestamp        : Apply current stream time to buffers
                        flags: readable, writable
                        Boolean. Default: false
  location            : Location of the file to read
                        flags: readable, writable, changeable only in NULL or READY state
                        String. Default: null
  name                : The name of the object
                        flags: readable, writable, 0x2000
                        String. Default: "filesrc0"
  num-buffers         : Number of buffers to output before sending EOS (-1 = unlimited)
                        flags: readable, writable
                        Integer. Range: -1 - 2147483647 Default: -1 
  parent              : The parent of the object
                        flags: readable, writable, 0x2000
                        Object of type "GstObject"
  typefind            : Run typefind before negotiating (deprecated, non-functional)
                        flags: readable, writable, deprecated
                        Boolean. Default: false

filesrc info is not much, you should try checking other elements’ info too.

Element Properties are what you can set from command line. Other things like Pads, Element Signals are often used in code implementation.

Queue

The above custom pipeline only play video. In able to process both video and audio at the same time, we will need an element called queue.

See the detail here.

The queue will create a new thread on the source pad to decouple the processing on sink and source pad.

Here is how to use it:

$ gst-launch-1.0 filesrc location=/home/user/Videos/trailer_1080p_h264_mp3.avi \
                 ! avidemux name=dm \
             dm. ! queue ! mpegaudioparse ! avdec_mp3 \
                 ! audioconvert ! audioresample ! pulsesink \
             dm. ! queue ! h264parse ! avdec_h264 ! videoconvert ! xvimagesink