Timestamp Calculation With GStreamer

The power of GStreamer brings usage difficulties in some cases. For instance, If you want to try something custom like calculating the absolute timestamp of the frames in the stream, this can be a bit tiring because of the many unknowns such as;
-which element provides the RTCP packet
-which element provides the RTP packet header
-will these information be accessed through the element’s buffer or events
-whether the time information obtained is the required RTP time or a time that GStreamer uses in its own operation.

As mentioned above, NTP timestamp and reference RTP time in RTCP packets are needed to calculate the absolute timestamp of a frame. The element via which we can access RTCP packets in GStreamer is the rtpbin element, so the rtpbin element must be included in the pipeline. The rtpbin combines the functions of several elements into a single element allowing for multiple RTP sessions and these will be synchronized together using RTCP SR packets.
Another requirement of calculating absolute timestamp of frames is time values in RTP headers. In GStreamer, these time values can be reached by adding a probe to the pads of the elements in pipelines. After these three time values are obtained, if the clockrate of the server is also known, absolute timestamp can be calculated for each frame. In the next section, a sample GStreamer pipeline and frame timestamp calculation steps are shown.

Sample Code

The pipeline below consists of rtspsrc, rtph264depay and appsink elements (Note that the pipeline is simplified for better understanding). While rtspsrc makes a connection to an RTSP server and reads the data, rtph264depay extracts H264 video from RTP packets. Also, appsink makes the application get a handle on GStreamer data. The rtpbin element is not included in this pipeline, but it will be created as rtspsrc manager.

First of all, we need to capture and connect the creation signal of the rtpbin element so that we can access it. In order to do this, rtspsrc element is taken from pipeline as in line 5. This element produces a “new-manager” signal after creating the manager. For the rtpbin element, a callback function is connected to this signal as in the 6th line. We will use the session information that we obtain in this function to access the RTCP SR.

Then we need to access the time values in the RTP packets. For this purpose, we can add a probe to the sink pad of the rtph264depay element. The rtph264depay element as in line 8 and the sink pad of this element as in line 9 are taken. We can connect the function by adding a probe to this obtained pad as in line 10.Then we need to access the time values in the RTP packets. For this purpose, we can add a probe to the sink pad of the rtph264depay element. The rtph264depay element as in line 8 and the sink pad of this element as in line 9 are taken. We can connect the function by adding a probe to this obtained pad as in line 10.Then we need to access the time values in the RTP packets. For this purpose, we can add a probe to the sink pad of the rtph264depay element. The rtph264depay element as in line 8 and the sink pad of this element as in line 9 are taken. We can connect the function by adding a probe to this obtained pad as in line 10.Then we need to access the time values in the RTP packets. For this purpose, we can add a probe to the sink pad of the rtph264depay element. The rtph264depay element as in line 8 and the sink pad of this element as in line 9 are taken. We can connect the function by adding a probe to this obtained pad as in line 10.