i.MX6 Video acceleration on Ubuntu Raring (and Debian)

Published on October 4, 2013

Archived Notice

This article has been archived and may contain broken links, photos and out-of-date information. If you have any questions, please Contact Us.

When we recently posted Linaro Raring images for i.MX6, we were surprised at the number responses regarding hardware acceleration, and especially video acceleration.

For a long time now, we've been saying that it could be done, but leaving it as an exercise for the reader. In other words, go read the manuals and figure it out. Based on the response, it's probably time to show some details.

It's not very pretty, but the following will show you how it can be done. I recommend a full cup of coffee and a SATA drive, because almost all of this will occur on the device, and native compilation is very slow on SD card.

Also note that these notes apply and have been tested on Debian Wheezy, except that the names of some of the prerequisites are different. Also note that I'm starting with raring-nano to make sure I capture all of them.

First, the pre-requisites

First, make sure your system is up-to-date: root@linaro-nano:~# apt-get update Then you'll need a compiler: root@linaro-nano:~# apt-get install build-essential And the ability to compile gstreamer plugins:

root@linaro-nano:~# apt-get install libgstreamer-plugins-base0.10-dev
root@linaro-nano:~# apt-get build-dep gst-plugins-base0.10

During the steps above, you're going to get messages like this: /sbin/ldconfig.real: /usr/lib/libOpenVG.so is not a symbolic link In English, this is a warning that the Vivante libraries we installed don't follow proper Linux shared-library versioning standards (where .so files are supposed to be symlinks to .so.version).

You can safely ignore them.

Then get the packages

The packages required for building the gstreamer plugins are as follows:

A set of kernel headers
firmware-imx-3.5.7-1.0.0
imx-lib-3.5.7-1.0.0
fsl-alsa-plugins-3.5.7-1.0.0
libfslcodec-3.5.7-1.0.0
libfslparser-3.5.7-1.0.0
libfslvpuwrap-3.5.7-1.0.0
gst-fsl-plugins-3.5.7-1.0.0

Kernel headers

Because some of these packages use hardware-specific APIs, they expect some i.MX6-specific header files to be present in /usr/include on the build machine.

There are two simple ways to accomplish this:

If you've already built the kernel, you can use the headers_install target in the kernel build tree to create a set of headers and simply copy them into /usr/include, or
You can use the Debian approach of make-kpkg to build a linux-headers-versionnum.deb package and use dpkg -i to install it.

To make this easy, we've uploaded a package to our cloud storage site with a recent version of the tar-ball version that you can grab and extract as shown below. We had some difficulty with make-kpkg generating an armhf package with a cross-compiler.

root@linaro-nano:~# wget https://commondatastorage.googleapis.com/boundarydevices.com/linux-headers-3.0.35-02829-gac24896_4.1.0.tar.gz
root@linaro-nano:~# tar zxvf linux-headers-3.0.35-02829-gac24896_4.1.0.tar.gz -C /usr

Getting the other packages from Freescale - the LTIB way

There are two routes to the other packages, which are distributed by Freescale. The first is to download two packages from the Freescale 3.0.35_4.1.0 release page:

L3.0.35_4.1.0_ER_SOURCE_BSP - the core BSP sources
L3.0.35_4.1.0_MM_CODECS - the codec sources

Inside the L3.0.35_4.1.0_ER_SOURCE_BSP package, you'll find tar-balls for both the imx-lib-3.0.35-4.1.0 and firmware-imx-3.0.35-4.1.0 packages, and you'll find tar-balls of the other packages inside L3.0.35_4.1.0_MM_CODECS package in the ltib codecs directory. Note that although there are .deb packages inside the codec package, we weren't successful in compiling from them.

Getting the other packages from Freescale - the Yocto way

Another way to get the codec and BSP packages is to download them from the Freescale Yocto repository at https://www.freescale.com/lgfiles/NMG/MAD/YOCTO/. This location allows for automated download, but because some of the sources require acceptance of a EULA, they're packaged as self-extracting shell scripts (.bin files).

Getting the packages is very easy using this route:

# first, the open-source codec package:
root@linaro-nano:~# wget https://www.freescale.com/lgfiles/NMG/MAD/YOCTO/gst-fsl-plugins-3.5.7-1.0.0.tar.gz
# then the restricted files
root@linaro-nano:~# for pkg in firmware-imx
                             imx-lib
                             fsl-alsa-plugins
                             libfslcodec
                             libfslparser
                             libfslvpuwrap ; do
                         wget https://www.freescale.com/lgfiles/NMG/MAD/YOCTO/${pkg}-3.5.7-1.0.0.bin ;
                     done

But extracting is a bit more complicated, requiring you to accept the EULA for each:

root@linaro-nano:~# tar zxf gst-fsl-plugins-3.5.7-1.0.0.tar.gz
root@linaro-nano:~# for pkg in firmware-imx
                             imx-lib
                             fsl-alsa-plugins
                             libfslcodec
                             libfslparser
                             libfslvpuwrap ; do
                         sh ${pkg}-3.5.7-1.0.0.bin ;
                     done

Building the packages

Building (or installing) the packages is relatively straightforward once you know the flags required for each.

The firmware-imx package simply requires copying files to the /lib/firmware directory: root@linaro-nano:~# cp -ravf firmware-imx-3.5.7-1.0.0/firmware/* /lib/firmware/ The imx-lib package seems to have failures building the rng library, so we've skipped it. It also requires the PLATFORM=IMX6Q flag:

root@linaro-nano:~# cd imx-lib-3.5.7-1.0.0/
root@linaro-nano:~/imx-lib-3.5.7-1.0.0# rm -rf rng
root@linaro-nano:~/imx-lib-3.5.7-1.0.0# make PLATFORM=IMX6Q all && make PLATFORM=IMX6Q install

The fsl-alsa-plugins package requires some autotools-fu before compilation:

root@linaro-nano:~# cd fsl-alsa-plugins-3.5.7-1.0.0/
root@linaro-nano:~/fsl-alsa-plugins-3.5.7-1.0.0# aclocal && autoconf && automake --add-missing
root@linaro-nano:~/fsl-alsa-plugins-3.5.7-1.0.0# ./configure --prefix=/usr
root@linaro-nano:~/fsl-alsa-plugins-3.5.7-1.0.0# make all && make DESTDIR=/ install

The libfslparser, libfslvpuwrap, and gst-fsl-plugins packages each have an autogen.sh script to take care of the autotools stuff:

root@linaro-nano:~# cd libfslcodec-3.5.7-1.0.0
root@linaro-nano:~/libfslcodec-3.5.7-1.0.0# ./autogen.sh --prefix=/usr && make all && make DESTDIR=/ install
root@linaro-nano:~/libfslcodec-3.5.7-1.0.0# cd ../libfslparser-3.5.7-1.0.0
root@linaro-nano:~/libfslparser-3.5.7-1.0.0# ./autogen.sh --prefix=/usr && make all && make DESTDIR=/ install
root@linaro-nano:~/libfslparser-3.5.7-1.0.0# cd ../libfslvpuwrap-3.5.7-1.0.0
root@linaro-nano:~/libfslvpuwrap-3.5.7-1.0.0# ./autogen.sh --prefix=/usr && make all && make DESTDIR=/ install
root@linaro-nano:~/libfslvpuwrap-3.5.7-1.0.0# cd ../gst-fsl-plugins-3.5.7-1.0.0
root@linaro-nano:~/gst-fsl-plugins-3.5.7-1.0.0# ./autogen.sh PLATFORM=MX6 --prefix=/usr
root@linaro-nano:~/gst-fsl-plugins-3.5.7-1.0.0# make all && make DESTDIR=/ install

Testing things out

To test things out, we downloaded the Sintel 1080P video file from the Blender Foundation's Sintel web-site and played the video using gplay. Note that before doing so, we also installed the base, good, bad, and ugly gstreamer plugins, and that we flushed the disk cache prior to starting playback:

root@linaro-nano:~# for t in base good bad ugly ; do
         apt-get install gstreamer0.10-plugins-${t} ;
done
root@linaro-nano:~# echo 3 > /proc/sys/vm/drop_caches
root@linaro-nano:~# gplay Sintel.2010.1080p.mkv

We also tested out camera capture for both our OV5642 5MP parallel camera and OV5640 5MP MIPI camera using the mfw_v4lsrc element:

root@linaro-nano:~# modprobe mxc_v4l2_capture
root@linaro-nano:~# gst-launch-0.10 mfw_v4lsrc capture-mode=4 ! mfw_v4lsink

Binary distribution

I'm sure a lot of you patient enough to get this far saying “I just want it to work!”.

I hear you, and I'm getting there.

If you look closely above, you'll see that everything except firmware-imx uses make install to install things into the root filesystem, and that most of the packages specify the DESTDIR variable to say where the output should go. The one exception is imx-lib, which wants the PLATFORM=IMX6Q flag and a destination directory of DEST_DIR.

We can use this to create an overlay of the binary bits in a temporary directory:

root@linaro-nano:~# mkdir -p /root/overlay/lib/firmware/
root@linaro-nano:~# cp -ravf firmware-imx-3.5.7-1.0.0/firmware/*
                        /root/overlay/lib/firmware/
root@linaro-nano:~# for pkg in fsl-alsa-plugins
      gst-fsl-plugins
      libfslcodec
      libfslparser
      libfslvpuwrap ; do
      cd ${pkg}-3.5.7-1.0.0 ;
      make DESTDIR=/root/overlay/ install ;
      cd ../ ;
done
root@linaro-nano:~# cd imx-lib-3.5.7-1.0.0
root@linaro-nano:~/imx-lib-3.5.7-1.0.0# make DEST_DIR=/root/overlay PLATFORM=IMX6Q install

And create a tar-ball of the result like so:

root@linaro-nano:~# cd overlay
root@linaro-nano:~/overlay# tar czvf ../imx6-gstreamer-plugins-raring-20131003.tar.gz *

We've uploaded the result of this, and you can grab it at:

imx6-gstreamer-plugins-raring-20131003.tar.gz

If you extract this on top of one of the previously uploaded images, you'll need to install gstreamer and the gst-plugins- packages as described above.

Recap and commentary

To recap, this post described the steps needed to get, build, and install most of the gstreamer plugins for i.MX6.

Note that there is one component not covered: the OpenGL image sink. That will have to wait for another day.

We also haven't yet tested all of the functionality to look for gaps between this and other platforms. It's entirely possible that there's some breakage because we haven't included patches like this one to the core GStreamer build.