IntelQuickSyncVideo

Hardware-Accelerated Video Playback on Intel CPUs in Ubuntu

Note this document covers Intel CPU/GPUs only.

There's a lot of complicated stuff written on this topic. This document aims to get you working efficient video playback on Ubuntu, as simply as possible, as well as outlining future plans.

It's important to know the difference between working and not working. If it's working then you should be able to play modern 4K or full HD videos with around 20% CPU usage or less on a laptop and 10% or less on a desktop. If it's much more than that then something has gone wrong. For example, in testing these instructions a properly configured Haswell desktop uses only 3% CPU to play 4K 60FPS H.264 video.

Alternate approaches do exist, but those have been observed to increase rather than decrease CPU usage when compared to the default software decoders. So remember to measure your CPU. A working solution should play the video and decrease your CPU usage significantly.

Quick Start 17.10

Ubuntu 17.10 (beta 2) introduces out-of-the box support for video acceleration that's much simpler than previous Ubuntu releases.

If you are installing Ubuntu 17.10 from scratch, make sure you connect to wifi/network during the setup. Assuming you have internet access while installing Ubuntu then simply ticking the "Install third-party software" option will install most of what you need.
If you already have Ubuntu 17.10 installed or failed to get third-party software installed because you weren't connected to the internet during setup then all you need to do to catch up is:
```
 $ sudo apt install ubuntu-restricted-addons
```
You now have working hardware accelerated video playback in totem ("Videos" app) only. You can verify this by installing the vainfo tool:
```
 $ sudo apt install vainfo
 $ vainfo
```
However totem has a few performance problems that are still yet to be fixed so for smoother playback and lower power usage we recommend mpv 0.26 instead, which in Ubuntu 17.10 is configured to provide working hardware acceleration out-of-the-box:
```
 $ sudo apt install mpv
 $ mpv myvideo.mp4
```
Unfortunately we cannot recommend gnome-mpv right now. Only mpv.

Quick Start 17.04

Here's how to make things work quickly. What "works" well is limited to a couple of particular video players. If you need web browser video, or MKV files for example, then you should look at the Future Roadmap instead.

Install Ubuntu. You may find that 17.10 or 17.04 is required in order to get a current acceleration driver that supports your hardware.
Preferably you're using X and not Wayland. This means either Unity 7 or GNOME is OK. Just don't use GNOME on Wayland because it's not yet fully working. That is; unless you're willing to use some ugly workarounds outlined below.
Install the Intel acceleration driver and tools:
```
 $ sudo apt install vainfo i965-va-driver
```
Check that acceleration is installed and supported:
```
 $ vainfo
```
You should see a list of video codec profiles that your system is capable of playing with full hardware acceleration.
Reboot. It's not yet clear why, but you might need to reboot before the best performance is achievable.

Option 1: Accelerated playback using GStreamer

Install the missing packages:

 $ sudo apt install gstreamer1.0-vaapi gstreamer1.0-libav gstreamer1.0-plugins-ugly gstreamer1.0-plugins-bad

Play your video using gst-play-1.0 for the best performance:
```
 $ gst-play-1.0 myvideo.mp4
```
or if you are using a Gnome Shell Wayland session then the required syntax is:
```
 $ env GST_GL_WINDOW=wayland gst-play-1.0 --videosink glimagesink myvideo.mp4
```
You can also use totem to play videos accelerated, but compared to gst-play-1.0 or mpv it is more power hungry, less smooth and corrupt:
```
 $ env GST_GL_WINDOW=wayland totem myvideo.mp4
```

Option 2: Accelerated playback using MPV

Install the missing packages:
```
 $ sudo apt install mpv
```
Play your video with mpv hardware acceleration turned on (it's not turned on by default!):
```
 $ mpv --hwdec=vaapi myvideo.mp4
```
or using autodetection:
```
 $ mpv --hwdec myvideo.mp4
```
The reason why mpv is recommended over other FFmpeg-based players is because FFmpeg hardware acceleration support is not automatically integrated and has to be implemented per application. It is known to be implemented and working properly in mpv only.
MPV can also work fully accelerated on Wayland, but again it doesn't work automatically. With Gnome Shell on Wayland you might need to force it:
```
 $ env -uDISPLAY mpv --hwdec --opengl-backend=wayland myvideo.mp4
```

Other video player apps won't work anywhere near as efficiently. VLC, Mplayer, Google Chrome and Firefox for example, will only use software decoding (much higher CPU) since they use FFmpeg directly rather than GStreamer. See the Future Roadmap below where we will try to fix those issues.

And TS files

Video files with the .ts extension are also playable with the same hardware acceleration after a few extra steps:

Install the ffmpeg command-line tool:
```
 $ sudo apt install -f ffmpeg
```
Make an MP4 file (this is fast and does not change the video or audio contents, just the file format):
```
 $ ffmpeg -i INFILE.ts -vcodec copy -acodec copy OUTFILE.mp4
```
Play your new file OUTFILE.mp4 using the above instructions.

Know Your Limits

If video playback appears frozen or causes high CPU usage, then it's likely you're trying to play a certain profile of video your CPU is a bit too old to handle (or you made a mistake in following the instructions).

The simplest answer is to run 'vainfo' which will list all the video codecs that your CPU/GPU can handle in hardware.

The complicated answer is to look up your CPU model (from 'lscpu') on http://ark.intel.com, find out what generation it is and then using that information you can check your CPU generation against this list or this table.

Medium Term: A more polished quick start

The above Quick Start section has a lot of caveats. Most of them are mentioned as links to bugs that need fixing. But there is also the problem of poor MKV and TS container file support in GStreamer that needs fixing. After some or all of these issues are addressed we will be able to recommend Totem as the efficient video player of choice.

Long Term: Future Roadmap

In future we would like all video playback in Ubuntu to be accelerated by default (on modern Intel chips at least), out of the box. There are two good ways to get there. Both of them involve the FFmpeg because the FFmpeg libraries are the core decoding libraries that almost all video players and web browsers use by default.

For video player apps, we will need to modify the standard Ubuntu/Debian ffmpeg packages to add acceleration. For web browsers however, they will also each need to be modified as each browser has it's own private copy of FFmpeg libraries. So FFmpeg is the focus of all future work.

You should also know that the Intel Quick Sync Video is what we're aiming to use. Quick Sync Video is the Intel hardware feature that accelerates video handling efficiently. So if you see "QSV" referred to in software instructions then those instructions are misleading you, and actually talking about something else that indirectly uses your Intel Quick Sync Video hardware. Also note that historically the video decoding functions have been referred to by Intel as Intel Clear Video HD, and this is still true for Intel technical documents. But generally the internet community refers to it under the heading of Quick Sync Video (QSV).

Future Approach #1: VA-API ("LibVA") decode

FFmpeg's wiki claims that there is no support for a VA-API decoder. This appears to be contradicted using the above mpv instructions until you realize that mpv reimplements VA-API support from scratch. But it also seems to be disproven by the existence of vaapi_decode.c and lots of related logic in the FFmpeg source code starting in version 3.3.

The trouble is that FFmpeg's hwaccel architecture (v1) requires per-app implementation, and that implementation is missing in common players like Mplayer, Chrome/Chromium and Firefox. Even FFmpeg's own ffplay tool doesn't support it yet. However rather than implementing the v1 hwaccel architecture in each app, it is probably advisable to take (or wait for) the approach the FFmpeg themselves have in mind: Hardware Acceleration (hwaccel) API v2.

Future Approach #2: Media SDK ("MSDK")

(This is the approach that the FFmpeg documentation rather confusingly refers to as "Intel QSV")

Intel's Media SDK is the company's professional offering with the aim of providing the highest performance and the most video features all accelerated on Intel chips.

It sounds great. You can download the SDK for free, and they have also published a first open source release of it too. But there are hidden reasons why we can't use any of this yet in Ubuntu. The problem is that pieces are missing. Both of them have a dependency on a custom proprietary fork of LibVA (VA-API). While this "missing" code is actually provided in the commercial download, it requires that you:

Overwrite official Ubuntu files, possibly breaking your system; and
Recompile all your video players yourself; and
Don't redistribute what you've made.

That won't work for us right now. And we don't recommend you try. It's very complicated and can easily break your Ubuntu installation.

So what about their open source version of Media SDK on Github? Well, that won't work either. It actually has a dependency on the commercial download in order to build at all, and with that has all the same restrictions as the commercial download.

Why is this a "Future Approach" then? Because in future we might be able to use the Media SDK after Intel removes the proprietary prerequisite problem.

YouTube

In-browser hardware-accelerated video decoding is coming in future. However you can play YouTube/web videos with full hardware acceleration in a separate window right now:

Follow the above Quick Start steps to get mpv installed and working.
Create a configuration file .config/mpv/mpv.conf in your home directory that explicitly tells mpv to use acceleration and to download only formats that most Intel CPUs can decode in hardware:
```
 hwdec=vaapi
 ytdl-format="bestvideo[ext=mp4]+bestaudio[ext=m4a]"
```
Drag (only works in Xorg right now) the link to your web video into the mpv window, or provide it on the mpv command line.

Overall Architecture (in Ubuntu 17.10)

What about NVIDIA and AMD?

They have different acceleration architectures with different instructions. So for the sake of simplicity, NVIDIA and AMD are not covered in this document. Still, long term future efforts for all CPU/GPU types should be focussed on FFmpeg for the same reasons outlined above.

Ubuntu Wiki