Youtube tutorial: https://youtu.

be/g7x2e9UsSc8
In this guide, we use vs2017 to build ffmpeg with x264, cuvid, nvenc, and libnpp plugins
1. Download and install Msys
https://sourceforge.net/projects/msys2/
2. Download and install yasm win64.exe(for general use on 64bit Windows, NOT for use with visual studio)
http://www.tortall.net/projects/yasm/releases/yasm-1.2.0-win64.exe
2-1 rename the yasm-1.2.0-win64.exe to yasm.exe
3. Compile x264 from source
Download x264 20170625 from https://download.videolan.org/pub/x264/snapshots/x264-snapshot-20170625-2245-stable.tar.bz2
3-1 Launch x64 Native Tools Command Prompt for VS2017
3-2 Type the following to launch Msys
C:\msys64\msys2_shell.cmd -mingw64 -use-full-path
3-3 Copy the yasm.exe from step 2-1 to /usr/bin
cp /C/...../yasm.exe /usr/bin/
3-4 Type the following to install make, pig-config and diffutils in Msys
pacman -S make pkg-config diffutils
3-5 go to the x264 source directory
cd /C/..../ffmpeg-3.4.2
3-6 Type
CC=cl ./configure --enable-static --prefix=/c/x264 --disable-cli
make
make install
3-7 The x264 build will be stored in C:/x264
4. Download and install CUDA toolkit
https://developer.nvidia.com/cuda-downloads
5. Compile ffmpeg from source
5-1 Download and extract ffmpeg 3.4.2 from http://ffmpeg.org/releases/ffmpeg-3.4.2.tar.bz2
5-2 Launch x64 Native Tools Command Prompt for VS2017
5-3 Type the following to launch Msys
C:\msys64\msys2_shell.cmd
5-4 In Msys, type the followings to copy cuda and x264 headers, libraries to /usr/local/include/ and /c/local/lib64/
respectively
mkdir /usr/local/include/
mkdir /usr/local/lib64
cp -R /c/Program\ Files/NVIDIA\ GPU\ Computing\ Toolkit/CUDA/v9.1/include/* /usr/local/include/
cp -R /c/Program\ Files/NVIDIA\ GPU\ Computing\ Toolkit/CUDA/v9.1/lib/x64/* /usr/local/lib64/
cp -R /c/x264/include/* /usr/local/include/
cp -R /c/x264/lib/* /usr/local/lib64/
5-5 Type the following to add the path of MS VC 2017 cl.exe and link.exe
export PATH="/C/Program Files (x86)/Microsoft Visual
Studio/2017/Community/VC/Tools/MSVC/14.12.25827/bin/Hostx64/x64/":$PATH
5-6 Type the followings to make sure cl and make correctly link to Microsoft cl.exe and link.exe
cl
link
5-7 Go to ffmpeg-3.4.2 source directory
cd /C/..../ffmpeg-3.4.2
5-8 Type
./configure \
--enable-gpl\
--enable-libx264\
--enable-cuda \
--enable-cuvid \
--enable-nvenc \
--enable-nonfree \
--enable-libnpp \
--prefix=/c/ffmpeg \
--extra-cflags=-I/usr/local/include \
--extra-ldflags=-LIBPATH:/usr/local/lib64 \
--toolchain=msvc
5-9 Complie and install in prefix directory
make
make install
5-10 The ffmpeg build will be stored in C:/ffmpeg
6. Launch a new command prompt in window
6-1 Go to build directory
cd C://ffmepg
6-2 Type the followings to make sure the ffmpeg complied with x264, scale_npp, nvenc, cuvid
ffmpeg –filters | findstr scale_npp
ffmpeg –encoders | findstr x264
ffmpeg –encoders | findstr nvenc
ffmpeg –decoders | findstr cuvid