Matt's Blog

JUNIPER, VMX

JUNIPER VMX GETTING STARTED

GUIDE (VMWARE)
JANUARY 7, 2016 | MDINHAM | 33 COMMENTS

As of Junos 15.1F4, Juniper are now of cially supporting vMX on Vmware.

The installation process has quite a few steps to it, so following on my my vMX Getting
Started Guide for KVM, here is a quick post showing you how to do it on your home lab
running Vmware Hypervisor ESXi 6.0.

ESXi Installation
Lets get started with the installation of ESXi. Im doing this running ESXi as a nested VM
on a Macbook, but the process would be the same if you were doing it on bare metal.
Register with Vmware and download the ESXi ISO from Vmware and then boot your
machine from the ISO. The installation of ESXi is a simple process. Go through the
installation steps one by one and reboot ESXi once the installation has completed.
Following the reboot ESXi will load up and if your management LAN is running DHCP
the host will have been assigned an IP address for management. You need to download
the VMware client to be able to manage ESXi free. Open a web browser and connect to
the ESXi IP download the tools as suggested, and then load up the client.
Once the client is loaded, rstly you should license the ESXi host. You can get a free
license from Vmware at the ESXi download page.
In the web client the license is applied at Home Inventory click con guration and
then Licensed Features. You can then click edit to apply the license.

vMX Installation
If you have a valid login, you can download vMX directly from the vMX download page.
Now load up the client for your ESXi server and login.
There is no OVA build currently, so several steps need to be done manually.

Copy Files to the Datastore

Before progressing any further you will need to extract the vMX package. All of the
vmdk les are located in the subdirectory /vmdk.
Software image for vMX VCP: jinstall64-vmx-15.1F4.15-domestic.vmdk
Software image for VCP le storage: vmxhdd.vmdk
Software image for VFP: vFPC-20151203.vmdk
metadata_usb.vmdk: Virtual hard disk with bootstrapping information. This is used
by the VCP.
Click the summary tab, select the datastore under Storage, right click and select Browse
Datastore.
Create a folder called vmx and then click the upload le button and upload all of the
vmdk les listed above to this new folder.

Set Up the vMX Network

If you are not familiar with vMX then at this point it would be a good idea to read over
my vMX Getting Started Guide for KVM, so that you understand the architecture of
vMX, and how the vMX virtual machines communicate with one another.
The VMware release is no different to the KVM release when it comes to the required
default networks. There are a minimum of three networks that will need to be
con gured:
Management network (br-ext)
Internal network for VCP and VFP communication (br-int)
Data interfaces
To create these networks, go back to the ESXi client, select the ESXi server and click
theCon gurationtab.Select Networking under Hardware. In the top right corner click
Add networking.
Management Network
1. Select Virtual Machine as the connection type and click next
2. Select Use vSwitch0 and click next

3. At port group properties, set network label to br-ext and click next

4. Now click nish

You will see the new port group br-ext has been added to the standard switch
vSwitch0.
Internal Network
Again, select Networking under Hardware. In the top right corner click Add networking.
1. Select Virtual Machine as the connection type and click next
2. This time select Create a vSphere standard switch and clear all physical NIC check
boxes, then click next

3. For network label, use br-int

4. You should now have a port group called br-int, with no adapters assigned
Data Network
Now add a data network, the process is repeated according to the number of data NICs
that you wish to add. Time to create a single adapter named p1p1.
Again, select Networking under Hardware. In the top right corner click Add networking.

1. Select Virtual Machine as the connection type and click next

2. Select Create a vSphere standard switch and add the physical NIC that you want to
use, and click next
3. Name the connection p1p1, click next, and nish
Repeat this process if you have any more data adapters to add to vMX.
Complete the network con guration
You will now see the 3 networks in the networking summary screen br-ext, br-int and
p1p1.
You must enable promiscuous mode in all vSwitches so that packets with any MAC
addresses can reach the vMX. e.g. for OSPF to work properly.

For each vSwitch, click properties, then select vSwitch and click edit. Select security and
change promiscuous mode to accept.

Set Up the vMX Virtual Machines

Just like vMX on KVM, there are two VMs that must be created the virtual control
plane (VCP) running the Junos OS, and the virtual forwarding plane (VFP) running an
x86 visualised release of Trio running on Wind River Linux.
The process for creating both of the virtual machines is very similar. Its a simple case of
following the VMware wizard and choosing the correct settings for the VM.
VCP

This process below outlines the steps required to create the VCP virtual machine.
1. Within the VMware client, select the ESXi host, right click, new virtual machine
2. Select to create a custom virtual machine, and press next
3. Give the machines a suitable name, e.g. vcp-vmx1
4. Select the datastore where you would like to store the VM and press next
5. Set the virtual machine version to 8
6. For the guest OS type, choose Other, Other (64-bit)
7. Select one virtual socket, and 1 cpu core per socket, to assign a total of 1 CPU core to
the VCP
8. Provision 2GB of memory
9. In the network setup, select 2 network adapters.
Asign br-ext as the 1st adapter and br-int as the 2nd adapter.
Set both to be e1000.
10. Select LSI Logic Parallel as the SCSI controller
11. When prompted to select the disk type, choose use an existing virtual disk, and then
on the next screen browse to the correct datastore and select the jinstall64-vmx15.1F4.15-domestic.vmdk image that you uploaded earlier
12. At the advanced options page, simply click next
13. Select to edit the virtual machine settings before completion and click continue
14. Now you need to add two more hard drives click Add, and then Hard Disk, this time
selecting vmxhdd.vmdk as the second drive
15. Repeat the add Hard Disk process again this time adding the metadata_usb.vmdk
image as the third drive.
NOTE: this 3rd hard drive is important if you dont con gure it then the rst time VCP
boots, VCP will setup as an olive not vMX!
You can now boot the VCP!
If the boot process appears to wait at Loading /boot/loader do not worry, on the
VMware release you dont see the full Junos OS boot process on the console.
VFP
This process below outlines the steps required to create the VFP virtual machine.
1. Within the VMware client, select the ESXi hosts, right click, new virtual machine
2. Select custom and press next
3. Give the machines a suitable name, e.g. vfp-vmx1

4. Select the datastore where you would like to store the VM and press next
5. Set the virtual machine version to 8
6. For the guest OS, choose Other, Other (64-bit)
7. When prompted to select the number of CPUs, for this build the minimum you can
choose is three virtual sockets, and 1 cpu core per socket, to give a total of 3 CPU
cores assigned to the VCP
8. Provision 8GB of memory
9. In the network setup, select at least 3 network adapters, assigning br-ext as the 1st
adapter and br-int as the 2nd adapter. Set them both to be e1000 adapters. The data
adapters can now be selected, set them to vmxnet3 or e1000 depending on your
preference. For better performance, Id suggest you use vmxnet3 because this is a
paravirtualization adapter.
NOTE: if you wish to use SR-IOV at the time of writing SR-IOV is not of cially
supported on VMware, only on KVM.
10. Select LSI Logic Parallel as the SCSI controller
11. When prompted to select the disk to use, choose use an existing virtual disk, and
then on the next screen browse to the datastore and select the vFPC20151203.vmdk image that you uploaded earlier (bear in mind the image naming has
changed from vPFE* to vFPC* in this latest release of vMX)
NOTE: on my build the Juniper supplied image needed to be converted to thick
provisioned using vmkfstools, otherwise the VM refused to boot (I was getting a
VMware error related to free space even though the drives were not full). You may
not have to do this! Thanks to @tomverhaeg for working through this strange issue
with me!
12. At the advanced options page, simply click next
13. At Ready to Complete, you can click nish and boot the VFP!
NOTE: VMware virtual console for the VFP does not show anything beyond Please
Wait: Booting if you wish to login to the VFP you will need to con gure serial console.
The process for setting up serial console is described here.

Veri cation
At this point if both machines have powered on successfully you should have a running
vMX.
Now login to the VCP and run the Junos command show chassis fpc. After a few
moments you should see the FPC as online and ge-* interfaces will appear.
Have fun!

For more information, please refer to the Juniper documentation on the VMware
release of vMX.

33 THOUGHTS ON JUNIPER VMX GETTING STARTED GUIDE (VMWARE)

Perry Young
JANUARY 7, 2016 AT 4:58 PM

The only comment I have is that this is awesome!

mdinham
JANUARY 7, 2016 AT 4:59 PM

Ha! Nice one Perry

RD
JANUARY 11, 2016 AT 11:53 PM

Does one need a p1p1, et al interface? I have one NIC on my ESXi server that is
cabled, and didnt have plans to utilize the 2nd NIC really.

mdinham
JANUARY 12, 2016 AT 10:20 AM

Thanks for reading, sure you could just use the one NIC its all about how you
do the vSwitch con g

RD
JANUARY 13, 2016 AT 12:24 AM

What does the p1p1 / data interfaces come up as within each vMX instance?
Id like to replicate the logical-systems EVPN lab that was in an older blog post, with
vMX running on ESXi.

mdinham

JANUARY 13, 2016 AT 9:50 AM

The data interfaces start at the 3rd interface assigned to the VFP and it will be
the ge-0/0/0 interface on Junos. 4th interface ge-0/0/1, etc

gabis
FEBRUARY 1, 2016 AT 4:14 PM

Hi,
Thanks for the tuto.
For the vFPC supplied image, it may be the hypervisor version that need a disk
conversion (I do have the same issue on ESXi6, I dont tried ESXi 5.5)
Command to convirt :
vmkfstools -i vFPCxxxx.vmdk vFPC-vmx.vmdk -d thin (or thick)
The disk was originally on sparse mode.
Gabis

Jide Akintola
FEBRUARY 5, 2016 AT 6:00 PM

Nicely written Matt, well done and thanks!

Keeran
FEBRUARY 11, 2016 AT 7:32 PM

Hi
Firstly great post! But i did come into an issue with the vFP, im using esxi 6 and i had
the same issue where i had to change the vmdk le to thick however when i reboot
the node, it gets stuck Booting the kernal.. Please wait: booting would you have
an ideas on how i could x this?
Thanks
Keeran

mdinham
FEBRUARY 12, 2016 AT 1:29 PM

Hi, thanks

You wont see any further progress on the console for the VFP, unless you attach
a serial console to monitor the boot. Do you see the VFP active within Junos?

Keeran
FEBRUARY 12, 2016 AT 1:53 PM

Hi Matt
Thanks for the update! I can see the vFPC in Junos so it looks like all is good
I was working on this all yesterday and happy to get this sorted today!
Cheers and thanks again
Keeran

maurips
FEBRUARY 12, 2016 AT 4:48 PM

Hi Matt,
great post!
I have followed your guide but at the end i am connecting to the console of the VCP
and it seems stuck at Loading /boot/loader for hours.
How can I x this? (I cannot use the serial connection through network because it is
a licensed feature not present in my essential vcenter license.
Thank you.
Maurizio

mdinham
FEBRUARY 12, 2016 AT 5:13 PM

Hi, thanks
As mentioned to Keeran also in the comments, you wont see any further
progress on the console for the VFP. Do you see the VFP active within Junos, and
any ge-* interfaces?

maurips
FEBRUARY 12, 2016 AT 5:48 PM

No, Unfortunately I have no way to See the console terminal, no junos, Nothing
at all. No way to type commands

Lab Manager
FEBRUARY 23, 2016 AT 5:01 PM

Hi Matt and thanks for this great post!

Ive installed both machines and went through all instructions with no issues.
Everything is up and still I cant see any FPC as online and ge-* interfaces
appearing, when Im running show chassis fpc from the VCP console.
Have I missed something? Maybe I should con gure something rst?
I cant understand from this manual how the separate machines can
communicate each other and how the VCP could see something from the
VFE.
Thanks in advance

mdinham
FEBRUARY 23, 2016 AT 5:45 PM

Hi thanks for reading it

Did you setup the internal vSwitch and use
that to connect the VCP and the VFP. This is in addition to the
management vSwitch.

Lab Manager
FEBRUARY 23, 2016 AT 5:52 PM

Thank you for the quick response

Indeed, I setup that vSwitch (br-int) and they both have an interface connected to
it.

mdinham
FEBRUARY 23, 2016 AT 6:04 PM

Great

Its possible that VFP has crashed during boot, is it possible to setup the

serial console? You might not be able to do this, depending on your VMware
license

Lab Manager
FEBRUARY 24, 2016 AT 9:09 AM

Ive setup the console as well, however how do I connect it? Where from?

mdinham
FEBRUARY 24, 2016 AT 9:32 AM

You access it by telnetting to the ESXi server IP, with the port as speci ed
when the serial console was con gured. Be aware though that the free ESXi
does not support serial console. Even though it can be con gured, the license
does not allow it to work

Lab Manager
FEBRUARY 24, 2016 AT 10:30 AM

Ive managed to access by Telnet and everything looks allright with the machine,
however when ifcon g, I see only the int and ext interfaces although Ive added 2
more interfaces for data. Maybe because they have a different adapter type? I
create them as your recommendation.

mdinham
FEBRUARY 24, 2016 AT 10:41 AM

From the VCP, are you able to ping the following from Junos
ping 128.0.0.16 routing-instance __juniper_private1__
128.0.0.16 is the VFP

maurips
FEBRUARY 24, 2016 AT 10:50 AM

Hi all.
1 vMX is working now in my ESXi5.5 environment (VCP and VFP can communicate
each other). I still have 3 issues:
1) Sometimes the VCP loses the VFP and I have to restart the VFP VM to restore
the communication between them
2) Sometimes the communication is working but the ge-0/0/0 interface is down
(again I have to restart the VFP VM to restore the ge-0/0/0)
3) I tried to install 2 vMX on the same host. Whenever I start the second vMX both
VCP lose the connection with their correspondent VFP, even if I created two
different port group for the br-int.
Any idea to x these issues?

Thank you.
Thanks

mdinham
FEBRUARY 24, 2016 AT 11:46 AM

Hi, thanks for reading the post. It looks like you are doing the right thing on the
setup and install. For this one, Id suggest that you raise a JTAC case if you have a
service contract, or if you post on http://forums.juniper.net in the vMX forum
then it might be better to continue the discussion there. I will look out for the
post. Thanks

Lab Manager
FEBRUARY 24, 2016 AT 11:16 AM

No I cant, even not to 127.0.0.1says Cant request assign requested address

Show route also brings nothing.

mdinham
FEBRUARY 24, 2016 AT 11:48 AM

Hi, is it OK if you post on http://forums.juniper.net in the vMX forum , as it might

be better to continue for us to the discussion there I will look out for the post
Please can you post the output of the ping and show chassis fpc thanks

Lab Manager
FEBRUARY 24, 2016 AT 12:23 PM

Just did it over there, by the author algosec

Vadim
MARCH 2, 2016 AT 3:39 PM

Hi, Do you try to get start vMX in unetlab ?

D
MARCH 19, 2016 AT 6:37 PM

Tried to boot it in Workstation by following the same principles. I think it should

work under Workstation as well, but the VFPC machine doesnt boot On the

serial console connection I see the following error messages while boot:
Waiting for root device to be ready
mount: mounting /dev/sda2 on /mnt failed: No such le or directory
Waiting for root device to be ready
mount: mounting /dev/sda2 on /mnt failed: No such le or directory
Waiting for root device to be ready
Any idea what might be wrong?

D
MARCH 19, 2016 AT 7:26 PM

Changed the HDD type to IDE and that xed the issue.

mdinham
MARCH 20, 2016 AT 10:48 AM

Awesome

Manoj Karmacharya
JUNE 22, 2016 AT 4:42 AM

Hi!,
show chassis fpc returns empty message

mdinham
JUNE 22, 2016 AT 8:09 AM

Hi thanks for reading the post

Please go to the Juniper vMX forum and make a post there. I will look out for it.
Please post details on the network/vSwitch con guration for the VCP and VFP,
con rming how many vNICs are assigned to each VM and each vSwitch.