You are on page 1of 10

Hernán J.

Larrea

NetApp Data Ontap


Simulator Cookbook
HOW TO BUILD YOUR OWN VIRTUAL, ALL FUNCTIONAL
STORAGE SIMULATOR, WITH UBUNTU OS AND ISCSI FEATURES.
NetApp Data Ontap Simulator Cookbook

CONTENT

Introduction.................................................................................................................................................. 3

Concepts ....................................................................................................................................................... 3

Ingredients.................................................................................................................................................... 3

Vmware Server 2 ...................................................................................................................................... 4

Ubuntu Server 10.04 ................................................................................................................................ 4

Ubuntu aditional packages ....................................................................................................................... 4

Data Ontap simulator 7.3.4 ...................................................................................................................... 4

Netapp licenses ........................................................................................................................................ 4

Model ........................................................................................................................................................... 4

Recipe ........................................................................................................................................................... 5

Installing licenses ...................................................................................................................................... 7

provisioning a lun through iscsi ................................................................................................................ 7

About me .................................................................................................................................................... 10

2 Hernán J. Larrea – www.hernanjlarrea.com.ar


NetApp Data Ontap Simulator Cookbook

INTRODUCTION

I have created this document as it has been a great experience to me working with the Data Ontap
simulator. It is very real, building the simulator as it is explained on this guide, will let you practice
exporting LUN via iSCSI, creating shares, working with snapshots, it is almost an all functional virtualized
storage.

If you want to learn some storage fundamentals or NetApp fundamentals, gain some practice on the
Data OnTap CLI or FilerView, this will help you to set up a very real, working, and all functional lab.

So, let’s cook….

CONCEPTS

Some stuff you need to get familiar to understand this guide if you have never worked with storage
solutions. If you are already familiar with Storage concepts or with NetApp technologies don’t bother
reading this section.

Storage fundamentals, it would be great first you to have some idea of what a storage box is, different
vendors, such as EMC, Hitachi and IBM, NetApp is just one more of them. And what is a SAN, and SAN
Connectivity, for real storage environments this is the cornerstone, it is a physical layer composed by FC
cables, FC switches and directors, even we are not going to virtualize a SAN (since it is not possible), and
we are going to use iSCSI instead, it would be great you to read some about other Storage Vendors and
most of all about SAN and SAN Connectivity, so you can get an idea the way it is in the real world, and
get a better understanding on the gaps between the real world and this simulator.

Filer, is the way NetApp call’s their storage boxes, doesn’t care if they are used only NAS (CIFS and NFS)
or only for LUN Mapping, or both, they call their boxes, filers.

FilerView, is the admin portal for the storage, an alternative for the CLI.

Aggregate, Volume, qtree, LUN, Snapshots, this is what you can do with the disks inside your storage,
how to arrange them to reach a technical objective. For example, you arrange your physical disks into
Aggregates, but the aggregates are composed by raid groups and these ones by physical disks,
understand the loop? Of course, an Aggregate can have more than one raid group. Once your aggregate
is created, you can create volumes inside of it, and inside these volumes you can create shares, qtrees,
luns or snapshots.

But as this is not a NetApp administration guide, I’m going to stop here talking about this, and just say
these are key concepts to understand in order to be able to operate you NetApp simulated Filer, if you
want to read more you can do it at NetApp’s NOW site, http://now.netapp.com.

Finally, I’ve mentioned this couple of times at this point, synthetic hardware, this nothing else but the
hardware you add to your VM.

INGREDIENTS

3 Hernán J. Larrea – www.hernanjlarrea.com.ar


NetApp Data Ontap Simulator Cookbook

What you will need for your recipe and where to get it.

VMWARE SERVER 2

Virtualization software, this is my choice, you can change it for any other that supports a Linux operating
system, mounting ISO images in synthetic CD hardware of your VM and more than one Ethernet
synthetic interface in your VM. VMWare Server 2 it is free, and you can get it from:
http://downloads.vmware.com/d/info/datacenter_downloads/vmware_server/2_0 just register and
download.

UBUNTU SERVER 10.04

This was also my choice, even it is documented by NetApp that is fully supported by their simulator, I
guess you can run another Linux flavor you like. It is also free, at the moment this document was written
10.04 was the latest version available. Download from: http://www.ubuntu.com/server/get-
ubuntu/download no need registering, just select 32 bits Server ISO and download it.

UBUNTU ADDITIONAL PACKAGES

Some packages you will need, as I choose Ubuntu as the base system I used apt in order to get the
packages, but if you choose another Linux flavor you might have to download the sources or
precompiled binaries and make them on you system.

• open-iscsi (necessary for mounting iSCSI exported LUNs from your virtualized storage)
• lsscsi (just to check when an iSCSI LUN has been mapped to the system)
• expect (necessary only for running the simulator in background)
• vim (just because I like it more than vim-tiny)

For Ubuntu choosers, all of them available from the Lucid repository.

DATA ONTAP SIMULATOR 7.3.4

The master piece, the simulator itself, you can get it from http://now.netapp.com (for this you must be
a NetApp’s member, partner or customer, as the company I work for is, I’ve been able to download it,
anyway there is a registration form available on the site, I haven’t tried if registering on through that
form makes also the download available).

You will get an ISO image file from where you will get all you need for your simulator. At the time this
document was written, Data Ontap 8 version had already been released but as it is for 64 bits I preferred
to keep working on 7.3.4. Also this new version seems is does not need to run on top a VM, of course
the ISO for this version is much bigger than for 7.3.4.

NETAPP LICENSES

Data Ontap simulator includes all the licenses or the products and features available in your filer, such as
Flex Clone, Snapmirror, Syncmirror, iSCSI, CIFs, NFS, and so on. You can find them inside the ISO image,
under ./docs/license_keys.htm

MODEL

4 Hernán J. Larrea – www.hernanjlarrea.com.ar


NetApp Data Ontap Simulator Cookbook

First of all, let’s take a quick view of what we are going to build and its layers (on the following diagram
I’m going to include only the relevant layers).

lun snaps
vol0 Volume (vol1)
aggr0 Aggregate (aggr1)
ns0
Data Ontap
iSCSI
Ubuntu
VM
eth0 eth1

Data Ontap root Aggregate and Volume

Starting from bottom to top, our VM, a common VM configured to run a Linux OS with 2 virtual NICs.
We will need 2 because by definition, the Data Ontap simulator, binds its network interface/s to physical
interfaces (actually this is a synthetic NIC but the simulator doesn’t know that). And once the ns0
interface from the storage is attached to the physical one, the server where the simulator is running it
won’t be able to access the storage through that interface, and as we are going to map luns though iSCSI
a second interface will help us to do so.

The VM NICs can be bridged, NAT, or host only connections to your hosting machine, that’s your call,
and also based on what your Virtualization software allows you. As we are going to download some
packages with apt, I recommend you to set at least one of them as bridged or NAT.

Moving forward, the OS, Ubuntu is this case, we will install Ubuntu Server with the Virtual Machine
mode, so a minimal system is installed, no graphical interface, no unneeded packages, no wasted space.

Finally the Data Ontap simulator and the open-iSCSI package for LUN Mappings. We are talking about
this much in depth in the following sections.

RECIPE

Here you will find the steps for the implementation of the VM and the Simulator, on some parts I’ll show
a high level description on what should be done, such as in the OS installation process, the guide is
meant to let you setup your lab to learn some storage and NetApp not Linux, internet is plenty of
documents to answer your questions and I’ll be pleased to answer any consult.

Also the storage once installed will be simple configured, no clustering, no virtual interfaces, you can
play with that if you want, but I’m not going to show that here.

1. Create your VM with 2 NICs, 1 configured as NAT or bridged at least. I recommend you to use
between 256 and 512 MB for RAM and 8 GB for disks will be ok.
2. Install Ubuntu server as minimal virtual machine mode, during the first screen, ater selecting
language, press F4 and select the installation mode.
3. Follow the installation steps for the OS.
4. Take note of which network interface will you select as primary when the installation prompts.

5 Hernán J. Larrea – www.hernanjlarrea.com.ar


NetApp Data Ontap Simulator Cookbook

5. When prompted about server functions to install, select openSSH and Samba, for remote
administration and filesharing with your desktop, these functions are not required pieces for
the puzzle but might make your life easy working with the simulator.
6. Once Ubuntu is installed I recommend you to take a snapshot of the VM, just in case something
goes wrong you do not have to run the installation all over again.
7. If installed openSSH configure it to accept connections over any of the network interfaces.
8. Edit /etc/network/interfaces to include your second interface. And if you want fix IPs for both
interfaces.
9. Restart networking daemon.
10. Install the following packages:
a. vim ; Run: apt-get install vim
b. open-iscsi ; Run: apt-get install open-iscsi
c. expect ; Run: apt-get install expect
d. lsscsi ; Run: apt-get install lsscsi
11. Configure iSCSI daemon to start automatically. Edit /etc/iscsi/iscsid.conf under Startup Settings,
change the startup mode to automatic.
12. Mount the Data Ontap simulator CD and run the installation script ./setup.sh following steps
(13 to 26 ) show the options given during the setup.sh script execution. Text between brackets
shows the default option, if you press return without entering any character this will be the
configured option. For these steps, comments will be in bold to make difference between
installation prompts and comments.
13. Where to install to? [/sim]:
14. Would you like to install as a cluster? [no]: I’ll write another document regard clustering
options.
15. Would you like full HTML/PDF FilerView documentation to be installed [yes]: n
16. Continue with installation? [no]: y
17. Your simulator has 2 disk(s). How many more would you like to add? [0]: 26
18. What disk size would you like to use? [a]: a Choose whether you feel comfortable with.
19. Disk adapter to put disks on? [0]:
20. Use DHCP on first boot? [yes]:
21. Ask for floppy boot? [no]:
22. Which network interface should the simulator use? [default]: eth1 Make sure to check an
interface different from the one you selected as default during the installation. Also to get
this working properly so you can access, the internet, the local network and the simulator
from the VM ensure the eth0 is first in the routing table (Run command: route) if not, just
delete the routes for eth0 and add them again.
23. How much memory would you like the simulator to use? [512]: 256 Set the amount of memory
you feel comfortable with inside the parameters given.
24. Create a new log for each session? [no]:
25. Overwrite the single log each time? [yes]:
26. Run /sim/runsim.sh to start the simulator. The first time you run the simulator it will ask you
to set settings relevant to your virtual storage functionality, anytime you want to run this
configuration again, you can run /sim/steup.sh.
27. Run /sim/runsim.sh to start the simulator and the first configuration. For next steps (28 to 48)
I’m going to show the prompts by the setup script, any comments will be made in bold to avoid
confusion between script’s prompts and comments.
28. Please enter the new hostname []: ontap01 Just choose a name that likes you.
29. Do you want to enable IPv6? [n]:

6 Hernán J. Larrea – www.hernanjlarrea.com.ar


NetApp Data Ontap Simulator Cookbook

30. Do you want to configure virtual network interfaces? [n]:


31. Please enter the IP address for Network Interface ns0 [192.168.0.106]:
32. Please enter the netmask for Network Interface ns0 [255.255.255.0]:
33. Please enter media type for ns0 {100tx-fd, auto} [auto]:
34. Please enter the IP address for Network Interface ns1 []:
35. Would you like to continue setup through the web interface? [n]:
36. Please enter the name or IP address of the IPv4 default gateway [192.168.0.1]:
37. Please enter the name or IP address of the administration host: 192.168.0.101
38. Please enter timezone [GMT]:
39. Where is the filer located? []: Argentina
40. What language will be used for multi-protocol files (Type ? for list)?:en_US
41. Do you want to run DNS resolver? [n]:
42. Do you want to run NIS client? [n]:
43. Do you want to configure the Shelf Alternate Control Path Management interface for SAS
shelves [n]:
New password:
Retype new password:
44. Do you want to make the system visible via WINS? [n]:
45. A filer can be configured for multiprotocol access (...) Selection (1-2)? [1]: 1
Enter the password for the root user []:
Retype the password:
46. The default name for this CIFS server is 'ONTAP01'. Would you like to change this name? [n]:
47. Data ONTAP CIFS services support four styles of user authentication. Selection (1-4)? [1]: 4
48. What is the name of the Workgroup? [WORKGROUP]:
49. At this point you should be being requested to enter the root password.
50. Once loged in as root you will be taken to the Data Ontap prompt “your storage name”> in my
case Ontap01>. Now you can start playing with your storage!

INSTALLING LICENSES

For most things we want to do here, we need to install some licenses, from your linux machine mount
the CD image and browse to ./doc and open the file named license_keys.htm. To install the licenses it is
really simple, we can do it through CLI or via FilerView. For our next objective we will need to install
iSCSI license, so go to the file, grab the code and run the following command in the Data Ontap CLI:

license add <code>

Repeat the procedure for every product or feature you want to enable. To get a summary on the
licenses installed just run the license command.

PROVISIONING A LUN THROUGH ISCSI

At this point your simulator should be up and running and you should have all disks as spares, to
accomplish this objective what we need to now is:

• Create a new Aggregate.


• Create a volume inside the Aggregate.
• Create a LUN inside the volume.
• Create an Initiator group and add our iSCSI address.

7 Hernán J. Larrea – www.hernanjlarrea.com.ar


NetApp Data Ontap Simulator Cookbook

• Map the lun and the initiator group.


• Log in from the OS to the storage over iSCSI and discover the LUN.
• Make a FS inside the new disk and mount it.
• Keep playing with your storage!

I’m going to describe how to do this through the CLI, but you might want to do it through the FilerView
all options are available from any of these interfaces.

So, as we already cooked, let’s decorate the plate…

Creating a new aggregate

First of all, as I mentioned in the concepts section, we know disks are arranged into raid groups, and
aggregates are composed by raid groups. First of all, lets check the status of our disks.

Ontap01>sysconfig –r

You will get an output with the details on the physical disks and how are they arranged in through the
different aggregates. (If no aggregate was created only aggr0, the root aggregate will be shown an all
the other disks will be shown as spares).

Ontap01>aggr create aggr_tst -r 8 -t raid4 16@16m

This command will create an aggregate called aggr_tst with a raid group size of 8 with only 1 disk of
parity per raid group (raid 4) and will include 16 disks of 16 MBs each. (So it will create 2 raid groups
with 8 disks each, but only 7 will be usable since each raid group has 1 disk or parity).

Creating a volume inside the aggregate

Now the aggregate is created, check the space on it:

ontap01> df -h -A aggr_tst
Aggregate total used avail capacity
aggr_tst 191MB 80KB 191MB 0%
aggr_tst/.snapshot 10MB 0MB 10MB 0%

There are 10MB (5%) reserved for aggregate snapshots as we won’t take any snapshot from the
aggregate let’s claim that space:

ontap01> snap reserve -A aggr_tst 0

ontap01> df -h -A aggr_tst
Aggregate total used avail capacity
aggr_tst 201MB 80KB 201MB 0%
aggr_tst/.snapshot 0TB 0TB 0TB ---%

Now we’ve got all the space for us, let’s create a 100MB volume inside this aggregate:

ontap01> vol create vol_tst aggr_tst 100m

This command will create a 100MB volume named vol_tst inside the aggregate aggr_tst. So, the volume
is created, let’s check the space on it:

ontap01> df -h vol_tst

8 Hernán J. Larrea – www.hernanjlarrea.com.ar


NetApp Data Ontap Simulator Cookbook

Filesystem total used avail capacity Mounted on


/vol/vol_tst/ 80MB 88KB 79MB 0% /vol/vol_tst/
/vol/vol_tst/.snapshot 20MB 0MB 20MB 0% /vol/vol_tst/.snapshot

Again we see space reserved for snapshots, we might want to take snapshots of this volume, so let’s
keep it, and create a LUN inside this volume.

Creating the LUN

Just run the following command:

ontap01> lun create -s 50m -t linux /vol/vol_tst/lun_tst


ontap01> lun show
/vol/vol_tst/lun_tst 50m (52428800) (r/w, online)

And the LUN is created. The lun create command includes the size of the lun (of course it can’t be bigger
than the volume where it is located), the type o OS which will access that LUN and finally the path
where you want to store it inside your filer.

Creating the iSCSI initiator group

First of all, on the storage side, check the iSCSI service is running:

ontap01> iscsi status


iSCSI service is running

In order to create the initiator group, you first must know the initiator’s iSCSI node name, go to the
Linux machine and read the content of /etc/iscsi/initiatorname.iscsi file:

InitiatorName=iqn.1993-08.org.debian:01:58cad98ae9c6

Then, create the initiator group with the igroup command:

ontap01> igroup create -i -t linux Initiators_tst


ontap01> igroup show
Initiators_tst (iSCSI) (ostype: linux):

About the create parameter or igoup command, you must specify it is an iScsi initiator goup type (-i
option) and the the host type and the group name. As you can see, it has been created but it’s empty,
let’s add the node name of the server.

ontap01> igroup add Initiators_tst iqn.1993-08.org.debian:01:58cad98ae9c6


ontap01> igroup show
Initiators_tst (iSCSI) (ostype: linux):
iqn.1993-08.org.debian:01:58cad98ae9c6 (not logged in)

Now, we need to map the LUN to the initiator group.

Mapping the LUN to the initiator group

Run the following command:

ontap01> lun map /vol/vol_tst/lun_tst Initiators_tst 1

and the mapping is done. You indicate which LUN to which group and which id will be set to the LUN
being mapped.

9 Hernán J. Larrea – www.hernanjlarrea.com.ar


NetApp Data Ontap Simulator Cookbook

Discovering the LUN on the server’s side

At this point our storage is presenting the LUN via iSCSI to the server, now the server must run the
necessary steps in order to discover the LUN.

simulator:/$ lsscsi
[1:0:0:0] cd/dvd NECVMWar VMware IDE CDR10 1.00 /dev/sr0
[2:0:0:0] disk VMware, VMware Virtual S 1.0 /dev/sda

As you see, we only have the cdrom device and the disk where the OS is installed. Let’s discover the
iSCSI target:

simulator:/# iscsiadm -m discovery -t st -p 192.168.233.131


192.168.233.131:3260,1000 iqn.1992-08.com.netapp:sn.99923498

Now check or SCSI devices, just to keep a pre discovery view

simulator:/# lsscsi
[1:0:0:0] cd/dvd NECVMWar VMware IDE CDR10 1.00 /dev/sr0
[2:0:0:0] disk VMware, VMware Virtual S 1.0 /dev/sda

Now restart the iSCSI service:

simulator:/# /etc/init.d/open-iscsi restart


* Disconnecting iSCSI targets [ OK ]
* Stopping iSCSI initiator service [ OK ]
* Starting iSCSI initiator service iscsid [ OK ]
ln: target `/lib/init/rw/sendsigs.omit.d/' is not a directory: No such file or directory
* Setting up iSCSI targets [ OK ]

Finally, list SCSI devices again:

root@simulator:/# lsscsi
[1:0:0:0] cd/dvd NECVMWar VMware IDE CDR10 1.00 /dev/sr0
[2:0:0:0] disk VMware, VMware Virtual S 1.0 /dev/sda
[3:0:0:1] disk NETAPP LUN 7340 /dev/sdb

Creating an FS inside the LUN

Run fdisk and create a new partition inside the LUN and then format the partition.

Run: fdisk /dev/sdb, create the partition and write the partition table to the disk.

Then run mkfs.ext4 (this is just another one of my choices), Run: mkfs.ext4 /dev/sdb1

Finally mount the partition, Run: mount /dev/sdb1 /mnt.

ABOUT ME

I’m Hernán J. Larrea, an IT guy, I’ve been working in IT for almost 7 years. I decided to address my career
in Wintel technologies, specially Active Directory, Backup and Recovery environments, Storage solutions
and a little programming in .NET. If you want to know more about me, I invite you to visit my personal
website www.hernanjlarrea.com.ar and if you want to email me, you can do it at hjlarrea@hotmail.com

Hernán J. Larrea

10 Hernán J. Larrea – www.hernanjlarrea.com.ar