Running the Netapp Ontap 7-Mode 8.2.1 Simulator

The below is copied from a set of notes taken a while ago. I’ll update these as I go.

For VMware Workstation

  • Grab the vsim_netapp-7m.tgz from the Netapp Support site, untar / unzip it
  • This will uncompress a bunch of vmdk files. Most of these files are “individual” disk which will appear on the storage controller
  • Load the VMX file in VMware Workstation

 For ESXi

  • Grab the vsim_esx-7m.tgz from the Netapp Support site, untar / unzip it
  • Enable ssh on the exsi server
  • Copy the tar to the datastore1
  • Uncompress the image (tar -xvzf)
  • Run vmkload_mod multiextent  (https://communities.netapp.com/thread/24329)

 

Common Instructions

  • Boot the vm
    Ctrl-c during boot

1

Select option 4 to start with a fresh config

Provide a hostname, don’t use IPv6 or interface groups
Then setup an IP address and default gateway on e0a, the remaining interfaces can be setup later

When requested don’t provide an admin host (otherwise you will be restricted to use this machine configure the netapp)

netapp_ip

Setup root password when requested
Log in as root
Change the network interfaces setting to connect the correct physical network (Usually e0a is mapped to network adaper 1)

netapp_settings

  • You should now be able to ping and ssh to the netapp using root / pwd
  • Now install OnCommand System Manager 3.1  (I tired to use 3.1.1 but it refused to authenticate correctly stick with the older version)
  • Add the netapp to OnCommand System Manager using the root username/password

If the command manager does not work (connection refused) then its probably because the httpd wasn’t enabled during initial config. Fix as follows:

1) ssh to netapp

netapp1> options httpd
httpd.access legacy
httpd.admin.access legacy
httpd.admin.enable off
httpd.admin.hostsequiv.enable off
httpd.admin.max_connections 512
httpd.admin.ssl.enable off
httpd.admin.top-page.authentication on
httpd.autoindex.enable off
httpd.bypass_traverse_checking off
httpd.enable off
httpd.ipv6.enable off
httpd.log.format common
httpd.method.trace.enable off
httpd.rootdir /vol/vol0/home/http
httpd.timeout 300
httpd.timewait.enable off
3) options httpd.admin.enable true (enable access)
4) options httpd.admin.ssl.enable true (enable secure access)
netapp1> options httpd
httpd.access legacy
httpd.admin.access legacy
httpd.admin.enable on
httpd.admin.hostsequiv.enable off
httpd.admin.max_connections 512
httpd.admin.ssl.enable on
httpd.admin.top-page.authentication on
httpd.autoindex.enable off
httpd.bypass_traverse_checking off
httpd.enable off
httpd.ipv6.enable off
httpd.log.format common
httpd.method.trace.enable off
httpd.rootdir /vol/vol0/home/http
httpd.timeout 300
httpd.timewait.enable off

Now install the licenses from the OnCommand System Manager.  Click Config -> System tools -> Licences -> Add then paste the codes. Note, the ESXi version has different codes to the Vmware Workstation version.

The Netapp is now ready to configure and use.

Read my Netapp Storage Concepts (for network engineers) for more information on how to use the netapp.

 

Disk Structure in the Simulator

The 8.2.1 simulator starts off with:

  • 28 disks (2 shelves with 14 disks each)
netapp01*> storage  show disk
DISK                  SHELF BAY SERIAL           VENDOR   MODEL      REV
--------------------- --------- ---------------- -------- ---------- ----
v4.16                   ?    ?  08561200         NETAPP   VD-1000MB- 0042
v4.17                   ?    ?  08561201         NETAPP   VD-1000MB- 0042
v4.18                   ?    ?  08561202         NETAPP   VD-1000MB- 0042
v4.19                   ?    ?  08561203         NETAPP   VD-1000MB- 0042
v4.20                   ?    ?  08561204         NETAPP   VD-1000MB- 0042
v4.21                   ?    ?  08561205         NETAPP   VD-1000MB- 0042
v4.22                   ?    ?  08561206         NETAPP   VD-1000MB- 0042
  • pool 0 with 14 assigned disks (leaving 14 unowned disks)
  • aggr0, containing plex0, and rg0 (RAID group) with 3 disks in a RAID-DP configuration (1 data disk)
netapp01*> aggr status -v
           Aggr State           Status                Options
          aggr0 online          raid_dp, aggr         root, diskroot, nosnap=off, raidtype=raid_dp,
                                64-bit                raidsize=16, ignore_inconsistent=off,
                                                      snapmirrored=off, resyncsnaptime=60,
                                                      fs_size_fixed=off, lost_write_protect=on,
                                                      ha_policy=cfo, hybrid_enabled=off,
                                                      percent_snapshot_space=0%,
                                                      free_space_realloc=off

                Volumes: vol0

                Plex /aggr0/plex0: online, normal, active
                    RAID group /aggr0/plex0/rg0: normal, block checksums
  • vol0 in aggr0 – thick provisioned 871.916MB in size
netapp01*> vol size vol0
vol size: Flexible volume 'vol0' has size 871916k.


In onCommand click Storage -> Disks

netapp_disk

 

 

Enable access to the OS

Enter advanced mode and unlock the diagnostic user. This will allow you to look at the operating system files/logs

 ssh as root users, Enter a password and confirm.
 priv set advanced
 useradmin diaguser unlock
 useradmin diaguser password

Then launch the systemshell and login as diag and enter the password you have just set:

systemshell

netapp_diag

 

 

References

Netapp Cheat Sheet – Lists most basic cli commands

ESXi Install guide

Add Shelves to the simulator

Installing FreeBSD 10.1 in VMWare

Install FreeBSD 10.1

Create a new VM machine with

  • OS Type Other, then select FreeBSD 64bit
  • 1 CPU
  • 512M DRAM
  • 50 G Disk with can be allocate as we go along

Download the a FreeBSD 10.1 AMD64 Boot ISO and boot off this image. Then select the following

  • Standard Install (others don’t fully install unless you pick all the options manually)
  • All distributions
  • Yes to the mouse
  • No packages (will build them from the ports)
  • Use the full disk for FreeBSD
  • Accept all the default mount sizes and options
  • Set the root password
  • Add your first user and invite them to the “wheel” group

Let the VM complete install and reboot. Now do the initial tasks

  • Update the ports & source tree using the process described here, make sure to use the correct RELENG tag
  • Add a cronjob to the root user to get this happening weekly. The crontab below does the following
    • Update the time
    • Update the ports tree
    • Update the docs
    • Update src tree
    • Update the locate database (which the locate command uses)
# Update time at 1am
0 1 * * * /usr/sbin/ntpdate au.pool.ntp.org > /dev/null 2>&1

# Update ports tree at 2am
0 2 * * * portsnap fetch update > /dev/null 2>&1

# Update docs tree at 3am
0 3 * * * svn update /usr/doc > /dev/null 2>&1

# Update src tree at 4am
0 4 * * * svn update /usr/src > /dev/null 2>&1

# Update locate db at 5am
0 4 * * * /usr/libexec/locate.updatedb > /dev/null 2>&1
  • Build a specific kernel if you like as shown here
  • Update your ip addressing and resolve.conf as required
  • Update your fstab & SMB mounts

I then I like to install the following packages:

  • Install zsh (/usr/ports/shells/zsh)
  • Install vim without X11 support (/usr/ports/editors/vim-lite)
    • The editors/vim includes gvim, the gtk version of vim.  If you just want the command-line vim, install editors/vim-lite
  • Install net-snmp (/usr/ports/net-mgmt/net-snmp/)
  • Install nmap (/usr/ports/security/nmap/)
  • Install calc (/usr/ports/math/calc)
  • Install whatmask (/usr/ports/net-mgmt/whatmask)
  • Install expect (/usr/ports/lang/expect)
  • Install wget (/usr/ports/ftp/wget)
  • Install quagga (/usr/ports/net/quagga)
  • Install SOAP:Lite (/usr/ports/net/p5-SOAP-Lite)
  • Install Dumper (/usr/ports/devel/p5-Data-Dumper)
  • Install XML:Xpath (/usr/ports/textproc/p5-XML-XPath)
  • Install Expect for perl (/usr/ports/lang/p5-Expect)
  • Install Net::SNMP (/usr/ports/net-mgmt/p5-Net-SNMP)
  • Install Socket (/usr/ports/net/p5-Socket)
  • Move the users crontabs

Displaying SNMP OIDs in a MIB

The last time i tackled this issue I used the snmptranslate utility which is part of the net-snmp package.

While I was looking around the Internet I found this post which describes a better alternative to the snmptranslate utility. It uses the smidump tool in the libsmi package. So I proceeded to try it on a FreeBSD as follows:

1) Install libsmi (as root)

cd /usr/ports/net-mgmt/libsmi/
make install clean

2) Download the latest VMware mibs from here

3) Run the tool (for a simple MIB)

~/vmware/mib/vmw# /usr/local/bin/smidump -f identifiers BRIDGE-MIB.mib
# BRIDGE-MIB list of identifiers (generated by smidump 0.4.8)

BRIDGE-MIB BridgeId                           type
BRIDGE-MIB Timeout                            type
BRIDGE-MIB dot1dBridge                        node         1.3.6.1.2.1.17
BRIDGE-MIB dot1dNotifications                 node         1.3.6.1.2.1.17.0
BRIDGE-MIB newRoot                            notification 1.3.6.1.2.1.17.0.1
BRIDGE-MIB topologyChange                     notification 1.3.6.1.2.1.17.0.2
BRIDGE-MIB dot1dBase                          node         1.3.6.1.2.1.17.1
BRIDGE-MIB dot1dBaseBridgeAddress             scalar       1.3.6.1.2.1.17.1.1
BRIDGE-MIB dot1dBaseNumPorts                  scalar       1.3.6.1.2.1.17.1.2
BRIDGE-MIB dot1dBaseType                      scalar       1.3.6.1.2.1.17.1.3
BRIDGE-MIB dot1dBasePortTable                 table        1.3.6.1.2.1.17.1.4
BRIDGE-MIB dot1dBasePortEntry                 row          1.3.6.1.2.1.17.1.4.1
<SNIP>

4)  IF you run this on a VMware mib you may get the error below. This indicates that you will need to supply the additional mibs in the correct order

~/vmware/mib/vmw# /usr/local/bin/smidump -f identifiers VMWARE-SYSTEM-MIB.mib
VMWARE-SYSTEM-MIB.mib:15: failed to locate MIB module `VMWARE-ROOT-MIB'
VMWARE-SYSTEM-MIB.mib:46: unknown object identifier label `vmwSystem'
smidump: module `VMWARE-SYSTEM-MIB.mib' contains errors, expect flawed output
smidump: aborting due to severe parsing errors
smidump: use the -k option to force continuation
 1:06PM agarg@zzz:~/vmware/mib/vmw# /usr/local/bin/smidump -f identifiers VMWARE-SYSTEM-MIB.mib VMWARE-ROOT-MIB.mib
VMWARE-SYSTEM-MIB.mib:15: failed to locate MIB module `VMWARE-ROOT-MIB'
VMWARE-SYSTEM-MIB.mib:46: unknown object identifier label `vmwSystem'
smidump: module `VMWARE-SYSTEM-MIB.mib' contains errors, expect flawed output
smidump: aborting due to severe parsing errors
smidump: use the -k option to force continuation
 1:06PM agarg@zzz:~/vmware/mib/vmw# /usr/local/bin/smidump -f identifiers VMWARE-ROOT-MIB.mib VMWARE-SYSTEM-MIB.mib
# VMWARE-ROOT-MIB list of identifiers (generated by smidump 0.4.8)

VMWARE-ROOT-MIB   vmware                   node         1.3.6.1.4.1.6876
VMWARE-ROOT-MIB   vmwNotifications         node         1.3.6.1.4.1.6876.0
VMWARE-ROOT-MIB   vmwSystem                node         1.3.6.1.4.1.6876.1
VMWARE-ROOT-MIB   vmwVirtMachines          node         1.3.6.1.4.1.6876.2
VMWARE-ROOT-MIB   vmwResources             node         1.3.6.1.4.1.6876.3
VMWARE-ROOT-MIB   vmwProductSpecific       node         1.3.6.1.4.1.6876.4
VMWARE-ROOT-MIB   vmwLdap                  node         1.3.6.1.4.1.6876.40
VMWARE-ROOT-MIB   vmwTraps                 node         1.3.6.1.4.1.6876.50

Understanding VMware Syslog messages

At work I’ve start to manage an existing vmware cluster. One of the first things I did was to setup a common syslog platform and send all the messages from the ESXi servers to it.

The syslogs messages from vmware are prefixed with process ID which identifies what process the syslogs relate to. The format of the syslog is:

: Message

For example:

Jul 23 00:05:01 # Jul 22 14:05:01 vs01.local hostd-probe: [FFAE2B70 warning 'Libs'] SSL_VerifyCbHelper: Certificate verification is disabled, s
o connection will proceed despite the error
Jul 23 00:05:01 # Jul 22 14:05:01 vs01.local Rhttpproxy: [FFEC1B70 verbose 'Proxy Req 34829'] New proxy client <io_obj p:0xffd16a68, h:21, <TCP
'127.0.0.1:80'>, >
Jul 23 00:05:01 # Jul 22 14:05:01 vs01.local hostd-probe: [FFB64B70 warning 'Libs'] SSL_VerifyCbHelper: Certificate verification is disabled, s
o connection will proceed despite the error
Jul 23 00:05:01 # Jul 22 14:05:01 vs01.local hostd-probe: [FFB64B70 warning 'Libs'] SSL_VerifyCbHelper: Certificate verification is disabled, s
o connection will proceed despite the error
Jul 23 00:05:01 # Jul 22 14:05:01 vs01.local hostd-probe: [FFB64B70 warning 'Libs'] SSL_VerifyCbHelper: Certificate verification is disabled, s
o connection will proceed despite the error
Jul 23 00:05:01 # Jul 22 14:05:01 vs01.local hostd-probe: [FFA7F7E0 quiet 'Default'] Successfully acquired hardware: PowerEdge R720
Jul 23 00:05:01 # Jul 22 14:05:01 vs01.local Hostd: [277C2B70 verbose 'SoapAdapter'] Responded to service state request
Jul 23 00:05:02 # Jul 22 14:05:01 vs01.local hostd-probe: [FFB23B70 info 'ThreadPool'] Thread delisted
Jul 23 00:05:02 # Jul 22 14:05:01 vs01.local hostd-probe: [FFA7F7E0 info 'ThreadPool'] Thread delisted
Jul 23 00:05:02 # Jul 22 14:05:01 vs01.local syslog[31439740]: hostd probing is done.
Jul 23 00:05:08 # Jul 22 14:05:08 vs01.local Vpxa: [3914AB70 verbose 'hostdstats'] Set internal stats for VM: 2 (vpxa VM id), 26149 (vpxd VM id
). Is FT primary? false
Jul 23 00:05:08 # Jul 22 14:05:08 vs01.local Vpxa: [3914AB70 verbose 'hostdstats'] Set internal stats for VM: 3 (vpxa VM id), 26150 (vpxd VM id
). Is FT primary? false
Jul 23 00:05:08 # Jul 22 14:05:08 vs01.local Vpxa: [3914AB70 verbose 'hostdstats'] Set internal stats for VM: 4 (vpxa VM id), 26151 (vpxd VM id
). Is FT primary? false

So far I’ve been able to identify the following processes and their usefulness.

Useful
Process
Details
Low VPXD It is Vcenter Server Service. If this service is stopped then we will not able to connect to Vcenter Server via Vsphere client.
Low VPXA It is the agent of Vcenter server. also known as mini vcenter server which is installed on the each esx server which is managed by Vcenter server. What are the management action we are performing on top of the vcenter server. (Like:- Increasing/Decreasing RAM & HDD, Making any type of changes in cluster,doing vmotion. This agent collects all information from the vcenter server and pass this information to the kernal of the esx server)
Low HOSTD This is the agent of ESX server, here VPXA pass the information to the HOSTD and hostd pass the information to ESX server.
Medium vmkernel Core VMkernel logs, including device discovery, storage and networking device and driver events, and virtual machine startup.
High vmkwarning selected error messages from vmkernel
Fdm vSphere High Availability logs, produced by the fdm service.
Rhttpproxy Web interface?
High vobd vob propagates kernel level errors to third-party applications. vobd is a daemon that VMware and third-party applications use for monitoring and troubleshooting.
Low syslog
Low crond standard cron messages
High heartbeat Awesome, a quick summary of ESXi host startup and shutdown, and an hourly heartbeat with uptime, number of virtual machines running, and service resource consumption. For more information, see Format of the ESXi 5.0 vmksummary log file (2004566).
Low cimslp CIM (Common Information Model) SLP (Service Location Protocol)

Build a Bootable OSX Mavericks VM in VMWare

This are some notes for myself. More details will follow as I test this process

1) Make bootable Mavericks ISO

You will need a working OSX VM or mac to do this step.

Download Mavericks from the app store and file is stored in /Applications/Install OSX Mavericks/InstallESD.dmg

Then follow the instructions below (from here) to make a bootable ISO

# Mount the installer image
hdiutil attach /Applications/Install\ OS\ X\ Mavericks.app/Contents/SharedSupport/InstallESD.dmg -noverify -nobrowse -mountpoint /Volumes/install_app

# Convert the boot image to a sparse bundle
hdiutil convert /Volumes/install_app/BaseSystem.dmg -format UDSP -o /tmp/Mavericks

# Increase the sparse bundle capacity to accommodate the packages
hdiutil resize -size 8g /tmp/Mavericks.sparseimage

# Mount the sparse bundle for package addition
hdiutil attach /tmp/Mavericks.sparseimage -noverify -nobrowse -mountpoint /Volumes/install_build

# Remove Package link and replace with actual files
rm /Volumes/install_build/System/Installation/Packages
cp -rp /Volumes/install_app/Packages /Volumes/install_build/System/Installation/

# Unmount the installer image
hdiutil detach /Volumes/install_app

# Unmount the sparse bundle
hdiutil detach /Volumes/install_build

# Resize the partition in the sparse bundle to remove any free space
hdiutil resize -size `hdiutil resize -limits /tmp/Mavericks.sparseimage | tail -n 1 | awk '{ print $1 }'`b /tmp/Mavericks.sparseimage

# Convert the sparse bundle to ISO/CD master
hdiutil convert /tmp/Mavericks.sparseimage -format UDTO -o /tmp/Mavericks

# Remove the sparse bundle
rm /tmp/Mavericks.sparseimage

# Rename the ISO and move it to the desktop
mv /tmp/Mavericks.cdr ~/Desktop/Mavericks.iso
2) VMWare unlocker
Install VMware unlocker from here.
3) In VMWare
Create a new virtual machine and set the parameters as follows:
a) Guest Operating System: Apple Mac OS X
b) Version: Mac OS X 10.8 64bit
c)  Processors 1, 2 Cores
c) Point the CDRROM to the ISO created in step 1
Power on the VM and install OSX
4) Install the VMware tools

Pick the latest version from the below and download the “darwin.zip.tar” file (it’s only about 2Mb), you can extract it and browse into the archive to find the ISO. Mount the ISO inside OSX and run the installer.

http://softwareupdate.vmware.com/cds/vmw-desktop/fusion/

5) Install Video Driver VMvgaII

See: http://www.insanelymac.com/forum/topic/188962-vmware-svga-ii-display-driver-for-os-x-v125/

Install the version for 10.9 directly from here http://sourceforge.net/projects/vmsvga2/files/Display/ , otherwise you will get lots of funky errors. EG Finder will not work, Safari will keep crashing.

6) Go to setting -> Display
Hold down the ALT key when selecting Scaled
This will present a larger number of resolutions to choose from. I needed 1600×900

VMware Time Syncronization

To prevent VMware from modifying the time on your guest machine, add the following options to your vmx file:

tools.syncTime = “FALSE”
time.synchronize.continue = “FALSE”
time.synchronize.restore = “FALSE”
time.synchronize.resume.disk = “FALSE”
time.synchronize.shrink = “FALSE”
time.synchronize.tools.startup = “FALSE”

Here is VMware’s description of these options.This info is reblogged from here.