Linux Basic Commands – File System Navigation

cd – Change directory

pwd – Print name of current working directory

ls – List directory contents

file – Print a brief description of the file’s contents.

CD Commands

cd command, is the main command you use to navigate around the file system.  While moving, the pwd command, will tell you where you are now in the current file system. ls command will list the files and directory contents in your current directory.

Root Directory

The Root Directory or “/” is the start of the file system directory, or parent of all directories.

Absolute Pathnames

An absolute pathname begins with the root directory and follows the tree branch by branch until the path to the desired directory or file is completed.  For example,

$ cd /etc/network    (changes your current directory to root --> etc --> network directory)
$ pwd
/etc/network

Relative Pathnames

The shell uses special symbols to represent relative positions in the file system tree.

• “.” symbol refers to the working directory
• “..” symbol refers to the parent directory of the current working directory.

1. “cd ..” means move up to parent directory
2. “cd ./bin” means from current directory move down to a directory name “bin”
3. “cd bin” and point 2 does the same thing.

In general if you do not specify a pathname to something, the working directory will be assumed.

cd Shortcuts

• cd
  Changes the working directory to your home directory
• cd -
  Changes the working directory to the previous working directory.
• cd ~user_name 
  Changes the working directory to the home directory of user-name.

Facts about Filenames

1. Filenames that begin with a period character are hidden. ls command will not list them unless you use ls -a.
2. Filenames are case sensitive.
3. There is no concept of “file extension”.
4. You can, but try not to use punctuation characters in filenames.  Limit the punctuation characters in the filename  you create to “.” period, “-“ dash and “_” underscore. 
5. You can, but try not to use spaces between filename. Try to use “_” (underscore) to replace your spaces between filename.

The ls Command

• ls
  List current directory.  List filename only.
• ls /usr
  List directory under /usr directory
• ls ~
  List home directory
• ls -a
  List all files, including hidden files.
• ls -l or ll
  List current directory with long format.  File permission rights, owner, group, size, date time last modified, filename, links.

The file Command

As there is no concept of “file extension”, you can use the “file” command to print out a brief description of the file’s contents.

$ file /bin/ls
ls: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=0xf31e99218b4d7034cf8257055686bca22f5a3c01, stripped

$file * (list all file type in the current directory)

Base Directories

• /
  The root directory.  Where everything begins.
• /boot
  Contains the Linux kernel (vmlinuz), initial RAM disk image (initrd.img) and boot loader (grub).
• /bin
  Contain common binaries programs. Example, bash, cat, chmod, chgrup, chown, date, grep, kill, less, login, ls, mkdir, rm, rmdir, ping, sed, su, tar.
• /sbin
  This directory contains system binaries. These are programs that perform system tasks that are generally
  reserved for the superuser. Example, fsck, ifconfig, ifdown, ifup, mkfs, reboot, reload, route, runlevel, start, stop, shutdown.
• /lib
  Contains shared library files used by the core system programs. These are similar to DLLs in Windows.
• /usr
  The /usr directory tree is likely the largest one on a Linux system. It contains all the programs and support files used
  by regular users.
• /usr/bin, contains the executable programs installed by your Linux distribution.
• /usr/sbin, contains more system administration programs
• /usr/lib, contains shared libraries for the programs in /usr/bin
• /opt
  The /opt directory is used to install “optional” software. This is mainly used to hold commercial software products
  that may be installed on your system.  Most of the software will be placed here after system setup.
• /etc
  The /etc directory contains all of the system-wide configuration files. It also contains a collection of shell
  scripts which start each of the system services at boot time. Everything in this directory should be readable text.
• /etc/crontab, a file that defines when automated jobs will run
• /etc/fstab, a table of storage devices and their associated mount points.
• /etc/passwd, a list of the user accounts
• /etc/hostname, define of system name
• /etc/hosts, IP to name hosts file
• /etc/profile, bash system-wide profile for users
• /dev
  This is a special directory which contains device nodes. “Everything is a file” also applies to devices. Here is where
  the kernel maintains a list of all the devices it understands.
• /media
  On modern Linux systems the /media directory will contain the mount points for removable media such USB
  drives, CD-ROMs, etc. that are mounted automatically at insertion.
• /mnt
  On older Linux systems, the /mnt directory contains mount points for removable devices that have been mounted
  manually.
• /proc
  The /proc directory is special. It's not a real file system in the sense of files stored on your hard drive. Rather, it is a
  virtual file system maintained by the Linux kernel. The “files” it contains are peepholes into the kernel itself. The
  files are readable and will give you a picture of how the kernel sees your computer.
• /lost+found
  Each formatted partition or device using a Linux file system, such as ext3, will have this directory. It is used in the case of a partial recovery from a file system corruption event. Unless something really bad has happened to your system, this directory will remain empty.
• /home
  In normal configurations, each user is given a directory in /home. Ordinary users can only write files in their home
  directories. This limitation protects the system from errant user activity.
• /root
  This is the home directory for the root account.
• /tmp
  The /tmp directory is intended for storage of temporary, transient files created by various programs. Some
  configurations cause this directory to be emptied each time the system is rebooted.
• /var
  The /var directory tree is where data that is likely to change is stored. Various databases, spool files, user mail, etc. are located here.
• /var/log, contains log files, records of various system activity. These are very important and should be monitored from time to time.

Linux Shell Basic – bash shell (note 1)

For most linux system, the default shell is bash, an acronym for Bourne Again Shell.  bash is an enhanced replacement for sh, the original Unix shell program written by Steve Bourne.

The shell is a program that takes your commands from the keyboard and gives them to the operating system to perform.

The bash shell is located in /bin directory.  In /bin directory, you will see a file named “bash”, this is the binary program for bash shell. 

Shell Operations

A shell is a program that interprets commands and allows a user to execute commands by typing them manually at a terminal. A shell is not an operating system. It is a way to interface with the operating system and run commands.

bash shell is just like DOS command prompt.  It has some built-in understanding of “shell commands” and “rules”.  shell script files are like the .bat files in DOS, it contains basic programming logic and utility commands for the shell to interpret and execute.

The following is a brief description of the shell’s operation when it reads and executes a command.  Basically, the shell does the following:

1. Reads its input from a file or interactive console.
2. Breaks the input into words and operators or refer to as tokens. 
3. Parses the input tokens into simple and compound commands.
4. Breaking the expanded tokens into lists of filenames and commands and arguments.
5. Performs any necessary redirections and removes the redirection operators and their operands from the argument list.
6. Executes the command.
7. Waits for the command to complete and collects its exit status.

For more details on bash shell, you can refer to http://www.gnu.org/software/bash/manual/bashref.html

How is BASH different form the DOS command prompt?

1. Case Sensitivity:
   In Linus, commands and filenames are case sensitive. 
2. “\” vs “/”:
   in DOS “/” is use for command argument delimiter and “\” is a directory separator.  In Linux, the “/” is the directory separator and “\” is an escape character (see special characters for BASH on the use of “\”).
3. Filenames:
   The DOS uses “eight . three” filename convention.  In Linux, there is no file extension.  Periods can be placed at any part of the filename.

Special Characters for BASH

• \
  Escape character.  If you want to reference a special character, you must “escape” it with a backslash first before you can put in the special character for BASH to understand.
• /
  Directory separator.
• .
  Current directory.  Or “hided” files when it is the first character in a filename.
• ..
  Parent directory.
• ~
  User’s home directory.
• *
  Represents 0 or more characters in a filename, or by itself, represents all files in a directory.
• ?
  Represents a single character in a filename.
• [ ]
  Can be used to represent a range of values. e.g [0-9] [A-Z]
• |
  ”Pipe”.  Redirect the output of one command into another command.
• >
  Redirect output of a command into a new file.
• >>
  Redirect output of a command onto the end of an existing file.  Or append.
• <
  Redirect the output of a command into a program.
• ;
  Command separator.  Allows you to execute multiple commands on a single line.
• &&
  Command separator as “;”, but only runs the second command if the first one finished without errors.
• &
  Execute a command in the background and immediately get your shell back.

Executing Commands

• Most common/utility commands are located in your shell’s “PATH”, meaning that you can just type the name of the program to execute it and the shell will search for the program in the “PATH” and do the execution.   
  You can run the following command to see your current shell’s “PATH”.
  # echo $PATH
• To execute commands that are not in your PATH, you have to give the complete locations of the command.
  /home/john/myprogram
  ./myprogram (Execute a program in the current directory)
  ~/myprogram (Execute a program from a personal directory)

Do note that you need “./” before you run any program that is not in your “PATH”

Ubuntu Server 13.1 - Install Telnet and SSH Server for remote connection (Post Setup 3)

By default, telnet server and ssh server is not install.  You will have to install it yourself. 
1. Before installation of telnet and ssh, do a ping test to ensure that you are able to connect to Internet from Ubuntu Server.
2. Update and refresh the latest software packages list in Ubuntu Server. This is to ensure that you will install the latest copy of Telnet Server and SSH Server.

$ sudo apt-get update

You will see a list of connections to http://us.archive.ubuntu.com ….  ended with “Reading package lists… Done”.
If you don’t have any error message, you are good to go.
3. Install Telnet Server

$ sudo apt-get install telnetd

After this step, you should be able to telnet to your Ubuntu Server.
If for any reasons that you need to restart your telnetd services, enter the following command

$ sudo /etc/initd/openbsd-inetd restart

4. Install SSH Server

$ sudo apt-get install openssh-server

After this step, you should be able to ssh to your Ubuntu Server.
If for any reason that you need to restart your ssh services, enter the following command

$ sudo /etc/init.d/ssh restart

Ubuntu Server 13.1 - Shutdown and Reboot (Post Setup 2)

Reboot after shutdown

$ sudo shutdown –r now
or
$ sudo reboot

Shutdown and Power off

$ sudo shutdown –P now
or
$ sudo poweroff

Ubuntu Server 13.1 – Basic Networking, Changing IP Address and Hostname (Post Setup 1)

Change IP address from DHCP to Static
The network interface config file is located at /etc/network/interfaces
The default setting of the interfaces file is

# This file describes the network interfaces available on your system

# and how to activate them. For more information, see interfaces(5).

# The loopback network interface

auto lo

iface lo inet loopback

# The primary network interface

auto eth0

iface eth0 inet dhcp

Modify the setting as per below after the entry iface eth0 inet.  I think the below setting is clear enough, don’t have to explain what is IP address, broadcast and gateway.

$ sudo su -

[sudo] password for xxx: (enter your password here for su to root)

$ nano /etc/network/interfaces

iface eth0 inet static

address 192.168.1.51

netmask 255.255.255.0

network 192.168.1.0

broadcast 192.168.1.255

gateway 192.168.1.1

dns-nameservers 192.168.200.1

dns-nameservers 165.21.83.88

$ service networking restart

Note: The /etc/resolv.conf name server entries will be replaced/updated with the dns-nameservers setting in /etc/network/interfaces after rebooting.  Therefore, no need to modify /etc/resolv.conf with the name server entries again.

 

Changing Hostname and Fully Qualified Domain Name (FQDN)

The host name file is located at /etc/hostname, you can modify the entry in this file to the required hostname. Do note that you will also have to modify the hostname in  /etc/hosts file for the 127.0.0.1 entry, if not your hostname and your hosts file entries will be different.

The command service networking restart doesn’t seem to update the hostname changes, best to reboot the system. 

ESXi 5.0 Load Balancing Test: Route based on the originating virtual port ID

Within ESXi 5.0, there are 4 methods of Load Balancing.  As stated in the help

Route based on the originating port ID
Select an uplink based on the virtual port where the traffic entered the standard switch.

Route based on ip hash
Select an uplink based on a hash of the source and destination IP addresses of each packet. For non-IP packets, whatever is at those offsets is used to compute the hash.

Route based on source MAC hash
Select an uplink based on a hash of the source Ethernet.

Use explicit failover order
Always use the highest order uplink from the list of Active adapters that passes failover detection criteria.


This test is to review what "Route based on source MAC hash" does and how it perform the required traffic load balancing.

ESXi 5.0 Load Balancing Test: Route based on IP hash

Within ESXi 5.0, there are 4 methods of Load Balancing.  As stated in the help

Route based on the originating port ID Select an uplink based on the virtual port where the traffic entered the standard switch.

Route based on ip hash Select an uplink based on a hash of the source and destination IP addresses of each packet. For non-IP packets, whatever is at those offsets is used to compute the hash.

Route based on source MAC hash Select an uplink based on a hash of the source Ethernet.

Use explicit failover order Always use the highest order uplink from the list of Active adapters that passes failover detection criteria.

The test below is to test and study what "Route based on IP hash" does and how it perform traffic load balancing.

ESXi 5.0 Load Balancing Test: Route based on the originating virtual port ID

Within ESXi 5.0, there are 4 methods of Load Balancing.  As stated in the help

Route based on the originating port ID
Select an uplink based on the virtual port where the traffic entered the standard switch.
Route based on ip hash
Select an uplink based on a hash of the source and destination IP addresses of each packet. For non-IP packets, whatever is at those offsets is used to compute the hash.

Route based on source MAC hash
Select an uplink based on a hash of the source Ethernet.
Use explicit failover order
Always use the highest order uplink from the list of Active adapters that passes failover detection criteria. This test is to review what "Route based on the originating port ID" does and how it perform the required traffic load balancing.

vSwitch Network Failover Detection Testing: Beacon Probing and Link Status

In ESXi 5.0, under vSwitch Network NIC Teaming, there is a Network Failover Detection field.  It has two setting, Beacon Probing and Link Status Only.

Under the help Section, it state the following,

Link Status only
Relies solely on the link status that the network adapter provides. This option detects failures, such as cable pulls and physical switch power failures, but not configuration errors, such as a physical switch port being blocked by spanning tree or mis-configured to the wrong VLAN or cable pulls on the other side of a physical switch.
Beacon Probing
Sends out and listens for beacon probes on all NICs in the team and uses this information, in addition to link status, to determine link failure. This option detects many of the failures mentioned above that are not detected by link status alone.
Note: Do not use beacon probing with IP-hash load balancing.
 
After three days of testing over weekends, I found Beacon Probing is very confusing, but it works.

Link Status is what stated above, nothing much to test.  But Beacon Probing is Beacon + Link State.

vSwitch Networking Security Testing - Part 2 MAC Address Changes

In ESXi 5.0 Networking Security, each vSwitch has the following Security Policy Tab.

• Promiscuous Mode: Accept or Reject
• MAC Address Changes: Accept or Reject
• Forged Transmits: Accept or Reject

In part 2 of this test lab, I am exploring the MAC Address Changes setting and the effect of setting it to Accept or Reject.

A Windows 2008 R2 Server VM is installed in the ESXi 5.0 Server and a Port Group is created for this VM.


Under the help menu of "Edit Security Policy for a vSphere Standard Switch".  It state the difference between these two setting

• MAC Address Changes
• Reject - If you set the MAC Address Changes to Reject and the guest operating system changes the MAC address of the adapter to anything other than what is in the .vmx configuration file, all inbound frames are dropped.
  If the Guest OS changes the MAC address back to match the MAC address in the .vmx configuration file, inbound frames are passed again.
• Accept - Changing the MAC address from the Guest OS has the intended effect: frames to the new MAC address are received.

• Forged Transmits 
• Reject - Any outbound frame with a source MAC address that is different from the one currently set on the adapter are dropped.
  
• Accept - No filtering is performed and all outbound frames are passed.

After reading this, I am totally confused.  My understanding from the statement is that if you change the MAC Address of the vNIC of the Guest VM, "Mac Address Changes: Reject" drop inbound traffic and "Forged Transmits: Reject" drop Outbound traffic.

The confusion is, reference to where? or from which perspective?  While doing networking for so many years, I came to understand that when we come to "Inbound" traffic and "Outbound" traffic, we must think like a Router or Switch.  If I am a Router, any traffic that is coming into me is an "Inbound" traffic, the traffic can be from any interfaces.  That goes the same to any traffic that is going out of me, which is an "Outbound" traffic, it can also be going out of any Interfaces.  That goes the same if your are a Server.

But in the case of a Server it's "Inbound" traffic is the Physical Switch "Outbound" traffic, and the Server "Outbound" traffic is the Physical Switch "Inbound" traffic.  When we read the traffic, we have to know where we stand.  Inside the Server? or Inside the Switch.  That is what I called perspective of things.

The problem with the above statement is that it does not state the "Inbound" and "Outbound" is from which perspective.  From the virtual machine or from the virtual switch. 

In any case, I am doing some test to find out.

vSwitch Networking Security Testing - Part 1 Promiscuous Mode

In ESXi 5.0 Networking Security, each vSwitch has the following Security Policy Tab.

• Promiscuous Mode: Accept or Reject
• MAC Address Changes: Accept or Reject
• Forged Transmits: Accept or Reject

In part 1 of this test lab, I am exploring the Promiscuous Mode setting and the effect of setting it to Accept or Reject.

A Windows 7 VM is installed in the ESXi 5.0 Server and connected to the same VLAN as the Management Traffic. 

WireShark is installed in the VM to capture the traffic in the vSwitch.  Microsoft Network Capture is used to open the WireShark captured packet.   MS NCap is use because NCap can do a better job in sorting TCP Session as compare to WireShark.

ESXi 5.0 Management Network Interface Testing - Part 2

Part 2 of the Management Network Interface Testing is about miscofig of ESXi Management Network Interface.

Misconfiguration happened very day and is part of life.  We learned from our mistake and gain experience from our mistake. 

In this part 2 of the test, three tests will be performed

• Duplicate IP address of current Management IP in the same network segment.
• Misconfig of Management Network IP.
• Delete of Management Network IP that is currently connecting to vCenter.

ESXi 5.0 Management Network Interface Testing - Part 1

During the course of upgrading my ESX 4.1 Server farm.  I notice that the behaviour of ESXi Management Network Interface is different from the Service Console of ESX 4.1.  With that, I decided to further test and understand the concept and deployment of  multiple Management Interface of ESXi 5.0.

Test Objectives
The test objective is to test the redundancy of ESXi Management Network Interface for vCenter connectivity and SSH.  Objective is to recover ESXi remotely should a portion of the network failed.

ESXi 5.0 Kickstart Installation Part 4 - The Kickstart File Result

ESX 5.0 Kickstart Installation

Below is the result of the Kickstart Script. 


vSwitch0

ESXi 5.0 Kickstart Installation Part 3 - The Kickstart File

# +---------------------------------------------------------------------------+
# | Kickstat File : ESX07
# +---------------------------------------------------------------------------+


# +---------------------------------------------------------------------------+
# | Start of ESXi 5.0 Kick Start Script (22 Sept 2011)
# +---------------------------------------------------------------------------+


# +---------------------------------------------------------------------------+
# | Accept License agreement
# +---------------------------------------------------------------------------+
vmaccepteula


# +---------------------------------------------------------------------------+
# | Disk Partitioning
# | Clear all partitions in first detected disk and overwrite any VMFS
# | partitions on the specified drives.
# +---------------------------------------------------------------------------+
clearpart --firstdisk --overwritevmfs

ESXi 5.0 Kickstart Installation Part 2 – Getting the Kickstart Script kicking

ESX 5.0 Kickstart Installation

OK.  I must say, looking into VMware “vSphere Installation and Setup” guide for help doesn’t do much help.  Or, you only got 10% of the script required. 

Limited VMware Documentation

For booting and locating the Kickstart file, you can look into vSphere 5.0 documentation

ESXi and vCenter Server 5.0 Documentation > vSphere Installation and Setup > Installing, Upgrading, or Migrating Hosts Using a Script > Enter Boot Options to Start an Installation or Upgrade Script

Under the section "Boot Options", you will find some commands option to tell ESXi where to locate the kickstart script.

As for the kickstart script, you can find some information in

ESXi and vCenter Server 5.0 Documentation > vSphere Installation and Setup > Installing, Upgrading, or Migrating Hosts Using a Script > About Installation and Upgrade Scripts
Under the section “Installation and Upgrade Script Commands”.   You will find some commands there, but it will not bring you far for your scripting customization. 

Most ESX 4.1 Kickstart Script Don't Work

With my experience in creating kickstart script in ESX 4.1, I decided to reuse most of my 4.1 kickstart script. To my surprise, most of the command failed. That left me with no choice but to learn how to write vSphere 5.0 kickstart from basic again. 

ESXi 5.0 Kickstart Installation Part 1 - Getting the USB Drive Ready

ESX 5.0 Kickstart Installation

As I have 7 ESX 4.1 Servers in my Environment each with many VLANs, iSCSI and NFS Setting, I need to find a way to automate the installation and create a script that will bring consistency setting across my ESX farm.  This is where I spend 2 days understanding the different options to automate this process.

I have dropped the use of DHCP+PXE+TFTP as it is too complicated to setup if I have another site to install.  In addition, I can’t bring my whole DHCP+PXE+TFTP Servers setup into another site.  In addition, if the installation site is in Customer environment, addition a DHCP+PXE+TFTP sound like a lot of things to explain and answer to customer.

Upgrade ESX 4.1 update 1 to ESXi 5

vSphere 5 is out! Manage to find some time this week to upgrade my ESX 4.1 update 1 environment to vSphere 5.  For those people out there using ESX(i) whitebox, below is my whitebox configuration that works with ESX(i) 4.1 and ESXi 5.

Item	Description	Qty	Cost	Total (SGD)
1	Gigabyte x58A-UD3R	1	$ 365	$ 365
2	Intel i7-960 3.2 GHz	1	$ 377	$ 377
3	Asus EAH5450 Silent 1 GB DDR3	1	$ 79	$ 79
4	Kingston 12800/1600 (3x4GB in a kit)	2	$ 219	$ 438
5	Seagate 1 TB 32MB 7200rpm	1	$ 82	$ 82
6	Seagate 1.5 TB 32MB 7200rpm	1	$ 105	$ 105
7	Andyson F650M 650W Modular 80+	1	$ 135	$ 135
8	Samsung S222 22x DvD+-RW	1	$ 22	$ 22
9	Intel Gigabit ET Dual Port Server Adapter E1G42ET	2	$ 225	$ 450
10	Normal Casing	1	$ 72	$ 72
	Total (in Singapore Dollars)			$2,125

Switching Lab Core Switch 2 Config

!

!
upgrade fpd auto
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname Core2

Switching Lab Core Router 1 Config

!

!
upgrade fpd auto
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname Core_R1