Monday, September 7, 2009

System Monitoring

File containing Boot messages:
/var/log/boot.msg

To view the content of current kernel buffer:
# dmesg

Few /proc files containing system info:

/proc/cpuinfo : View processor information
/proc/meminfo : View memory information
/proc/devices : View the devices configured on your linux machine
/pro/ioports : View the I/O ports
/pro/interrupts : View the IRQ assignments
/proc/dma : View the DMA channels
/proc/bus/pci/devices : View the PCI information
/proc/scsi/scsi : View the scsi information

Fiew Commands to view hardware details:

To display the list of hardware(devices) installed in your system:
# hwinfo

To display hard drive info and manage its paramets:
# hdparm

To list the current disk partitions:
# fdisk

To display the CPU and I/O statistics:
# iostat

To display the list of PCI buses and the devices connected to it:
# lspci

To collect the system information in HTML or ASCII format:
# siga

To prepare system information using Perl:
# sitar

To display the current time, system uptime, # of users on the system:
# uptime

To list the network ports and their offered services:
# netstat -patune

To display current kernel version:
# uname -a

To display the cpu, memory, swap usage :
# top

To display the system memory and swap usage:
# free

To display memory, swap, io, system and cpu usage:
# vmstat

To display the kernel type:
# getconf LONG_BIT

Networking

To start network services:
# /etc/rc.d/init.d/network start

To stop network services:
# /etc/rc.d/init.d/network stop

To restart network services:
# /etc/rc.d/init.d/network restart

To view the network configuraiton:
# ifconfig [or]
# ip addr show

To configure an interface :
# ifconfig eth0 192.168.0.1 netmask 255.255.255.0 up
# Modify the file /etc/sysconfig/network-scripts/ifcfg-eth0

To configure alias on eth0 :
# ifconfig eth0:1 192.168.0.2 netmask 255.255.255.0 up

To list routing table :
# route

To add the default gateway to the routing table:
# route add default gw 192.168.0.15
[or]
# ip route add default via 192.168.0.5

To add a route for a specific host:
# route add -host 11.0.0.2 gw 192.168.0.115

To add a route for a specific network:
# route add -net 11.0.0.0 gw 192.168.0.115

Daemon used for dynamin routing/ Uses RIP: routed

Command to start routed daemon:
# rcrouted start

Configure a network card:
# ip link set mtu 9000 dev eth0

To show the network links:
# ip link show

To view ARP cache:
# arp -n

To remove an ARP entry:
# arp -d 192.168.1.1

To modifiy the host name resolution order, Modify the below file:
# cat /etc/host.conf
order hosts,bind

To modify the DNS server info:
# cat /etc/resolv.conf
search openna.com
nameserver 218.104.16.6
nameserver 218.104.16.12

To configure networking in Linux, modify the below file:
# cat /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=yes
HOSTNAME=myserver
GATEWAY=192.168.1.1
GATEWAYDEV=eth0

To enable IP Forwarding:

For RHEL 6.2,
a. Add the below lines to /etc/sysctl.conf file
# Enable packet forwarding
net.ipv4.ip_forward = 1

b. Restart network services
# /etc/rc.d/init.d/network restart

For RHEL version lesser than 6.2,
a. Modify /etc/sysconfig/network
b. Restart network services

Disk Partitioning

To list the current partitions:
# fdisk -l

To list the partitions on a disk:
# fdisk -l /dev/sdb

To create partition on a disk:
# fdisk /dev/sdb
-> type n
-> type 'e' for extended or
'p' for primary partition
-> Enter the first cylinder number
-> Enter the last cylinder or size in KB,MB or GB
-> Enter 'p' to verify the partition table
-> Finally save the partition table by entering 'w'

Command used to inform the kernel of partition table changes:
# partprobe

Filesystems

Redhat Enterprise Linux supports the below type of filesystems.

1. Local or Disk based Filesystems such as ext2, ext3, ext4, xfs.
2. Memory based filesystems such as tmpfs, proc, ramfs.
2. Network based Filesystem such as nfs, cifs.
3. Shared Storage based Filesystem such asgfs1, gfs2.

Disk Based Filesystems:

Ext2 Filesystem :

The ext2 or second extended filesystem was used long back and now-a-days we don't create these type of filesystem because of size restrictions and slow performance.
A strange limitation of Ext2 is it supports only upto the date January 18, 2038.
Some limitations of ext2 are
Max File Size = 16GB-2TB
Max Filesystem Size = 2-32 TB

Ext3 Filesystem :

Ext3 filesystem has been the most commonly used filesystem. It is actually an journalled Ext2 filesystem.
Ext3 filesystems can be remounted as Ext2 filesystems without any conversion.
On high-end storage devices, Ext3 has some limitations in that it can only scale to a maximum of 16TB. On a 1TB S-ATA drive, the performance of the Ext3 filesystem repair utility (fsck), which
is used to verify and repair the filesystem after a crash, is extremely long. For many users that require high availability, the Ext3 filesystem typically supports close to 2-4TB of storage.

Ext4 Filesystem :

Ext4 is the fourth generation of the Ext filesystem family and is the default filesystem in Red Hat Enterprise Linux 6. Ext4 has been offered in test previews since Red Hat Enterprise Linux 5.4, giving customers confidence in its maturity. Some features of Ext4 are extent-based metadata, delayed allocation, large storage support, journal check-summing. Filesystem repair time (fsck) in Ext4 is much faster than in Ext2 and Ext3. Currently, Red Hat’s maximum supported size for Ext4 is 16TB in both Red Hat Enterprise Linux 5 and Red Hat Enterprise Linux 6

XFS Filesystem:

XFS is a robust and mature 64-bit journaling filesystem that supports very large files and filesystems on a single host. Red Hat’s maximum supported XFS filesystem image is 100TB. XFS is one of the highest performing filesystems on large systems with enterprise workloads.

Some features of XFS are

• delayed allocation
• dynamically allocated inodes
• b-tree indexing for scalability of free space management
• ability to support a large number of concurrent operations
• extensive run-time metadata consistency checking
• sophisticated metadata read-ahead algorithms
• tightly integrated backup and restore utilities
• online defragmentation
• online filesystem growing
• comprehensive diagnostics capabilities
• scalable and fast repair utilities

And also the other OS based filesystems (given below) are also supported in Redhat linux such as

UFS: Used in Solaris and early BSD operating systems. Linux provides read and write support is experimental.

FAT: Windows based filesystem.

NTFS: Windows NT based filesystem. Unlike FAT, it is a b-tree file system, meaning it has a performance and reliability advantage over FAT.

Which filesystem should I use ?
For large server/storage device, you can go for XFS.
Generally Ext3/Ext4 should be fine for most of the filesystems.
If your existing workload has performed well with Ext3, staying with Ext3 on Red Hat Enterprise Linux 5 or migrating to Ext4 on Red Hat Enterprise Linux 6 should provide you and your applications with a very familiar environment. Two key advantages of Ext4 over Ext3 on the same storage include faster filesystem check and repair times and higher streaming read and write performance on high-speed devices.

In general Ext3 or Ext4 is better if an application uses a single read/write thread and small files, while XFS shines when an application uses multiple read/write threads and bigger files.


Configuration Files:
1. /etx/fstab - You need to edit this file to automate the file system mounts:

2. /etc/mtab - An entry will be added/removed to this file every time a filesystem gets mounted/unmounted.


Now let us look at the commands to manage the filesystems.


Q. How to list current mounts:
A.
# mount
# cat /etc/mtab

Q. How to create an Ext4 filesystem on a disk/partition/LV :
A.
# mkfs -t ext4 /dev/sdb
# mkfs -t ext4 /dev/sdb1
# mkfs -t ext4 /dev/vg00/lvol1

Here mention the file system type after the flag 't'.

Above command can be used to create any type of file system
Whereas below commands are restricted to specific types only.

EXT2 : mke2fs or mkfs.ext2
EXT3 : mkfs.ext3
Reiserfs: mkfs.reiserfs
vfat : mkfs.vfat
xfs : mkfs.xfs
MSDos : mkdosfs or mkfs.msdos
minix : mkfs.minix
xiafs : mkfs.xiafs
NTFS : mkntfs

Q. How to run file system check ( you need unmount a filesystem before proceeding with fsck command) :
A. fsck -t ext2 /dev/sdb1

Q. How to check all file systems except the mounted ones:
A.
# fsck -m -A

File system specific fsck commands:
EXT2 : fsck.ext2 or e2fsck
Reiserfs: reiserfsck
JFS : fsck.jfs
xfs : fsck.xfs
xiafs : fsck.xiafs
minix : fsck.minix

Q. How to mount a ext2 file system:
A.
#mount -t ext2 /dev/sdb1 /mnt/usb

Q. How to mount all filesystems:
A.
# mount -a

Q. To mount all ext2 file systems:
A.
# mount -a -t ext2

Q. Unmount a file system:
A.
# umount /mnt/usb

Q. To unmount all the mounted filesystems (except / and currently used ones):
A. 
# umount -a

Q. To list open files on a mounted file system:
A.
# /usr/bin/lsof +D /mnt/usb

Q. To adjust tunable filesystem paramets on ext2/ext3 filesystems:
A.
# tune2fs /dev/sdb1

Q. To list the contents of the filesystem superblock:
A.  You can use any of the below commands.

# tune2fs -l /dev/sdb1
# dumpe2fs -b /dev/sdb1

Q. To extend /data filesystem (without unmounting) :
A.

# ext2online -v /data

Q. How to display the filesystem size, usage, free space, mount points, etc..
A:
# df -h 

Q. How to shrink a filesystem
A.  You can not do an online filesystem shrink. 
You need to unmount a filesystem before reducing its size.
Perform the following steps (1 to 7) to reduce a filesystem size.

1. Unmount the filesystem
# umount /mntpoint

2. Do filesystem check
# e2fsck -f /dev/mapper/vgname-lvname

3. Reduce the filesystem to 25GB
# resize2fs /dev/mapper/vgname-lvname 25G

4. Reduce the logical volume 
# lvresize -L 25G /dev/mapper/vgname-lvname

5. Verify the LV Size
# lvdisplay /dev/vgname/lvname

6. Mount the filesystem
# mntpoint

7. Verify the filesystem size
# df -h /mntpoint

Package(RPM) Management

RPM:
- Is a tool for the management of source files, packages, build process
- Includes a distribution method and formart for binery files, including pre and postinstall scrpits

Database of the installed packages - /var/lib/rpm
To access the database, use the rpm command

To install a package:
# rpm -ivh package-name.rpm

To upgrade a package:
# rpm -Uvh package-name.rpm

To erase/uninstall a package:
# rpm -evh package-name.rpm

To freshen a package: [ Only installs if the old version was installed ]
# rpm -F package-name.rpm

Query a package:
# rpm -q package-name

To list all the installed packages:
# rpm -qa

To show the package owning a specific file:
# rpm -qf /usr/bin/ls


To list the files owned by a package:
# rpm -ql package-name.rpm

To display package information:
# rpm -qi package-name.rpm

To verifiy a package:
# rpm -V package-name

To verify the package signature:
# rpm --checksig package-name.rpm

To import a public key for a package:
# rpm --import path-to-public-key-file

Integrated package Management:
RdHat: redhat-config-packages
SuSE: yast

Red Hat Online Update:
- Create and manage account and systems thru http://rhn.redhat.com
- Register individual systems with
# up2date --register
- User 'up2date' command to update the systems

SuSE Online Update:
- Command:
'you' or
'yast' -> Online Update
- It downloads and installs the patches freom any SuSE mirror server
- You can create internal mirrors in your network

Volume Group

Create a VG:
# vgcreate vg00 /dev/hda1 /dev/hdb1

Create a VG by mentioning the PE size:
# vgcreate -s 10M vg01 /dev/hda1 /dev/hdb1

To display all the VGs:
# vgdisplay

To display more info on a particular VG:
# vgdisplay -v vgname

To display only active VGs:
# vgdisplay -A

Activate a VG :
# vgchange -a y vgname

De-activate a VG:
# vgchange -a n vgname

Extend a VG:
# vgextend vgname /dev/hdb1 /dev/hdc2

Reduce a VG:
# vgreduce vgname /dev/hdc2

Remove a VG:
# vgremove vgname

Import a VG into a system:
# vgimport vgname

Export a VG from a system:
# vgexport vgname

To learn new VGs:
# vgscan

To backup VGDA to /etc/lvm/backup/vg_name:
# vgcfgbackup vgname

To restore VG meta data from /etc/lvm/backup/vg_name or a file to the disk
# vgcfgrestore -f filename -n vgname vgname

To recreate VG directory and logical volume special files:
# vgmknodes vgname

Logical volume

To report information about logical volumes:
# lvs

To create a logical volume in an existing volume group:

Method 1 - By mentioning the size:
# lvcreate -L 1G -n lv01 vg01 /dev/sdb1

Method 2 - By mentioning the # of LEs:
# lvcreate -l 32 -n lv01 vg01

You can now use /dev/vg01/lv01 as a block device111111

To scan all disks for Logical volumes:
# lvscan

To change the minor number of a LV:
# lvchange --minor 98 lv_name

To change the permission:

Read-Only:
# lvchange -p r lv_name

Read/Write:
# lvchange --permission r/w lv_name

To expand a LV:
# lvresize -L +1G lv_name

To reduce the size of a LV:
# lvresize -L -512M lv_name

To display the attributes of a LV:
# lvdisplay -v /dev/datavg/loglv

To remove a LV:
# lvremove /dev/vg01/lv01

Physical Volume

Physical Volume:

You can create PV on a hard disk or partiiosn (of type '8e' / Linux LVM)

To initialize a PV on a partition:
# pvcreate /dev/sdb1

To initialize multiple PVs:
# pvcreate /dev/sdb1 /dev/sdb2 /dev/sdb3

To forcibly initialize a PV on a disk :
# pvcreate -f /dev/sdb

Above command initializes a PV by putting a volume group descriptor area a.k.a VGDA at the start of the PV

To show the information about a PV:
# pvdisplay /dev/sdb1

To report information about PVs:
# pvs

To scan all disks for Physical volumes:
# pvscan

To scan/show PVs only for exported VGs:
# pvscan -e

To scan/show PVs not belonging to any VG:
# pvscan -n

To expand a PV after enlarging the partition with fdisk:
# pvresize /dev/sdb1

To shrink a PV on a partition prior to shrinking the partition with fdisk:
# pvresize --setphysicalvolumesize 40G /dev/sdb1

To view PEs:
# pvdata -E /dev/vg1/lv1

To move PEs between PVs :
# pvmove -n lv1 /dev/sdd1 /dev/sdd2

LVM Limits

Max # of VG - 99
Max # of LV - 256
Max size of a LV - 512MB - 2TB (32bit)
Max size of a LV - 512MB - 1PB (64bit)
Max # of LE - 65534
Max # of PE - 65534

Startup and Shutdown

1. Run Levels:

0 - To halt a system
1,s,S - Single user mod
2 - Multiuser, no network
3 - Multiuser with network, but no XWindows
4 - Unused
5 - Multiuser with XWindows and network
6 - Reboot

2. To switch to single user run level:
# telinit 5

3. Directory containing Runlevel Scripts:

0 - /etc/init.d/rc0.d
1 - /etc/init.d/rc1.d
2 - /etc/init.d/rc2.d
3 - /etc/init.d/rc3.d
4 - /etc/init.d/rc4.d
5 - /etc/init.d/rc5.d
6 - /etc/init.d/rc6.d

4. Boot Loaders:
a. LILO
b. GRUB

5. Config Files for Boot Loaders :
LILO: /etc/lilo.conf

GRUB: /boot/grub/menu.lst

6. To set password for grub:

a.
# grub
grub> md5crypt
Password: ibmlnx
Encrypted: $1$JGuR//$kjOu7Q9kTZk01wFKWQqMb/
grub> quit

b.
# vi /boot/grub/menu.lst
Add the following line below the “splashimage” line (Red Hat)
After you see title SUSE Linux 10.0 on the next line type lock.
On the following line type password md5 [the encrypted password you copied from previous shell]:

###Don't change this comment YaST2 identifier: Original name: linux###
title SUSE Linux 10.0
lock
password md5 $1$Rdv455345ga345GvIRgXWxcF1Vjb7tZ//
root (hd0,1)
kernel /boot/vmlinuz root=/dev/hda2 vga=0x31a selinux=0
resume=/dev/hda1 splash=silent showopts
initrd /boot/initrd

7. To shutdown a system :
# shutdown now
# halt

8. To reboot a system :
# shutdown -r now
# reboot

9. To install GRB :
# /sbin/grub-install /dev/sda

Devices

Charcter / Raw Devices:
- A Raw device is a device that allows only "serial" access

Block Devices:

- A Block device is any device that allows "random" access
- Ex: Hard disk, disk partitions, RAM, Logical Volumes, RAID volumes
- It can be used for file systems, swaps, RAID, LVM
- Have a special file in /dev

Floppy Disks:
- Floppy disks have special files from /dev/fd0, /dev/fd1 .. upto 8 files/devices

Hard Disks:
- Mostly used types are IDE and SCSI

IDE:
- IDE Disks have special files from /dev/hda, /dev/hdb .. upto 8 files/devices
- Maximum 2 disks on 1 bus, 2 buses on 1 adapter, 2 adapters in a system.


SCSI:
- Depending on the scsi type, you can have maximum 7 or 15 disks on 1 bus
- SCSI Disks have special files from /dev/sda, /dev/sdb .sdz, then sdaa thru sddx. upt0 128 files/devices
- Also includes SD, DVD-ROM, tapes, ..
- Linux kernel supports a total of 128 SCSI disks by default

Partitions:
- Maximum of 4 primary partitions
- you can have 1 extended partition as a primary partition
- An extended partition can have multiple logcal partitions
- By deafult, linux supports 59 logical partitions on IDE and 11 logical on SCSI disks
- Partitions can be created using "fdisk /dev/sdx" command
- To view current partitions, use "fdisk -l" command
- Each partition can be of types Linux, Linux Swap, Linux LVM depending upon the usage

RAM Disk:
- Is a block device created in memory

- Is temporary, gets deleted after reboot

- By default Linux supports upto 16 RAM disks and maximum 255 RAM disks

- Use 'dd' command to create a RAM disk
# dd if=/dev/zero of=/dev/ram1 bs=1k count=4096

- To delete a RAM disk,
# freeramdisk /dev/ram1
- initrd (Initial RAM disk) is a compressed ramdisk image which is mounted as initial root filesystem
- Use 'mkinitrd' command to create initial ram disk

X Window for Linux

X Window for Linux

- Graphical User Interface of Unix
- Runs in a sperate user space in linux
- Users client-server architecture

Few X-window based applications,
1. xterm
2. xeyes
3. xbanner
4. xcalc
5. xedit

Default X Window Server in Linux:
XFree86

Configuration:
- Only root can do the configuration
- Config File: /etc/X11/XF86Config or XF86Config-4
- Command used to configure:
1. XFree86 -configure
2. For Redhat Linux: redfat-config-xfree86
3. For SUSE Linux: SAX2, YAST

Command to start X session:
# startx

To start second session:
# startx --:1

Window Manager:
- Available Window Managers:
Fvwm95, Afterstep, KDE, GNOME
- $HOME/.xsession contains your default window manager
- Use 'swtichdesk' command to change your window manager

Session MAnager:

- Is a program that manages X sessions

- Starts XFree86m displays a graphical login prompt

- Authenticates the user logins, then starts the users favourite window manager

- When a user logouts, it restarts XFree86 and displays a login prompt for the next user

- Available Session Managers:
a. xdm - /etc/X11/xdm/Xaccess, /etc/X11/xdm/xdm-config
b. kdm - /etc/kde/kdm/kdmrc, /etc/kde/kdm/Xaccess
c. gdm - /etc/X11/gdm/gdm.conf

- Started by init process in runlevel 5