Xen on SLES10

Xen consists of two major components.

• Virtual Machine Monitor (VMM) - A layer between physical hardware and VM. In general term, this is called hypervisor.
• Xen Tools - a set of command line application used to administer VM.

VMM must be loaded before any of VMs are started. VM in xen are called as domains. Privileged domain, called domain0 is an interface to communicate directly with an adminstrator to access the physical hardware of the host machine.

An unprivileged domain is called domainU. To install Xen, the following package is needed.

• xen
• xen-tools
• kernel-xen
• xen-doc* (optional)

After the installation, the reboot is required to boot into Xen bootloader. In case the system is not booting properly, you can switch back to the non-virtualized system and disable the firewall.

• rcSuSEfirewall2 stop
• insserv -r SuSEfirewall2_setup
• insserv -r SuSEfirewall2_init
• insserv -r SuSEfirewall_final (conditional)

To manage Xen domains at the commands line.

• Xen configuration file is located at: /etc/xen/vm
• There are examples of template configuration file for a single domain: /etc/xen/examples
• xm is the administration tools, it communicates with the xend management processes running on domain0 linux installation.
• To start a vm, the create command is used: xm create -c -f /data/xen/SLES10-WebServer.conf
• To list the currently running Xen domains: xm list
• The list command contains the following fields: name, domid, memory, vcpus, state (r - running, b - blocked, domain has been created, but it is blocked when a domain is waiting for IO or nothing to do, p - pause, s - shutdown, c - crashed)
• To connects you with the terminal of running domain: xm console domain_id
• To disconnect from a terminal: use the keystrokes: Ctrl-]
• To interrupt the execution a domain temporarily: xm pause domain_id
• To unpause: xm unpause domain_id
• To shutdown: xm shutdown domain_id
• To force shutdown a not responding domain: xm destroy domain_id
• To save the state of a domain for a longer time: xm save domain_id
• To restore a domain from a resulting file: xm restore filename
• To change the memory allocation for a domain: xm mem-set domain_id amount_of_memory_in_megabytes

Automate Domain Startup and shutdown

• SLES10 comes with a start script called, xendomains, which included in the xen-tools package.
• If domain0 is booted, all the domains with the configuration files located under /etc/xen/auto will be started.
• If domain0 is shutdown, or rebooted, all the running Xen domains will be shutdown.
• The script can be adjusted /etc/sysconfig/xendomains for some configuration options like: XENDOMAINS_MIGRATE. It will migrate domain automatically to a different host when a domain0 is shutdown. The flags has to be set to the IP address of the target machine.

Understand Xen Networking

• domain0 is controlling the physical network interfaces of the host system. Unprivileged domains are connected to domain0 through Virtual Ethernet adapters.
• domainU (VE) <---->(VE)domain0(Physical Ethernet)<------>Network(LAN)
• The default mechanism to connect VE and PE in domain0 is bridging.
• When a new domain is created, the following steps happened.
• 1. Xen provides a VE to the new domain
• 2. xend creates a new VE in domain0
• 3. both VE are connected through a virtual point to point connection.
• 4. The VE in domain0 is added to the bridge with the physical interface.
• xend performs the networking changes with the help of scripts: /etc/xen/scripts.
• The script to initiate the bridge xenbr0: /etc/xen/scripts/nework-bridge
• The script to start and add the VE: /etc/xen/scripts/vif-bridge
• To configure the network scripts are used by xend: /etc/xen/xend-config.sxp
• Command: ip a is prompted on domain0 will list down all the different interface naming schema.
• 1. peth - physical interfaces in domain0. peth devices are connected to the network bridge
• 2. vif - virtual interfaces which are part of the bridge. E.g: vif6.0 is connected to the first vif in domain 6.
• 3. veth - virtual interfaces are connected to the vif interfaces of domain0 (vif0.x)
• 4. eth0 - the default veth interface is named eth0 and connected with vif0.0. This is the default network interface of domain0.
• 5. xenbr0 - the default bridge that connects virtual and physical interfaces.

Migrate a Guest Domain

• 2 methods: Use domain save and restore or Use migration and live migration.
• Domain Save and Restore
• 1. suspend the domain: xm save domain_id filename
• 2. copy the file to the new host system, then restore the domain: xm restore filename
• Migration and Live Migration
• 1. xm migrate domain_id target_host
• 2. By adding --live migration flag, the downtime during the migration can be reduced.
• 3. /etc/xen/xend-config.sxp with 2 importants flags: xend-relocation-server yes, xend-relocation-hosts-allow '^localhost$'

VI Cursor Movement Commands

xargs and find

xargs and find

find and xargs do go very well together: find to locate what you're looking for, and xargs to run the same command on each of the things found.

Traditionally, an advantage to xargs was its ability to handle long command lines before failing, unlike some other commands. This command:

rm `find tmp -maxdepth 1 -name '*.mp3'`

is intended to remove all tmp/*.mp3 files (and ignore any subdirectories), but can fail with an "Argument list too long" message. This exact equivalent:

find tmp -maxdepth 1 -name '*.mp3' -maxdepth 1 | xargs rm

does exactly the same thing but will avoid the problem by batching arguments up. More modern kernels (since 2.6.23) shouldn't have this issue, but it's wise to make your scripts as portable as possible; and the xargs version is also easier on the eye.

You can also manually batch arguments if needed, using the -n option.

find tmp -maxdepth 1 -name '*.mp3' -maxdepth 1 | xargs -n1 rm

will pass one argument at a time to rm. This is also useful if you're using the -p option as you can confirm one file at a time rather than all at once.

Filenames containing whitespace can also cause problems; xargs and find can deal with this, using GNU extensions to both to break on the null character rather than on whitespace:

find tmp -maxdepth 1 -name *.mp3 -print0 | xargs -0 rm

You must use these options either on both find and xargs or on neither, or you'll get odd results.

Another common use of xargs with find is to combine it with grep. For example,

find . -name '*.pl' | xargs grep -L '^use strict'

will search all the *.pl files in the current directory and subdirectories, and print the names of any that don't have a line starting with 'use strict'. Enforce good practice in your scripting!

Configure Special File Permissions

Name: sticky bit, 1, t
Users can only delete files when they are owner, or when they are root or owner of the directory. This is usually applied to be /tmp directory.

Name: SGID (Set GroupID), 2, s
When a program is run, this sets the group ID of the process to that of the group of the file. Files created in this directory belong to the group to which the directory belongs and not the primary group of the users. New directories created in this directories inherit the SGID bit.

Name: SUID (Set UserID), 4, s
Sets the user ID of the process to that of the owner of the file when the program is run.

UUID

1. If you don't know the UUID of your disk, you can find it by using one of the several commands below:

host # vol_id /dev/sda3
...
ID_FS_UUID=a1331d73-d640-4bac-97b4-cf33a375ae5b
...

or:

host # blkid /dev/sda3 <-- Leave blank to show all disks
/dev/sda3: LABEL="/" UUID="a1331d73-d640-4bac-97b4-cf33a375ae5b" SEC_TYPE="ext3" TYPE="ext2"

also:

host # ls -l /dev/disk/by-uuid|grep sda3
lrwxrwxrwx 1 root root 10 11. Okt 18:02 a1331d73-d640-4bac-97b4-cf33a375ae5b-> ../../sda3

2. If you prefer to generate your own UUID's (see above), you can use the uuidgen command and couple it with tune2fs to change the default UUID assigned to your disk by the system, like this:

host # uuidgen
1d721189-7b71-4315-95a7-1c3abc90d379
host # tune2fs -U 1d721189-7b71-4315-95a7-1c3abc90d379 /dev/sda3

3. Then again, if you already know the UUID, you might want to find out what disk it's associated with. You can generally get this information with the "findfs" command, like so:

host # findfs UUID=a1331d73-d640-4bac-97b4-cf33a375ae5b
/dev/sda3

Of course, using some of the commands above and grepping out part of the UUID will also get you your answer, like:

host # ls -l /dev/disk/by-uuid|grep a1331d73-d640-4bac-97b4-cf33a375ae5b
lrwxrwxrwx 1 root root 10 11. Okt 18:02 a1331d73-d640-4bac-97b4-cf33a375ae5b-> ../../sda3

or

host # blkid|grep a1331d73-d640-4bac-97b4-cf33a375ae5b <-- remember that blkid with no arguments returns all of the system disk
/dev/sda3: LABEL="/" UUID="a1331d73-d640-4bac-97b4-cf33a375ae5b" SEC_TYPE="ext3" TYPE="ext2"

4. And, lastly (for this post, at least ;), you can mount your disks using the UUID, and even incorporate that automated UUID mounting into your /etc/fstab. To mount directly from the command line, you can do something like this:

host # mount -U a1331d73-d640-4bac-97b4-cf33a375ae5b /directory/you/mount/this/disk/on

and you could instruct your system to mount this partition by UUID from within the fstab, as well. It works basically the same way that the LABEL keyword does:

host # cat /etc/fstab
...
UUID=a1331d73-d640-4bac-97b4-cf33a375ae5b /directory/you/mount/this/disk/on ext3fs defaults 1 1


And, at this point, you should be able to figure your way around using UUID's to manipulate your disk on Linux with no problem. Enjoy, and please "be careful" :)

Expect and SSH

#!/usr/bin/expect -f

set username [lrange $argv 0 0]
set password [lrange $argv 1 1]
set ipaddr [lrange $argv 2 2]
set timeout -1

spawn ssh $username@$ipaddr
match_max 100000

expect {
"*?(yes/no)*" { send -- "yes\r" }
"*?assword:*" { send -- "$password\r" }
}

set timeout 10
expect {
"*?assword:*" { send -- "$password\r" }
}

expect "$username*" { send -- "sudo su - \r" }
interact

System Global Area

The SGA consists of several memory structures:
1. Shared Pool
2. Database Buffer Cache
3. Redo Log Buffer
4. Other structures (for example, lock and latch management, statistical data)

Shared Pool
It consists of two key performance-related memory structures, Library Cache, Data Dictionary Cache. Because the Shared Pool can be shared globally, such as reusable SQL execution plans, PL/SQL packages, procedures, functions, and cursor information, it must be sized to accommodate the needs of both the fixed and variables areas. Memory allocation for the Shared Pool is determined by the SHARED_POOL_SIZE initialization parameter. It can be dynamically resized using ALTER SYSTEM SET. After performance analysis, this can be adjusted but the total SGA size cannot exceed SGA_MAX_SIZE.

Libray Cache
It stores information about eh most recently used SQL and PL/SQL statements. It enables the sharing of commonly used statements which is managed by the least recently used (LRU) algorithm.

Data Dictionary Cache
This is a collection of the most recently used definitions in the database, which includes information about database files, tables, indexes, columns, users, privileges, and other database objects. During the parse phase, the server process looks at the data dictionary for information to resolve object names and validate access. By caching data dctionary information into memory improves response time on queries and DML.

There are two additional memory structures that can be configured within the SGA:
1. Large Pool
2. Java Pool

Oracle Architecture Diagram

 

Oracle Primary Components

Oracle server:
There are several files, processes and memory structures in an Oracle server, however, not all of them are used when processing a SQL statement. Some are used to improve the performance of the databse, to ensure that the database can be recovered in the event of a software of hardware error, ot to perform other tasks necessary to maintain the database. The Oracle server consists of an Oracle instance and an Oracle database

Oracle Instance:
An Oracle instance is the combination of the background processes and memory structures. The instance must be started to access the data in the database. Every time an instance is started, a System Global Area (SGA) is allocated and Oracle background processes are started. Background processes perform functions on behalf of the invoking procceses. They consolidate functions that would otherwise be handled my multiple Oracle programs running for each user. The background processes perform input/output (I/O) and monitor other ORacle processes to provide increased parallelism for better performance and reliability. An Oracle instance consists of the System Global Area (SGA) memory structure and the background processes that are used to manage a database. An instance is identified by using methods specific to each operating system. The instance can open and use only one database at a time.

Oracle Database:
The general purpose of a database is to store and retrieve related information. an Oracle database has a logical and a physical structure. The physical structure of the database is a set of operating system files in the database. An Oracle database consists of three file types.
1. Data files containing the actual data in the database.
2. Online redo log files containing a record of changes made to the database to enable recovery of the data in case of failures.
3. Control files containing information necessary to maintain and verify database integrity.

Memory Structure
Oracle's memory structure consists of two memory areas known as
1. System Global Area (SGA): Allocated at instance startup, and is a fundamental component of an Oracle instance.
2. Program Global Area (PGA): Allocated when the server process is started.

Migration ASM from /dev/sdc to /dev/sdb

1. Startup ASM on sdc
[root@oracle01 ~]# fdisk -l /dev/sdc

Disk /dev/sdc: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdc1 1 487 3911796 83 Linux
/dev/sdc2 488 974 3911827+ 83 Linux
[root@oracle01 ~]# su - oracle
[oracle@oracle01.myserver.com /home/oracle]
$ export ORACLE_SID=+ASM
[oracle@oracle01.myserver.com /home/oracle]
$ sqlplus / as sysdba

SQL*Plus: Release 10.2.0.1.0 - Production on Wed Jan 9 22:10:09 2008

Copyright (c) 1982, 2005, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup mount
ASM instance started

Total System Global Area 83886080 bytes
Fixed Size 1217836 bytes
Variable Size 57502420 bytes
ASM Cache 25165824 bytes
ASM diskgroups mounted
SQL> exit
Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
[oracle@oracle01.myserver.com /home/oracle]
$ asmcmd
ASMCMD> lsdg
State Type Rebal Unbal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Name
MOUNTED NORMAL N N 512 4096 1048576 7640 5284 0 2642 0 DATA/
ASMCMD> exit

2. Create filesystem onto sdb and do a dd sdc[1-2] to files (data_lun1, data_lun2)

[root@oracle01 ~]# fdisk -l /dev/sdb

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdb1 1 1305 10482381 83 Linux
[root@oracle01 ~]# mkfs /dev/sdb1
mke2fs 1.35 (28-Feb-2004)
max_blocks 2683489280, rsv_groups = 81894, rsv_gdb = 639
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
1310720 inodes, 2620595 blocks
131029 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2684354560
80 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Writing inode tables: done
inode.i_blocks = 46016, i_size = 4243456
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 33 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[root@oracle01 ~]# mount /dev/sdb1 /mydata/
[root@oracle01 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
7.3G 3.5G 3.5G 51% /
/dev/sda1 99M 9.0M 85M 10% /boot
none 126M 0 126M 0% /dev/shm
/dev/sdb1 9.9G 23M 9.4G 1% /mydata
[root@oracle01 ~]# dd if=/dev/sdc1 of=/mydata/data_lun1 bs=1M
3820+1 records in
3820+1 records out
[root@oracle01 ~]# dd if=/dev/sdc2 of=/mydata/data_lun2 bs=1M
3820+1 records in
3820+1 records out

3. Shutdown ASM

[root@oracle01 ~]# su - oracle
[oracle@oracle01.myserver.com /home/oracle]
$ export ORACLE_SID=+ASM
[oracle@oracle01.myserver.com /home/oracle]
$ sqlplus / as sysdba

SQL*Plus: Release 10.2.0.1.0 - Production on Wed Jan 9 22:20:36 2008

Copyright (c) 1982, 2005, Oracle. All rights reserved.


Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options

SQL> shutdown immediate
ASM diskgroups dismounted
ASM instance shutdown

4. Delete /dev/sdc1 and /dev/sdc2. Create a filesystem of 9.9G.

[root@oracle01 ~]# fdisk /dev/sdc

The number of cylinders for this disk is set to 1305.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4):
Value out of range.
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1305, default 1305): 1305

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@oracle01 ~]# partprobe
Warning: Unable to open /dev/hdc read-write (Read-only file system). /dev/hdc has been opened read-only.
[root@oracle01 ~]# mkfs /dev/sdc1
mke2fs 1.35 (28-Feb-2004)
max_blocks 2683489280, rsv_groups = 81894, rsv_gdb = 639
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
1310720 inodes, 2620595 blocks
131029 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2684354560
80 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Writing inode tables: done
inode.i_blocks = 46016, i_size = 4243456
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 24 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[root@oracle01 ~]# mkdir /mydata2
[root@oracle01 ~]# mount /dev/sdc1 /mydata2
[root@oracle01 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
7.3G 3.5G 3.5G 51% /
/dev/sda1 99M 9.0M 85M 10% /boot
none 126M 0 126M 0% /dev/shm
/dev/sdb1 9.9G 7.5G 1.9G 81% /mydata
/dev/sdc1 9.9G 23M 9.4G 1% /mydata2

5. Move data_lun[1-2] to this filesystem /mydata2

[root@oracle01 ~]# cp /mydata/* /mydata2/
cp: omitting directory `/mydata/lost+found'
[root@oracle01 ~]# cd /mydata2
[root@oracle01 mydata2]# ls -sh
total 7.5G
3.8G data_lun1 3.8G data_lun2 16K lost+found


6. Creating /dev/sdb[1-2]

[root@oracle01 mydata2]# fdisk /dev/sdb

The number of cylinders for this disk is set to 1305.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdb1 1 1305 10482381 83 Linux

Command (m for help): d
Selected partition 1

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1305, default 1305): 487

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (488-1305, default 488):
Using default value 488
Last cylinder or +size or +sizeM or +sizeK (488-1305, default 1305): 974

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

7. dd the data_lun[1-2] back to /dev/sdb[1-2]

[root@oracle01 mydata2]# fdisk -l /dev/sdb

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdb1 1 487 3911796 83 Linux
/dev/sdb2 488 974 3911827+ 83 Linux

[root@oracle01 mydata2]# dd if=/mydata2/data_lun1 of=/dev/sdb1 bs=1M
3820+1 records in
3820+1 records out
[root@oracle01 mydata2]# dd if=/mydata2/data_lun2 of=/dev/sdb2 bs=1M
3820+1 records in
3820+1 records out
[root@oracle01 asm]# ln -s /dev/sdb1 /u02/oradata/asm/data_lun1
[root@oracle01 asm]# ln -s /dev/sdb2 /u02/oradata/asm/data_lun2
[root@oracle01 asm]# chown -R oracle:oinstall /u02/oradata/asm/data_lun*
[root@oracle01 asm]# chmod -R 660 /u02/oradata/asm/data_lun*

8. Start and mount ASM on /dev/sdb[1-2]

[root@oracle01 asm]# su - oracle
[oracle@oracle01.myserver.com /home/oracle]
$ export ORACLE_SID=+ASM
[oracle@oracle01.myserver.com /home/oracle]
$ sqlplus / as sysdba

SQL*Plus: Release 10.2.0.1.0 - Production on Wed Jan 9 22:58:11 2008

Copyright (c) 1982, 2005, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup mount
ASM instance started

Total System Global Area 83886080 bytes
Fixed Size 1217836 bytes
Variable Size 57502420 bytes
ASM Cache 25165824 bytes
ASM diskgroups mounted
SQL> exit
Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
[oracle@oracle01.myserver.com /home/oracle]
$ asmcmd
ASMCMD> lsdg
State Type Rebal Unbal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Name
MOUNTED NORMAL N N 512 4096 1048576 7640 5284 0 2642 0 DATA/
ASMCMD>

9. Start Database instance

[oracle@oracle01.myserver.com /home/oracle]
$ export ORACLE_SID=testdb
[oracle@oracle01.myserver.com /home/oracle]
$ sqlplus / as sysdba

SQL*Plus: Release 10.2.0.1.0 - Production on Wed Jan 9 23:00:00 2008

Copyright (c) 1982, 2005, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup open
ORACLE instance started.

Total System Global Area 167772160 bytes
Fixed Size 1218316 bytes
Variable Size 71305460 bytes
Database Buffers 92274688 bytes
Redo Buffers 2973696 bytes
Database mounted.
Database opened.
SQL>