Apache on SLES10

To setup a basic apache webserver, you need to install

• apache2
• apache2-prefork
• apache2-example-pages (optional)

Commands to administer apache2

• To start apache2: rcapache2 start
• To stop: rcapache2 stop
• To let apache starts at boot time: insserv apache2
• To reload the config file: rcapache2 reload
• To restart daemon: rcapache2 restart
• To examine the syntax of the config files: apache2ctl configtest
• The default directory: /srv/www/htdocs. It is called DocumentRoot of the web server.
• wwwrun is the user that executing apache2

Apache2 configuration files

• /etc/apache2/httpd.conf - The main apache configuration file.
• /etc/apache2/default-server.conf - Contains the basic web server setup. All options set in this file can be overwritten by other configuration files.
• /etc/apache2/vhost.d - The directory for virtual host setups.
• /etc/apache2/uuid - Sets the user and group id for apache. By default, apache uses wwwrun(uid) and www(gid)
• /etc/apache2/server-tuning.conf - tune the performance of apache2.
• /etc/apache2/errors.conf - configure the behavior of apache when a request cannot be performed.
• /etc/apache2/ssl-global.conf - configure the connection encryption with SSL.

Create RSA Key Pair

• cat /dev/random > /tmp/random Stop this command after a few seconds by pressing Ctrl-C
• To create the server key: openssl genrsa -des3 -rand /tmp/random 1024 -out server.key
• Sign the public key to create a certification: openssl req -new -x509 -key server.key -out server.crt
• Copy the server.key to /etc/apache2/ssl.key/* and copy the server.crt to /etc/apache2/ssl.crt/*
• To configure the main server, you need to add the following lines to /etc/apache2/default-server.conf
• SSLEngine on
• SSLCertificateFile /etc/apache2/ssl.crt/server.crt
• SSLCertificateKeyFile /etc/apache2/ssl.key/server.key
• APACHE_START_TIMEOUT="10" - This setting extends the start timeout of apache so that have more time to enter the passphrase of the private RSA key.
• APACHE_SERVER_FLAGS="SSL" - This enables some directives that are necessary for SSL encrytion. It lets apache listen on port 443 instead of only to port 80.

Xen on SLES10

Xen consists of two major components.

• Virtual Machine Monitor (VMM) - A layer between physical hardware and VM. In general term, this is called hypervisor.
• Xen Tools - a set of command line application used to administer VM.

VMM must be loaded before any of VMs are started. VM in xen are called as domains. Privileged domain, called domain0 is an interface to communicate directly with an adminstrator to access the physical hardware of the host machine.

An unprivileged domain is called domainU. To install Xen, the following package is needed.

• xen
• xen-tools
• kernel-xen
• xen-doc* (optional)

After the installation, the reboot is required to boot into Xen bootloader. In case the system is not booting properly, you can switch back to the non-virtualized system and disable the firewall.

• rcSuSEfirewall2 stop
• insserv -r SuSEfirewall2_setup
• insserv -r SuSEfirewall2_init
• insserv -r SuSEfirewall_final (conditional)

To manage Xen domains at the commands line.

• Xen configuration file is located at: /etc/xen/vm
• There are examples of template configuration file for a single domain: /etc/xen/examples
• xm is the administration tools, it communicates with the xend management processes running on domain0 linux installation.
• To start a vm, the create command is used: xm create -c -f /data/xen/SLES10-WebServer.conf
• To list the currently running Xen domains: xm list
• The list command contains the following fields: name, domid, memory, vcpus, state (r - running, b - blocked, domain has been created, but it is blocked when a domain is waiting for IO or nothing to do, p - pause, s - shutdown, c - crashed)
• To connects you with the terminal of running domain: xm console domain_id
• To disconnect from a terminal: use the keystrokes: Ctrl-]
• To interrupt the execution a domain temporarily: xm pause domain_id
• To unpause: xm unpause domain_id
• To shutdown: xm shutdown domain_id
• To force shutdown a not responding domain: xm destroy domain_id
• To save the state of a domain for a longer time: xm save domain_id
• To restore a domain from a resulting file: xm restore filename
• To change the memory allocation for a domain: xm mem-set domain_id amount_of_memory_in_megabytes

Automate Domain Startup and shutdown

• SLES10 comes with a start script called, xendomains, which included in the xen-tools package.
• If domain0 is booted, all the domains with the configuration files located under /etc/xen/auto will be started.
• If domain0 is shutdown, or rebooted, all the running Xen domains will be shutdown.
• The script can be adjusted /etc/sysconfig/xendomains for some configuration options like: XENDOMAINS_MIGRATE. It will migrate domain automatically to a different host when a domain0 is shutdown. The flags has to be set to the IP address of the target machine.

Understand Xen Networking

• domain0 is controlling the physical network interfaces of the host system. Unprivileged domains are connected to domain0 through Virtual Ethernet adapters.
• domainU (VE) <---->(VE)domain0(Physical Ethernet)<------>Network(LAN)
• The default mechanism to connect VE and PE in domain0 is bridging.
• When a new domain is created, the following steps happened.
• 1. Xen provides a VE to the new domain
• 2. xend creates a new VE in domain0
• 3. both VE are connected through a virtual point to point connection.
• 4. The VE in domain0 is added to the bridge with the physical interface.
• xend performs the networking changes with the help of scripts: /etc/xen/scripts.
• The script to initiate the bridge xenbr0: /etc/xen/scripts/nework-bridge
• The script to start and add the VE: /etc/xen/scripts/vif-bridge
• To configure the network scripts are used by xend: /etc/xen/xend-config.sxp
• Command: ip a is prompted on domain0 will list down all the different interface naming schema.
• 1. peth - physical interfaces in domain0. peth devices are connected to the network bridge
• 2. vif - virtual interfaces which are part of the bridge. E.g: vif6.0 is connected to the first vif in domain 6.
• 3. veth - virtual interfaces are connected to the vif interfaces of domain0 (vif0.x)
• 4. eth0 - the default veth interface is named eth0 and connected with vif0.0. This is the default network interface of domain0.
• 5. xenbr0 - the default bridge that connects virtual and physical interfaces.

Migrate a Guest Domain

• 2 methods: Use domain save and restore or Use migration and live migration.
• Domain Save and Restore
• 1. suspend the domain: xm save domain_id filename
• 2. copy the file to the new host system, then restore the domain: xm restore filename
• Migration and Live Migration
• 1. xm migrate domain_id target_host
• 2. By adding --live migration flag, the downtime during the migration can be reduced.
• 3. /etc/xen/xend-config.sxp with 2 importants flags: xend-relocation-server yes, xend-relocation-hosts-allow '^localhost$'

VI Cursor Movement Commands

xargs and find

xargs and find

find and xargs do go very well together: find to locate what you're looking for, and xargs to run the same command on each of the things found.

Traditionally, an advantage to xargs was its ability to handle long command lines before failing, unlike some other commands. This command:

rm `find tmp -maxdepth 1 -name '*.mp3'`

is intended to remove all tmp/*.mp3 files (and ignore any subdirectories), but can fail with an "Argument list too long" message. This exact equivalent:

find tmp -maxdepth 1 -name '*.mp3' -maxdepth 1 | xargs rm

does exactly the same thing but will avoid the problem by batching arguments up. More modern kernels (since 2.6.23) shouldn't have this issue, but it's wise to make your scripts as portable as possible; and the xargs version is also easier on the eye.

You can also manually batch arguments if needed, using the -n option.

find tmp -maxdepth 1 -name '*.mp3' -maxdepth 1 | xargs -n1 rm

will pass one argument at a time to rm. This is also useful if you're using the -p option as you can confirm one file at a time rather than all at once.

Filenames containing whitespace can also cause problems; xargs and find can deal with this, using GNU extensions to both to break on the null character rather than on whitespace:

find tmp -maxdepth 1 -name *.mp3 -print0 | xargs -0 rm

You must use these options either on both find and xargs or on neither, or you'll get odd results.

Another common use of xargs with find is to combine it with grep. For example,

find . -name '*.pl' | xargs grep -L '^use strict'

will search all the *.pl files in the current directory and subdirectories, and print the names of any that don't have a line starting with 'use strict'. Enforce good practice in your scripting!

Configure Special File Permissions

Name: sticky bit, 1, t
Users can only delete files when they are owner, or when they are root or owner of the directory. This is usually applied to be /tmp directory.

Name: SGID (Set GroupID), 2, s
When a program is run, this sets the group ID of the process to that of the group of the file. Files created in this directory belong to the group to which the directory belongs and not the primary group of the users. New directories created in this directories inherit the SGID bit.

Name: SUID (Set UserID), 4, s
Sets the user ID of the process to that of the owner of the file when the program is run.

UUID

1. If you don't know the UUID of your disk, you can find it by using one of the several commands below:

host # vol_id /dev/sda3
...
ID_FS_UUID=a1331d73-d640-4bac-97b4-cf33a375ae5b
...

or:

host # blkid /dev/sda3 <-- Leave blank to show all disks
/dev/sda3: LABEL="/" UUID="a1331d73-d640-4bac-97b4-cf33a375ae5b" SEC_TYPE="ext3" TYPE="ext2"

also:

host # ls -l /dev/disk/by-uuid|grep sda3
lrwxrwxrwx 1 root root 10 11. Okt 18:02 a1331d73-d640-4bac-97b4-cf33a375ae5b-> ../../sda3

2. If you prefer to generate your own UUID's (see above), you can use the uuidgen command and couple it with tune2fs to change the default UUID assigned to your disk by the system, like this:

host # uuidgen
1d721189-7b71-4315-95a7-1c3abc90d379
host # tune2fs -U 1d721189-7b71-4315-95a7-1c3abc90d379 /dev/sda3

3. Then again, if you already know the UUID, you might want to find out what disk it's associated with. You can generally get this information with the "findfs" command, like so:

host # findfs UUID=a1331d73-d640-4bac-97b4-cf33a375ae5b
/dev/sda3

Of course, using some of the commands above and grepping out part of the UUID will also get you your answer, like:

host # ls -l /dev/disk/by-uuid|grep a1331d73-d640-4bac-97b4-cf33a375ae5b
lrwxrwxrwx 1 root root 10 11. Okt 18:02 a1331d73-d640-4bac-97b4-cf33a375ae5b-> ../../sda3

or

host # blkid|grep a1331d73-d640-4bac-97b4-cf33a375ae5b <-- remember that blkid with no arguments returns all of the system disk
/dev/sda3: LABEL="/" UUID="a1331d73-d640-4bac-97b4-cf33a375ae5b" SEC_TYPE="ext3" TYPE="ext2"

4. And, lastly (for this post, at least ;), you can mount your disks using the UUID, and even incorporate that automated UUID mounting into your /etc/fstab. To mount directly from the command line, you can do something like this:

host # mount -U a1331d73-d640-4bac-97b4-cf33a375ae5b /directory/you/mount/this/disk/on

and you could instruct your system to mount this partition by UUID from within the fstab, as well. It works basically the same way that the LABEL keyword does:

host # cat /etc/fstab
...
UUID=a1331d73-d640-4bac-97b4-cf33a375ae5b /directory/you/mount/this/disk/on ext3fs defaults 1 1


And, at this point, you should be able to figure your way around using UUID's to manipulate your disk on Linux with no problem. Enjoy, and please "be careful" :)

Expect and SSH

#!/usr/bin/expect -f

set username [lrange $argv 0 0]
set password [lrange $argv 1 1]
set ipaddr [lrange $argv 2 2]
set timeout -1

spawn ssh $username@$ipaddr
match_max 100000

expect {
"*?(yes/no)*" { send -- "yes\r" }
"*?assword:*" { send -- "$password\r" }
}

set timeout 10
expect {
"*?assword:*" { send -- "$password\r" }
}

expect "$username*" { send -- "sudo su - \r" }
interact

System Global Area

The SGA consists of several memory structures:
1. Shared Pool
2. Database Buffer Cache
3. Redo Log Buffer
4. Other structures (for example, lock and latch management, statistical data)

Shared Pool
It consists of two key performance-related memory structures, Library Cache, Data Dictionary Cache. Because the Shared Pool can be shared globally, such as reusable SQL execution plans, PL/SQL packages, procedures, functions, and cursor information, it must be sized to accommodate the needs of both the fixed and variables areas. Memory allocation for the Shared Pool is determined by the SHARED_POOL_SIZE initialization parameter. It can be dynamically resized using ALTER SYSTEM SET. After performance analysis, this can be adjusted but the total SGA size cannot exceed SGA_MAX_SIZE.

Libray Cache
It stores information about eh most recently used SQL and PL/SQL statements. It enables the sharing of commonly used statements which is managed by the least recently used (LRU) algorithm.

Data Dictionary Cache
This is a collection of the most recently used definitions in the database, which includes information about database files, tables, indexes, columns, users, privileges, and other database objects. During the parse phase, the server process looks at the data dictionary for information to resolve object names and validate access. By caching data dctionary information into memory improves response time on queries and DML.

There are two additional memory structures that can be configured within the SGA:
1. Large Pool
2. Java Pool

Oracle Architecture Diagram

 

Oracle Primary Components

Oracle server:
There are several files, processes and memory structures in an Oracle server, however, not all of them are used when processing a SQL statement. Some are used to improve the performance of the databse, to ensure that the database can be recovered in the event of a software of hardware error, ot to perform other tasks necessary to maintain the database. The Oracle server consists of an Oracle instance and an Oracle database

Oracle Instance:
An Oracle instance is the combination of the background processes and memory structures. The instance must be started to access the data in the database. Every time an instance is started, a System Global Area (SGA) is allocated and Oracle background processes are started. Background processes perform functions on behalf of the invoking procceses. They consolidate functions that would otherwise be handled my multiple Oracle programs running for each user. The background processes perform input/output (I/O) and monitor other ORacle processes to provide increased parallelism for better performance and reliability. An Oracle instance consists of the System Global Area (SGA) memory structure and the background processes that are used to manage a database. An instance is identified by using methods specific to each operating system. The instance can open and use only one database at a time.

Oracle Database:
The general purpose of a database is to store and retrieve related information. an Oracle database has a logical and a physical structure. The physical structure of the database is a set of operating system files in the database. An Oracle database consists of three file types.
1. Data files containing the actual data in the database.
2. Online redo log files containing a record of changes made to the database to enable recovery of the data in case of failures.
3. Control files containing information necessary to maintain and verify database integrity.

Memory Structure
Oracle's memory structure consists of two memory areas known as
1. System Global Area (SGA): Allocated at instance startup, and is a fundamental component of an Oracle instance.
2. Program Global Area (PGA): Allocated when the server process is started.