Sunday, May 12, 2013

Openldap bug on centos6,4

Openldap bug on centos6,4

When you are setting up the new openldap server, specifically with the following combination.

[root@centos6-01 openldap]# cat /etc/centos-release
CentOS release 6.4 (Final)

[root@centos6-01 openldap]# uname -r
2.6.32-358.6.1.el6.x86_64

[root@centos6-01 openldap]# rpm -qa | grep openldap-servers
openldap-servers-2.4.23-32.el6_4.1.x86_64

Most of the problem that you will hit when you are trying to add a first ldap entry from a ldif file was "ldap_bind invalid credentials (49)". Luckily, when you are doing a search from google, you should be able to find this link to help you to solve the problem.  http://www.ezylinux.com/en/red-hat-6-how-to-fixes-ldapadd-ldap_bind-invalid-credentials-49/. However, right after you have applied the commands, I just found "ANOTHER" bug on the daemon script as depicted as below. Even though the daemon script yielded an error message, but the slapd process will still spawned out, which I think it is a bit ugly.

[root@centos6-01 openldap]# /etc/init.d/slapd start
ln: accessing `/var/run/openldap/slapd.pid': No such file or directory

[root@centos6-01 openldap]# pidof slapd
1857
[root@centos6-01 openldap]# ps -eaf | grep slapd
ldap      1857     1  0 15:07 ?        00:00:00 /usr/sbin/slapd -h  ldap:/// ldapi:/// -u ldap

I keep on finding for bug list reported from the redhat bugzilla, but I have no luck. So, I decided to hack the daemon script, /etc/init.d/slapd. In this script, you will find some lines sound as below.

 # Start daemons.
        echo -n $"Starting $prog: "
        daemon --pidfile=$pidfile --check=$prog $ulimit ${slapd} -h "\"$harg\"" -u ${user} $OPTIONS $SLAPD_OPTIONS
        RETVAL=$?
        if [ $RETVAL -eq 0 ]; then
                touch $lockfile
                ln $slapd_pidfile $pidfile
        fi

So, this is my hacked version. Please pay more attention on the highlighted lines.

 # Start daemons.
        echo -n $"Starting $prog: "
        daemon --pidfile=$pidfile --check=$prog $ulimit ${slapd} -h "\"$harg\"" -u ${user} $OPTIONS $SLAPD_OPTIONS
        pidof ${slapd} > $pidfile
        RETVAL=$?
        if [ $RETVAL -eq 0 ]; then
                touch $lockfile
                #ln $slapd_pidfile $pidfile
                ln $pidfile $slapd_pidfile
        fi

With this, I can start/stop my daemon with no problem.

[root@centos6-01 cn=config]# /etc/init.d/slapd start
Starting slapd:                                            [  OK  ]
[root@centos6-01 cn=config]# /etc/init.d/slapd stop
Stopping slapd:                                            [  OK  ]


I am not sure if Centos community aware of this bug. But, I am just sharing it out, in hoping that the bug will be fixed in future. 

Thursday, May 9, 2013

Network devices comes with UUID? What is nmcli? PART 1

Network devices comes with UUID? What is nmcli? PART 1

The whole notion to explore on the network configuration was because of the curiosity when I bounced the network services and found out "something" new.

 [root@localhost network-scripts]# /etc/init.d/network restart
Shutting down interface eth0:  Device state: 3 (disconnected)
                                                           [  OK  ]
Shutting down loopback interface:                          [  OK  ]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface eth0:  Active connection state: activating
Active connection path: /org/freedesktop/NetworkManager/ActiveConnection/10
state: activated
Connection activated

                                                           [  OK  ]


So, what is the highlighted lines telling me? I have never seen this before in my past 10 years using Linux. So, when I dig more on it, I found out it was actually invoked by the /sbin/ifup command at the /etc/init.d/network script, something like ifup eth0 boot. If you drill down the script in the debug mode, then you will realize ifup is another wrapper script on top of nmcli command, where it called out something like this

nmcli con up UUID 2b58ae6b-8ad0-4fcc-ad99-582fa16d2fd9

Well, this is pretty new to me. Network configuration that ties to a UUID? That's awesome. Is nmcli (network manager cli) going to replace ifconfig command? Personally, I have a lot of doubts. Let's find out more..

This is how a ifcfg-eth0 looks like in Centos6.4

[root@localhost network-scripts]# cat ifcfg-eth0
DEVICE="eth0"
BOOTPROTO="dhcp"
HWADDR="00:0C:29:10:6A:D3"
IPV6INIT="yes"
NM_CONTROLLED="yes"
ONBOOT="yes"
TYPE="Ethernet"
UUID="2b58ae6b-8ad0-4fcc-ad99-582fa16d2fd9"



Now, nmcli tagging each of the NIC with a UUID. With that in place, nmcli make use of this and prints out something like this for you as a more user-friendly format. From the man file, it tells us that there are three main objects that governed by nmcli which is nm, con, dev.

nm: Network Manager. Use this object to inquire and change state of NetworkManager.

con: Connection. Get information about NetworkManager's connections.

dev: Device. Get information about devices.

[root@localhost network-scripts]# nmcli nm status
RUNNING         STATE           WIFI-HARDWARE   WIFI       WWAN-HARDWARE   WWAN
running         connected       enabled         enabled    enabled         enabled


[root@localhost network-scripts]# nmcli con status
NAME                      UUID                                   DEVICES    SCOPE    DEFAULT  VPN
System eth0               2b58ae6b-8ad0-4fcc-ad99-582fa16d2fd9   eth0       system   yes      no


[root@localhost network-scripts]# nmcli dev
DEVICE     TYPE              STATE
eth0       802-3-ethernet    connected


[root@localhost network-scripts]# nmcli dev list
GENERAL.DEVICE:                 eth0
GENERAL.TYPE:                   802-3-ethernet
GENERAL.DRIVER:                 e1000
GENERAL.HWADDR:                 00:0C:29:10:6A:D3
GENERAL.STATE:                  connected
CAPABILITIES.CARRIER-DETECT:    yes
CAPABILITIES.SPEED:             1000 Mb/s
WIRED-PROPERTIES.CARRIER:       on
IP4-SETTINGS.ADDRESS:           192.168.195.133
IP4-SETTINGS.PREFIX:            24 (255.255.255.0)
IP4-SETTINGS.GATEWAY:           192.168.195.2
IP4-DNS1.DNS:                   192.168.195.2


In sync with this command, there is another cool command, called, nm-tool.

[root@localhost network-scripts]# nm-tool

NetworkManager Tool

State: connected

- Device: eth0  [System eth0] --------------------------------------------------
  Type:              Wired
  Driver:            e1000
  State:             connected
  Default:           yes
  HW Address:        00:0C:29:10:6A:D3

  Capabilities:
    Carrier Detect:  yes
    Speed:           1000 Mb/s

  Wired Properties
    Carrier:         on

  IPv4 Settings:
    Address:         192.168.195.133
    Prefix:          24 (255.255.255.0)
    Gateway:         192.168.195.2

    DNS:             192.168.195.2



From here, we can see there are a lot of useful information, e.g  kernel module, e1000 is used, speed, carrier state,  HW address.

Conclusion: I don't really see the need of nmcli, why it is being used? Isn't that another command to the native the command like ip? But, it is worth to reread the ifup script, and do a comparison between the old ifup and the new ifup script.

Mount CDROM in CENTOS? Hassle?

Mount CDROM in CENTOS? Hassle?

What a hassle finding the CD ROM device in Centos. The new release of Centos putting CD ROM device at /dev/sr0. Let look at the below, then you will know what is going on.

[root@localhost proc]# find /dev -name cdrom
/dev/cdrom
/dev/.udev/links/cdrom

[root@localhost proc]# ls -al /dev/cdrom
lrwxrwxrwx. 1 root root 3 May  9 13:46 /dev/cdrom -> sr0

[root@localhost proc]# ls -al /dev/| grep sr0
lrwxrwxrwx.  1 root root           3 May  9 13:46 cdrom -> sr0
lrwxrwxrwx.  1 root root           3 May  9 13:46 dvd -> sr0
lrwxrwxrwx.  1 root root           3 May  9 13:46 scd0 -> sr0
brw-rw----.  1 root cdrom    11,   0 May  9 13:46 sr0

[root@localhost proc]# mount -r -t iso9660 /dev/sr0 /mnt

[root@localhost proc]# cd /mnt/

[root@localhost mnt]# ls -al
total 685
dr-xr-xr-x.  7 root root   4096 Mar  5 10:33 .
dr-xr-xr-x. 22 root root   4096 May  9 13:44 ..
-r--r--r--.  2 root root     14 Mar  5 10:28 CentOS_BuildTag
-r--r--r--.  1 root root     31 Mar  5 10:31 .discinfo
dr-xr-xr-x.  3 root root   2048 Mar  4 17:05 EFI
-r--r--r--.  2 root root    212 Mar  2 09:46 EULA
-r--r--r--.  2 root root  18009 Mar  2 09:46 GPL
dr-xr-xr-x.  3 root root   2048 Mar  4 17:05 images
dr-xr-xr-x.  2 root root   2048 Mar  4 17:05 isolinux
dr-xr-xr-x.  2 root root 649216 Mar  5 10:31 Packages
-r--r--r--.  2 root root   1354 Mar  2 09:45 RELEASE-NOTES-en-US.html
dr-xr-xr-x.  2 root root   4096 Mar  5 10:33 repodata
-r--r--r--.  2 root root   1706 Mar  2 09:46 RPM-GPG-KEY-CentOS-6
-r--r--r--.  2 root root   1730 Mar  2 09:46 RPM-GPG-KEY-CentOS-Debug-6
-r--r--r--.  2 root root   1730 Mar  2 09:46 RPM-GPG-KEY-CentOS-Security-6
-r--r--r--.  2 root root   1734 Mar  2 09:46 RPM-GPG-KEY-CentOS-Testing-6
-r--r--r--.  1 root root   3380 Mar  5 10:33 TRANS.TBL
-r--r--r--.  1 root root    338 Mar  5 10:33 .treeinfo

Voila, I am done. Next remember, you can source for /dev/sr0. For me, this is something new.

Monday, May 6, 2013

Linux dm-multipath on local disk?


Linux dm-multipath on local disk?

Compiled by : Hiu, Yen-Onn (yenonn@gmail.com), 7th May 2013

Problem: If you are running a Linux, RHEL5 or RHEL6, some machines have local scsi disks that been detected by dm-multipath. It is a known fault on the configuration and strongly not recommended for local disks. Please read this link from Redhat for future clarification. (https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/DM_Multipath/#ignore_localdisk_procedure)

This is the scenario when you first login to a login problematic Linux/dm-multipath server.  You should see the multipathing is acquiring the / and /boot mount points. As the result, we can’t stop the multipathd because some of the disks are being used by the system.

Even with some of the /etc/multipath.conf blacklisted local scsi disk, we still can see the local disks are not been ignored by the dm-multipath.

[root@s11t0008c ~]# uname -r
2.6.32-220.4.2.el6.x86_64

[hiuy@s11t0008c ~]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/mpathap3 1008M  459M  498M  48% /
tmpfs                  32G     0   32G   0% /dev/shm
/dev/mapper/mpathap1  186M   59M  118M  34% /boot
/dev/mapper/vg00-home
                      4.0G  158M  3.6G   5% /home
/dev/mapper/vg00-opt  4.0G  1.7G  2.1G  45% /opt
/dev/mapper/vg00-tmp  4.0G  138M  3.7G   4% /tmp
/dev/mapper/vg00-usr  4.0G  1.3G  2.6G  33% /usr
/dev/mapper/vg00-var  4.0G  634M  3.2G  17% /var
/dev/mapper/vg00-crash
                      7.7G  923M  6.4G  13% /var/crash
tmpfs                 4.0K     0  4.0K   0% /dev/vx


This is the snippet of the /etc/multipath.conf
blacklist {
#       devnode ".*"
        devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
        devnode "^hd[a-z]"
        devnode "^dcssblk[0-9]*"
        device {
                vendor "DGC"
                product "LUNZ"
        }
        device {
                vendor "IBM"
                product "S/390.*"
        }
        # don't count normal SATA devices as multipaths
        device {
                vendor  "ATA"
        }
        # don't count 3ware devices as multipaths
        device {
                vendor  "3ware"
        }
        device {
                vendor  "AMCC"
        }
        # nor highpoint devices
        device {
                vendor  "HP.*"
        }
        wwid 3600508b1001030363945374330300f00
}

[root@s11t0008c ~]# multipath -ll
mpatha (3600508b1001030363945374330300f00) dm-0 HP,LOGICAL VOLUME
size=279G features='1 queue_if_no_path' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  `- 0:0:0:1 sda 8:0 active ready running
[root@s11t0008c sysconfig]# cat /etc/fstab

#
# /etc/fstab
# Created by anaconda on Fri Sep 28 08:12:03 2012
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=e65ac106-53a5-4f0d-bd5e-ad563883c7d8 /                       ext4    defaults        1 1
UUID=4d5a5077-8582-4f3d-8708-2588644686d1 /boot                   ext3    defaults        1 2
/dev/mapper/vg00-home   /home                   ext4    defaults        1 2
/dev/mapper/vg00-opt    /opt                    ext4    defaults        1 2
/dev/mapper/vg00-tmp    /tmp                    ext4    defaults        1 2
/dev/mapper/vg00-usr    /usr                    ext4    defaults        1 2
/dev/mapper/vg00-var    /var                    ext4    defaults        1 2
UUID=5caa587b-f2d4-4063-9db3-f9b2901e816d swap                    swap    defaults        0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
/dev/vg00/crash /var/crash      ext3    defaults        2       2

In order to trace the UUID on the / mount point, we can query the blkid table on the devices that had been registered. The

[root@s11t0008c ~]# blkid -U e65ac106-53a5-4f0d-bd5e-ad563883c7d8
/dev/mapper/mpathap3

Solution to the problem


You have to make sure that you have local disks that had been blacklisted. In this case, you can specify individual devices by their WWID (world-wide Identification) with the wwid entry in the blacklist section of the configuration file.

For example:
blacklist {
        wwid 3600508b1001030363945374330300f00
}

To verify the devices had been blacklisted you can use the command “multipath –v4” and you should see the output as below.

===== paths list =====
uuid                              hcil    dev  dev_t  pri dm_st chk_st vend/pr
3600508b1001030363945374330300f00 0:0:0:1 sda  8:0    1   undef ready  HP,LOGI
3600508b4000756cf0000a000029d0000 1:0:0:1 sdb  8:16   10  undef ready  HP,HSV2
.
.
May 06 23:01:06 | sda: (HP:LOGICAL VOLUME) vendor/product blacklisted
May 06 23:01:06 | sdb: (HP:HSV210) vendor/product blacklisted
.
.

Then, you have to recompile the initramdisk, to make sure that during the booting (init), the HP logical disk (local disk) is blacklisted.

1.       Backup your initial initramdisk

[root@s11t0008c boot]# cp -p /boot/initramfs-`uname -r`.img /boot/initramfs-`uname -r`.img.bak

2.       Generate the new initramdisk and reboot the server.

[root@s11t0008c boot]# mkinitrd –force /boot/initramfs-`uname -r`.img `uname -r`
[root@s11t0008c boot]# init 6

Post configuration/verification


Once you are done with the reboot, you should see this.
[root@s11t0008c ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3            1008M  459M  499M  48% /
tmpfs                  32G     0   32G   0% /dev/shm
/dev/sda1             186M  168M  9.2M  95% /boot
/dev/mapper/vg00-home
                      4.0G  158M  3.6G   5% /home
/dev/mapper/vg00-opt  4.0G  1.7G  2.1G  45% /opt
/dev/mapper/vg00-tmp  4.0G  138M  3.7G   4% /tmp
/dev/mapper/vg00-usr  4.0G  1.3G  2.6G  33% /usr
/dev/mapper/vg00-var  4.0G  637M  3.2G  17% /var
/dev/mapper/vg00-crash
                      7.7G  923M  6.4G  13% /var/crash
tmpfs                 4.0K     0  4.0K   0% /dev/vx

[root@s11t0008c ~]# blkid -U e65ac106-53a5-4f0d-bd5e-ad563883c7d8
/dev/sda3

[root@s11t0008c ~]# mpathconf
multipath is enabled
find_multipaths is disabled
user_friendly_names is enabled
dm_multipath module is loaded
multipathd is chkconfiged on

[root@s11t0008c ~]# multipath -ll
[root@s11t0008c ~]#