hi all,
I have some of my linux servers having network packets dropped especially when it is on the receiving parts. It can be noticed at the ifconfig command. With this dragging on the OS, it will degrade the performance. I have tried many way to solve the problem. Some of action that I did was upgrade the firmware version, install of driver, and even bugging the network engineer, claiming the problem was originate from network piece. Basically, I am desperated but, not able to find the answer.
[hiuy@nasilemak~]$ ifconfig
bond0 Link encap:Ethernet HWaddr 8C:DC:D4:0D:22:50
inet addr:10.104.192.1 Bcast:10.104.192.255 Mask:255.255.255.0
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:1798666372532 errors:31860797 dropped:806 overruns:31859889 frame:908
TX packets:1350275497458 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:366956654314385 (333.7 TiB) TX bytes:348508200310892 (316.9 TiB)
At last, I have seen some internet posting that Centos 6, couldn't handling the CPU interrupts well. if you can watch "cat the /proc/interrupts". Many of the kernel requests are performed by a single CPU, causing it too overwhelming. The result of this, you have to spread the load evenly to multiple CPUs. Also, it has been a waste if you are not using it.
The answer of this problem will be installing a package called, irqbalance, then start it right away. The result is heavenly revealed. My network problem is gone. Also, when interrupts are spread across all of the CPUs. Magically resolved my problem.
Wednesday, March 23, 2016
Working with Docker
hi all,
Horay!! I have dockerized my graphite engine. Basically, I crafted a docker image, called centos6.6-graphite-web-base. Then, I extended a couple of paths such that a couple of containers can write onto the same share paths and files. Now, I can spawn out a couple of carbon instances, listening on a specific port numbers with no mercy at all. Its just work! Docker rock!!!!!
I used the supervisor http://supervisord.org/, to launch the processes. It is neat.
CONTAINER_NAME="GRAPHITE1"
CONTAINER_PORT="80"
if docker ps | grep -q $CONTAINER_NAME
then
echo "Container is created: $CONTAINER_NAME"
else
echo -n "Creating container: $CONTAINER_NAME - "
docker run -d --restart=on-failure:3 \
-it \
--memory-swap=-1 \
--name $CONTAINER_NAME \
--hostname $CONTAINER_NAME \
-p $CONTAINER_PORT:$CONTAINER_PORT \
-v $WHISPER_STORAGE_PATH:/opt/graphite/storage/whisper \
-v $HTTPD_CONF_PATH:/etc/httpd/conf/httpd.conf:rw \
hiuy/centos6.6-graphite-web-base:0.0 \
/usr/local/bin/svscan /etc/my_services
echo ""
fi
CONTAINER_NAME="CARBON1"
CONTAINER_PORT="56260"
if docker ps | grep -q $CONTAINER_NAME
then
echo "Container is created: $CONTAINER_NAME"
else
echo -n "Creating container: $CONTAINER_NAME - "
docker run -d --restart=on-failure:3 \
-it \
--memory-swap=-1 \
--name $CONTAINER_NAME \
--hostname $CONTAINER_NAME \
--env port=$CONTAINER_PORT \
-p $CONTAINER_PORT:$CONTAINER_PORT \
-v $WHISPER_CONF_PATH:/opt/graphite/conf/storage-schemas.conf:ro \
-v $WHISPER_STORAGE_PATH:/opt/graphite/storage/whisper \
hiuy/centos6.6-carbon-base:0.0 \
/usr/local/bin/svscan /etc/my_services
fi
Horay!! I have dockerized my graphite engine. Basically, I crafted a docker image, called centos6.6-graphite-web-base. Then, I extended a couple of paths such that a couple of containers can write onto the same share paths and files. Now, I can spawn out a couple of carbon instances, listening on a specific port numbers with no mercy at all. Its just work! Docker rock!!!!!
I used the supervisor http://supervisord.org/
CONTAINER_NAME="GRAPHITE1"
CONTAINER_PORT="80"
if docker ps | grep -q $CONTAINER_NAME
then
echo "Container is created: $CONTAINER_NAME"
else
echo -n "Creating container: $CONTAINER_NAME - "
docker run -d --restart=on-failure:3 \
-it \
--memory-swap=-1 \
--name $CONTAINER_NAME \
--hostname $CONTAINER_NAME \
-p $CONTAINER_PORT:$CONTAINER_PORT \
-v $WHISPER_STORAGE_PATH:/opt/graphite/storage/whisper \
-v $HTTPD_CONF_PATH:/etc/httpd/conf/httpd.conf:rw \
hiuy/centos6.6-graphite-web-base:0.0 \
/usr/local/bin/svscan /etc/my_services
echo ""
fi
CONTAINER_NAME="CARBON1"
CONTAINER_PORT="56260"
if docker ps | grep -q $CONTAINER_NAME
then
echo "Container is created: $CONTAINER_NAME"
else
echo -n "Creating container: $CONTAINER_NAME - "
docker run -d --restart=on-failure:3 \
-it \
--memory-swap=-1 \
--name $CONTAINER_NAME \
--hostname $CONTAINER_NAME \
--env port=$CONTAINER_PORT \
-p $CONTAINER_PORT:$CONTAINER_PORT \
-v $WHISPER_CONF_PATH:/opt/graphite/conf/storage-schemas.conf:ro \
-v $WHISPER_STORAGE_PATH:/opt/graphite/storage/whisper \
hiuy/centos6.6-carbon-base:0.0 \
/usr/local/bin/svscan /etc/my_services
fi
Cassandra: fighting with hints, not fun!!
hi all,
I am not sure if you have experiences when supporting a multi-DC cassandra cluster. The cluster will be running well when network is not a problem. Disaster happens whenever there is a network hiccup. One of the reason behind this will be cassandra hints accumulation. Some or more nodes in the cluster are accumulating the hints for the remote DC, causing the performance degradation. Basically, you will notice your java GC is getting crazy, and having a long pauses. Ouch!!! Some times, you need to disable the hints in order to keep the cluster running. This is a pain. Hints are not visible from any of the nodetool command. So, I coded a simple script to count the accumulated system hints. With this, you can measure the level of hints that been accumulated. If possible, you can disable the hints via the nodetool command, then resume the services when the network is back to normal. In any occasion, you don't wish your cluster node to keep any hints.
#!/bin/bash
HINT_TEMP_FILE="/tmp/hints.txt"
STATUS_TEMP_FILE="/tmp/status"
echo " * Collecting system hints ..."
echo -e "select target_id from system.hints limit 1000000;\n" | /opt/cassandra/bin/cqlsh `hostname -f` > $HINT_TEMP_FILE
sleep 2
echo " * Collecting nodes status ..."
/opt/cassandra/bin/nodetool status > $STATUS_TEMP_FILE
echo " * Calculating system hints ..."
if wc -l $HINT_TEMP_FILE | grep -q "^0"
then
echo " Good news!!! No hints are pending!!!"
exit 0
fi
echo ""
echo " * Total hints: `grep -E "\w{8}-(\w{4}-){3}\w{12}" $HINT_TEMP_FILE | wc -l`"
for i in `grep -E "\w{8}-(\w{4}-){3}\w{12}" $HINT_TEMP_FILE | sort -u`
do
ip=`grep $i $STATUS_TEMP_FILE| awk '{print $2}'`
myhost=`host $ip | awk '{print $NF}'`
echo -e "Host id: $i \
IP: $ip \
Hostname: $myhost \
Count: `grep $i $HINT_TEMP_FILE | wc -l`"
done
I am not sure if you have experiences when supporting a multi-DC cassandra cluster. The cluster will be running well when network is not a problem. Disaster happens whenever there is a network hiccup. One of the reason behind this will be cassandra hints accumulation. Some or more nodes in the cluster are accumulating the hints for the remote DC, causing the performance degradation. Basically, you will notice your java GC is getting crazy, and having a long pauses. Ouch!!! Some times, you need to disable the hints in order to keep the cluster running. This is a pain. Hints are not visible from any of the nodetool command. So, I coded a simple script to count the accumulated system hints. With this, you can measure the level of hints that been accumulated. If possible, you can disable the hints via the nodetool command, then resume the services when the network is back to normal. In any occasion, you don't wish your cluster node to keep any hints.
#!/bin/bash
HINT_TEMP_FILE="/tmp/hints.txt"
STATUS_TEMP_FILE="/tmp/status"
echo " * Collecting system hints ..."
echo -e "select target_id from system.hints limit 1000000;\n" | /opt/cassandra/bin/cqlsh `hostname -f` > $HINT_TEMP_FILE
sleep 2
echo " * Collecting nodes status ..."
/opt/cassandra/bin/nodetool status > $STATUS_TEMP_FILE
echo " * Calculating system hints ..."
if wc -l $HINT_TEMP_FILE | grep -q "^0"
then
echo " Good news!!! No hints are pending!!!"
exit 0
fi
echo ""
echo " * Total hints: `grep -E "\w{8}-(\w{4}-){3}\w{12}" $HINT_TEMP_FILE | wc -l`"
for i in `grep -E "\w{8}-(\w{4}-){3}\w{12}" $HINT_TEMP_FILE | sort -u`
do
ip=`grep $i $STATUS_TEMP_FILE| awk '{print $2}'`
myhost=`host $ip | awk '{print $NF}'`
echo -e "Host id: $i \
IP: $ip \
Hostname: $myhost \
Count: `grep $i $HINT_TEMP_FILE | wc -l`"
done
Bash in action: multithreading
hi all,
I hit a bottleneck on a for loop statement, when I have a function that can be carried out concurrently for some numbers of thread. But, I don't know how to code it in bash. I bumped into this site: http://stackoverflow.com/questions/1455695/forking-multi-threaded-processes-bash, which inspired me to code it. So, at the last I manage to code a multithreading function using bash. Here is how it looks like. I hope it helps some of the devops engineers to solve their problems.
[hiuy@nasilemak ~]$ ./test.sh
Waiting for jobs: 1199 1200 1201
Waiting for jobs: 1199 1200 1201
Waiting for jobs: 1199 1200 1201
Waiting for jobs: 1199 1200 1201
Waiting for jobs: 1199 1200 1201
Waiting for jobs: 1199 1200 1201
Job is Done -- 1199
Waiting for jobs: 1200 1201 1445
Waiting for jobs: 1200 1201 1445
Waiting for jobs: 1200 1201 1445
Waiting for jobs: 1200 1201 1445
Waiting for jobs: 1200 1201 1445
Job is Done -- 1200
Waiting for jobs: 1201 1445 2099
Waiting for jobs: 1201 1445 2099
Waiting for jobs: 1201 1445 2099
Waiting for jobs: 1201 1445 2099
Waiting for jobs: 1201 1445 2099
Job is Done -- 1201
Waiting for jobs: 1445 2099 2118
Job is Done -- 2099
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Job is Done -- 1445
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Job is Done -- 2118
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Job is Done -- 2121
Here is the code
#!/bin/bash
TOTAL_THREAD=3
declare -a pids
function wait_pid()
{
pid=$1
if [ ! -z $pid ] && kill -0 $pid &>/dev/null
then
#this is a valid pid, then add it
pids=(${pids[@]} $pid)
if [ "${#pids[@]}" -lt "$TOTAL_THREAD" ]
then
return 0
elif [ "${#pids[@]}" -eq "$TOTAL_THREAD" ]
then
while [ "${#pids[@]}" -eq "$TOTAL_THREAD" ]
do
if [ "${#pids[@]}" -lt $TOTAL_THREAD ]
then
break
fi
echo "Waiting for jobs: ${pids[@]}"
local range=$(eval echo {0..$((${#pids[@]}-1))})
local i
for i in $range; do
if ! kill -0 ${pids[$i]} 2> /dev/null; then
echo "Job is Done -- ${pids[$i]}"
unset pids[$i]
fi
done
pids=("${pids[@]}") # Expunge nulls created by unset.
sleep 1
done
return 0
fi
fi
}
function wait_complete()
{
while [ "${#pids[@]}" -gt "0" ]
do
echo "Waiting for jobs: ${pids[@]}"
local range=$(eval echo {0..$((${#pids[@]}-1))})
local i
for i in $range; do
if ! kill -0 ${pids[$i]} 2> /dev/null; then
echo "Job is Done -- ${pids[$i]}"
unset pids[$i]
fi
done
pids=("${pids[@]}") # Expunge nulls created by unset.
sleep 1
done
}
for s in `echo "5 10 15 25 5 25 60"`
do
sleep $s &
wait_pid $!
done
wait_complete
I hit a bottleneck on a for loop statement, when I have a function that can be carried out concurrently for some numbers of thread. But, I don't know how to code it in bash. I bumped into this site: http://stackoverflow.com/questions/1455695/forking-multi-threaded-processes-bash, which inspired me to code it. So, at the last I manage to code a multithreading function using bash. Here is how it looks like. I hope it helps some of the devops engineers to solve their problems.
[hiuy@nasilemak ~]$ ./test.sh
Waiting for jobs: 1199 1200 1201
Waiting for jobs: 1199 1200 1201
Waiting for jobs: 1199 1200 1201
Waiting for jobs: 1199 1200 1201
Waiting for jobs: 1199 1200 1201
Waiting for jobs: 1199 1200 1201
Job is Done -- 1199
Waiting for jobs: 1200 1201 1445
Waiting for jobs: 1200 1201 1445
Waiting for jobs: 1200 1201 1445
Waiting for jobs: 1200 1201 1445
Waiting for jobs: 1200 1201 1445
Job is Done -- 1200
Waiting for jobs: 1201 1445 2099
Waiting for jobs: 1201 1445 2099
Waiting for jobs: 1201 1445 2099
Waiting for jobs: 1201 1445 2099
Waiting for jobs: 1201 1445 2099
Job is Done -- 1201
Waiting for jobs: 1445 2099 2118
Job is Done -- 2099
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Waiting for jobs: 1445 2118 2121
Job is Done -- 1445
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Waiting for jobs: 2118 2121
Job is Done -- 2118
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Waiting for jobs: 2121
Job is Done -- 2121
Here is the code
#!/bin/bash
TOTAL_THREAD=3
declare -a pids
function wait_pid()
{
pid=$1
if [ ! -z $pid ] && kill -0 $pid &>/dev/null
then
#this is a valid pid, then add it
pids=(${pids[@]} $pid)
if [ "${#pids[@]}" -lt "$TOTAL_THREAD" ]
then
return 0
elif [ "${#pids[@]}" -eq "$TOTAL_THREAD" ]
then
while [ "${#pids[@]}" -eq "$TOTAL_THREAD" ]
do
if [ "${#pids[@]}" -lt $TOTAL_THREAD ]
then
break
fi
echo "Waiting for jobs: ${pids[@]}"
local range=$(eval echo {0..$((${#pids[@]}-1))})
local i
for i in $range; do
if ! kill -0 ${pids[$i]} 2> /dev/null; then
echo "Job is Done -- ${pids[$i]}"
unset pids[$i]
fi
done
pids=("${pids[@]}") # Expunge nulls created by unset.
sleep 1
done
return 0
fi
fi
}
function wait_complete()
{
while [ "${#pids[@]}" -gt "0" ]
do
echo "Waiting for jobs: ${pids[@]}"
local range=$(eval echo {0..$((${#pids[@]}-1))})
local i
for i in $range; do
if ! kill -0 ${pids[$i]} 2> /dev/null; then
echo "Job is Done -- ${pids[$i]}"
unset pids[$i]
fi
done
pids=("${pids[@]}") # Expunge nulls created by unset.
sleep 1
done
}
for s in `echo "5 10 15 25 5 25 60"`
do
sleep $s &
wait_pid $!
done
wait_complete
Cassandra sstableupgrade function
hi all,
I guess this is a good common function that will help a lot of engineers who will work on cassandra upgrade. Personally, I coded a sstableupgrade function which I think it is neat and stable. So, I would like to share it to all. I have used it for my cassandra upgrade from version 1.2.8 to 2.0.14.
function sstable_upgrade()
{
echo_header "SSTABLE UPGRADE"
MYSNAPSHOT_NAME=""
if [ ! -z "$1" ]
then
MYSNAPSHOT_NAME="$1"
fi
SSTABLEUPGRADE_BIN="/opt/cassandra-2.0.14/bin/sstableupgrade"
sudo sed -i 's|256M|2G|g' $SSTABLEUPGRADE_BIN
echo_info "Harvesting keyspaces and column family to $CASSANDRA_BACKUP/backup_keyspace.out"
if sudo find /data/ -name "*-ic*Data.db" | grep -v snapshot* | awk -F"/" '{print $4" "$5}' | sort -u > $CASSANDRA_BACKUP/backup_keyspace.out
then
while read line
do
echo_info "SSTABLE upgrade: $SSTABLEUPGRADE_BIN $line $SNAPSHOT_NAME"
sudo -tt -u $CASS_USER /bin/bash -c "$SSTABLEUPGRADE_BIN $line $MYSNAPSHOT_NAME"
sleep 5
done < $CASSANDRA_BACKUP/backup_keyspace.out
else
echo_warn "Fail harvesting keyspaces and column families"
exit 1
fi
}
Here will be output
> INFO: SSTABLE upgrade: /opt/cassandra-2.0.14/bin/sstableupgrade system schema_keyspaces
Found 2 sstables that need upgrading.
Upgrading SSTableReader(path='/data/data02/system/schema_keyspaces/system-schema_keyspaces-ic-214-Data.db')
Upgrade of SSTableReader(path='/data/data02/system/schema_keyspaces/system-schema_keyspaces-ic-214-Data.db') complete.
Upgrading SSTableReader(path='/data/data02/system/schema_keyspaces/system-schema_keyspaces-ic-213-Data.db')
Upgrade of SSTableReader(path='/data/data02/system/schema_keyspaces/system-schema_keyspaces-ic-213-Data.db') complete.
> INFO: SSTABLE upgrade: /opt/cassandra-2.0.14/bin/sstableupgrade system_traces events
Found 2 sstables that need upgrading.
Upgrading SSTableReader(path='/data/data03/system_traces/events/system_traces-events-ic-1-Data.db')
Upgrade of SSTableReader(path='/data/data03/system_traces/events/system_traces-events-ic-1-Data.db') complete.
Upgrading SSTableReader(path='/data/data02/system_traces/events/system_traces-events-ic-2-Data.db')
Upgrade of SSTableReader(path='/data/data02/system_traces/events/system_traces-events-ic-2-Data.db') complete.
> INFO: SSTABLE upgrade: /opt/cassandra-2.0.14/bin/sstableupgrade system_traces sessions
Found 2 sstables that need upgrading.
Upgrading SSTableReader(path='/data/data03/system_traces/sessions/system_traces-sessions-ic-1-Data.db')
Upgrade of SSTableReader(path='/data/data03/system_traces/sessions/system_traces-sessions-ic-1-Data.db') complete.
Upgrading SSTableReader(path='/data/data03/system_traces/sessions/system_traces-sessions-ic-2-Data.db')
Upgrade of SSTableReader(path='/data/data03/system_traces/sessions/system_traces-sessions-ic-2-Data.db') complete.
> INFO: Done
I guess this is a good common function that will help a lot of engineers who will work on cassandra upgrade. Personally, I coded a sstableupgrade function which I think it is neat and stable. So, I would like to share it to all. I have used it for my cassandra upgrade from version 1.2.8 to 2.0.14.
function sstable_upgrade()
{
echo_header "SSTABLE UPGRADE"
MYSNAPSHOT_NAME=""
if [ ! -z "$1" ]
then
MYSNAPSHOT_NAME="$1"
fi
SSTABLEUPGRADE_BIN="/opt/cassandra-2.0.14/bin/sstableupgrade"
sudo sed -i 's|256M|2G|g' $SSTABLEUPGRADE_BIN
echo_info "Harvesting keyspaces and column family to $CASSANDRA_BACKUP/backup_keyspace.out"
if sudo find /data/ -name "*-ic*Data.db" | grep -v snapshot* | awk -F"/" '{print $4" "$5}' | sort -u > $CASSANDRA_BACKUP/backup_keyspace.out
then
while read line
do
echo_info "SSTABLE upgrade: $SSTABLEUPGRADE_BIN $line $SNAPSHOT_NAME"
sudo -tt -u $CASS_USER /bin/bash -c "$SSTABLEUPGRADE_BIN $line $MYSNAPSHOT_NAME"
sleep 5
done < $CASSANDRA_BACKUP/backup_keyspace.out
else
echo_warn "Fail harvesting keyspaces and column families"
exit 1
fi
}
Here will be output
> INFO: SSTABLE upgrade: /opt/cassandra-2.0.14/bin/sstableupgrade system schema_keyspaces
Found 2 sstables that need upgrading.
Upgrading SSTableReader(path='/data/data02/system/schema_keyspaces/system-schema_keyspaces-ic-214-Data.db')
Upgrade of SSTableReader(path='/data/data02/system/schema_keyspaces/system-schema_keyspaces-ic-214-Data.db') complete.
Upgrading SSTableReader(path='/data/data02/system/schema_keyspaces/system-schema_keyspaces-ic-213-Data.db')
Upgrade of SSTableReader(path='/data/data02/system/schema_keyspaces/system-schema_keyspaces-ic-213-Data.db') complete.
> INFO: SSTABLE upgrade: /opt/cassandra-2.0.14/bin/sstableupgrade system_traces events
Found 2 sstables that need upgrading.
Upgrading SSTableReader(path='/data/data03/system_traces/events/system_traces-events-ic-1-Data.db')
Upgrade of SSTableReader(path='/data/data03/system_traces/events/system_traces-events-ic-1-Data.db') complete.
Upgrading SSTableReader(path='/data/data02/system_traces/events/system_traces-events-ic-2-Data.db')
Upgrade of SSTableReader(path='/data/data02/system_traces/events/system_traces-events-ic-2-Data.db') complete.
> INFO: SSTABLE upgrade: /opt/cassandra-2.0.14/bin/sstableupgrade system_traces sessions
Found 2 sstables that need upgrading.
Upgrading SSTableReader(path='/data/data03/system_traces/sessions/system_traces-sessions-ic-1-Data.db')
Upgrade of SSTableReader(path='/data/data03/system_traces/sessions/system_traces-sessions-ic-1-Data.db') complete.
Upgrading SSTableReader(path='/data/data03/system_traces/sessions/system_traces-sessions-ic-2-Data.db')
Upgrade of SSTableReader(path='/data/data03/system_traces/sessions/system_traces-sessions-ic-2-Data.db') complete.
> INFO: Done
Tuesday, December 15, 2015
Cassandra Reaper installation
hi all,
Cassandra reaper is an excellent tool that helps cassandra data replication/repair happen seamlessly. It is a project from spotify and we can easily find it from github projects.
https://github.com/spotify/cassandra-reaper.git
I would like to take the opportunity to list down the installation steps.
1. Setting up the postgresdb
yum -y install postgresql-server.x86_64
/etc/init.d/postgresql initdb
/etc/init.d/postgresql start
[root@lab ~]# sudo su - postgres -c psql
psql (8.4.20)
Type "help" for help.
postgres=#
2. Install git and maven
yum -y install git
wget http://mirror.reverse.net/pub/apache/maven/maven-3/3.3.3/binaries/apache-maven-3.3.3-bin.zip
unzip apache-maven-3.3.3-bin.zip
3. Downloading and compiling cassandra-reaper and cassandra-reaper-ui
yum -y install python-argparse
yum -y install python-requests
git clone https://github.com/spotify/cassandra-reaper.git
mkdir -p /root/cassandra-reaper/src/main/resources/assets
cd /root/cassandra-reaper/src/main/resources/assets
wget https://github.com/spodkowinski/cassandra-reaper-ui/releases/download/v0.2.0/cassandra-reaper-ui-0.2.zip
unzip cassandra-reaper-ui-0.2.zip
cd ~/cassandra-reaper
~/apache-maven-3.3.3/bin/mvn package
4. Create the reaper schema. The schema is located at cassandra-reaper/src/main/db/ directory. It is called, reaper_db.sql
psql -h localhost -U reaper -d reaper_db < reaper_db.sql
5. Almost done. Now, we can create the main application directories and start the app.
mkdir -p /usr/share/cassandra-reaper/ /etc/sportify
cp cassandra-reaper-0.2.2-SNAPSHOT.jar /usr/share/cassandra-reaper/
chown root:root /usr/share/cassandra-reaper/cassandra-reaper-0.2.2-SNAPSHOT.jar
mv cassandra-reaper.yaml /etc/spotify/
chown root:root /etc/spotify/cassandra-reaper.yaml
mv spreaper /usr/bin/
chmod +x cassandra-reaper
6. Now, you can start the app.
./cassandra-reaper.
Cassandra reaper is an excellent tool that helps cassandra data replication/repair happen seamlessly. It is a project from spotify and we can easily find it from github projects.
https://github.com/spotify/cassandra-reaper.git
I would like to take the opportunity to list down the installation steps.
1. Setting up the postgresdb
yum -y install postgresql-server.x86_64
/etc/init.d/postgresql initdb
/etc/init.d/postgresql start
[root@lab ~]# sudo su - postgres -c psql
psql (8.4.20)
Type "help" for help.
postgres=#
2. Install git and maven
yum -y install git
wget http://mirror.reverse.net/pub/apache/maven/maven-3/3.3.3/binaries/apache-maven-3.3.3-bin.zip
unzip apache-maven-3.3.3-bin.zip
3. Downloading and compiling cassandra-reaper and cassandra-reaper-ui
yum -y install python-argparse
yum -y install python-requests
git clone https://github.com/spotify/cassandra-reaper.git
mkdir -p /root/cassandra-reaper/src/main/resources/assets
cd /root/cassandra-reaper/src/main/resources/assets
wget https://github.com/spodkowinski/cassandra-reaper-ui/releases/download/v0.2.0/cassandra-reaper-ui-0.2.zip
unzip cassandra-reaper-ui-0.2.zip
cd ~/cassandra-reaper
~/apache-maven-3.3.3/bin/mvn package
4. Create the reaper schema. The schema is located at cassandra-reaper/src/main/db/ directory. It is called, reaper_db.sql
psql -h localhost -U reaper -d reaper_db < reaper_db.sql
5. Almost done. Now, we can create the main application directories and start the app.
mkdir -p /usr/share/cassandra-reaper/ /etc/sportify
cp cassandra-reaper-0.2.2-SNAPSHOT.jar /usr/share/cassandra-reaper/
chown root:root /usr/share/cassandra-reaper/cassandra-reaper-0.2.2-SNAPSHOT.jar
mv cassandra-reaper.yaml /etc/spotify/
chown root:root /etc/spotify/cassandra-reaper.yaml
mv spreaper /usr/bin/
chmod +x cassandra-reaper
6. Now, you can start the app.
./cassandra-reaper.
Wednesday, April 22, 2015
Installing the collectd for a client node
hi all,
2. Now, you need to edit a couples of lines at the /etc/collectd.conf. It is simple.
3. Make sure you change your setting accordingly to suit your need. After all, just start the service like normal.
Voila, you are done!
Please get back to your graphite server, and take a look at the graphite dropdown tree, it should populate your new collectd host. If you want to repeat the same things, on let says, 200 servers? you can always use my pyssh.py script to login to the servers concurrently and make changes and apply the change as you like.
Next I guess, you would like to explore on the presentation layer, like Graphana. With graphana installed, you can show/manipulate the metrics in a narrow scope to fit the need of monitoring needs to you and your team. Or an more intuitive tool like logstash to give you more analytical view on your system log. Or you need a alerting system like graphite-beacon to generate alerts in a different formats.
Congrats!
I would like to list down the steps on how I do the collectd installation on the client node.
1. Download the relevant RPM. I am choosing the latest collectd version, which is 5.4.1. The default package that came from EPEL repo is on version 4, which is a bit outdated. The reason behind it collectd on version 4 is not comes with the graphite plugin. So, to take the shortcut, without compiling the binary from scratch of tarball, you can ought to be like me, and download/install the ready made RPM available from the internet.
wget http://anfadmin.ucsd.edu/linux/CentOS/6/x86_64/collectd-5.4.1-1.el6.x86_64.rpm
wget http://anfadmin.ucsd.edu/linux/CentOS/6/x86_64/libcollectdclient-5.4.1-1.el6.x86_64.rpm
rpm -ivh libcollectdclient-5.4.1-1.el6.x86_64.rpm collectd-5.4.1-1.el6.x86_64.rpm
vi /etc/collectd.conf
Hostname "centos66"
Host "graphite_hostname"
Port "2003"
Protocol "tcp"
LogSendErrors true
Prefix "collectd."
#Postfix "collectd"
StoreRates true
AlwaysAppendDS false
EscapeCharacter "."
/etc/init.d/collectd start
Please get back to your graphite server, and take a look at the graphite dropdown tree, it should populate your new collectd host. If you want to repeat the same things, on let says, 200 servers? you can always use my pyssh.py script to login to the servers concurrently and make changes and apply the change as you like.
Next I guess, you would like to explore on the presentation layer, like Graphana. With graphana installed, you can show/manipulate the metrics in a narrow scope to fit the need of monitoring needs to you and your team. Or an more intuitive tool like logstash to give you more analytical view on your system log. Or you need a alerting system like graphite-beacon to generate alerts in a different formats.
Congrats!
Tuesday, April 21, 2015
Multiple instance for carbon to leverage the traffic
hi,
if you are keen to leverage your carbon traffics, or may be you wish to categorise the metrics from the different sources. Then, you can follow these steps.
1. The configuration file is /opt/graphite/conf/carbon.conf
2. Please look closely at line until you see this. please uncomment them when you need to spawn another carbon instance.
# To configure special settings for the carbon-cache instance 'b', uncomment this:
[cache:b]
LINE_RECEIVER_PORT = 2103
PICKLE_RECEIVER_PORT = 2104
CACHE_QUERY_PORT = 7102
3. Make sure disable the UDP_LISTENER.
# Set this to True to enable the UDP listener. By default this is off
# because it is very common to run multiple carbon daemons and managing
# another (rarely used) port for every carbon instance is not fun.
ENABLE_UDP_LISTENER = False
UDP_RECEIVER_INTERFACE = 0.0.0.0
UDP_RECEIVER_PORT = 2003
4. Once you are done. you need to top up the daemon file like this.
start() {
echo -n $"Starting $prog: "
PYTHONPATH=/usr/local/lib/python2.6/dist-packages/ /opt/graphite/bin/carbon-cache.py --instance=a start
PYTHONPATH=/usr/local/lib/python2.6/dist-packages/ /opt/graphite/bin/carbon-cache.py --instance=b start
RETVAL=$?
.
.
.
stop() {
echo -n $"Stopping $prog: "
PYTHONPATH=/usr/local/lib/python2.6/dist-packages/ /opt/graphite/bin/carbon-cache.py --instance=a stop > /dev/null 2>&1
PYTHONPATH=/usr/local/lib/python2.6/dist-packages/ /opt/graphite/bin/carbon-cache.py --instance=b stop > /dev/null 2>&1
.
.
.
status()
if you are keen to leverage your carbon traffics, or may be you wish to categorise the metrics from the different sources. Then, you can follow these steps.
1. The configuration file is /opt/graphite/conf/carbon.conf
2. Please look closely at line until you see this. please uncomment them when you need to spawn another carbon instance.
# To configure special settings for the carbon-cache instance 'b', uncomment this:
[cache:b]
LINE_RECEIVER_PORT = 2103
PICKLE_RECEIVER_PORT = 2104
CACHE_QUERY_PORT = 7102
3. Make sure disable the UDP_LISTENER.
# Set this to True to enable the UDP listener. By default this is off
# because it is very common to run multiple carbon daemons and managing
# another (rarely used) port for every carbon instance is not fun.
ENABLE_UDP_LISTENER = False
UDP_RECEIVER_INTERFACE = 0.0.0.0
UDP_RECEIVER_PORT = 2003
4. Once you are done. you need to top up the daemon file like this.
start() {
echo -n $"Starting $prog: "
PYTHONPATH=/usr/local/lib/python2.6/dist-packages/ /opt/graphite/bin/carbon-cache.py --instance=a start
PYTHONPATH=/usr/local/lib/python2.6/dist-packages/ /opt/graphite/bin/carbon-cache.py --instance=b start
RETVAL=$?
.
.
.
stop() {
echo -n $"Stopping $prog: "
PYTHONPATH=/usr/local/lib/python2.6/dist-packages/ /opt/graphite/bin/carbon-cache.py --instance=a stop > /dev/null 2>&1
PYTHONPATH=/usr/local/lib/python2.6/dist-packages/ /opt/graphite/bin/carbon-cache.py --instance=b stop > /dev/null 2>&1
.
.
.
status()
Setting up a graphite service in CENTOS 6.6
hi all,
I just want list down the steps of graphite installation on CENTOS6.6.
First you have to install the pre-requisite of the packages.
1. yum -y install epel-release python-pip python-devel gcc libev libev-devel pycairo rrdtool-python mod_wsgi git httpd
2. pip install django==1.6.8 carbon whisper graphite-web django-tagging pytz
3. pip install twisted --upgrade
4. chmod a+w /opt/graphite/storage
Start copying the conf files and make changes.
5. cp /opt/graphite/conf/carbon.conf.example /opt/graphite/conf/carbon.conf
6. cp /opt/graphite/conf/graphite.wsgi.example /opt/graphite/conf/graphite.wsgi
I will commented the lines on the search index portion. for example like this.
# READ THIS
# Initializing the search index can be very expensive, please include
# the WSGIImportScript directive pointing to this script in your vhost
# config to ensure the index is preloaded before any requests are handed
# to the process.
#from graphite.logger import log
#log.info("graphite.wsgi - pid %d - reloading search index" % os.getpid())
#import graphite.metrics.search
7. cp /opt/graphite/conf/storage-schemas.conf.example /opt/graphite/conf/storage-schemas.conf
8. cp /opt/graphite/webapp/graphite/local_settings.py.example /opt/graphite/webapp/graphite/local_settings.py
Once you are done with the copying local_settings.py, please update that TIME_ZONE and SECRET_KEY. please use the tzselect if you don't know how to set your time zone. Secret key can be any strings that you can think of.
Now you are start working on your apache setting.
9. cp /opt/graphite/examples/example-graphite-vhost.conf /etc/httpd/conf.d/graphite-vhost.conf
Comment out the extra mod_wsgi mod from another conf file.
10. cat /etc/httpd/conf.d/wsgi.conf
#LoadModule wsgi_module modules/mod_wsgi.so
On the default /etc/httpd/conf/httpd.conf, you have to update the ServerName. After all you can start your web server service.
11. ServerName graph:80
12. chkconfig httpd on && /etc/init.d/httpd start
Now you can start initialise your django framework. Please complete all the questions asked during the initial setup. Put in your admin credential and email.
13. python manage.py syncdb
Now you can start working on the carbon daemon and start it. Get the free daemon file from the github.
14. wget -q https://github.com/dcarley/graphite-rpms/blob/master/carbon-0.9.8/carbon.init -O /etc/init.d/carbon && /etc/init.d/carbon start
Next you will need to verify your carbon port is listening and your web port is running.
15. netstat -atun | grep 2003
tcp 0 0 0.0.0.0:2003 0.0.0.0:* LISTEN
16. netstat -atun | grep 80
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
Wait for a while, after you have started the services. You need to look into your graphite storage for any new whisper file been created. Keep prompt the directory for any new created whisper files.
17. [root@graph storage]# find /opt/graphite/storage/ -name *.wsp
/opt/graphite/storage/whisper/carbon/agents/graph-a/updateOperations.wsp
/opt/graphite/storage/whisper/carbon/agents/graph-a/errors.wsp
/opt/graphite/storage/whisper/carbon/agents/graph-a/cpuUsage.wsp
/opt/graphite/storage/whisper/carbon/agents/graph-a/avgUpdateTime.wsp
/opt/graphite/storage/whisper/carbon/agents/graph-a/cache/bulk_queries.wsp
/opt/graphite/storage/whisper/carbon/agents/graph-a/cache/queries.wsp
Once all done, your graphite should be up and running. you can open your browser and go to your graphite web service to verify it is running. At this moment, you should not seeing any data injection from any servers yet. you need to continue working on collectd installation on the client nodes.
I just want list down the steps of graphite installation on CENTOS6.6.
First you have to install the pre-requisite of the packages.
1. yum -y install epel-release python-pip python-devel gcc libev libev-devel pycairo rrdtool-python mod_wsgi git httpd
2. pip install django==1.6.8 carbon whisper graphite-web django-tagging pytz
3. pip install twisted --upgrade
4. chmod a+w /opt/graphite/storage
Start copying the conf files and make changes.
5. cp /opt/graphite/conf/carbon.conf.example /opt/graphite/conf/carbon.conf
6. cp /opt/graphite/conf/graphite.wsgi.example /opt/graphite/conf/graphite.wsgi
I will commented the lines on the search index portion. for example like this.
# READ THIS
# Initializing the search index can be very expensive, please include
# the WSGIImportScript directive pointing to this script in your vhost
# config to ensure the index is preloaded before any requests are handed
# to the process.
#from graphite.logger import log
#log.info("graphite.wsgi - pid %d - reloading search index" % os.getpid())
#import graphite.metrics.search
7. cp /opt/graphite/conf/storage-schemas.conf.example /opt/graphite/conf/storage-schemas.conf
8. cp /opt/graphite/webapp/graphite/local_settings.py.example /opt/graphite/webapp/graphite/local_settings.py
Once you are done with the copying local_settings.py, please update that TIME_ZONE and SECRET_KEY. please use the tzselect if you don't know how to set your time zone. Secret key can be any strings that you can think of.
Now you are start working on your apache setting.
9. cp /opt/graphite/examples/example-graphite-vhost.conf /etc/httpd/conf.d/graphite-vhost.conf
Comment out the extra mod_wsgi mod from another conf file.
10. cat /etc/httpd/conf.d/wsgi.conf
#LoadModule wsgi_module modules/mod_wsgi.so
On the default /etc/httpd/conf/httpd.conf, you have to update the ServerName. After all you can start your web server service.
11. ServerName graph:80
12. chkconfig httpd on && /etc/init.d/httpd start
Now you can start initialise your django framework. Please complete all the questions asked during the initial setup. Put in your admin credential and email.
13. python manage.py syncdb
Now you can start working on the carbon daemon and start it. Get the free daemon file from the github.
14. wget -q https://github.com/dcarley/graphite-rpms/blob/master/carbon-0.9.8/carbon.init -O /etc/init.d/carbon && /etc/init.d/carbon start
Next you will need to verify your carbon port is listening and your web port is running.
15. netstat -atun | grep 2003
tcp 0 0 0.0.0.0:2003 0.0.0.0:* LISTEN
16. netstat -atun | grep 80
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
Wait for a while, after you have started the services. You need to look into your graphite storage for any new whisper file been created. Keep prompt the directory for any new created whisper files.
17. [root@graph storage]# find /opt/graphite/storage/ -name *.wsp
/opt/graphite/storage/whisper/carbon/agents/graph-a/updateOperations.wsp
/opt/graphite/storage/whisper/carbon/agents/graph-a/errors.wsp
/opt/graphite/storage/whisper/carbon/agents/graph-a/cpuUsage.wsp
/opt/graphite/storage/whisper/carbon/agents/graph-a/avgUpdateTime.wsp
/opt/graphite/storage/whisper/carbon/agents/graph-a/cache/bulk_queries.wsp
/opt/graphite/storage/whisper/carbon/agents/graph-a/cache/queries.wsp
Once all done, your graphite should be up and running. you can open your browser and go to your graphite web service to verify it is running. At this moment, you should not seeing any data injection from any servers yet. you need to continue working on collectd installation on the client nodes.
Friday, July 4, 2014
Hadoop default configuration parameters and values
hi all,
This is the default configuration for hadoop-1.2.1. Feel free to take this and have a reference.
dfs.access.time.precision=3600000
dfs.balance.bandwidthPerSec=1048576
dfs.block.access.key.update.interval=600
dfs.block.access.token.enable=false
dfs.block.access.token.lifetime=600
dfs.blockreport.initialDelay=0
dfs.blockreport.intervalMsec=3600000
dfs.block.size=67108864
dfs.client.block.write.retries=3
dfs.client.use.datanode.hostname=false
dfs.data.dir=${hadoop.tmp.dir}/dfs/data
dfs.datanode.address=0.0.0.0:50010
dfs.datanode.data.dir.perm=755
dfs.datanode.dns.interface=default
dfs.datanode.dns.nameserver=default
dfs.datanode.drop.cache.behind.reads=false
dfs.datanode.drop.cache.behind.writes=false
dfs.datanode.du.reserved=0
dfs.datanode.failed.volumes.tolerated=0
dfs.datanode.handler.count=3
dfs.datanode.http.address=0.0.0.0:50075
dfs.datanode.https.address=0.0.0.0:50475
dfs.datanode.ipc.address=0.0.0.0:50020
dfs.datanode.max.xcievers=4096
dfs.datanode.readahead.bytes=4193404
dfs.datanode.sync.behind.writes=false
dfs.datanode.use.datanode.hostname=false
dfs.default.chunk.view.size=32768
dfs.df.interval=60000
dfs.heartbeat.interval=3
dfs.http.address=0.0.0.0:50070
dfs.https.address=0.0.0.0:50470
dfs.https.client.keystore.resource=ssl-client.xml
dfs.https.enable=false
dfs.https.need.client.auth=false
dfs.https.server.keystore.resource=ssl-server.xml
dfs.image.transfer.bandwidthPerSec=0
dfs.max.objects=0
dfs.name.dir=${hadoop.tmp.dir}/dfs/name
dfs.name.edits.dir=${dfs.name.dir}
dfs.namenode.avoid.read.stale.datanode=false
dfs.namenode.avoid.write.stale.datanode=false
dfs.namenode.decommission.interval=30
dfs.namenode.decommission.nodes.per.interval=5
dfs.namenode.delegation.key.update-interval=86400000
dfs.namenode.delegation.token.max-lifetime=604800000
dfs.namenode.delegation.token.renew-interval=86400000
dfs.namenode.edits.toleration.length=0
dfs.namenode.handler.count=10
dfs.namenode.invalidate.work.pct.per.iteration=0.32f
dfs.namenode.kerberos.internal.spnego.principal=${dfs.web.authentication.kerberos.principal}
dfs.namenode.logging.level=info
dfs.namenode.replication.work.multiplier.per.iteration=2
dfs.namenode.safemode.min.datanodes=0
dfs.namenode.stale.datanode.interval=30000
dfs.namenode.write.stale.datanode.ratio=0.5f
dfs.permissions.supergroup=supergroup
dfs.permissions=true
dfs.replication=3
dfs.replication.considerLoad=true
dfs.replication.interval=3
dfs.replication.max=512
dfs.replication.min=1
dfs.safemode.extension=30000
dfs.safemode.threshold.pct=0.999f
dfs.secondary.http.address=0.0.0.0:50090
dfs.secondary.namenode.kerberos.internal.spnego.principal=${dfs.web.authentication.kerberos.principal}
dfs.webhdfs.enabled=false
dfs.web.ugi=webuser,webgroup
fs.checkpoint.dir=${hadoop.tmp.dir}/dfs/namesecondary
fs.checkpoint.edits.dir=${fs.checkpoint.dir}
fs.checkpoint.period=3600
fs.checkpoint.size=67108864
fs.default.name=file:///
fs.file.impl=org.apache.hadoop.fs.LocalFileSystem
fs.ftp.impl=org.apache.hadoop.fs.ftp.FTPFileSystem
fs.har.impl.disable.cache=true
fs.har.impl=org.apache.hadoop.fs.HarFileSystem
fs.hdfs.impl=org.apache.hadoop.hdfs.DistributedFileSystem
fs.hftp.impl=org.apache.hadoop.hdfs.HftpFileSystem
fs.hsftp.impl=org.apache.hadoop.hdfs.HsftpFileSystem
fs.kfs.impl=org.apache.hadoop.fs.kfs.KosmosFileSystem
fs.ramfs.impl=org.apache.hadoop.fs.InMemoryFileSystem
fs.s3.block.size=67108864
fs.s3.buffer.dir=${hadoop.tmp.dir}/s3
fs.s3.impl=org.apache.hadoop.fs.s3.S3FileSystem
fs.s3.maxRetries=4
fs.s3n.impl=org.apache.hadoop.fs.s3native.NativeS3FileSystem
fs.s3.sleepTimeSeconds=10
fs.trash.interval=0
fs.webhdfs.impl=org.apache.hadoop.hdfs.web.WebHdfsFileSystem
hadoop.http.authentication.kerberos.keytab=${user.home}/hadoop.keytab
hadoop.http.authentication.kerberos.principal=HTTP/localhost@LOCALHOST
hadoop.http.authentication.signature.secret.file=${user.home}/hadoop-http-auth-signature-secret
hadoop.http.authentication.simple.anonymous.allowed=true
hadoop.http.authentication.token.validity=36000
hadoop.http.authentication.type=simple
hadoop.jetty.logs.serve.aliases=true
hadoop.logfile.count=10
hadoop.logfile.size=10000000
hadoop.native.lib=true
hadoop.relaxed.worker.version.check=false
hadoop.rpc.socket.factory.class.default=org.apache.hadoop.net.StandardSocketFactory
hadoop.security.authentication=simple
hadoop.security.authorization=false
hadoop.security.group.mapping=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
hadoop.security.instrumentation.requires.admin=false
hadoop.security.token.service.use_ip=true
hadoop.security.uid.cache.secs=14400
hadoop.security.use-weak-http-crypto=false
hadoop.skip.worker.version.check=false
hadoop.tmp.dir=/tmp/hadoop-${user.name}
hadoop.util.hash.type=murmur
io.bytes.per.checksum=512
io.compression.codecs=org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.BZip2Codec,org.apache.hadoop.io.compress.SnappyCodec
io.file.buffer.size=4096
io.mapfile.bloom.error.rate=0.005
io.mapfile.bloom.size=1048576
io.map.index.skip=0
io.seqfile.compress.blocksize=1000000
io.seqfile.lazydecompress=true
io.seqfile.sorter.recordlimit=1000000
io.serializations=org.apache.hadoop.io.serializer.WritableSerialization
io.skip.checksum.errors=false
io.sort.factor=10
io.sort.mb=100
io.sort.record.percent=0.05
io.sort.spill.percent=0.80
ipc.client.connection.maxidletime=10000
ipc.client.connect.max.retries=10
ipc.client.fallback-to-simple-auth-allowed=false
ipc.client.idlethreshold=4000
ipc.client.kill.max=10
ipc.client.tcpnodelay=false
ipc.server.listen.queue.size=128
ipc.server.tcpnodelay=false
jobclient.output.filter=FAILED
job.end.retry.attempts=0
job.end.retry.interval=30000
keep.failed.task.files=false
local.cache.size=10737418240
mapred.acls.enabled=false
mapred.child.java.opts=-Xmx200m
mapred.child.tmp=./tmp
mapred.cluster.map.memory.mb=-1
mapred.cluster.max.map.memory.mb=-1
mapred.cluster.max.reduce.memory.mb=-1
mapred.cluster.reduce.memory.mb=-1
mapred.combine.recordsBeforeProgress=10000
mapred.compress.map.output=false
mapred.disk.healthChecker.interval=60000
mapred.healthChecker.interval=60000
mapred.healthChecker.script.timeout=600000
mapred.heartbeats.in.second=100
mapred.inmem.merge.threshold=1000
mapred.job.map.memory.mb=-1
mapred.job.queue.name=default
mapred.job.reduce.input.buffer.percent=0.0
mapred.job.reduce.memory.mb=-1
mapred.job.reuse.jvm.num.tasks=1
mapred.job.shuffle.input.buffer.percent=0.70
mapred.job.shuffle.merge.percent=0.66
mapred.jobtracker.blacklist.fault-bucket-width=15
mapred.jobtracker.blacklist.fault-timeout-window=180
mapred.jobtracker.completeuserjobs.maximum=100
mapred.job.tracker.handler.count=10
mapred.job.tracker.http.address=0.0.0.0:50030
mapred.jobtracker.job.history.block.size=3145728
mapred.job.tracker.jobhistory.lru.cache.size=5
mapred.jobtracker.jobSchedulable=org.apache.hadoop.mapred.JobSchedulable
mapred.job.tracker=local
mapred.jobtracker.maxtasks.per.job=-1
mapred.jobtracker.nodegroup.aware=false
mapred.job.tracker.persist.jobstatus.active=false
mapred.job.tracker.persist.jobstatus.dir=/jobtracker/jobsInfo
mapred.job.tracker.persist.jobstatus.hours=0
mapred.jobtracker.restart.recover=false
mapred.job.tracker.retiredjobs.cache.size=1000
mapred.jobtracker.taskScheduler=org.apache.hadoop.mapred.JobQueueTaskScheduler
mapred.line.input.format.linespermap=1
mapred.local.dir=${hadoop.tmp.dir}/mapred/local
mapred.local.dir.minspacekill=0
mapred.local.dir.minspacestart=0
mapred.map.max.attempts=4
mapred.map.output.compression.codec=org.apache.hadoop.io.compress.DefaultCodec
mapred.map.tasks=2
mapred.map.tasks.speculative.execution=true
mapred.max.tracker.blacklists=4
mapred.max.tracker.failures=4
mapred.merge.recordsBeforeProgress=10000
mapred.min.split.size=0
mapred.output.compress=false
mapred.output.compression.codec=org.apache.hadoop.io.compress.DefaultCodec
mapred.output.compression.type=RECORD
mapred.queue.default.state=RUNNING
mapred.queue.names=default
mapred.reduce.max.attempts=4
mapred.reduce.parallel.copies=5
mapred.reduce.slowstart.completed.maps=0.05
mapred.reduce.tasks=1
mapred.reduce.tasks.speculative.execution=true
mapred.skip.attempts.to.start.skipping=2
mapred.skip.map.auto.incr.proc.count=true
mapred.skip.map.max.skip.records=0
mapred.skip.reduce.auto.incr.proc.count=true
mapred.skip.reduce.max.skip.groups=0
mapred.submit.replication=10
mapred.system.dir=${hadoop.tmp.dir}/mapred/system
mapred.task.cache.levels=2
mapred.task.profile=false
mapred.task.profile.maps=0-2
mapred.task.profile.reduces=0-2
mapred.task.timeout=600000
mapred.tasktracker.dns.interface=default
mapred.tasktracker.dns.nameserver=default
mapred.tasktracker.expiry.interval=600000
mapred.task.tracker.http.address=0.0.0.0:50060
mapred.tasktracker.indexcache.mb=10
mapred.tasktracker.map.tasks.maximum=2
mapred.tasktracker.reduce.tasks.maximum=2
mapred.task.tracker.report.address=127.0.0.1:0
mapred.task.tracker.task-controller=org.apache.hadoop.mapred.DefaultTaskController
mapred.tasktracker.taskmemorymanager.monitoring-interval=5000
mapred.tasktracker.tasks.sleeptime-before-sigkill=5000
mapred.temp.dir=${hadoop.tmp.dir}/mapred/temp
mapreduce.ifile.readahead.bytes=4194304
mapreduce.ifile.readahead=true
mapreduce.job.acl-modify-job=
mapreduce.job.acl-view-job=
mapreduce.job.complete.cancel.delegation.tokens=true
mapreduce.job.counters.counter.name.max=64
mapreduce.job.counters.group.name.max=128
mapreduce.job.counters.groups.max=50
mapreduce.job.counters.max=120
mapreduce.jobhistory.cleaner.interval-ms=86400000
mapreduce.jobhistory.max-age-ms=2592000000
mapreduce.job.restart.recover=true
mapreduce.job.split.metainfo.maxsize=10000000
mapreduce.jobtracker.staging.root.dir=${hadoop.tmp.dir}/mapred/staging
mapreduce.reduce.input.limit=-1
mapreduce.reduce.shuffle.connect.timeout=180000
mapreduce.reduce.shuffle.maxfetchfailures=10
mapreduce.reduce.shuffle.read.timeout=180000
mapreduce.tasktracker.outofband.heartbeat.damper=1000000
mapreduce.tasktracker.outofband.heartbeat=false
mapred.used.genericoptionsparser=true
mapred.user.jobconf.limit=5242880
mapred.userlog.limit.kb=0
mapred.userlog.retain.hours=24
map.sort.class=org.apache.hadoop.util.QuickSort
net.topology.impl=org.apache.hadoop.net.NetworkTopology
tasktracker.http.threads=40
topology.node.switch.mapping.impl=org.apache.hadoop.net.ScriptBasedMapping
topology.script.number.args=100
webinterface.private.actions=false
This is the default configuration for hadoop-1.2.1. Feel free to take this and have a reference.
dfs.access.time.precision=3600000
dfs.balance.bandwidthPerSec=1048576
dfs.block.access.key.update.interval=600
dfs.block.access.token.enable=false
dfs.block.access.token.lifetime=600
dfs.blockreport.initialDelay=0
dfs.blockreport.intervalMsec=3600000
dfs.block.size=67108864
dfs.client.block.write.retries=3
dfs.client.use.datanode.hostname=false
dfs.data.dir=${hadoop.tmp.dir}/dfs/data
dfs.datanode.address=0.0.0.0:50010
dfs.datanode.data.dir.perm=755
dfs.datanode.dns.interface=default
dfs.datanode.dns.nameserver=default
dfs.datanode.drop.cache.behind.reads=false
dfs.datanode.drop.cache.behind.writes=false
dfs.datanode.du.reserved=0
dfs.datanode.failed.volumes.tolerated=0
dfs.datanode.handler.count=3
dfs.datanode.http.address=0.0.0.0:50075
dfs.datanode.https.address=0.0.0.0:50475
dfs.datanode.ipc.address=0.0.0.0:50020
dfs.datanode.max.xcievers=4096
dfs.datanode.readahead.bytes=4193404
dfs.datanode.sync.behind.writes=false
dfs.datanode.use.datanode.hostname=false
dfs.default.chunk.view.size=32768
dfs.df.interval=60000
dfs.heartbeat.interval=3
dfs.http.address=0.0.0.0:50070
dfs.https.address=0.0.0.0:50470
dfs.https.client.keystore.resource=ssl-client.xml
dfs.https.enable=false
dfs.https.need.client.auth=false
dfs.https.server.keystore.resource=ssl-server.xml
dfs.image.transfer.bandwidthPerSec=0
dfs.max.objects=0
dfs.name.dir=${hadoop.tmp.dir}/dfs/name
dfs.name.edits.dir=${dfs.name.dir}
dfs.namenode.avoid.read.stale.datanode=false
dfs.namenode.avoid.write.stale.datanode=false
dfs.namenode.decommission.interval=30
dfs.namenode.decommission.nodes.per.interval=5
dfs.namenode.delegation.key.update-interval=86400000
dfs.namenode.delegation.token.max-lifetime=604800000
dfs.namenode.delegation.token.renew-interval=86400000
dfs.namenode.edits.toleration.length=0
dfs.namenode.handler.count=10
dfs.namenode.invalidate.work.pct.per.iteration=0.32f
dfs.namenode.kerberos.internal.spnego.principal=${dfs.web.authentication.kerberos.principal}
dfs.namenode.logging.level=info
dfs.namenode.replication.work.multiplier.per.iteration=2
dfs.namenode.safemode.min.datanodes=0
dfs.namenode.stale.datanode.interval=30000
dfs.namenode.write.stale.datanode.ratio=0.5f
dfs.permissions.supergroup=supergroup
dfs.permissions=true
dfs.replication=3
dfs.replication.considerLoad=true
dfs.replication.interval=3
dfs.replication.max=512
dfs.replication.min=1
dfs.safemode.extension=30000
dfs.safemode.threshold.pct=0.999f
dfs.secondary.http.address=0.0.0.0:50090
dfs.secondary.namenode.kerberos.internal.spnego.principal=${dfs.web.authentication.kerberos.principal}
dfs.webhdfs.enabled=false
dfs.web.ugi=webuser,webgroup
fs.checkpoint.dir=${hadoop.tmp.dir}/dfs/namesecondary
fs.checkpoint.edits.dir=${fs.checkpoint.dir}
fs.checkpoint.period=3600
fs.checkpoint.size=67108864
fs.default.name=file:///
fs.file.impl=org.apache.hadoop.fs.LocalFileSystem
fs.ftp.impl=org.apache.hadoop.fs.ftp.FTPFileSystem
fs.har.impl.disable.cache=true
fs.har.impl=org.apache.hadoop.fs.HarFileSystem
fs.hdfs.impl=org.apache.hadoop.hdfs.DistributedFileSystem
fs.hftp.impl=org.apache.hadoop.hdfs.HftpFileSystem
fs.hsftp.impl=org.apache.hadoop.hdfs.HsftpFileSystem
fs.kfs.impl=org.apache.hadoop.fs.kfs.KosmosFileSystem
fs.ramfs.impl=org.apache.hadoop.fs.InMemoryFileSystem
fs.s3.block.size=67108864
fs.s3.buffer.dir=${hadoop.tmp.dir}/s3
fs.s3.impl=org.apache.hadoop.fs.s3.S3FileSystem
fs.s3.maxRetries=4
fs.s3n.impl=org.apache.hadoop.fs.s3native.NativeS3FileSystem
fs.s3.sleepTimeSeconds=10
fs.trash.interval=0
fs.webhdfs.impl=org.apache.hadoop.hdfs.web.WebHdfsFileSystem
hadoop.http.authentication.kerberos.keytab=${user.home}/hadoop.keytab
hadoop.http.authentication.kerberos.principal=HTTP/localhost@LOCALHOST
hadoop.http.authentication.signature.secret.file=${user.home}/hadoop-http-auth-signature-secret
hadoop.http.authentication.simple.anonymous.allowed=true
hadoop.http.authentication.token.validity=36000
hadoop.http.authentication.type=simple
hadoop.jetty.logs.serve.aliases=true
hadoop.logfile.count=10
hadoop.logfile.size=10000000
hadoop.native.lib=true
hadoop.relaxed.worker.version.check=false
hadoop.rpc.socket.factory.class.default=org.apache.hadoop.net.StandardSocketFactory
hadoop.security.authentication=simple
hadoop.security.authorization=false
hadoop.security.group.mapping=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
hadoop.security.instrumentation.requires.admin=false
hadoop.security.token.service.use_ip=true
hadoop.security.uid.cache.secs=14400
hadoop.security.use-weak-http-crypto=false
hadoop.skip.worker.version.check=false
hadoop.tmp.dir=/tmp/hadoop-${user.name}
hadoop.util.hash.type=murmur
io.bytes.per.checksum=512
io.compression.codecs=org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.BZip2Codec,org.apache.hadoop.io.compress.SnappyCodec
io.file.buffer.size=4096
io.mapfile.bloom.error.rate=0.005
io.mapfile.bloom.size=1048576
io.map.index.skip=0
io.seqfile.compress.blocksize=1000000
io.seqfile.lazydecompress=true
io.seqfile.sorter.recordlimit=1000000
io.serializations=org.apache.hadoop.io.serializer.WritableSerialization
io.skip.checksum.errors=false
io.sort.factor=10
io.sort.mb=100
io.sort.record.percent=0.05
io.sort.spill.percent=0.80
ipc.client.connection.maxidletime=10000
ipc.client.connect.max.retries=10
ipc.client.fallback-to-simple-auth-allowed=false
ipc.client.idlethreshold=4000
ipc.client.kill.max=10
ipc.client.tcpnodelay=false
ipc.server.listen.queue.size=128
ipc.server.tcpnodelay=false
jobclient.output.filter=FAILED
job.end.retry.attempts=0
job.end.retry.interval=30000
keep.failed.task.files=false
local.cache.size=10737418240
mapred.acls.enabled=false
mapred.child.java.opts=-Xmx200m
mapred.child.tmp=./tmp
mapred.cluster.map.memory.mb=-1
mapred.cluster.max.map.memory.mb=-1
mapred.cluster.max.reduce.memory.mb=-1
mapred.cluster.reduce.memory.mb=-1
mapred.combine.recordsBeforeProgress=10000
mapred.compress.map.output=false
mapred.disk.healthChecker.interval=60000
mapred.healthChecker.interval=60000
mapred.healthChecker.script.timeout=600000
mapred.heartbeats.in.second=100
mapred.inmem.merge.threshold=1000
mapred.job.map.memory.mb=-1
mapred.job.queue.name=default
mapred.job.reduce.input.buffer.percent=0.0
mapred.job.reduce.memory.mb=-1
mapred.job.reuse.jvm.num.tasks=1
mapred.job.shuffle.input.buffer.percent=0.70
mapred.job.shuffle.merge.percent=0.66
mapred.jobtracker.blacklist.fault-bucket-width=15
mapred.jobtracker.blacklist.fault-timeout-window=180
mapred.jobtracker.completeuserjobs.maximum=100
mapred.job.tracker.handler.count=10
mapred.job.tracker.http.address=0.0.0.0:50030
mapred.jobtracker.job.history.block.size=3145728
mapred.job.tracker.jobhistory.lru.cache.size=5
mapred.jobtracker.jobSchedulable=org.apache.hadoop.mapred.JobSchedulable
mapred.job.tracker=local
mapred.jobtracker.maxtasks.per.job=-1
mapred.jobtracker.nodegroup.aware=false
mapred.job.tracker.persist.jobstatus.active=false
mapred.job.tracker.persist.jobstatus.dir=/jobtracker/jobsInfo
mapred.job.tracker.persist.jobstatus.hours=0
mapred.jobtracker.restart.recover=false
mapred.job.tracker.retiredjobs.cache.size=1000
mapred.jobtracker.taskScheduler=org.apache.hadoop.mapred.JobQueueTaskScheduler
mapred.line.input.format.linespermap=1
mapred.local.dir=${hadoop.tmp.dir}/mapred/local
mapred.local.dir.minspacekill=0
mapred.local.dir.minspacestart=0
mapred.map.max.attempts=4
mapred.map.output.compression.codec=org.apache.hadoop.io.compress.DefaultCodec
mapred.map.tasks=2
mapred.map.tasks.speculative.execution=true
mapred.max.tracker.blacklists=4
mapred.max.tracker.failures=4
mapred.merge.recordsBeforeProgress=10000
mapred.min.split.size=0
mapred.output.compress=false
mapred.output.compression.codec=org.apache.hadoop.io.compress.DefaultCodec
mapred.output.compression.type=RECORD
mapred.queue.default.state=RUNNING
mapred.queue.names=default
mapred.reduce.max.attempts=4
mapred.reduce.parallel.copies=5
mapred.reduce.slowstart.completed.maps=0.05
mapred.reduce.tasks=1
mapred.reduce.tasks.speculative.execution=true
mapred.skip.attempts.to.start.skipping=2
mapred.skip.map.auto.incr.proc.count=true
mapred.skip.map.max.skip.records=0
mapred.skip.reduce.auto.incr.proc.count=true
mapred.skip.reduce.max.skip.groups=0
mapred.submit.replication=10
mapred.system.dir=${hadoop.tmp.dir}/mapred/system
mapred.task.cache.levels=2
mapred.task.profile=false
mapred.task.profile.maps=0-2
mapred.task.profile.reduces=0-2
mapred.task.timeout=600000
mapred.tasktracker.dns.interface=default
mapred.tasktracker.dns.nameserver=default
mapred.tasktracker.expiry.interval=600000
mapred.task.tracker.http.address=0.0.0.0:50060
mapred.tasktracker.indexcache.mb=10
mapred.tasktracker.map.tasks.maximum=2
mapred.tasktracker.reduce.tasks.maximum=2
mapred.task.tracker.report.address=127.0.0.1:0
mapred.task.tracker.task-controller=org.apache.hadoop.mapred.DefaultTaskController
mapred.tasktracker.taskmemorymanager.monitoring-interval=5000
mapred.tasktracker.tasks.sleeptime-before-sigkill=5000
mapred.temp.dir=${hadoop.tmp.dir}/mapred/temp
mapreduce.ifile.readahead.bytes=4194304
mapreduce.ifile.readahead=true
mapreduce.job.acl-modify-job=
mapreduce.job.acl-view-job=
mapreduce.job.complete.cancel.delegation.tokens=true
mapreduce.job.counters.counter.name.max=64
mapreduce.job.counters.group.name.max=128
mapreduce.job.counters.groups.max=50
mapreduce.job.counters.max=120
mapreduce.jobhistory.cleaner.interval-ms=86400000
mapreduce.jobhistory.max-age-ms=2592000000
mapreduce.job.restart.recover=true
mapreduce.job.split.metainfo.maxsize=10000000
mapreduce.jobtracker.staging.root.dir=${hadoop.tmp.dir}/mapred/staging
mapreduce.reduce.input.limit=-1
mapreduce.reduce.shuffle.connect.timeout=180000
mapreduce.reduce.shuffle.maxfetchfailures=10
mapreduce.reduce.shuffle.read.timeout=180000
mapreduce.tasktracker.outofband.heartbeat.damper=1000000
mapreduce.tasktracker.outofband.heartbeat=false
mapred.used.genericoptionsparser=true
mapred.user.jobconf.limit=5242880
mapred.userlog.limit.kb=0
mapred.userlog.retain.hours=24
map.sort.class=org.apache.hadoop.util.QuickSort
net.topology.impl=org.apache.hadoop.net.NetworkTopology
tasktracker.http.threads=40
topology.node.switch.mapping.impl=org.apache.hadoop.net.ScriptBasedMapping
topology.script.number.args=100
webinterface.private.actions=false
Subscribe to:
Posts (Atom)