Friday, October 23, 2009

Forking in PHP on Mac OSX using pcntl_fork

1)sudo port selfupdate
2)sudo port update outdated
3)sudo port install php5-pcntl
4)Try the following example

$pid = pcntl_fork();

if($pid) {
// parent process runs what is here
print "parent\n";
}
else {
// child process runs what is here
print "child\n";
}
?>

Tuesday, June 09, 2009

Init script to mount EBS and attach elastic IPs to EC2 instances

Modified init script to mount EBS automatically to your EC2 instances.  It also dynamically attaches your instances to an elastic ip.  You have to provide both EBS volume id and elastic ip as userdata separated by pipe symbol for this script to work.  Sample userdata 

vol-12345678|171.213.234.123

You have to use 

update-rc.d mountec2vol start 47 S . stop 35 0 6 . start 37 6 .

to add this script to startup. This script also additional code to mount mysql data directory and nginx directories from EBS. You can skip that part.




#! /bin/sh
#
# /etc/init.d/mountec2vol
#
# chkconfig: 234 20 50
# description: Assigns an EC2 EBS Volume to a device and mounts the device
#
# To add this as a service run:
# /sbin/chkconfig --add mountec2vol
#
# VARS
#
DEV="/dev/sdh"
MOUNT_POINT="/vol"
EC2_LOG='/var/log/ec2.log';
SELF=$(cd $(dirname $0); pwd -P)/$(basename $0)

export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.13/
export EC2_PRIVATE_KEY=/root/.ec2/pk.pem
export EC2_CERT=/root/.ec2/cert.pem
export EC2_HOME=/usr/local/src/ec2-api-tools-1.3-36506/
PATH=$PATH:$HOME/bin:$EC2_HOME/bin
MAX_TRIES=60

attach_ebs()
{
/bin/echo "=====================================" | tee -a $EC2_LOG
/bin/echo "Date: "`date` | tee -a $EC2_LOG;
/bin/echo "Attaching Elastic Block Store Volumes." | tee -a $EC2_LOG
ec2-attach-volume $VOL -i $INSTANCE -d $DEV 1>&2 2>>$EC2_LOG
CTR=0
while [ ! -e "$DEV" ]; do
/bin/sleep 1
CTR=`expr $CTR + 1`
if [ $CTR -eq $MAX_TRIES ]
then
/bin/echo "WARNING: Cannot attach volume $VOL to $DEV -- Giving up after $MAX_TRIES attempts" | tee -a $EC2_LOG
exit 1
fi
done
}
mount_ebs()
{
/bin/echo "Checking if $MOUNT_POINT is present" | tee -a $EC2_LOG
if [ ! -d $MOUNT_POINT ]; then
mkdir $MOUNT_POINT 1>&2 2>>$EC2_LOG
fi
/bin/echo "Mounting EBS to /vol " | tee -a $EC2_LOG
/bin/mount $DEV $MOUNT_POINT 1>&2 2>>$EC2_LOG
}
mount_mysql_dirs()
{
/bin/echo "Mounting mysql dirs" | tee -a $EC2_LOG
/bin/mount /etc/mysql/ 1>&2 2>>$EC2_LOG
/bin/mount /var/lib/mysql/ 1>&2 2>>$EC2_LOG
/bin/mount /var/log/mysql/ 1>&2 2>>$EC2_LOG
/bin/mount /usr/local/nginx/ 1>&2 2>>$EC2_LOG
}
unmount_mysql_dirs()
{
set +e;
/bin/echo "stopping mysql" | tee -a $EC2_LOG
/etc/init.d/mysql stop;
set -e;
/bin/echo "Unmounting mysql dirs" | tee -a $EC2_LOG
/bin/umount /etc/mysql/ 1>&2 2>>$EC2_LOG
/bin/umount /var/lib/mysql/ 1>&2 2>>$EC2_LOG
/bin/umount /var/log/mysql/ 1>&2 2>>$EC2_LOG
}
unmount_nginx_dirs()
{
set +e;
/bin/echo "stopping nginx" | tee -a $EC2_LOG
/etc/init.d/nginx stop;
set -e;
/bin/echo "Unmounting nginx dirs" | tee -a $EC2_LOG
/bin/umount /usr/local/nginx/ 1>&2 2>>$EC2_LOG
}
unmount_ebs()
{
/bin/echo "Unmounting Elastic Block Store Volumes." | tee -a $EC2_LOG
/bin/umount $MOUNT_POINT 1>&2 2>>$EC2_LOG
}
detach_ebs()
{
/bin/echo "Detaching Elastic Block Store Volumes." | tee -a $EC2_LOG
ec2-detach-volume $VOL 1>&2 2>>$EC2_LOG
}
populate_aws_data()
{
/bin/echo "Populating AWS data" | tee -a $EC2_LOG
INSTANCE=`curl http://169.254.169.254/latest/meta-data/instance-id 2> /dev/null`
USERDATA=`curl http://169.254.169.254/1.0/user-data 2>/dev/null`;
}
parse_user_args()
{
/bin/echo "Parsing user_args " | tee -a $EC2_LOG
VOL=${USERDATA%|*};
IP=${USERDATA#*|};
echo "usedata vol ip";
echo $USERDATA;
echo $VOL;
echo $IP;
}
bind_ip()
{
/bin/echo "Associating ip address: $IP to $INSTANCE" | tee -a $EC2_LOG
ec2-associate-address -i $INSTANCE $IP;
}

# start/stop functions for OS

case "$1" in

start)
populate_aws_data;
parse_user_args;
bind_ip;
attach_ebs;
mount_ebs;
mount_mysql_dirs;
sh -x /vol/ec2_stuff/everytime_by_mountec2.sh;
;;
stop)
unmount_mysql_dirs;
unmount_nginx_dirs;
unmount_ebs;
detach_ebs;
;;
restart)
set +e;
$SELF stop;
set -e;
sleep 60;
$SELF start
;;
*)
echo "Usage: $0 {start|stop|restart}" | tee -a $EC2_LOG
exit 1

esac

exit 0




Tuesday, May 26, 2009

Auto Mounting EBS on EC2

I assume that you have already created a EC2 instance (On top of your custom AMI),  EBS volume (both in the same availability zone) . Below are some instructions to automount EBS on EC2. 


AutoMount Init Script
===========================
http://developer.amazonwebservices.com/connect/thread.jspa?threadID=25071&start=0&tstart=0

1)apt-get install sun-java6-jre sun-java6-bin sun-java6-source
2)cd /usr/local/src/
3)wget http://developer.amazonwebservices.com/connect/servlet/JiveServlet/download/30-25071-102530-2022/mountec2vol
4)wget http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip
unzip ec2-api-tools.zip
5)edit mountec2vol
set JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.13/  or equivalent
set EC2_PRIVATE_KEY EC2_CERT
set EC2_HOME=/usr/local/src/ec2-api-tools-1.3-36506/
set VOL and DEV
6)cp mountec2vol /etc/init.d/
7)chmod +x /etc/init.d/mountec2vol
8)test  mountec2vol
/etc/init.d/mountec2vol start
/etc/init.d/mountec2vol stop
9)update-rc.d mountec2vol start 15 2 3 4 5 . start 47 S . start 35 0 1 6 .
10)Bundle the new image
ec2-bundle-vol -d /mnt -k /mnt/pk-balbal.pem -c /mnt/cert-balhbalh.pem -u userid  -r i386  -p automountingebs
11)upload the new image
ec2-upload-bundle  --url https://s3.amazonaws.com/subdir/ -b yourbucket  -m /mnt/automountingebs.xml -a accessid -s yoursecret
12)register image
ec2-register yourbucket /subdir/automountingebs.xml



Some additional info to create EBS 
=========================
On local machine
================
ec2-create-volume -z us-east-1c -s 10
ec2-describe-volumes vol-VVVV1111
ec2-attach-volume -d /dev/sdh -i i-IIII1111 vol-VVVV1111

On EC2 Instance
===============
apt-get install -y xfsprogs
modprobe xfs
mkfs.xfs /dev/sdh

echo "/dev/sdh /vol xfs noatime 0 0" >> /etc/fstab

mkdir /vol
mount /vol

http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1663


AutoMount
=========
http://www.ioncannon.net/system-administration/199/automounting-amazon-ebs-volumes-on-ec2-instances/

Monday, May 25, 2009

Nginx + PHPFPM + Memcache + Mysql + XCache + Monit + Logrotate

Step by Step
============
1)install debian (in case of EC2 - grab a public debian image and create a instance using it)
2)apt-get update
3)apt-get install openssh-server
4)apt-get install vim screen rsync rcconf telnet emacs locate unzip
5)set up .bashrc and  .vimrc
6)apt-get install sysstat subversion sudo
7)add new users and configure sudo
8)aptitude install postfix
9)apt-get install mysql-server mysql-client
10) mysql_secure_installation 
11)aptitude install build-essential libtool libltdl3-dev libgd-dev libmcrypt-dev libxml2-dev libmysqlclient15-dev flex m4 awk automake autoconf bison make libbz2-dev libpcre3-dev libssl-dev zlib1g-dev re2c
12)install php with fpm, curl, mysql

  Install PHP with FPM, CURL, Mysql etc
  =============
  1)sudo apt-get install libcurl4-openssl-dev(if 4 is not found, intall libcurl3-openssol-dev)
  2)sudo apt-get install libjpeg62
  3)sudo apt-get install libjpeg62-dev
  4)sudo apt-get install libpng12-dev
  5)cd /usr/local/src/
  6)wget http://in.php.net/distributions/php-5.2.8.tar.bz2
  7)wget http://php-fpm.anight.org/downloads/head/php-5.2.8-fpm-0.5.10.diff.gz
  8)gunzip php-5.2.8-fpm-0.5.10.diff.gz 
  9)tar -jxvpf php-5.2.8.tar.bz2
  10)patch -d php-5.2.8 -p1 <>
  11)./configure  --enable-fastcgi --enable-fpm --with-mcrypt --with-zlib --enable-mbstring --with-openssl --with-mysql --with-mysql-sock --with-gd --with-jpeg-dir=/usr/lib --enable-gd-native-ttf --without-sqlite --disable-pdo --disable-reflection --with-curl --with-mysqli=/usr/bin/mysql_config
  9)make
  9)make test
Test whether both mysql and mysqli are supported
  10)sudo make install


13)vi /usr/local/etc/php-fpm.conf 
:63
www-data
:66
www-data
You may want to add: /usr/local/lib/php to your php.ini include_path

14)install  xcache

   Install XCahce
   =============
   1)wget http://xcache.lighttpd.net/pub/Releases/1.2.2/xcache-1.2.2.tar.gz
   2)tar -zxvf xcache-1.2.2.tar.gz 
   3)cd xcache-1.2.2
   4)phpize
   5)./configure --with-php-config=/usr/local/bin/php-config --enable-xcache
   6)make install


15)install nginx

  Install Nginx
  ==============
  1)wget http://sysoev.ru/nginx/nginx-0.6.36.tar.gz
  2)tar xzvf nginx
  3)cd nginx
  4)./configure --sbin-path=/usr/local/sbin --with-http_ssl_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module
  5)make
  6)make install


16)configure nginx

    Configure Nginx
    ==============
    1)cd /usr/local/nginx/
    2)vi conf/nginx.conf
    read for more details
    http://www.yawn.it/2008/04/30/nginx-php-php-fpm-on-debian-etch-40/
    Enable forwarding php requests to php-fpm by uncommenting the handling of .php files
    edit fastcgi_param as follows
    fastcgi_param SCRIPT_FILENAME /usr/local/nginx/html$fastcgi_script_name;
    https://calomel.org/nginx.html
    http://articles.slicehost.com/2008/5/15/ubuntu-hardy-nginx-configuration
    http://articles.slicehost.com/2008/5/16/ubuntu-hardy-nginx-virtual-hosts
    3)change log format to include response time 
    http://wiki.nginx.org/NginxHttpUpstreamModule#.24upstream_response_time



17)configure xcache

    Configure XCache
    ================
    1)vi /usr/local/lib/php.ini
    add the following contents 

    magic_quotes_gpc=0
    [xcache-common]
    zend_extension = /usr/local/lib/php/extensions/no-debug-non-zts-20060613/xcache.so
    [xcache]
    xcache.shm_scheme = "mmap"
    xcache.size = 64M
    date.timezone = “Europe/Rome”
    default_charset = “utf-8″


18)install memcache extension for php

  Install Memcache extension for php
  ===================================
  compile php with memcache support
  http://www.mail-archive.com/php-install@lists.php.net/msg15223.html
  1)wget http://pecl.php.net/get/memcache-2.2.5.tgz
  2)tar -zxvf memcache-2.2.5.tgz
  3)cd memcache-2.2.5
  4)phpize 
  5)./configure 
  6)make && make test
  7)$ sudo make install
  Installing shared extensions:     /usr/local/lib/php/extensions/no-debug-non-zts-20060613/

  8)vi /usr/local/lib/php.ini
  Add the following to php.ini
  extension=memcache.so
  display_startup_errors = On
  extension_dir = "/usr/local/lib/php/extensions/no-debug-non-zts-20060613/"

  9)run memcache test case in genlib/testcases/basic_memcache.php 


19)test nginx - add init scripts 

Test nginx add  init.d scripts
=======================================
1)nginx
2)php-fpm start
3)create a info.php file and test it.
4)sudo kill `cat /usr/local/nginx/logs/nginx.pid`
5)add init.d scripts - follow instructions in 
http://articles.slicehost.com/2007/10/19/debian-etch-adding-an-nginx-init-script/

20)test php-fpm - add init script 

Test php-fpm add init.d scripts
===============================
1)cd /etc/init.d/
2)ln -s /usr/local/sbin/php-fpm php-fpm
3)/usr/sbin/update-rc.d -f php-fpm defaults

21)optimize

optimize
=======
Find out the number of connections to phpfpm from nginx
$netstat -np | grep 127.0.0.1:9000 | grep -v TIME_WAIT
set appropraite value for variables in php-fpm.conf 
set request_terminate_timeout to 10s
set request_slowlog_timeout to 6s
set user and group to  www-data 
set max_children to 20
perform ab testing and optimize  worker_processes, worker_connections in nginx and max_children in php-fpm.conf

22)add logrotate scripts for nginx,php-fpm

       logrotate scripts
       ================
       1)create logroate conf for nginx and php-fpm - copy logroate files from conf_files dir 
       2)add entries for nginx and php-fpm logrotation at /etc/cron.daily/logrotate (not needed if /etc/logrotate.d is included in the logrotate.conf files, just make sure logrotate is called daily)

23)fine tune php-fpm.conf - enable slow query - set max execution time
24)fine tune mysql - enable slow query - log queries that don't use index
25)check if query cache is turned on in mysql - optimize it

Optimize Query Cache in mysql
============================
1) show variables like 'query%';
2) show status like 'qc%';
http://dev.mysql.com/tech-resources/articles/mysql-query-cache.html

26)install monit

  Monit
  =====
  1)apt-get install monit
  2)vim /etc/monit/monitrc  - configure it
  3)create /etc/monit/monit.d directory - add sshd,nginx monit config files here
  4)add include /etc/monit/monit.d/* option in /etc/monit/monitrc
  3)monit -t
  4)vim /etc/default/monit  set startup=1
  5)test monit