11
May 11

Testing with DotCloud

Summer is almost here and it was really the time to clean up my blog a bit. Now there is a much more readable theme and all the CityDeal hamburger campaigns are gone. Hopefully nobody will be missing those?

I’ve been writing already about deploying applications instead of virtual machines and how I would like to move away from the machine instance and to a some kind of a service where you could select a system and the service would deploy and manage the needed computing resources. It now seems like this is starting to become a reality as there are providers like DotCloud and PHP Fog who take care of the infra while the customer just deploys the code.

I requested a beta invite to DotCloud which in contrary to PHP Fog is somewhat more multipurpose with other services than the PHP only. Actually, currently they have java, mysql, nodejs, perl, php, php-worker, postgresql, python, python-worker, rabbitmq, redis, ruby, ruby-worker, smtp, static as the deployment options. They also have some nice tutorials about how to use the system. The system is operated mostly locally from CLI and the code is pushed to dotcloud which seems like a nice way to do it as it is analogous to for example editing files locally and then using FTP to transfer the files to a web server.

As a test run, I went with installing WordPress as there is a good tutorial for that and only a few minutes later I had a running blog in DotCloud. I’m really liking this!


21
Apr 11

US-EAST-1 region outage 21st of April

Quora is down, Reddit is in emergency read only mode. Quite severe this is then!

According to the first investigation (from the AWS health dashboard) the reason for outage was a networking event which caused a large number of EBS volumes being re-mirrored. This caused capacity problems in the affected region. Also there were problems with one control plane which made it difficult to create new EBS volumes and instances. Control plane is a piece of router architecture which is responsible of drawing the network map, if you did not know it… I certainly did not know before.

Of course, there are plenty of other services impacted by the outage and I guess this is a great time to see how different services have been designed to sustain a degradation of some underlying components. Quora is totally dead (well, there is the notification to users) and Reddit is in read only mode. I give my points to Reddit as they have managed to fail gracefully to a cached read only mode.

Funny thing, just today I was reading a text by James Hamilton which is spot on this situation. I need to say I am surprised Quora did not have a fail over to a different location as the other location in US seems to be ok.


25
Jan 11

Working with Amazon Route 53

I wanted to get a fi-domain as I am building a site for our housing company. It’s very much a pro bono work, but interesting nevertheless. To be honest, this is the first time I have to register a fi-domain and man, it’s not as easy as getting a com or similar domain with DynDNS etc. You need to be a Finnish citizen to be allowed to get one for starters and made sure you are not violating any possible trademarks or even more, some real people with your domain name.

I would perhaps been ok if a DynDNS type of service would exists (well, now as I write this it probably does) in Finland, but the ones I came across were mostly just taking orders and not like dynamically updating their resources… but can’t of course be totally sure. Anyway, I decided to give Amazon Route 53 a go as it is new and I do appreciate the possibility to update the records on command line. Or well, I perhaps did not investigate really too much before signing up.

First I had to though register the fi-domain with Ficora and that took around a day to get the credentials on paper. Yes. On paper. The next step was to register the name and give them two (at this point fictious) name servers. Then I was on my way to Route 53. The first look at the Getting Started Guide is not very encouraging. Need to create some files which contain the access keys and the actual requests. Need to run a perl script to actually create the records. Good thing I bought my first Mac just a few months ago as with Windows this would have sucked.

So the first thing was to create the .aws-secret file which contains your AWS Secret Access Keys it looks something like this:

%awsSecretAccessKeys = (
“my-keys” => {
id => “JISEGIOJDFGSLSDKFG”,
key => “KSLDFSDFGSDFGSasdfsdASFDSDF”,
},
);

And it really needs to be named .aws-secret and have only read permissions as the dnscurl.pl checks this.

Then create the zone you have registered:

<CreateHostedZoneRequest xmlns=”https://route53.amazonaws.com/doc/2010-10-01/”>
<Name>YOURDOMAIN.fi.</Name>
<CallerReference>SOMETHINGRANDOMHERE</CallerReference>
<HostedZoneConfig>
<Comment>Creating first zone</Comment>
</HostedZoneConfig>
</CreateHostedZoneRequest>

Then download dnscurl.pl from the AWS developer tools and run it with these parameters:

dnscurl.pl –keyname my-keys — -X POST -H “Content-Type: text/xml; charset=UTF-8″ –upload-file MyCreateRequest.xml https://route53.amazonaws.com/2010-10-01/hostedzone

You should get something like this in return:

<CreateHostedZoneResponse xmlns=”https://route53.amazonaws.com/doc/2010-10-01/”><HostedZone><Id>/hostedzone/34LJSKFSJGSDFKJ</Id><Name>YOURDOMAIN.fi.</Name><CallerReference>JIjasdmfasfw4af3233</CallerReference><Config><Comment>Creating first zone</Comment></Config></HostedZone><ChangeInfo><Id>/change/23ILKSFJDLSK</Id><Status>PENDING</Status><SubmittedAt>2011-01-24T20:48:47.715Z</SubmittedAt></ChangeInfo><DelegationSet><NameServers><NameServer>ns-1778.awsdns-30.co.uk</NameServer><NameServer>ns-372.awsdns-44.com</NameServer><NameServer>ns-1621.awsdns-38.org</NameServer><NameServer>ns-534.awsdns-04.net</NameServer></NameServers></DelegationSet></CreateHostedZoneResponse>

Here are the real name servers which I had to give to Ficora and it happily said them being ok, so fi-domain is well supported by AWS! Yey!

Then you can start adding records to your zone. First need to create the MyRecordsRequest.xml for the records which could look like this:

<?xml version=”1.0″ encoding=”UTF-8″?>
<ChangeResourceRecordSetsRequest xmlns=”https://route53.amazonaws.com/doc/2010-10-01/”>
<ChangeBatch>
<Comment>
Create A-record
</Comment>
<Changes>
<Change>
<Action>CREATE</Action>
<ResourceRecordSet>
<Name>www.yourdomain.fi.</Name>
<Type>A</Type>
<TTL>14400</TTL>
<ResourceRecords>
<ResourceRecord>
<Value>192.0.0.111</Value>
</ResourceRecord>
</ResourceRecords>
</ResourceRecordSet>
</Change>
</Changes>
</ChangeBatch>
</ChangeResourceRecordSetsRequest>

dnscurl.pl –keyname my-keys — -H “Content-Type: text/xml; charset=UTF-8″ -X POST –upload-file ./MyRecordsRequest.xml https://route53.amazonaws.com/2010-10-01/hostedzone/34LJSKFSJGSDFKJ/rrset

And you should get a response like this:
0.0%
<?xml version=”1.0″?>
<ChangeResourceRecordSetsResponse xmlns=”https://route53.amazonaws.com/doc/2010-10-01/”><ChangeInfo><Id>/change/C3FMNWCVL1YW40</Id><Status>PENDING</Status><SubmittedAt>2011-01-25T19:16:24.181Z</SubmittedAt></ChangeInfo></ChangeResourceRecordSetsResponse>

I got a few problems with “root is not authorized to perform: route53:ChangeResourceRecordSets on resource” because I did not have ./ in front of the MyRecordsRequest.xml, so remember to have it there.


23
Jan 11

Pageload performance with Aegir

I’ve been working with Aegir recently to get some ideas how to host multiple Drupal sites in a somewhat automated and scalable way. To my surprise the performance was actually a lot better with the t1.micro instance which runs the Aegir platform than what I get from the small instance. A video below has a demo when running the identical sites on both servers:

The pageload was around 2.6 seconds with the standard Drupal installation and around a second faster with the Aegir platform. The site on Aegir was dump from the other site taken a few days ago with no significant differences. Have to say I really really like Aegir!


23
Jan 11

Installing Aegir on Fedora 8

These are more like notes for myself, but I am happy to share if someone else would be interested. No warranties though, if you can’t get Aegir running with these notes :)

Get the packages:

yum install httpd php php-mysql php-gd mysql-server postfix sudo rsync git-core unzip alpine screen

Set the PHP.ini memory limits, 192M works ok

Fix your DNS and how your host resolves:

edit /etc/hosts

x.x.x.x FQDN hostname

hostname FQDN

Install MTA (I run Postfix, so just copypaste from other server)

upload posfix configuration to /etc/postfix

Edit the /etc/postfix/canonical

This will rewrite the name which Drupal uses to send emails to what you need

create canonical db

postmap /etc/postfix/canonical
service postfix start

Create a group for the Aegir user and which Apache uses as well:

groupadd www-data

edit /etc/httpd/conf/httpd.conf to run as www-data

The Aegir user home folder needs to have 755 as the permissions (at least this works)

adduser -r –groups www-data –home-dir /var/home/aegiruser aegiruser

Edit the httpd.conf to have the Aegir confs included

Include /var/home/providus/config/apache.conf

Edit sudoers file

aegiruser ALL=NOPASSWD: /usr/sbin/apachectl

start mysqld

/usr/bin/mysqladmin -u root password ‘password’

wget -O /tmp/install.sh ‘http://git.aegirproject.org/?p=provision.git;a=blob_plain;f=install.sh.txt;hb=provision-0.4-beta2′

su -s /bin/sh aegiruser -c “sh /tmp/install.sh”

./drush/drush.php dl drupal