Over the year that we've been perfecting our site, we have always had a beta version up and running.
For a bootstrapped startup, its vitally important to save wherever possible, and we've tried lots of possibilities. Here are some that might help you decide what hosting is right for you (from cheap to expensive)
Note: I haven't written about cloud based hosting here, you'll find a good article on cloud based hosting at this link.
1. Free or shared hosting:
Free or shared hosting providers such as godaddy or bluehost (recommended) can provide you really cheap hosting (under $10) per month, your own dot com domain, and huge amounts of data transfer, storage and even shell access.
The downside? You won't be able to run cpu intensive tasks, and your data transfer rates may not be impressive.
Recommended For: When your site when its just taking shape: mostly static, and few users.
Also, your blog
Cost: $10 or less per month
2. Home based hosting:
It is possible to use a spare computer (or your desktop that's always on) as a webserver that can serve out your site.
It isn't too difficult, needs average technical skills, and might not cost you a penny!
You can do this without a static ip, by using a service like dyndns, and can get free domains (non-dot-com) or dot-com paid domains.
Make sure, before you go for this, that you won't run into data overages with your ISP (most ISPs in the USA provide unlimited bandwidth, so it should not be a problem).
Also, there can be potential slow-down of your home based network. You can read on a possible situation here, and how to fix it.
Recommended For: Users with website setup skills. Excellent low cost/high performance initial investment
Cost: $30 or less per year. Excluding internet charges.
3. VPS Hosting:
VPS (Virtual Private Server) hosting can be the next step from home based hosting or shared hosting - especially when you aren't satisfied with the performance of the system or the network.
VPS hosting gives you shell access, and you are guaranteed a minimum RAM and system performance.
This would be a stop-gap arrangement before going for a dedicated server or managed service.
For VPS/Dedicated hosting, I recommend softlayer.
Recommended For: When you need more performance from the system - CPU or bandwidth
Cost: $50 - $100 per month.
3. Dedicated and Managed Hosting:
Once your website is on its way to success, or you want a site that you have full control over, and is ready to scale - consider dedicated or managed hosting.
Managed hosting is one, where the company makes sure your server is running just fine, and you only need to keep your code in place. They worry about patches, alerts etc.
Dedicated hosting, is where you get a server with an operating system of your choice. You do everything else!
You can get any configuration you like, but the price keeps going up. I do recommend starting with a lower end server with more RAM, and then going up with configuration as you need.
Recommended For: More scalability and advanced technical skills
Cost: $100 - ?? per month.
4. Co-located Hosting:
For the technically inclined, co-located hosting is the premium hosting solution.
In co-location, you own your own server, and possibly firewalls, and install them in the co-location facility.
Apart from having guaranteed power and network bandwidth, you also get secure entry to your server, fire protection, video surveillance etc. As you can imagine, co-located hosting is more than just about getting a dedicated port for your server.
Once your site is up and running, and needs a stable facility for your computer - which you manage - co-location is the option for you.
Recommended For: Ownership, high security and performance, and advanced technical skills
Cost: $150 - ?? per month.
I hope you find this blog useful. Do leave me feedback and ask any questions.
Wednesday, December 9, 2009
Why an in-home server can slow down the network
Even with the best (in-home) technologies, the server can hog your home network, and bring everything down to a crawl. Case in point, my 5MBPS connection started to feel like dialup, as I browsed in slow motion.
How can this happen? And how do you prevent this?
High active ip connections.
Here is what can happen with a poorly configured server:
Let's say you setup a Linksys WRT-64G router which you use at home, with your new webserver on a lan port. Over time, your server will establish new connections with users (based on the apache settings). These connections, and the router timeout limits, can severely hamper the overall network performance.
The reason? Connections aren't killed after they are used. And the router runs out of memory trying to keep track of all the connections.
The really odd symptom, is that when you open a new browser window, it struggles to connect to a website. But your other windows are running better.
Here is a simple remedy to the problem:
1. Router setting changes:
First lets check if your router is complaining:
a. Check in your router status page for the number of Active ip connections. Typically this should not be more than 30 per computer using the router.
b. If the Active ip connections, are high (500+), then check if most of the active connections are from your server (in dd-wrt, you can do this by going to Status > Click on the number against 'Active IP Connections'.
c. If you find it is your server, you've hit jackpot. Time to set things straight:
Set the router to terminate unnecessary persistent connections.
You can do this by reducing the TCP and UDP timeouts (I recommend TCP timeout 600 seconds and UDP timeout 120 seconds).
This will prevent the router from keeping unnecessary connections on.
2. Server setting changes:
Apache likes to setup a lot of connections so you get a decent amount of multi-user performance from your website. If yours is a new website, likely it will take time to get bombarded by too many users.
Given that you want to share your home network with your server, these settings will help reduce the demand from the server:
a. Go to the apache config file (in ubuntu, it is: /etc/apache2/apache2.conf), and modify the mpm worker and preform modules as follows. (If you already have the same Server settings, then considering reducing them further)
<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxClients 100
MaxRequestsPerChild 500
</IfModule>
<IfModule mpm_worker_module>
StartServers 2
MaxClients 150
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 0
</IfModule>
b. Restart your apache server (in Ubuntu: sudo apache2ctl restart)
That's it! And your apache server will run much lighter now - especially with fewer extra servers lurking around.
3. Check if everything looks good:
You should now see a drop in number of active ip connections.
Also, soon your in-home network will get back to its peppy self.
I hope this helps you!
How can this happen? And how do you prevent this?
High active ip connections.
Here is what can happen with a poorly configured server:
Let's say you setup a Linksys WRT-64G router which you use at home, with your new webserver on a lan port. Over time, your server will establish new connections with users (based on the apache settings). These connections, and the router timeout limits, can severely hamper the overall network performance.
The reason? Connections aren't killed after they are used. And the router runs out of memory trying to keep track of all the connections.
The really odd symptom, is that when you open a new browser window, it struggles to connect to a website. But your other windows are running better.
Here is a simple remedy to the problem:
1. Router setting changes:
First lets check if your router is complaining:
a. Check in your router status page for the number of Active ip connections. Typically this should not be more than 30 per computer using the router.
b. If the Active ip connections, are high (500+), then check if most of the active connections are from your server (in dd-wrt, you can do this by going to Status > Click on the number against 'Active IP Connections'.
c. If you find it is your server, you've hit jackpot. Time to set things straight:
Set the router to terminate unnecessary persistent connections.
You can do this by reducing the TCP and UDP timeouts (I recommend TCP timeout 600 seconds and UDP timeout 120 seconds).
This will prevent the router from keeping unnecessary connections on.
2. Server setting changes:
Apache likes to setup a lot of connections so you get a decent amount of multi-user performance from your website. If yours is a new website, likely it will take time to get bombarded by too many users.
Given that you want to share your home network with your server, these settings will help reduce the demand from the server:
a. Go to the apache config file (in ubuntu, it is: /etc/apache2/apache2.conf), and modify the mpm worker and preform modules as follows. (If you already have the same Server settings, then considering reducing them further)
<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxClients 100
MaxRequestsPerChild 500
</IfModule>
<IfModule mpm_worker_module>
StartServers 2
MaxClients 150
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 0
</IfModule>
b. Restart your apache server (in Ubuntu: sudo apache2ctl restart)
That's it! And your apache server will run much lighter now - especially with fewer extra servers lurking around.
3. Check if everything looks good:
You should now see a drop in number of active ip connections.
Also, soon your in-home network will get back to its peppy self.
I hope this helps you!
Saturday, December 5, 2009
The mysql slow query log
The first step to tuning your database, is to find out what's running slow.
I recommend doing tuning your queries before changing your MySql memory allocation, sessions etc.
Its always better to treat the problem ground up (by finding rogue queries) rather than put a quick fix and forget about it (e.g. allocating more memory or setting sessions). Over time, the quick fixes will lead you to buy more hardware and cost more.
1. Setup the MySQL Slow query log:
To setup the log, first uncomment the log_slow_queries and long_query_time parameters:
# Here you can see queries with especially long duration
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 5
#log-queries-not-using-indexes
This will set the slow query log to: /var/log/mysql/mysql-slow.log and will log any queries that take more than 5 seconds.
To start with, you should set the long_query_time to a higher value, so you don't get overwhelmed with too many slow queries. Later you can bring this down to 2 seconds or so.
And later you should also set the 'log-queries-not-using-indexes, to identify queries that will slow significantly as data load increases.
2. Restart mysql:
In ubuntu, simply run the command:
sudo /etc/init.d/mysql restart
This will restart the mysql server and start logging your slow queries.
3. Use your database, and keep checking for slow queries!
I hope this helps you in tuning your MySql database from the ground up.
I recommend doing tuning your queries before changing your MySql memory allocation, sessions etc.
Its always better to treat the problem ground up (by finding rogue queries) rather than put a quick fix and forget about it (e.g. allocating more memory or setting sessions). Over time, the quick fixes will lead you to buy more hardware and cost more.
1. Setup the MySQL Slow query log:
To setup the log, first uncomment the log_slow_queries and long_query_time parameters:
# Here you can see queries with especially long duration
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 5
#log-queries-not-using-indexes
This will set the slow query log to: /var/log/mysql/mysql-slow.log and will log any queries that take more than 5 seconds.
To start with, you should set the long_query_time to a higher value, so you don't get overwhelmed with too many slow queries. Later you can bring this down to 2 seconds or so.
And later you should also set the 'log-queries-not-using-indexes, to identify queries that will slow significantly as data load increases.
2. Restart mysql:
In ubuntu, simply run the command:
sudo /etc/init.d/mysql restart
This will restart the mysql server and start logging your slow queries.
3. Use your database, and keep checking for slow queries!
I hope this helps you in tuning your MySql database from the ground up.
LAMP: Why does a page take too long to open?
If you're wondering why some pages take ages to open on your LAMP site, consider checking all the queries that hit your mysql database when you open that page.
Sometimes it is not slow queries, but too many reasonably fast queries, that are slowing you down.
Note: I suggest you try using your slow query log first! Here is a post that will help you.
For checking the database for all queries, you should be the only user on the database.
1. To know *all* that's hitting your mysql database, simply uncomment this command in your my.cnf file (in Ubuntu, it is located at: /etc/mysql; you need to be root or run 'sudo /etc/mysql/my.cnf')
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
log = /var/log/mysql/mysql.log
2. Once you've uncommented this line, restart mysql server:
In ubuntu: sudo /etc/init.d/mysql restart
3. Clear startup logs:
Most likely you don't want anything to do with the startup queries, so simply clear the log:
sudo echo " " > /var/log/mysql/mysql.log
4. Run your test: Go ahead and open your troublesome page in your browser.
5. As soon as the page is opened fully, copy out the log (here it copies to your home directory):
sudo cp /var/log/mysql/mysql.log ~/slowpage.log
If you have more troublesome pages, clear the log again and copy it to another location, for analysis.
6. Once you're done with getting your logs, comment the log setup in my.cnf once more, and restart mysql again (steps 1 and 2). This will keep your mysql running fast.
7. Now go ahead and analyze what queries are run when you go to a particular page.
All the best with your tuning efforts!
Sometimes it is not slow queries, but too many reasonably fast queries, that are slowing you down.
Note: I suggest you try using your slow query log first! Here is a post that will help you.
For checking the database for all queries, you should be the only user on the database.
1. To know *all* that's hitting your mysql database, simply uncomment this command in your my.cnf file (in Ubuntu, it is located at: /etc/mysql; you need to be root or run 'sudo /etc/mysql/my.cnf')
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
log = /var/log/mysql/mysql.log
2. Once you've uncommented this line, restart mysql server:
In ubuntu: sudo /etc/init.d/mysql restart
3. Clear startup logs:
Most likely you don't want anything to do with the startup queries, so simply clear the log:
sudo echo " " > /var/log/mysql/mysql.log
4. Run your test: Go ahead and open your troublesome page in your browser.
5. As soon as the page is opened fully, copy out the log (here it copies to your home directory):
sudo cp /var/log/mysql/mysql.log ~/slowpage.log
If you have more troublesome pages, clear the log again and copy it to another location, for analysis.
6. Once you're done with getting your logs, comment the log setup in my.cnf once more, and restart mysql again (steps 1 and 2). This will keep your mysql running fast.
7. Now go ahead and analyze what queries are run when you go to a particular page.
All the best with your tuning efforts!
MySql Session
To find out what is going on in your MySql database, you can check the sessions that are presently running, with the command:
show processlist;
show processlist;
Thursday, November 5, 2009
Apache MySQL Performance dip
Separating the MySQL and Apache servers has slowed my site down, instead of speeding it up! Any suggestions?
I have hosted a LAMP website on a 2GHz AMD Opteron server running with 2GB of RAM.
As the data needs grew, the MySQL database seemed to need its own area, and could not serve the apache server fast enough.
So I split the MySQL database to run on its own 2GHz AMD Opteron/2GB RAM server - with a single-hop connection between the two of them at the data center.
But performance has dipped - any clicks take a much longer time to show a result from the database.
Any ideas how to get better performance from the separate Apache/MySQL server setup?
Thanks!
I have hosted a LAMP website on a 2GHz AMD Opteron server running with 2GB of RAM.
As the data needs grew, the MySQL database seemed to need its own area, and could not serve the apache server fast enough.
So I split the MySQL database to run on its own 2GHz AMD Opteron/2GB RAM server - with a single-hop connection between the two of them at the data center.
But performance has dipped - any clicks take a much longer time to show a result from the database.
Any ideas how to get better performance from the separate Apache/MySQL server setup?
Thanks!
Saturday, October 31, 2009
Password Management Using htpasswd
In Linux, several components use non-system standard passwords.
Some of these are svn, and ftp (e.g. vsftpd).
Here are the basics to get you through setting and resetting these passwords:
For svn, refer this guide.
For vsftpd, refer this guide.
Basically, htpasswd can be used to create encrypted passwords for such modules. Here are some basics to htpasswd:
1. htpasswd uses a password encrypted file, using an SSL certificate.
You can create your own ssl certificate too. But for public/production environments, you'd want to get a certificate from a third party so users don't get a security alert.
2. The first time, use the -c flag to set a password, as follows. You'll need to do this as root (or prefix sudo in ubuntu)
htpasswd -c /etc/apache2/my_passwd.passwd username
3. Subsequently, use the -m flag to modify this file for adding/editing users:
htpasswd -m /etc/apache2/my_passwd.passwd username
Some of these are svn, and ftp (e.g. vsftpd).
Here are the basics to get you through setting and resetting these passwords:
For svn, refer this guide.
For vsftpd, refer this guide.
Basically, htpasswd can be used to create encrypted passwords for such modules. Here are some basics to htpasswd:
1. htpasswd uses a password encrypted file, using an SSL certificate.
You can create your own ssl certificate too. But for public/production environments, you'd want to get a certificate from a third party so users don't get a security alert.
2. The first time, use the -c flag to set a password, as follows. You'll need to do this as root (or prefix sudo in ubuntu)
htpasswd -c /etc/apache2/my_passwd.passwd username
3. Subsequently, use the -m flag to modify this file for adding/editing users:
htpasswd -m /etc/apache2/my_passwd.passwd username
Subscribe to:
Posts (Atom)
Posts
