A single web application server is often more than enough for serving thousands of users if appropriate measures are taken to optimize application stack and server software. Here's the things I've found are essential to web application performance.
Database performance
Number one thing with database performance is to try to minimize database I/O access. Memory access is fast, hard disk access is slow. Common RDBM's typically persist to hard drive, so developer can utilize caching to greatly speed up database reads.
- Use DB software's own caching so that as much data as possible are cached at memory. With the usual suspects, MySQL and InnoDB, this means setting 'innodb_buffer_pool_size' to be maybe half your server's total RAM.
- Cache database queries and results. Use DBAL that handles prepared statements. Store query results to memory cache.
- Cache individual HTTP server responses with slow or frequently accessed DB queries into some key-value storage. Even a simple query done several times per pageload puts a considerable load on your database software.
Also make sure, that you have proper primary keys and indexes in place.
Reverse proxying
Use reverse proxy software to sit between your client and your server, so that proxy serves pre-rendered pages of your site very efficiently and saves your server from rendering response to same request over and over again.
Use a framework, that sets caching related HTTP headers, so that static pages are stored in cache for a long time and need to be rendered rarely, and dynamic pages with quickly changing information are stored in cache only for a short time. Remember though, that if you have a busy site, even a short cache time, say 5 seconds, can have a big impact because you only have to render the page every five seconds, instead of every request.
Your framework ideally should also be able to handle setting ESI (Edge Side Includes) tags, so that you can set up different expiry times for different parts of the page.
Popular reverse proxy software are Varnish, Squid and Nginx, which is also a very quick HTTP server.
Optimize page loading times
While speeding up page loading times positively enhances your site's user experience, it can have an effect of your server's performance, as well. Few things to concentrate on:
- Utilize client side caching for static resources. Set appropriate expiry times for infrequently changing static resources (CSS/JS/images etc.), so your clients can load them from their own cache between requests.
- Compress your responses with gzip compression to reduce the amount of traffic you send.
- Minify Javascript and CSS to reduce traffic.
Google, Yahoo etc. have some excellent tools for finding out and measuring your site's performance. Here's old but still valid advice: http://googlewebmastercentral.blogspot.fi/2010/04/using-site-speed-in-web-search-ranking.html
Language opcode caching
Most languages used for web application programming are dynamic, meaning they are interpreted at runtime. Compiler might need to translate the high-level source code to some intermediary form (bytecode), which can then be executed.
If your compiler doesn't do this, you want to set up and use bytecode caching to avoid compiling it with every request. For example, Python stores the bytecode automatically, while PHP does not.