Software
Since 2001, derkeiler.com is running 24 hours a day.
On our servers, thousands of processes perform complex tasks 365 days a year.
This wouldn’t be possible without brilliant pieces of software.
On this page I want to present useful programs which I’ve been using for years, and which just work flawlessly.
FreeBSD
Everyone knows that choosing an operating system is one of the most important decisions when building an IT infrastructure.
In my eyes everyone should choose the operating system which fits his needs best. I’ve chosen FreeBSD for my servers for the following reasons:
- Clear and complete Documentation:
Have you ever tried to find documentation for let’s say debian linux to perform a simple task (e.g. adding a startscript)? After searching hundreds of manuals, HOWTOs and FAQs you have probably ended on google searching the net to find a way to solve your problem. Other operating systems or distributions have similar problems, because writing documentation is not the funniest job to do, so sometimes nobody does it. FreeBSD has a several hundred page long handbook which is 100% up-to-date and covers all topics from installing the os to getting flash and java in your firefox working. If you ever wondered what man pages are good for: Have a look at the tuning-manpage of FreeBSD: It’s a perfect example on how useful manpages can be. - Clean codebase
I thought of writing some thoughts about another well known operating system, which code base is (at least in my eyes) ... different. But you as a reader can get your own picture by reading our archive of this particular os, or just do a grep -R fu.. /usr/src. The FreeBSD code (the part I read) is well documented and relatively easy to understand. - Defined release process
Most IT professionals know what project management is, and how a defined development and test plan can improve the quality of a product. The FreeBSD release engineering process gives me a good feeling, because it shows that the development team of the os plans before they code (not the other way around). - Constancy
This attribute is true for nearly all unix operating systems. It’s important for me because I want to use my time to learn things which bring me forward. If the operating system I use would change the way I have to operate it every three years, I would have to use much of my time to learn new mechanical skills just to do the same things I did with the old version of the operating system. I personally find this kind of improvements quite useless. - Stability
If the hardware is ok, FreeBSD just runs. The only os/hardware combination I personally used which is even more stable is SUN Solaris. - Price: FreeBSD … free. The fun running the greatest text adventure ever created … priceless.
Apache
It’s by far the most popular webserver used on the internet, and it’s stability is unquestioned.
On the derkeiler.com-Cluster more than 50 different apache instances are running 24 hours a day, and it never failed.
mod_gzip
Content compression is one of the coolest http extensions ever created, and it’s surprising that not many people are using it.
There is much to write about mod_gzip. Fortunately I already wrote an article about it for Linux Magazine. You can find it here (there is also a german version available).
Perl
To start with the biggest problem:
Perl is very flexible. There are most of the time several ways of doing something in perl. The problem is that it’s easy to do something, but hard to do things right. If you use perl in a critical environment it’s up to you as a programmer to follow good programming practices like declaring variables, the language itself doesn’t force you to do so.
Beside this, perl is a great language to perform most tasks. It’s robust, has a c-like syntax, garbage collection, infinite modules and is quite fast.
If you want to learn perl: Get a book which sets a high value on clean coding style (most perl books I know don’t do this). The free book Learning perl the Hard Way from Green Tea Press is a positive example.
mod_perl
As seen above Perl is used on our sites to perform various tasks, including offering you dynamic content. For every page request at least 5 different Perl programs are started.
A widely used technique to implement dynamic content into a webpage is CGI. With CGI, for every request to a program this program has to be started, executed and terminated.
Especially with interpreted programs this technique has a large performance penalty: The start of the interpreter and compilation of a script normally takes longer or as long as the execution of an average CGI program.
To overcome this overhead mod_perl makes the perl interpreter a part of apache. Furthermore it’s possible to compile all perl scripts on apache startup. This way you are able to trade memory (which is relative cheap these days) for speed.
MySQL
MySQL is a high performance relational database.
Back in 2000 when I first stumbled across this database it did not offered much features beside having SQL as query language (even MS Access had referential integrity this time).
Anyway i started using it, and have never lost a single dataset which was stored in a MySQL database.
Over the years MySQL became a full featured database with features like replication, transactions and referential integrity.
On the derkeiler.com-cluster the MySQL servers handle about 150 SQL querys per second without problems.
If you run your own MySQL server I recommend the book High Performance MySQL. It’s one of the best computer books I ever read and saved me from buying new hardware.
memcached
memcached is a high-performance, distributed memory object caching system.
It’s used to store information which has to be accessed fast, but is not important enough to be stored in a database.
Cacti
Cacti is a system monitoring solution which provides graphs about various system parameters.
It’s not designed to be a alerting system, but it’s great to provide long-term trends.
I use it to monitor about 50 parameters of the system, from bandwidth usage to the behavior of the MySQL database.
The gathered information is especially useful to optimize and tune the system.
