Share

Postgres Connection Pools: Pgpool vs. PgBouncer

21 Feb 2008, 17:03

We use Postgres for a lot of stuff here. Since it's a multi-process database server (unlike MySQL which is multi-threaded), creating a direct connection to Postgres is relatively slow.

This can be worked around by using a connection pool server in front of the DB. Up until recently we used pgpool for this. But it has a few disadvantages:

* It's multi-process itself (although it preforks), so it eats quite a lot of RAM.
* Once you reach the maximum number of processes, it "queues" incoming connections in the OS accept queue, which is ugly.
* It has poor (but improving) instrumentation.

Recently we switched to using PgBouncer from our friends at Skype. It fixes all these problems quite nicely:

* It uses single-process event-based socket handling, using the excellent libevent like memcached. This means it uses several orders of magnitude less RAM than pgpool.
* It has configurable internal queuing, so your clients' connections don't time out if they're in the queue.
* It has good instrumentation in the form of an internal pseudo-database you can query.

Now here are some pretty graphs. Spot where we changed:



Comments

  • lozzd

    Hont.

    21 Feb 2008, 17:08
  • muz

    Do want.

    22 Feb 2008, 10:22
  • iz-sai

    Wow, cool, did not know you use Postgres! Are last.fm running entirely on PostgreSQL or this is all about minor databases?

    4 Jun 2008, 14:15
  • kkaiv

    quadmother & 30GB does not seem like a minor database :) anyway it's nice to see that you have found our soft beneficial

    10 Feb 2009, 20:15
See all 5 comments
Leave a comment. Log in to Last.fm or sign up (it’s free).