[ic] more on scalability and st=text versus db, academic discussion

Jeff Dafoe interchange-users@icdevgroup.org
Wed Oct 30 17:42:00 2002


    I was thinking more about interchange's incredible scalability due to
the fact that it can do product scans in the text files instead of the SQL
database.  Postgresql and mysql are essentially non-scalable past a single
machine (neither currently support synchronous multimaster replication)
without having to hack whatever software is using the dbms (IC, in our
case).  They also both fork a backend process for each connection.  This can
be a serious limitation for scalability, you can deploy multiple web servers
but if they all have to connect to the same database server that server can
rapidly become the bottleneck after about 400 concurrent connections.
    In comes Interchange to save the day.  You can do product scans (which
probably account for the majority of the database traffic on a typical
ecommerce website) against the text files and other non-SQL-based indexes.
These files can be easily replicated across machines.  By not relying on the
SQL server for product scans Interchange solves, in advance, issues that can
cause serious growing pains for ecommerce sites.

Jeff