[ic] Newbie, need info on importing 500,000+ items

[jojo@buchonline.net]

On 15 Jan, Company InterNet Services wrote:
> I'm a webmaster for a large bookstore and want to upgrade our database system (speed). I've read about the minivend system though
> never had a change to get it.
> 
> I seen the demo allows you to add 'Single' items, however I have nearly half a million items currently in a flatfile database
> Pipe(|) delimited. (I can have it delimited with any char. if needed)
> 
> I looked though all the archives but didn't find much that could help me. (mostly excel export problems)
> 
> When this question is answered (and no problems are foreseen) I'll go ahead and
> install the software (I don't want to install then uninstall if it is not the solution I need)
> 
> Ken Lyons
> CIS-Tech
> Webmaster@ChristianBookServices.com

>From the doc (old Interchange Version 4.5)
-----snip-----
Interchange Database Operation

NOTE: In the following descriptions, we will use the following terms somewhat interchangeably: 

key, code

      Either one is a reference to the key for the database. In Interchange this is often the product code or SKU, which is the part number for the product.
      Other key values may be used to generate relationships to other database tables. 

      It is required that the key be the first column of an ASCII source file for GDBM, Berkeley DB, or in-memory built-in database formats. It is also strongly
      suggested that you keep that practice for SQL databases, since Interchange's import, export, and search facilities will work much better with that practice.
field, column

      This is a column of the database. One of the columns is always the key -- Interchange prefers that the key be the first column. Field is an
      interchangeable reference. 
table, database

      A table in the database. Because of the evolution of Interchange from a single-table database to an access method for an unlimited number of tables (and
      databases, for that matter), we will sometimes refer to a table as a database. The only time database refers to something different is when describing that
      concept as it relates to SQL -- where a database contains a series of tables. Interchange cannot create SQL databases, but given the proper
      permissions it may drop and create tables within that database. 

If necessary, Interchange reads the data to place in tables from standard ASCII-delimited files. All of these ASCII source files are kept in the products directory,
normally products in the catalog directory (where catalog.cfg is). 

The ASCII files can have ^M (carriage return) characters if desired, but must have a newline character at the end of the line to work -- Mac users uploading
files must use ASCII mode, not binary mode! 

Interchange's default ASCII delimiter is TAB. 

IMPORTANT NOTE: The items must be separated by a single delimiter. The items are lined up for your reading convenience. 

TAB

      Fields separated by ^I characters. No whitespace should be at the beginning of the line. 

          code    description             price   image
          SH543   Men's fine cotton shirt 14.95   shirts.jpg

PIPE

      Fields separated by | characters. No whitespace should be at the beginning of the line. 

          code|description|price|image
          SH543|Men's fine cotton shirt|14.95|shirts.jpg

CSV

      Fields enclosed in quotes, separated by commas. No whitespace should be at the beginning of the line. 

          "code","description","price","image"
          "SH543","Men's fine cotton shirt","14.95","shirts.jpg"

NOTE: Using the default TAB delimiter is highly recommended if you are planning on searching the ASCII source file of the database. PIPE works fairly well, but
CSV delimiter schemes cause problems with searching. 
-----snip-----

Joachim

-- 
Hans-Joachim Leidinger
buch online                 jojo@buchonline.net
Munscheidstr. 14            FAX: +49 209 1671441
45886 Gelsenkirchen         FAX: 0209 1671441