[ic] Newbie, need info on importing 500,000+ items

[BF]

On Mon, 15 Jan 2001, Company InterNet Services wrote:

> I'm a webmaster for a large bookstore and want to upgrade our database system 
> (speed). I've read about the minivend system though never had a change
> to get it.
> 
> I seen the demo allows you to add 'Single' items, however I have nearly half 
> a million items currently in a flatfile database Pipe(|) delimited. (I
> can have it delimited with any char. if needed)
> 
> I looked though all the archives but didn't find much that could help me. 
> (mostly excel export problems)
> 
> When this question is answered (and no problems are foreseen) I'll go ahead 
> and install the software (I don't want to install then uninstall if it
> is not the solution I need)
> 
> Ken Lyons
> CIS-Tech
> Webmaster@ChristianBookServices.com
> 
> 

The import/export shouldn't be the problem. But how to edit a single
record with the new IC admin UI, is not yet clear to me. 

There is one answer of Mike Heins with regards to large product tables
which gives a hint. 

Mike Heins wrote (Jan. 10th, Subject line: [ic]/[mv] Large Product tables

> It is easy to shoot yourself in the foot with any application and a
> large database. I know, I have done it many times. Interchange's admin
> UI is optimized for DBs that are 50,000 records or less. If you have
> that many records, you need SQL to use the IC admin tool with any
> success. Even then, you have to make sure that proper indexing is done
> on all fields that are selected/sorted on.
>
> [snip] 
>
> The Interchange application itself will not repeal the laws of physics,
> but if used in a reasonable manner it is capable of working with very
> large database tables. There are sites which have as many as a million
> items, and at least one that has 500,000 and uses GDBM. They don't try
> and run the item editor on that, though! And they use external
> import/export  programs and don't try to do auto-import and export.

He gave this answer with regards to product tables of 3 million records.
There is no problem to run a GDBM table with 500 000 60 800 000 records,
when you index on all fields you want to search for, use binary searches
and use the static page build feature in addition. 

With Mike's answer I am still not clear, if, in order to use IC's new
admin UI and be able to edit single records, he advises to split
a 500 000 records product table in 10 separated  50 000 record tables
and go from there.

I wonder if there is are bookstores using the older MV releases who have
already upgraded to IC and use its admin UI. Would be nice to hear from
them about their experience.
 
Birgitt