Re: [mv] Help with external export scripts for databases

[Ed LaFrance <edl@newmediaems.com>]

******    message to minivend-users from Ed LaFrance <edl@newmediaems.com>     ******

At 06:25 PM 11/30/00 -0500, you wrote:
>******    message to minivend-users from Mike Heins 
><mikeh@minivend.com>     ******
>
>Quoting Ed LaFrance (edl@newmediaems.com):
> >
> > I am working on some Perl scripts for exporting gdbm and db databases for
> > an older, busy minivend site (3.14), the idea being that these could be 
> run
> > externally from command line, crontab, etc 'offline' from the site.

[snip]


>I have always exported by placing in a page:
>
>[if session arg eq password]
>Export
>     [loop arg="products pricing transactions etc etc"]
>         [loop-code]=[tag export [loop-code] /tmp/[loop-code].tab TAB][/tag]
>     [/loop]
>[else]
>Need the password sent as argument.
>[/else]
>[/if]
>
>Then you just run a cron job that calls:
>
>         GET http://localhost/cgi-bin/catalog/pagename?;password
>
>Much easier than writing a script, so much so that I never bothered
>to write one. 8-)

[snip]

...Easier, but vunerable in this case.  The problem is that their databases 
are so huge that this method always ends with hammered session error in 
error.log and an internal server error in the browser, even with Hammerlock 
and PidCheck set to 300 seconds or more - and this is with just one of the 
big files, there are several - in the interative loop example above, it 
would be all over before it even started.  I am guessing that 'GET' method 
would be vunerable in this way as well.  We are working on some 
preventative measures, like rotating out old data to prevent data bloat in 
the first place, etc, and the machine is quite robust: multi-Gig memory, 
quad processors, Ultra-wide SCSI HD, etc.

So being able to export the data from outside Minivend would be nice. 
Dumping the contents of a straight DB or GDBM file is basic; unpacking the 
value element of the GDBM which have been packed with Storable is giving 
the script fits and I can't figure out why.  If anyone out there has done 
this, I would very much appreciate hearing from you.  I have various 
things, following the example of what is done in Vend/Table/GDBM.pm, but to 
no avail:

Storable binary image v56.97 more recent than I am (v1.0) at 
blib/lib/Storable.pm (autosplit into blib/lib/auto/Storable/thaw.al) line 
238, at export_gdbm line 19



- Ed L.

 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
New Media E.M.S.               Software Solutions for Business
463 Main St., Suite D          edl@newmediaems.com
Placerville, CA  95667         http://www.newmediaems.com
(530) 622-9421                 (530) 622-9426 Fax
(866) 519-4680 Toll-Free
 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

-
To unsubscribe from the list, DO NOT REPLY to this message.  Instead, send
email with 'UNSUBSCRIBE minivend-users' in the body to Majordomo@minivend.com.
Archive of past messages: http://www.minivend.com/minivend/minivend-list