From: arw at ifu.net (arw)
Date: Mon, 5 Apr 1999 16:37:00 GMT
Subject: GadFly - MemoryError
Message-ID: <199904051641.MAA12603@ifu.ifu.net>
Content-Length: 1862
X-UID: 1704                                                  

I don't know why you have this problem.
Gadfly should not construct the cross product with
the equality you mention present.
(b1.URL = b2.URL)

Please try adding an index on URL on the table
in question.  Also, maybe try increasing your virtual
memory (page file size, or whatever).

Good luck.  fwiw, I've definitely run tables with
100k rows in Gadfly on a 64Meg machine.
   -- Aaron Watters

ps: let me know how it goes.

----Original Message-----
   >From:     	Oleg Broytmann <phd at sun.med.ru>
   >To:         	arw at ifu.net
   >Cc:         	Python Mailing List <python-list at cwi.nl>
   >Subject:     	GadFly - MemoryError
   >Reply-To:     	phd at sun.med.ru
   >Date:    	Saturday, April 03, 1999 9:35 AM
   >
   >Hello!
   >
   >   I tried to add yeat another database backend to my project "Bookmarks
   >database". My database contains now about 3000 URLs, not too much, I think.
   >I subclass by BookmarksParser to parse bookmarks.html into gadfly database
   >and got a database of 500 Kbytes - very small database, I hope.
   >   Then I tried to find duplicates (there are duplicates). I ran the query:
   >
   >SELECT b1.rec_no, b2.rec_no, b1.URL
   >   FROM bookmarks b1, bookmarks b2
   >WHERE b1.URL = b2.URL
   >AND   b1.rec_no < b2.rec_no
   >
   >   GadFly eats all memory and fails with MemoryError. Is it that gadfly
   >really constructs multiplication of b1 and b2, thus getting 3000*3000
   >pairs? Or I just did something wrong?
   >   I tried to create indicies, but this was not of any help.
   >
   >   BTW, where are indicies stored? After creating ones I didn't find
   >additional files - only one file in the database changed.
   >
   >Oleg.
   >---- 
   >    Oleg Broytmann  National Research Surgery Centre  http://sun.med.ru/~phd/
   >           Programmers don't die, they just GOSUB without RETURN.
   >
   >
   >





