76 lines
2.4 KiB
Plaintext
76 lines
2.4 KiB
Plaintext
From: arw at ifu.net (arw)
|
|
Date: Fri, 9 Apr 1999 16:20:00 GMT
|
|
Subject: GadFly - MemoryError
|
|
Message-ID: <199904091623.MAA24288@ifu.ifu.net>
|
|
Content-Length: 2193
|
|
X-UID: 1629
|
|
|
|
My guess is that you have a lot of url matches and
|
|
gadfly is doing what you ask it to do. Could be wrong.
|
|
Rewriting the query might help, as suggested elsewhere.
|
|
If you want to discuss this further I don't think we need
|
|
copy the list. With 3000 entries you have a potential
|
|
9M matches.
|
|
|
|
Please try this
|
|
|
|
select url, count(*) from table group by url order by 1 desc
|
|
|
|
And have a look at how many redundancies you have. This
|
|
query should run faster, I hope.
|
|
|
|
We could take this kind of discussion offline, if you like. :)
|
|
-- Aaron Watters
|
|
|
|
----Original Message-----
|
|
>From: Oleg Broytmann <phd at sun.med.ru>
|
|
>To: arw <arw at ifu.net>
|
|
>Cc: Python Mailing List <python-list at cwi.nl>
|
|
>Subject: RE: GadFly - MemoryError
|
|
>Reply-To: phd at sun.med.ru
|
|
>Date: Wednesday, April 07, 1999 4:16 AM
|
|
>
|
|
>Hello!
|
|
>
|
|
>On Mon, 5 Apr 1999, arw wrote:
|
|
>> I don't know why you have this problem.
|
|
>> Gadfly should not construct the cross product with
|
|
>> the equality you mention present.
|
|
>> (b1.URL = b2.URL)
|
|
>
|
|
> I freed some memory and the program worked. It ate 30 Megs while running,
|
|
>so I was in need of memory. But it ran 30 minutes (on 3000 rows!)
|
|
>
|
|
>> Please try adding an index on URL on the table
|
|
>> in question. Also, maybe try increasing your virtual
|
|
>> memory (page file size, or whatever).
|
|
>
|
|
> Indices or compiled kjbuckets was not of big help - instead of running
|
|
>35 minutes the program ran 30. I think, my computer spent more time
|
|
>swapping, not working.
|
|
>
|
|
>> Good luck. fwiw, I've definitely run tables with
|
|
>> 100k rows in Gadfly on a 64Meg machine.
|
|
>> -- Aaron Watters
|
|
>
|
|
> You are in luck.
|
|
>
|
|
>> ps: let me know how it goes.
|
|
>
|
|
> Not so good. I think I would not use gadfly, at least for this project
|
|
>(currently I am using very simple "database", that stored in text files. Of
|
|
>course, I cannot make complex requests, sucj as joins).
|
|
>
|
|
>Oleg.
|
|
>----
|
|
> Oleg Broytmann National Research Surgery Centre http://sun.med.ru/~phd/
|
|
> Programmers don't die, they just GOSUB without RETURN.
|
|
>
|
|
>
|
|
>
|
|
|
|
|
|
|
|
|
|
|