68 lines
2.8 KiB
Plaintext
68 lines
2.8 KiB
Plaintext
From: tismer at appliedbiometrics.com (Christian Tismer)
|
|
Date: Mon, 19 Apr 1999 21:06:06 GMT
|
|
Subject: Memory and swapping question
|
|
References: <371B5ED8.A9C82170@appliedbiometrics.com> <7fg1ep$t5s$1@nnrp1.dejanews.com>
|
|
Message-ID: <371B9ABE.3D2D8543@appliedbiometrics.com>
|
|
Content-Length: 2517
|
|
X-UID: 67
|
|
|
|
|
|
aaron_watters at my-dejanews.com wrote:
|
|
>
|
|
> In article <371B5ED8.A9C82170 at appliedbiometrics.com>,
|
|
> Christian Tismer <tismer at appliedbiometrics.com> wrote:
|
|
> > due to a question which came up in the tutor list, I'd like
|
|
> > to ask if somebody can explain the following:....
|
|
>
|
|
> <chomp><chomp>timings on making huge lists of integers...
|
|
>
|
|
> > On my system, creation takes about 10 times as for big/2,
|
|
> > this is ok. But the del takes at least three times as long.
|
|
> > Besides the fact that integers are never really disposed but
|
|
> > build up a freelist, why is deletion so much slower now?
|
|
>
|
|
> Could be wrong, but this may be a case of a famous database
|
|
> problem. The OS (typically) swaps out pages by picking the "least recently
|
|
> used" page, but when you are decreffing (scanning) a HUGE list of sequentially
|
|
> allocated objects this guarantees that the page you need next will
|
|
> be swapped out by the time you get to it. Yikes! Allocation is faster
|
|
> because you are really only "paging things out" (the first time)
|
|
> and the write to the disk can be buffered until the disk is
|
|
> ready, allowing the program to proceed (?I think?).
|
|
|
|
Exactly.
|
|
In this case, things are even worse:
|
|
Iin the de-allocation phase, the internal integer cache
|
|
is scanned in order, to return the integers to the
|
|
freelist. This treats a stack-like structure as a heap,
|
|
making integer deallocation like a list.reverse() on
|
|
the cache file.
|
|
It also applies to other objects which are created
|
|
in-order and referenced by the list. The same disk trashing.
|
|
Which means that the malloc routines use a similar,
|
|
stack-like freelist, at least on Win98.
|
|
|
|
> This is one reason why Oracle and Sybase, etc, like to do their own
|
|
> memory and disk management ("gimme them sectors -- don't need no
|
|
> g.d. filesystem, thanks!"). Just a guess, but a not completely
|
|
> uneducated one.
|
|
|
|
Well, I changed the deallocation strategy of lists to free
|
|
objects beginning from the end. (1 line in listobject.c)
|
|
Now, all my examples run as expected, with del no longer
|
|
being more expensive than create.
|
|
|
|
cheers - chris
|
|
|
|
--
|
|
Christian Tismer :^) <mailto:tismer at appliedbiometrics.com>
|
|
Applied Biometrics GmbH : Have a break! Take a ride on Python's
|
|
Kaiserin-Augusta-Allee 101 : *Starship* http://starship.python.net
|
|
10553 Berlin : PGP key -> http://wwwkeys.pgp.net
|
|
PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF
|
|
we're tired of banana software - shipped green, ripens at home
|
|
|
|
|
|
|
|
|