253 lines
7.9 KiB
Plaintext
253 lines
7.9 KiB
Plaintext
From: mwh21 at cam.ac.uk (Michael Hudson)
|
|
Date: 30 Apr 1999 16:11:39 +0100
|
|
Subject: Python IS slow ! [was] Re: Python too slow for real world
|
|
References: <613145F79272D211914B0020AFF6401914DAD8@gandalf.digicool.com> <p5g15lmb35.fsf@bidra241.bbn.hp.com> <slrn7ieipq.8uk.wtanksle@dolphin.openprojects.net> <7g9qmo$luf$1@news.udel.edu> <372965CA.2B3FD2FE@appliedbiometrics.com> <19990430080805.B776752@vislab.epa.gov> <3729A9F3.75B2692B@appliedbiometrics.com>
|
|
Message-ID: <m3u2tyw3bo.fsf@atrus.jesus.cam.ac.uk>
|
|
Content-Length: 7498
|
|
X-UID: 1248
|
|
|
|
Christian Tismer <tismer at appliedbiometrics.com> writes:
|
|
|
|
> Randall Hopper wrote:
|
|
> >
|
|
> > Christian Tismer:
|
|
> > |Terry Reedy wrote:
|
|
> > |> A batch-mode optimizer analyzing an entire file (module) should be able to
|
|
> > |> detect whether or not function names are rebound.
|
|
> > |>
|
|
> > |> Perhaps module bindings should be considered immutable from outside the
|
|
> > |> module unless explicitly declared otherwise.
|
|
> > |
|
|
> > |I'm thinking of no new keyword, but a mechanism which allows me to lock a
|
|
> > |namespace somehow.
|
|
> >
|
|
> > I like this idea in concept. Though I would prefer a way to have
|
|
> > namespaces "lock by default". Examples: After a class definition, the
|
|
> > class function dictionary is locked. After a module is fully read, all
|
|
> > references are bound and the module namespace is locked. etc.
|
|
>
|
|
> Well, I wouldn't do that by default. By default, everything
|
|
> could stay as it is. First of all, this would not break any
|
|
> existing code. Then, many people will want to
|
|
> fine tune their modules, and they are perhaps not done
|
|
> after a class definition was ready.
|
|
>
|
|
> Then, what would you do with classes which depend on each
|
|
> other? You cannot lock them immediately, this would fail.
|
|
> Locking them after they both are defined is fine, since
|
|
> everything is defined then. With minimum effort and no
|
|
> language changes, this will be needed.
|
|
>
|
|
> Then think of all the more difficult systems which need
|
|
> more effort to become configured. The xml parsers together
|
|
> with SAX are an example. If I wanted to lock this, then
|
|
> this must be done with care. One would also not lock the mixin
|
|
> classes, but only the final workhorse class, bound with
|
|
> the correctly selected parser, and so on.
|
|
>
|
|
> It might also be necessary to find a way to specify which
|
|
> attributes may be locked and which not, since there exist
|
|
> indeed cases where Python's super-flexibility is needed :-)
|
|
>
|
|
> Depending on how exactly will be implemented, a single line
|
|
> at the end of a module should suffice to accomplish this stuff
|
|
> for the standard cases. Fine adjustment would take a little more.
|
|
> As a side effect, locking a module would also find all
|
|
> referenced but undefined symbols.
|
|
>
|
|
> Anyway, this is still no cakewalk and quite a lot of code
|
|
> is involved. Needs much more thinking...
|
|
|
|
I think I can do this. Not to classes yet, but given a function, I can
|
|
make you another function where all the global lookups are bound to
|
|
the values found at that moment.
|
|
|
|
It's used like this:
|
|
|
|
>>> def f(x):
|
|
... return x+y
|
|
...
|
|
>>> import closure
|
|
>>> g=closure.bind(f,y=1)
|
|
>>> f(1)
|
|
Traceback (innermost last):
|
|
File "<stdin>", line 1, in ?
|
|
File "<stdin>", line 2, in f
|
|
NameError: y
|
|
>>> g(1)
|
|
2
|
|
|
|
It's then easy to write a function that binds all the variables found
|
|
in the current environment:
|
|
|
|
def bind_now(func):
|
|
try:
|
|
raise ""
|
|
except:
|
|
import sys
|
|
frame=sys.exc_traceback.tb_frame.f_back
|
|
l=apply(bind,(func,),frame.f_locals)
|
|
g=apply(bind,(l,),frame.f_globals)
|
|
return g
|
|
|
|
This gets used like so:
|
|
|
|
>>> import closure
|
|
>>> y=1
|
|
>>> def f(x):
|
|
... return x+y
|
|
...
|
|
>>> f(0)
|
|
1
|
|
>>> g=closure.bind_now (f)
|
|
>>> y=2
|
|
>>> f(0)
|
|
2
|
|
>>> g(0)
|
|
1
|
|
>>>
|
|
|
|
Is this what you wanted?
|
|
|
|
A word of warning: this code is nasty, *nasty*, NASTY. Possibly the
|
|
most horrible thing you will see perpetrated in Python this year. It
|
|
applies regular expressions to strings of bytecode...
|
|
|
|
I made Python core repeatedly when debugging it.
|
|
|
|
However, it works. The returned functions are very fast. I wrote this
|
|
package because I wanted to avoid both the tackiness of the `default
|
|
argument hack' and the performance penalty of using classes to fake
|
|
closures.
|
|
|
|
As to `sealing' classes in this fashion, I guess it could be
|
|
done. You'd need to look for patterns of LOAD_FAST 0 (the first
|
|
argument is the zeroth local) followed by LOAD_ATTR. You could then
|
|
calculate this and insert a LOAD_CONST instead. The thing is, this
|
|
would replace two argumented bytecode with one, changing the length of
|
|
the codestring and you'd need to recompute jumps. I haven't had the
|
|
bloody-mindedness to get this to work yet.
|
|
|
|
Code follows...
|
|
|
|
HTH
|
|
Michael
|
|
|
|
import new,string,re
|
|
|
|
def copy_code_with_changes(codeobject,
|
|
argcount=None,
|
|
nlocals=None,
|
|
stacksize=None,
|
|
flags=None,
|
|
code=None,
|
|
consts=None,
|
|
names=None,
|
|
varnames=None,
|
|
filename=None,
|
|
name=None,
|
|
firstlineno=None,
|
|
lnotab=None):
|
|
if argcount is None: argcount = codeobject.co_argcount
|
|
if nlocals is None: nlocals = codeobject.co_nlocals
|
|
if stacksize is None: stacksize = codeobject.co_stacksize
|
|
if flags is None: flags = codeobject.co_flags
|
|
if code is None: code = codeobject.co_code
|
|
if consts is None: consts = codeobject.co_consts
|
|
if names is None: names = codeobject.co_names
|
|
if varnames is None: varnames = codeobject.co_varnames
|
|
if filename is None: filename = codeobject.co_filename
|
|
if name is None: name = codeobject.co_name
|
|
if firstlineno is None: firstlineno = codeobject.co_firstlineno
|
|
if lnotab is None: lnotab = codeobject.co_lnotab
|
|
return new.code(argcount,
|
|
nlocals,
|
|
stacksize,
|
|
flags,
|
|
code,
|
|
consts,
|
|
names,
|
|
varnames,
|
|
filename,
|
|
name,
|
|
firstlineno,
|
|
lnotab)
|
|
|
|
def encode_16(n):
|
|
return '\\%03o\\%03o'%(n%256,n/256)
|
|
|
|
LOAD_CONST=chr(100)
|
|
LOAD_GLOBAL=chr(116)
|
|
|
|
def munge_code(code,vars):
|
|
codestring=code.co_code
|
|
names=list(code.co_names)
|
|
consts=list(code.co_consts)
|
|
for var,value in vars.items():
|
|
try:
|
|
index=names.index(var)
|
|
except ValueError:
|
|
continue
|
|
codestring=re.sub(LOAD_GLOBAL+encode_16(index),
|
|
LOAD_CONST+encode_16(len(consts)),
|
|
codestring)
|
|
consts.append(value)
|
|
return copy_code_with_changes(
|
|
code,
|
|
consts=tuple(consts),
|
|
code=codestring)
|
|
|
|
def bind(func,**vars):
|
|
newfunc=new.function(
|
|
munge_code(func.func_code,vars),
|
|
func.func_globals,
|
|
func.func_name)
|
|
newfunc.__doc__=func.__doc__
|
|
newfunc.func_defaults=func.func_defaults
|
|
newfunc.func_doc=func.func_doc
|
|
return newfunc
|
|
|
|
def bind_locals(func):
|
|
try:
|
|
raise ""
|
|
except:
|
|
import sys
|
|
frame=sys.exc_traceback.tb_frame.f_back
|
|
l=apply(bind,(func,),frame.f_locals)
|
|
frame=None
|
|
return l
|
|
|
|
def bind_now(func):
|
|
try:
|
|
raise ""
|
|
except:
|
|
import sys
|
|
frame=sys.exc_traceback.tb_frame.f_back
|
|
l=apply(bind,(func,),frame.f_locals)
|
|
g=apply(bind,(l,),frame.f_globals)
|
|
return g
|
|
|
|
## examples
|
|
|
|
def make_adder(n):
|
|
def adder(x):
|
|
return x+n
|
|
return bind_locals(adder)
|
|
|
|
def make_balance(initial_amount):
|
|
def withdraw(amount):
|
|
if current[0]<amount:
|
|
raise "debt!"
|
|
else:
|
|
current[0]=current[0]-amount
|
|
return current[0]
|
|
return bind
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|