From: mwh21 at cam.ac.uk (Michael Hudson) Date: 30 Apr 1999 16:11:39 +0100 Subject: Python IS slow ! [was] Re: Python too slow for real world References: <613145F79272D211914B0020AFF6401914DAD8@gandalf.digicool.com> <7g9qmo$luf$1@news.udel.edu> <372965CA.2B3FD2FE@appliedbiometrics.com> <19990430080805.B776752@vislab.epa.gov> <3729A9F3.75B2692B@appliedbiometrics.com> Message-ID: Content-Length: 7498 X-UID: 1248 Christian Tismer writes: > Randall Hopper wrote: > > > > Christian Tismer: > > |Terry Reedy wrote: > > |> A batch-mode optimizer analyzing an entire file (module) should be able to > > |> detect whether or not function names are rebound. > > |> > > |> Perhaps module bindings should be considered immutable from outside the > > |> module unless explicitly declared otherwise. > > | > > |I'm thinking of no new keyword, but a mechanism which allows me to lock a > > |namespace somehow. > > > > I like this idea in concept. Though I would prefer a way to have > > namespaces "lock by default". Examples: After a class definition, the > > class function dictionary is locked. After a module is fully read, all > > references are bound and the module namespace is locked. etc. > > Well, I wouldn't do that by default. By default, everything > could stay as it is. First of all, this would not break any > existing code. Then, many people will want to > fine tune their modules, and they are perhaps not done > after a class definition was ready. > > Then, what would you do with classes which depend on each > other? You cannot lock them immediately, this would fail. > Locking them after they both are defined is fine, since > everything is defined then. With minimum effort and no > language changes, this will be needed. > > Then think of all the more difficult systems which need > more effort to become configured. The xml parsers together > with SAX are an example. If I wanted to lock this, then > this must be done with care. One would also not lock the mixin > classes, but only the final workhorse class, bound with > the correctly selected parser, and so on. > > It might also be necessary to find a way to specify which > attributes may be locked and which not, since there exist > indeed cases where Python's super-flexibility is needed :-) > > Depending on how exactly will be implemented, a single line > at the end of a module should suffice to accomplish this stuff > for the standard cases. Fine adjustment would take a little more. > As a side effect, locking a module would also find all > referenced but undefined symbols. > > Anyway, this is still no cakewalk and quite a lot of code > is involved. Needs much more thinking... I think I can do this. Not to classes yet, but given a function, I can make you another function where all the global lookups are bound to the values found at that moment. It's used like this: >>> def f(x): ... return x+y ... >>> import closure >>> g=closure.bind(f,y=1) >>> f(1) Traceback (innermost last): File "", line 1, in ? File "", line 2, in f NameError: y >>> g(1) 2 It's then easy to write a function that binds all the variables found in the current environment: def bind_now(func): try: raise "" except: import sys frame=sys.exc_traceback.tb_frame.f_back l=apply(bind,(func,),frame.f_locals) g=apply(bind,(l,),frame.f_globals) return g This gets used like so: >>> import closure >>> y=1 >>> def f(x): ... return x+y ... >>> f(0) 1 >>> g=closure.bind_now (f) >>> y=2 >>> f(0) 2 >>> g(0) 1 >>> Is this what you wanted? A word of warning: this code is nasty, *nasty*, NASTY. Possibly the most horrible thing you will see perpetrated in Python this year. It applies regular expressions to strings of bytecode... I made Python core repeatedly when debugging it. However, it works. The returned functions are very fast. I wrote this package because I wanted to avoid both the tackiness of the `default argument hack' and the performance penalty of using classes to fake closures. As to `sealing' classes in this fashion, I guess it could be done. You'd need to look for patterns of LOAD_FAST 0 (the first argument is the zeroth local) followed by LOAD_ATTR. You could then calculate this and insert a LOAD_CONST instead. The thing is, this would replace two argumented bytecode with one, changing the length of the codestring and you'd need to recompute jumps. I haven't had the bloody-mindedness to get this to work yet. Code follows... HTH Michael import new,string,re def copy_code_with_changes(codeobject, argcount=None, nlocals=None, stacksize=None, flags=None, code=None, consts=None, names=None, varnames=None, filename=None, name=None, firstlineno=None, lnotab=None): if argcount is None: argcount = codeobject.co_argcount if nlocals is None: nlocals = codeobject.co_nlocals if stacksize is None: stacksize = codeobject.co_stacksize if flags is None: flags = codeobject.co_flags if code is None: code = codeobject.co_code if consts is None: consts = codeobject.co_consts if names is None: names = codeobject.co_names if varnames is None: varnames = codeobject.co_varnames if filename is None: filename = codeobject.co_filename if name is None: name = codeobject.co_name if firstlineno is None: firstlineno = codeobject.co_firstlineno if lnotab is None: lnotab = codeobject.co_lnotab return new.code(argcount, nlocals, stacksize, flags, code, consts, names, varnames, filename, name, firstlineno, lnotab) def encode_16(n): return '\\%03o\\%03o'%(n%256,n/256) LOAD_CONST=chr(100) LOAD_GLOBAL=chr(116) def munge_code(code,vars): codestring=code.co_code names=list(code.co_names) consts=list(code.co_consts) for var,value in vars.items(): try: index=names.index(var) except ValueError: continue codestring=re.sub(LOAD_GLOBAL+encode_16(index), LOAD_CONST+encode_16(len(consts)), codestring) consts.append(value) return copy_code_with_changes( code, consts=tuple(consts), code=codestring) def bind(func,**vars): newfunc=new.function( munge_code(func.func_code,vars), func.func_globals, func.func_name) newfunc.__doc__=func.__doc__ newfunc.func_defaults=func.func_defaults newfunc.func_doc=func.func_doc return newfunc def bind_locals(func): try: raise "" except: import sys frame=sys.exc_traceback.tb_frame.f_back l=apply(bind,(func,),frame.f_locals) frame=None return l def bind_now(func): try: raise "" except: import sys frame=sys.exc_traceback.tb_frame.f_back l=apply(bind,(func,),frame.f_locals) g=apply(bind,(l,),frame.f_globals) return g ## examples def make_adder(n): def adder(x): return x+n return bind_locals(adder) def make_balance(initial_amount): def withdraw(amount): if current[0]