'In Python, how can I get the global variables that are used in a function?
I'm trying to collect info on crashes and I am having trouble figuring out how to get the globals that are being used in the crashed function.
import inspect
fun = 222
other = "junk"
def test():
global fun
harold = 888 + fun
try:
harold/0
except:
frames = inspect.trace()
print "Local variables:"
print frames[0][0].f_locals
print "All global variables, not what I want!"
print frames[0][0].f_globals
test()
test() only uses "fun" but f_globals gives all the available globals. Is there some way to get just the globals that are being used by this function?
Solution 1:[1]
Check this out
a = 10
def test():
global a
a = 12
b = 12
print "co_argcount = ",test.__code__.co_argcount
print "co_cellvars = ",test.__code__.co_cellvars
print "co_code = ",test.__code__.co_code
print "co_consts = ",test.__code__.co_consts
print "co_filename = ",test.__code__.co_filename
print "co_firstlineno = ",test.__code__.co_firstlineno
print "co_flags = ",test.__code__.co_flags
print "co_freevars = ",test.__code__.co_freevars
print "co_lnotab = ",test.__code__.co_lnotab
print "co_name = ",test.__code__.co_name
print "co_names = ",test.__code__.co_names
print "co_nlocals = ",test.__code__.co_nlocals
print "co_stacksize = ",test.__code__.co_stacksize
print "co_varnames = ",test.__code__.co_varnames
Solution 2:[2]
I needed that also myself. This is my solution. The non-fast path covers most cases you are probably interested in.
def iterGlobalsUsedInFunc(f, fast=False, loadsOnly=True):
if hasattr(f, "func_code"): code = f.func_code
else: code = f
if fast:
# co_names is the list of all names which are used.
# These are mostly the globals. These are also attrib names, so these are more...
for name in code.co_names:
yield name
else:
# Use the disassembly. Note that this will still not
# find dynamic lookups to `globals()`
# (which is anyway not possible to detect always).
import dis
ops = ["LOAD_GLOBAL"]
if not loadsOnly:
ops += ["STORE_GLOBAL", "DELETE_GLOBAL"]
ops = map(dis.opmap.__getitem__, ops)
i = 0
while i < len(code.co_code):
op = ord(code.co_code[i])
i += 1
if op >= dis.HAVE_ARGUMENT:
oparg = ord(code.co_code[i]) + ord(code.co_code[i+1])*256
i += 2
else:
oparg = None
if op in ops:
name = code.co_names[oparg]
yield name
# iterate through sub code objects
import types
for subcode in code.co_consts:
if isinstance(subcode, types.CodeType):
for g in iterGlobalsUsedInFunc(subcode, fast=fast, loadsOnly=loadsOnly):
yield g
An updated version might be here.
My use case:
I have some module (songdb) which has some global database objects and I wanted to lazily load them once I called a function which uses the global database variable. I could have manually decorated such functions with a lazy loader or I could automatically detect which functions need it by my iterGlobalsUsedInFunc function.
This is basically the code (full code; was actually extended for classes now), where init automatically decorates such functions:
DBs = {
"songDb": "songs.db",
"songHashDb": "songHashs.db",
"songSearchIndexDb": "songSearchIndex.db",
}
for db in DBs.keys(): globals()[db] = None
def usedDbsInFunc(f):
dbs = []
for name in utils.iterGlobalsUsedInFunc(f, loadsOnly=True):
if name in DBs:
dbs += [name]
return dbs
def init():
import types
for fname in globals().keys():
f = globals()[fname]
if not isinstance(f, types.FunctionType): continue
dbs = usedDbsInFunc(f)
if not dbs: continue
globals()[fname] = lazyInitDb(*dbs)(f)
def initDb(db):
if not globals()[db]:
globals()[db] = DB(DBs[db])
def lazyInitDb(*dbs):
def decorator(f):
def decorated(*args, **kwargs):
for db in dbs:
initDb(db)
return f(*args, **kwargs)
return decorated
return decorator
Another solution would have been to use an object proxy which lazily loads the database. I have used that elsewhere in this project, thus I have also implemented such object proxy; if you are interested, see here: utils.py:ObjectProxy.
Solution 3:[3]
A dirty way would be to use inspect.getsourcelines() and search for lines containing global <varname>. There are no good methods for this, at least not in inspect module.
Solution 4:[4]
As you already found out, the property f_globals gives you the global namespace in which the function was defined.
From what I can see, the only way to find out which global variables are actually used is to disassemble the function's byte code with dis; look for the byte codes STORE_NAME, STORE_GLOBAL, DELETE_GLOBAL, etc.
Solution 5:[5]
If using Colab / Jupyter
In one cell you run dis redirecting the output to a variable
%%capture dis_output
func_to_check=my_own_function
# dis : Disassembler for Python bytecode
from dis import dis
dis(func_to_check)
And then you can filter its content to detect the use of GLOBALS. Here an example
Version 1 (safer than v2)
# Then grep will find the use of GLOBALS
# Method 1 (kind of safer)
with open('dis_output.txt', 'w') as f:
f.writelines(dis_output.stdout)
! cat dis_output.txt | grep -i global
Version 2 (not so safe)
# Method 2 (not so safe)
! echo "{dis_output.stdout}" | grep -i global
Results (example)
14 4 LOAD_GLOBAL 0 (AudioLibrary)
36 LOAD_GLOBAL 2 (userLanguageAudio)
50 LOAD_GLOBAL 2 (userLanguageAudio)
62 LOAD_GLOBAL 3 (LESSON_FILES_DIR)
27 76 LOAD_GLOBAL 4 (get_ipython)
30 96 LOAD_GLOBAL 6 (pread)
31 104 LOAD_GLOBAL 7 (print)
34 >> 124 LOAD_GLOBAL 7 (print)
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Rumple Stiltskin |
| Solution 2 | |
| Solution 3 | pajton |
| Solution 4 | Aaron Digulla |
| Solution 5 | Community |