python script that imports matplotlib succeeds butfrozen binary of script fails

Question

My script needed to import numpy, sklearn, and matplotlib but I couldn't install sklearn. A very helpful response to my question http://https://stackoverflow.com/questions/38733220/difference-between-scikit-learn-and-sklearn explained that I needed to reinstall numpy. Using pip to update numpy failed because OS X 10.11 SIP prevented uninstalling the current numpy. The very helpful answer to a question about pip and SIP by mfripp http://https://apple.stackexchange.com/questions/209572/how-to-use-pip-after-the-os-x-el-capitan-upgrade provided a detailed solution to the problem. I followed those instruction exactly and used pip to reinstall numpy, matplotlib, scipy and sklearn for all users.

When I ran my completed script using the command

python DistMatPlot.py Random10A.matrix Random10A.pdf

the script ran perfectly, writing all expected output files.

However, I always saw:

"/Library/Python/2.7/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment. warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')"

which I had never seen with other matplotlib scripts before updating numpy, matplotlib, etc. The 2 second delay was only mildly annoying.

I compiled a frozen binary using pyinstaller and during the compiling I got several messages similar to that above. The resulting frozen binary run failed with the command:

./DistMatPlot Random10A.matrix Random10A.pdf

produced the following:

/var/folders/8x/7_zp_33h8xj6td0059b72p9h0000gp/T/_MEIhIysTV/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment. Traceback (most recent call last): File "", line 13, in File "/Library/Python/2.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module exec(bytecode, module.dict) File "matplotlib/pyplot.py", line 114, in File "matplotlib/backends/init.py", line 32, in pylab_setup File "/Library/Python/2.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module exec(bytecode, module.dict) File "matplotlib/backends/backend_macosx.py", line 24, in File "/Library/Python/2.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 546, in load_module module = imp.load_module(fullname, fp, filename, ext_tuple) RuntimeError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework. See the Python documentation for more information on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework, or try one of the other backends. If you are Working with Matplotlib in a virtual enviroment see 'Working with Matplotlib in Virtual environments' in the Matplotlib FAQ DistMatPlot returned -1

I have looked at similar questions and tried their suggested solutions to no avail.

(1) Why does matplotlib need to rebuild a font cache each time it is run?

(2) Why does the frozen binary fail when the script itself succeeds? Do I need some additional option other than -F when running pyinstaller?

Answer 1

The issue with the frozen binary turned out to be "RuntimeError: Python is not installed as a framework."

Several posts discussed that issue and suggested adding these two lines before "import matplotlib.pyplot as plt":

import matplotlib
matplotlib.use('TkAgg')

That did not work, but this slight modification did work: import matplotlib
matplotlib.use('Agg')

I suspect that 'Agg" may be specific to either OS X or to the version of python that is included with OS X 10.11

Answer 2

Pyinstaller can have problems compiling some scripts, particularly if the use either the scipy or matplotlib modules. Compiling appears to succeed, but the compiled script may fail to execute. If the error message indicates that a module, e,g, scipy.stats, could not be found try compiling with the hiddenimport option: pyinstaller -F --hiddenimport=scipy.stats --hiddenimport=_sysconfigdata GrowthRates.py

When compiling scripts that use scipy or matplotlib Anaconda python helps a lot: OSX: python27Env Windows: Use Anaconda python Linux: Use Anaconda python