Create conda environment: "Found conflicts!" when solving environment and "Finding shortest conflict path" running forever

Question

I was provided with an environment.ubuntu.yml file to be able to create a conda environment. However, running conda create env --file environment.ubuntu.yml I get the following output:

conda env create --file environment.ubuntu.yml
Collecting package metadata (repodata.json): done
Solving environment: - 
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
Examining fontconfig:   5%|▉                 | 10/202 [00:00<00:00, 5393.91it/ ]
Comparing specs that have this dependency:   0%|         | 0/12 [00:00<?, ?it/s]
Finding shortest confli|  path for fontconfig==2.13.1=he4413a7_1000:  12%|▏| 1/8
Finding shortest conflict path for fontconfig==2.13.1=he4413a7_1000:  25%|▎| 2/8
Finding shortest conflict pa/  for fontconfig[version='>=2.13.0,<3.0a0']:  25%|
Finding shortest conflict path for fontconfig[version='>=2.13.0,<3.0a0']:  38%|
Finding shortest conf- ct path for fontconfig[version='>=2.11.1']:  38%|▍| 3/8 [
Finding shortest conflict path for fontconfig[version='>=2.11.1']:  50%|▌| 4/8 [
Finding shortest conflict pa|  for fontconfig[version='>=2.12.4,<3.0a0']:  50%|
...

The Finding shortest conflict path goes on for a long time - apparently it is trying to resolve some dependency issues. Running for 2 hours it is not clear if it will ever stop.

My question is: what is happening here and can I somehow manually speed up this process, e.g. by removing unnecessary dependencies? Since I did not create the environment I don't know which are actually necessary.

The reason why I'm curious is because it is checking conflict paths like fontconfig[version='>=2.12.6,<3.0a0'] when the one in dependencies is 2.13.1 and I have installed it through 2.13.1

Please find below the .yml file for reference.

name: cea
channels:
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free
  - dlr-sc
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
  - conda-forge
  - conda-forge/label/gcc7
  - defaults
dependencies:
  - attrs=19.1.0=py_0
  - backports=1.0=py_2
  - backports.functools_lru_cache=1.5=py_1
  - backports_abc=0.5=py_1
  - boost-cpp=1.68.0=h11c811c_1000
  - bzip2=1.0.6=h14c3975_1002
  - ca-certificates=2019.3.9=hecc5488_0
  - cairo=1.14.12=h80bd089_1005
  - certifi=2019.3.9=py27_0
  - click=7.0=py_0
  - click-plugins=1.0.4=py_0
  - cligj=0.5.0=py_0
  - curl=7.64.1=hf8cf82a_0
  - cycler=0.10.0=py_1
  - descartes=1.1.0=py_2
  - enum34=1.1.6=py27_1001
  - expat=2.2.5=hf484d3e_1002
  - fiona=1.8.6=py27hf242f0b_2
  - fontconfig=2.13.1=he4413a7_1000
  - freetype=2.10.0=he983fc9_0
  - freexl=1.0.5=h14c3975_1002
  - functools32=3.2.3.2=py_3
  - futures=3.2.0=py27_1000
  - gdal=2.4.1=py27h5f563d9_8
  - geopandas=0.4.1=py_1
  - geos=3.7.1=hf484d3e_1000
  - geotiff=1.4.3=h1105359_1000
  - gettext=0.19.8.1=hc5be6a0_1002
  - giflib=5.1.7=h516909a_1
  - glib=2.56.2=had28632_1001
  - hdf4=4.2.13=h9a582f1_1002
  - hdf5=1.10.4=nompi_h3c11f04_1106
  - icu=58.2=hf484d3e_1000
  - jpeg=9c=h14c3975_1001
  - json-c=0.13.1=h14c3975_1001
  - kealib=1.4.10=h1978553_1003
  - kiwisolver=1.0.1=py27h6bb024c_1002
  - krb5=1.16.3=h05b26f9_1001
  - libblas=3.8.0=4_openblas
  - libcblas=3.8.0=4_openblas
  - libcurl=7.64.1=hda55be3_0
  - libdap4=3.19.1=hd48c02d_1000
  - libedit=3.1.20170329=hf8c457e_1001
  - libffi=3.2.1=he1b5a44_1006
  - libgdal=2.4.1=heae24aa_8
  - libiconv=1.15=h516909a_1005
  - libkml=1.3.0=h328b03d_1009
  - liblapack=3.8.0=4_openblas
  - libnetcdf=4.6.2=hbdf4f91_1001
  - libpng=1.6.36=h84994c4_1000
  - libpq=11.2=h4770945_0
  - libspatialindex=1.9.0=he1b5a44_0
  - libspatialite=4.3.0a=hb5ec416_1026
  - libssh2=1.8.2=h22169c7_2
  - libtiff=4.0.10=h648cc4a_1001
  - libuuid=2.32.1=h14c3975_1000
  - libxcb=1.13=h14c3975_1002
  - libxml2=2.9.8=h143f9aa_1005
  - mapclassify=1.0.1=py27_1
  - matplotlib=2.2.3=py27h8a2030e_1
  - matplotlib-base=2.2.3=py27h60b886d_1
  - munch=2.3.2=py_0
  - ncurses=6.1=hf484d3e_1002
  - numpy=1.16.2=py27h8b7e671_1
  - openblas=0.3.5=h9ac9557_1001
  - openjpeg=2.3.0=hf38bd82_1003
  - openssl=1.1.1b=h14c3975_1
  - pandas=0.24.2=py27hf484d3e_0
  - pip=19.0.3=py27_0
  - pixman=0.34.0=h14c3975_1003
  - poppler=0.67.0=h2fc8fa2_1002
  - poppler-data=0.4.9=1
  - postgresql=11.2=h61314c7_0
  - proj4=5.2.0=h14c3975_1001
  - pthread-stubs=0.4=h14c3975_1001
  - pyparsing=2.3.1=py_0
  - pyproj=1.9.6=py27hc0953d3_1000
  - pyqt=5.6.0=py27h13b7fb3_1008
  - python=2.7.15=h721da81_1008
  - python-dateutil=2.8.0=py_0
  - pytz=2018.9=py_0
  - readline=7.0=hf8c457e_1001
  - rtree=0.8.3=py27h666c49c_1002
  - scipy=1.2.1=py27h09a28d5_1
  - setuptools=40.8.0=py27_0
  - shapely=1.6.4=py27h2afed24_1003
  - singledispatch=3.4.0.3=py27_1000
  - sip=4.18.1=py27hf484d3e_1000
  - six=1.12.0=py27_1000
  - sqlite=3.26.0=h67949de_1001
  - subprocess32=3.5.3=py27h14c3975_0
  - tbb=2019.5=hc9558a2_0
  - tk=8.6.9=h84994c4_1001
  - tornado=5.1.1=py27h14c3975_1000
  - tzcode=2018g=h14c3975_1001
  - wheel=0.33.1=py27_0
  - xerces-c=3.2.2=hac72e42_1001
  - xorg-kbproto=1.0.7=h14c3975_1002
  - xorg-libice=1.0.9=h516909a_1004
  - xorg-libsm=1.2.3=h84519dc_1000
  - xorg-libx11=1.6.7=h14c3975_1000
  - xorg-libxau=1.0.9=h14c3975_0
  - xorg-libxdmcp=1.1.3=h516909a_0
  - xorg-libxext=1.3.4=h516909a_0
  - xorg-libxrender=0.9.10=h516909a_1002
  - xorg-renderproto=0.11.1=h14c3975_1002
  - xorg-xextproto=7.3.0=h14c3975_1002
  - xorg-xproto=7.0.31=h14c3975_1007
  - xz=5.2.4=h14c3975_1001
  - zlib=1.2.11=h14c3975_1004
  - freeimageplus=3.18.0=hf484d3e_2
  - gl2ps=1.3.8=h14c3975_4
  - oce=0.17.2=h6bb024c_10
  - pythonocc-core=0.17=py27_1
  - mkl=2017.0.3=0
  - mkl-service=1.1.2=py27_3
  - dbus=1.13.2=h714fa37_1
  - gst-plugins-base=1.14.0=hbbd80ab_1
  - gstreamer=1.14.0=hb453b48_1
  - libgcc-ng=8.2.0=hdf63c60_1
  - libgfortran-ng=7.3.0=hdf63c60_0
  - libstdcxx-ng=8.2.0=hdf63c60_1
  - pcre=8.43=he6710b0_0
  - qt=5.6.3=h8bf5577_3
  - pip:
    - alabaster==0.7.12
    - babel==2.6.0
    - backports.shutil-get-terminal-size==1.0.0
    - bleach==3.1.0
    - chardet==3.0.4
    - cloudpickle==0.8.1
    - configparser==3.7.4
    - cvxopt==1.2.3
    - decorator==4.4.0
    - defusedxml==0.5.0
    - docutils==0.14
    - doit==0.29.0
    - entrypoints==0.3
    - ephem==3.7.6.0
    - funcsigs==1.0.2
    - h5py==2.9.0
    - idna==2.8
    - imagesize==1.1.0
    - importlib-resources==1.0.2
    - ipaddress==1.0.22
    - ipykernel==4.10.0
    - ipython==5.8.0
    - ipython-genutils==0.2.0
    - ipywidgets==7.4.2
    - jinja2==2.10
    - joblib==0.12.5
    - jsonschema==3.0.1
    - jupyter==1.0.0
    - jupyter-client==5.2.4
    - jupyter-console==5.2.0
    - jupyter-core==4.4.0
    - llvmlite==0.28.0
    - lxml==4.3.3
    - markupsafe==1.1.1
    - mistune==0.8.4
    - mock==2.0.0
    - nbconvert==5.4.1
    - nbformat==4.4.0
    - networkx==2.2
    - notebook==5.7.8
    - numba==0.43.1
    - packaging==19.0
    - pandocfilters==1.4.2
    - pathlib2==2.3.3
    - patsy==0.5.1
    - pbr==5.1.3
    - pexpect==4.6.0
    - pickleshare==0.7.5
    - pkginfo==1.5.0.1
    - plotly==3.7.1
    - prometheus-client==0.6.0
    - prompt-toolkit==1.0.15
    - ptyprocess==0.6.0
    - py4design==0.27
    - pycollada==0.6
    - pygments==2.3.1
    - pyinotify==0.9.6
    - pymc3==3.6
    - pymf==0.1.9
    - pyrsistent==0.14.11
    - pysal==1.14.4.post2
    - pyshp==2.1.0
    - pyyaml==5.1
    - pyzmq==18.0.1
    - qtconsole==4.4.3
    - readme-renderer==24.0
    - requests==2.21.0
    - requests-toolbelt==0.9.1
    - retrying==1.3.3
    - salib==1.2
    - scandir==1.10.0
    - scikit-learn==0.20.3
    - seaborn==0.9.0
    - send2trash==1.5.0
    - simplegeneric==0.8.1
    - snowballstemmer==1.2.1
    - sphinx==1.8.5
    - sphinxcontrib-websupport==1.1.0
    - terminado==0.8.2
    - testpath==0.4.2
    - theano==1.0.4
    - timezonefinder==4.0.1
    - tqdm==4.31.1
    - traitlets==4.3.2
    - twine==1.13.0
    - typing==3.6.6
    - urllib3==1.24.1
    - utm==0.4.2
    - wcwidth==0.1.7
    - webencodings==0.5.1
    - widgetsnbextension==3.4.2
    - xlrd==1.2.0
    - xlwt==1.3.0
    - deap==1.3.0

Answer 1

This error and the subsequent behaviour is very likely coming from the Anaconda bug which from time to time causes inconsistencies in the local environment. As of Jan 26th, 2020 the bug was not yet resolved.

For me the same problem manifests profoundly on Mac.

Answer 2

In general, the answer to this question is that you should create your .yml from hand by scratch, with only the required channels and constraints. You should relax the version constraints so that they only look like 1.19 instead of 1.19.10

However, despite following this advice, I have almost exactly your problem right now. It spent many hours on fontconfig and I left it overnight before giving up. My spec is pretty simple:

channels:
  - conda-forge
  - usgs-astrogeology
  - defaults
dependencies:
  - python=3.6
  - isis3
  - gdal=2.3

I'll edit this when I figure it out.

EDIT: This problem is under discussion here (specific to the software I was trying to set up, probably not helpful for others) https://github.com/USGS-Astrogeology/ISIS3/issues/3570

Answer 3

For me adding the following to the yaml file worked:

channels:
  - conda-forge
  - defaults
  - conda-forge/label/broken

Answer 4

In my case, I had replaced some of the build versions (the ...=he4413a7_1000 part) as the ones in the original file had been relabelled as broken on conda-forge. However, even though the version number hadn't changed for these dependencies, their subdependencies had changed, causing all these conflicts. The bottom line is: don't mess with dependencies manually.

I solved the problem by keeping the original build version specifications, and adding conda-forge/labels/broken to the channels part of the .ymlfile.

Answer 5

I had the same problem, here is how I sped my environment creation. The commands are in *nix, but I'll tell what they are doing.

I copied the enviroment.yml' to environment-clean.yml` and started to edit the second file. I maintained the "name" and "channels" sections.

Now I checked which libraries were used by the code. Since it were just a bunch of Jupyter notebooks, all the files were in the same dir. Maybe you will have to do it recursively. Te command below list the imports and remove and repetition:

grep import -h *.ipynb |grep -v undefined|sort |uniq

Now I manually edited the environment-clean.yml file. First include the python version that were originally used. You can find it from the id after each package. So py39h8c16a72_0 will tell you that you are using python 3.9. Put - python=3.9 as a dependency. A nice feature of pandas is that you can have environments with different python versions.

Then I removed everything except the directly imported files. I maintained the version number, and removed the id after the name. So - pandas=1.3.1=py39h8c16a72_0 would become - pandas=1.3.1.

No dependency of dependency were left in the file.

Now just create it:

conda env create -f environment-clean.yml

Conda environments aren't really multiplatform. My suggestion to always create them using the command conda env export --from-history>path-to-yml.yml, so just the explicitly installed packages will be included in the environment file. No secondary dependencies will be listed in the yml file. You must install them fixing the version number. If you use this yml file to install with conda env create --file path-to-yml.yml then conda will then take care of downloading the dependencies as necessary and you shouldn't have any conflicts.

Answer 6

I have met this issue and solved it. Firstly, I made my anaconda version to be consistent with where the yaml file export from. This may be useless but i consider that consistent conda version can promise consistent base environment. Secondly, i deleted all dependencies but some 'main' packages which are i actually want to use in my yaml file.That means let these 'main' package handle dependency issues.

Answer 7

I was also faced with the same problem on windows 10, using pycharm with anaconda (python 3.7) latest build as of feb 17,2020. Strangely enough toggling the "use conda package manager" on the project interpreter page in pycharm's settings allowed me to update numpy and install packages normally. Try installing a package without the conda package manager and with the conda package manager. Good luck!

Answer 8

For me worked best to do:

which python (confirming that it was in the anaconda3 bin)

python --version
conda search "^python$"

Note which version matches the version (here 3.7.3) in use for Anaconda (since I'm making a new environment, it stands to reason that I could use the same version of Python that Anaconda itself is using). Note that my listing from the conda channels (conda search above) showed only one 3.7.3 line.

Then do this:

conda create -n myenv python=3.7.3 anaconda

Took about 1 minute to run the last command :)