Cannot compile pyethash python package which requires C99 compiler (AFAIU). Error - Cannot open include file: 'alloca.h'

Question

Problem

When installing pyethash manually or with pip I get the same kind of error:

fatal error C1083: Cannot open include file: 'alloca.h': No such file or directory error: command 'C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\cl.exe' failed with exit status 2

Related and tried already:

• error: Unable to find vcvarsall.bat
• Errors while building/installing C module for Python 2.7

Similar problem with no answer:

• comment42289832_26127562
• Setup script exited with error cl.exe' failed with exit status 2

Other facts:

• Successfully installed Crypto and scrypt which require C++ compiler.
• There are another 3 files mentioned in pyethash core.c source file headers which are absent anywhere on my drive:
  • alloca.h
  • stdint.h
  • stdlib.h

System

python 2.7.11 (v2.7.11:6d1b6a68f775, Dec 5 2015, 20:40:30) [MSC v.1500 64 bit (AMD64)], windows 8.1 x64

...\ethash-master> python setup.py install

PS C:\pyethereum\ethash-master> python setup.py install
running install
running build
running build_ext
building 'pyethash' extension
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -IC:\Python2
7\include -IC:\Python27\PC /Tcsrc/python/core.c /Fobuild\temp.win-amd64-2.7\Release\src/python/core.obj -Isrc/ -std=gnu9
9 -Wall
cl : Command line warning D9002 : ignoring unknown option '-std=gnu99'
core.c
c:\program files (x86)\microsoft visual studio 9.0\vc\include\codeanalysis\sourceannotations.h(81) : warning C4820: 'Pre
Attribute' : '4' bytes padding added after data member 'Access'
c:\program files (x86)\microsoft visual studio 9.0\vc\include\codeanalysis\sourceannotations.h(96) : warning C4820: 'Pre
Attribute' : '4' bytes padding added after data member 'NullTerminated'
c:\program files (x86)\microsoft visual studio 9.0\vc\include\codeanalysis\sourceannotations.h(112) : warning C4820: 'Po
stAttribute' : '4' bytes padding added after data member 'Access'
c:\program files (x86)\microsoft visual studio 9.0\vc\include\codeanalysis\sourceannotations.h(191) : warning C4820: 'Pr
eRangeAttribute' : '4' bytes padding added after data member 'Deref'
c:\program files (x86)\microsoft visual studio 9.0\vc\include\codeanalysis\sourceannotations.h(203) : warning C4820: 'Po
stRangeAttribute' : '4' bytes padding added after data member 'Deref'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\io.h(60) : warning C4820: '_finddata32i64_t' : '4' bytes p
adding added after data member 'name'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\io.h(64) : warning C4820: '_finddata64i32_t' : '4' bytes p
adding added after data member 'attrib'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\io.h(73) : warning C4820: '__finddata64_t' : '4' bytes pad
ding added after data member 'attrib'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\io.h(78) : warning C4820: '__finddata64_t' : '4' bytes pad
ding added after data member 'name'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\io.h(126) : warning C4820: '_wfinddata64i32_t' : '4' bytes
 padding added after data member 'attrib'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\io.h(131) : warning C4820: '_wfinddata64i32_t' : '4' bytes
 padding added after data member 'name'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\io.h(135) : warning C4820: '_wfinddata64_t' : '4' bytes pa
dding added after data member 'attrib'
C:\Program Files\Microsoft SDKs\Windows\v7.0\include\basetsd.h(114) : warning C4668: '__midl' is not defined as a prepro
cessor macro, replacing with '0' for '#if/#elif'
C:\Program Files\Microsoft SDKs\Windows\v7.0\include\basetsd.h(424) : warning C4668: '_WIN32_WINNT' is not defined as a
preprocessor macro, replacing with '0' for '#if/#elif'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\stdio.h(62) : warning C4820: '_iobuf' : '4' bytes padding
added after data member '_cnt'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\stdio.h(381) : warning C4255: '_get_printf_count_output' :
 no function prototype given: converting '()' to '(void)'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\stdlib.h(215) : warning C4255: '_get_purecall_handler' : n
o function prototype given: converting '()' to '(void)'
c:\python27\include\pyport.h(206) : warning C4668: 'SIZEOF_PID_T' is not defined as a preprocessor macro, replacing with
 '0' for '#if/#elif'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\math.h(41) : warning C4820: '_exception' : '4' bytes paddi
ng added after data member 'type'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\sys/stat.h(111) : warning C4820: '_stat32' : '2' bytes pad
ding added after data member 'st_gid'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\sys/stat.h(127) : warning C4820: 'stat' : '2' bytes paddin
g added after data member 'st_gid'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\sys/stat.h(143) : warning C4820: '_stat32i64' : '2' bytes
padding added after data member 'st_gid'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\sys/stat.h(144) : warning C4820: '_stat32i64' : '4' bytes
padding added after data member 'st_rdev'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\sys/stat.h(148) : warning C4820: '_stat32i64' : '4' bytes
padding added after data member 'st_ctime'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\sys/stat.h(157) : warning C4820: '_stat64i32' : '2' bytes
padding added after data member 'st_gid'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\sys/stat.h(171) : warning C4820: '_stat64' : '2' bytes pad
ding added after data member 'st_gid'
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\sys/stat.h(172) : warning C4820: '_stat64' : '4' bytes pad
ding added after data member 'st_rdev'
c:\python27\include\object.h(358) : warning C4820: '_typeobject' : '4' bytes padding added after data member 'tp_flags'
c:\python27\include\object.h(411) : warning C4820: '_typeobject' : '4' bytes padding added after data member 'tp_version
_tag'
c:\python27\include\unicodeobject.h(420) : warning C4820: '<unnamed-tag>' : '4' bytes padding added after data member 'h
ash'
c:\python27\include\intobject.h(26) : warning C4820: '<unnamed-tag>' : '4' bytes padding added after data member 'ob_iva
l'
c:\python27\include\stringobject.h(49) : warning C4820: '<unnamed-tag>' : '7' bytes padding added after data member 'ob_
sval'
c:\python27\include\bytearrayobject.h(26) : warning C4820: '<unnamed-tag>' : '4' bytes padding added after data member '
ob_exports'
c:\python27\include\setobject.h(26) : warning C4820: '<unnamed-tag>' : '4' bytes padding added after data member 'hash'
c:\python27\include\setobject.h(56) : warning C4820: '_setobject' : '4' bytes padding added after data member 'hash'
c:\python27\include\methodobject.h(42) : warning C4820: 'PyMethodDef' : '4' bytes padding added after data member 'ml_fl
ags'
c:\python27\include\fileobject.h(26) : warning C4820: '<unnamed-tag>' : '4' bytes padding added after data member 'f_ski
pnextlf'
c:\python27\include\fileobject.h(33) : warning C4820: '<unnamed-tag>' : '4' bytes padding added after data member 'writa
ble'
c:\python27\include\genobject.h(23) : warning C4820: '<unnamed-tag>' : '4' bytes padding added after data member 'gi_run
ning'
c:\python27\include\descrobject.h(28) : warning C4820: 'wrapperbase' : '4' bytes padding added after data member 'offset
'
c:\python27\include\descrobject.h(32) : warning C4820: 'wrapperbase' : '4' bytes padding added after data member 'flags'

c:\python27\include\weakrefobject.h(37) : warning C4820: '_PyWeakReference' : '4' bytes padding added after data member
'hash'
c:\python27\include\pystate.h(70) : warning C4820: '_ts' : '4' bytes padding added after data member 'use_tracing'
c:\python27\include\import.h(61) : warning C4820: '_frozen' : '4' bytes padding added after data member 'size'
c:\python27\include\code.h(26) : warning C4820: '<unnamed-tag>' : '4' bytes padding added after data member 'co_firstlin
eno'
src/python/core.c(2) : fatal error C1083: Cannot open include file: 'alloca.h': No such file or directory
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 9.0\\VC\\BIN\\amd64\\cl.exe' failed with exit status 2

Question

How can I compile this package? (Please help, it is 4th day now!)

Answer 1

alloca allocates automatic memory and as Jens Gustedt noted is not standardised.

MSVCRT declares it in the <malloc.h> header. Its implementation on Windows aligns with the common behavior on UNIX systems, so it should work as expected. Other parts of the code might be more tightly coupled to UNIX though and may require a rewrite.

The other two header are standard C headers and should be in the INCLUDE directory which your compiler automatically searches.

Answer 2

If your Python 2.7 interpreter is compiled with a recent version (as yours appear to be, i.e. [MSC v.1500 64 bit (AMD64)]), you can install a newer version of MSVC (Visual Studio Community Edition is free) and modify Python 2.7's msvxx99.py, inserting the following directly at the start of the find_vcvarsall function in the file <python>/lib/distutils/msvc9compiler.py (where <python> is usually C:\Python27 or similar). This should fix matters for any Python 2 modules.

Note, you will need to change the path if your installation resides on a drive other than C:. The code is intentionally simple, and does not try to get smart by using the registry or vswhere etc, and will fall back on the original implementation if a newer version of MSVC cannot be found.

def find_vcvarsall(version):
    # --[insert starts here]--
    import glob
    paths = glob.glob(r"C:\Program Files (x86)\Microsoft Visual Studio\20*\*\vc\Auxiliary\Build\vcvarsall.bat")
    if paths:
        print("{} possible locations for vcvarsall.bat found:".format(len(paths)))
        for i, path in enumerate(paths):
            print("\t[{}] {}".format(i, path))
        if len(paths) > 1:
            print("picking one at random:")
            paths.shuffle()
        path = paths[0]
        print("selecting:  {}".format(path))
        return path

    # otherwise, do what it would have done normally

Answer 3

I think it is x86 / x64 compability error.

I've used python 3.9.0 and these steps worked for me (with py-evm project on python https://github.com/ethereum/py-evm):

1. Download pyethash 0.1.27: https://pypi.org/project/pyethash/0.1.27/#files and unpack archive in lib folder in your project. I've unpacked at /virtualenv/Lib/site-packages/pyethash-0.1.27 folder

2. Download ethash from git: https://github.com/ethereum/ethash

3. Copy all files form ethash folder to /virtualenv/Lib/site-packages/pyethash-0.1.27

4. Change /pyethash-0.1.27/src/libethash/mmap.h add this code to the end of file:

#pragma comment(lib, "Shell32.lib")

5. Change pyethash-0.1.27\src\python\core.c file - replace #include <alloca.h> by the code:

#if defined(_WIN32) || defined(WIN32)
#include <malloc.h>
#else
#include <alloca.h>
#endif

6. Zip /virtualenv/Lib/site-packages/pyethash-0.1.27 folder and remove that folder (zip archive will stay).

7. Run pip install for new created zip /virtualenv/Lib/site-packages/pyethash-0.1.27.zip

8. It can conflict with versions: ethash have 0.1.23 version instead of 0.1.27, i just changed my requerements to 0.1.23 and it worked for me. You can try to copy not all files (from ethhash to pyethhash), just files one by one that breaks the compilation.