How to collect source files with CMake without globbing?

Question

The CMake documentation explicitly states that file(GLOB ...) is not recommended to collect source files for a build, but it doesn't mention what the recommended method actually is.

Specifying every source file manually sounds a little bit too manually to me. So, what is the right method to collect source files, if not file(GLOB ...)?

Answer 1

I use GLOB for exactly that and every time I add a file I run

touch ../src/CMakeLists.txt

The next make command will re-scan the directories.

"There is no way for CMake to generate a build system that knows when a new source file has been added" Really? Okay, so tell it!

It's not 100% automatic but a damn sight easier than adding files manually.

Answer 2

I use cog, a python module. Here is a sample to collect .cpp file:

The CMakeLists.txt:

set(SRC_FILES "")

# [[[cog
#    import cog, glob
#    for src in glob.glob('*.cpp'):
#       if "skeleton" in src: continue
#       cog.outl("SET(SRC_FILES ${SRC_FILES} %s)" % src)
# ]]]
# [[[end]]]

add_library(mylib STATIC  ${SRC_FILES})

And then, run:

python -m cogapp -r CMakeLists.txt

The CMakeLists.txt file will be updated in place.

For how to install cog and other usage, please read the article from the author.

Answer 3

I use a conventional CMakeLists.txt and a python script to update it. I run the python script manually after adding files.

import os
import re       

def relFiles(base, sub):
    fullSub = os.path.join(base,sub)
    abs = [os.path.join(dp, f) for dp, dn, fn in os.walk(fullSub) for f in fn]
    return [os.path.relpath(f, base) for f in abs]

def updateAddLibrary(cmakelistsDir, subs):
    cmakelists = os.path.join(cmakelistsDir, "CMakeLists.txt")
    listings = [relFiles(cmakelistsDir, sub) for sub in subs]
    files = [f for listing in listings for f in listing] #flatten
    with open(cmakelists, 'r') as file:
        text = file.read()
    sources = "".join(["    %s\n" % f.replace('\\', '/') for f in files])
    text = re.sub(r"add_library\s*\(\s*([^\s\)]+).*?\)",
           r"add_library(\1\n%s)" % sources, 
           text, 1, re.DOTALL)
    with open(cmakelists, "w") as file:
        file.write(text)

dir = os.path.dirname(os.path.abspath(__file__))
updateAddLibrary(dir, ['inc','src'])

Example before:

...
add_library(MyLib
    inc/a.h
) 
...

after:

...
add_library(MyLib
    inc/a.h
    inc/sub/b.h
    src/a.cpp
)        
...

Answer 4

Following douyw's answer, thank you for your answer.

Not a Cmake expert, don't want to be one, I spent a fxxking 3 hours trying to deal with GLOB(disabled) and aux_source_directory(Not even close to GLOB), and douyw save my life.

I add the recursive file walking, and it's working in my project:

Firstly, install the cogapp python module (python -m pip install cogapp)

set(SRC_FILES "")

# [[[cog
#   import cog, os
#   for root, _, files in os.walk(".", topdown=False):
#       for f in files: 
#           if not "_unittest" in f: continue
#           if not f.endswith(".cpp"): continue
#           cog.outl('SET(SRC_FILES ${SRC_FILES} "%s")' % os.path.join(root, f).replace('\\', '/'))
# ]]]
# [[[end]]]

run: python -m cogapp -r CMakeLists.txt

• The upper lines add all "*_unittest.cpp" to the list
• You can change the middle lines to make your own rule, just plain python. Using regex is better, but simple string searching can do the job on the above situation.
• Notice the last line, it needs to replace // to change to the usable universal separator. You may generate your CMakeList.txt in windows.
• You may replace SRC_FILES with whatever you want.
• The python command may be run under Jenkins/TeamCity, triggered by svn/git commit. Then we can automatically add new files.