How to make a linux shared object (library) runnable on its own?

Question

Noticing that gcc -shared creates an executable file, I just got the weird idea to check what happens when I try to run it ... well the result was a segfault for my own lib. So, being curious about that, I tried to "run" the glibc (/lib/x86_64-linux-gnu/libc.so.6 on my system). Sure enough, it didn't crash but provided me some output:

GNU C Library (Debian GLIBC 2.19-18) stable release version 2.19, by Roland McGrath et al.
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.8.4.
Compiled on a Linux 3.16.7 system on 2015-04-14.
Available extensions:
    crypt add-on version 2.1 by Michael Glad and others
    GNU Libidn by Simon Josefsson
    Native POSIX Threads Library by Ulrich Drepper et al
    BIND-8.2.3-T5B
libc ABIs: UNIQUE IFUNC
For bug reporting instructions, please see:
<http://www.debian.org/Bugs/>.

So my question here is: what is the magic behind this? I can't just define a main symbol in a library -- or can I?

Answer 1

While linking with -shared gcc strips start files, and some objects (like cout) will not be initialized. So, std::cout << "Abc" << std::endl will cause SEGFAULT.

Approach 1

(simplest way to create executable library)

To fix it change linker options. The simplest way - run gcc to build executable with -v option (verbose) and see the linker command line. In this command line you should remove -z now, -pie (if present) and add -shared. The sources must be anyway compiled with -fPIC (not -fPIE).

Let's try. For example we have the following x.cpp:

#include <iostream>

// The next line is required, while building executable gcc will
// anyway include full path to ld-linux-x86-64.so.2:
extern "C" const char interp_section[] __attribute__((section(".interp"))) = "/lib64/ld-linux-x86-64.so.2";

// some "library" function
extern "C"  __attribute__((visibility("default"))) int aaa() {
    std::cout << "AAA" << std::endl;
    return 1234;
}

// use main in a common way
int main() {
    std::cout << "Abc" << std::endl;
}

Firstly compile this file via g++ -c x.cpp -fPIC. Then will link it dumping command-line via g++ x.o -o x -v.

We will get correct executable, which can't be dynamically loaded as a shared library. Check this by python script check_x.py:

import ctypes
d = ctypes.cdll.LoadLibrary('./x')
print(d.aaa())

Running $ ./x will be successful. Running $ python check_x.py will fail with OSError: ./x: cannot dynamically load position-independent executable.

While linking g++ calls collect2 linker wraper which calls ld. You can see command-line for collect2 in the output of last g++ command like this:

/usr/lib/gcc/x86_64-linux-gnu/11/collect2 -plugin /usr/lib/gcc/x86_64-linux-gnu/11/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/11/lto-wrapper -plugin-opt=-fresolution=/tmp/ccqDN9Df.res -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc --build-id --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -pie -z now -z relro -o x /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/Scrt1.o /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/11/crtbeginS.o -L/usr/lib/gcc/x86_64-linux-gnu/11 -L/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/11/../../.. x.o -lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/lib/gcc/x86_64-linux-gnu/11/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/crtn.o

Find there -pie -z now and replace with -shared. After running this command you will get new x executable, which will wonderfully work as an executable and as a shared library:

$ ./x
Abc
$ python3 check_x.py
AAA
1234

This approach has disadvantages: it is hard to do replacement automatically. Also before calling collect2 GCC will create a temporary file for LTO plugin (link-time optimization). This temporary file will be missing while you running the command manually.

Approach 2

(applicable way to create executable library)

The idea is to change linker for GCC to own wrapper which will correct arguments for collect2. We will use the following Python script collect3.py as linker:

#!/usr/bin/python3
import subprocess, sys, os

marker = '--_wrapper_make_runnable_so'

def sublist_index(haystack, needle):
    for i in range(len(haystack) - len(needle)):
        if haystack[i:i+len(needle)] == needle: return i

def remove_sublist(haystack, needle):
    idx = sublist_index(haystack, needle)
    if idx is None: return haystack

    return haystack[:idx] + haystack[idx+len(needle):]

def fix_args(args):
    #print("!!BEFORE REPLACE ", *args)
    if marker not in args:
         return args

    args = remove_sublist(args, [marker])
    args = remove_sublist(args, ['-z', 'now'])
    args = remove_sublist(args, ['-pie'])

    args.append('-shared')
    #print("!!AFTER REPLACE ", *args)

    return args

# get search paths for linker directly from gcc
def findPaths(prefix = "programs: ="):
    for line in subprocess.run(['gcc', '-print-search-dirs'], stdout=subprocess.PIPE).stdout.decode('utf-8').split('\n'):
        if line.startswith(prefix): return line[len(prefix):].split(':')

# get search paths for linker directly from gcc
def findLinker(linker_name = 'collect2'):
    for p in findPaths():
        candidate = os.path.join(p, linker_name)
        #print("!!CHECKING LINKER ", candidate)
        if os.path.exists(candidate) : return candidate

if __name__=='__main__':
    args = sys.argv[1:]
    args = fix_args(args)
    exit(subprocess.call([findLinker(), *args]))

This script will replace arguments and call true linker. To switch linker we will create the file specs.txt with the following content:

*linker:
<full path to>/collect3.py

To tell our fake linker that we want to correct arguments we will use the additional argument --_wrapper_make_runnable_so. So, the complete command line will be the following:

g++ -specs=specs.txt -Wl,--_wrapper_make_runnable_so x.o -o x

(we suppose that you want to link existing x.o).

After this you can both run the target x and use it as dynamic library.