What does this mean: 'sem_wait@@GLIBC_2.2.5 () from /lib64/libpthread.so.0'?

Question

I am running a C++ program which calls a python plugin, but at some late stage when it calls the python, it generates a core dump and I used gdb to open it and do a backtrace, it shows something related to multithreads issues.

What does exactly this mean?

The full info is below:

# gdb src/api/tests/api_test core.118173 
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-120.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /data0/src/api/tests/api_test...done.
[New LWP 118173]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `./src/api/tests/api_test --gtest_filter=predictTest.predict'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007fd6170eec44 in sem_wait@@GLIBC_2.2.5 () from /lib64/libpthread.so.0
Missing separate debuginfos, use: debuginfo-install glibc-2.17-324.el7_9.x86_64 libgcc-4.8.5-44.el7.x86_64 libstdc++-4.8.5-44.el7.x86_64 python-libs-2.7.5-90.el7.x86_64
(gdb) bt
#0  0x00007fd6170eec44 in sem_wait@@GLIBC_2.2.5 () from /lib64/libpthread.so.0
#1  0x00007fd616e26d55 in PyThread_acquire_lock () from /lib64/libpython2.7.so.1.0
#2  0x00007fd616e151e0 in new_threadstate () from /lib64/libpython2.7.so.1.0
#3  0x00007fd616e15a8c in PyGILState_Ensure () from /lib64/libpython2.7.so.1.0
#4  0x0000000000711c34 in PythonPlugin::abstract_feature (this=0x11f7c3d0, type=0, input=..., aResult=...) at /root/src/module/fusion/PythonPlugin.cpp:173
#5  0x0000000000501079 in FusionLayer::abstract_feature (this=0x3fd399a0, type=0, input=..., aResult=...) at /root/src/module/fusion/FusionLayer.cpp:181
#6  0x0000000000475216 in Predict::predict_field (this=0x2b87d80, type=0, input=..., output=...) at /root/src/api/Predict.cpp:106
#7  0x00000000004757d5 in Predict::process (this=0x2b87d80, 
    input_buffer=0x2a85bb8 "{\"input\":\"[{\\\"action\\\":\\\"original\\\",\\\"att\\\":1,\\\"city\\\":\\\"1000\\\",\\\"filter\\\":76,\\\"lbs\\\":\\\"\\\",\\\"m_retweettext\\\":\\\"\\\",\\\"m_summary\\\":\\\"\\\",\\\"m_text\\\":\\\"张庭夫妇\\\",\\\"m_title\\\":\\\"\\\",\\\"mblog_dupnum\\\":0,\\\"m"..., input_len=684, out_buffer=0x7ffe22c2fba0 "", out_len=40960)
    at /root/src/api/Predict.cpp:164
#8  0x000000000046dd4f in predictTest_predict_Test::TestBody (this=0x2bf83e0) at /root/src/api/tests/predict_test.cpp:127
#9  0x000000000083a3b2 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) ()
#10 0x0000000000836782 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) ()
#11 0x000000000082781b in testing::Test::Run() ()
#12 0x0000000000827ff6 in testing::TestInfo::Run() ()
#13 0x0000000000828686 in testing::TestCase::Run() ()
#14 0x000000000082ed36 in testing::internal::UnitTestImpl::RunAllTests() ()
#15 0x000000000083b678 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) ()
#16 0x0000000000837436 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) ()
#17 0x000000000082d9c8 in testing::UnitTest::Run() ()
#18 0x0000000000822253 in RUN_ALL_TESTS() ()
#19 0x00000000008221ee in main ()

It involves multithreads and maybe hard to fix, but offer some advices or suggestions.

Answer 1

What does exactly this mean?

It means that your program crashed in the sem_wait function, most likely because you passed it invalid sem pointer.

To see what value was passed in, and where inside sem_wait() your program crashed, do what GDB told you to do:

sudo debuginfo-install glibc-2.17-324.el7_9.x86_64 libgcc-4.8.5-44.el7.x86_64 libstdc++-4.8.5-44.el7.x86_64 python-libs-2.7.5-90.el7.x86_64