Sending a request to my http server hangs indefinitely in CI environment (CircleCI andgithub actions). Works as expected locally

Question

So I’m writing a basic http server in c++ for school. At the moment my service can receive a request and send a basic response. I’m writing a basic acceptance test with python requests and executing it in a shell script by running the webserver in the background and then running the python client. Locally (on Mac) running the tests it works fine, also on a custom ubuntu docker container, which is also used for my config.yml file. But when running the tests on circleCI the client seems to be accepted but the GET request is not being received and the program hangs. Does anyone know why this is and how to solve it? Please not that the include code is still being developed so be kind :slight_smile:

My config.yml:

version: 2

jobs:
  build:
    docker:
      - image: "rynosaurusrex/ci_for_codam:webserv"
    steps:
      - checkout
      - run:
          name: Build
          command: 'echo Building'
      - run:
          name: Unit-Test
          command: "make test && ./unit_test"
      - run:
          name: Acceptance-Test
          command: make acceptence

My basic python GET request:

import time
import requests
from requests import Session
from enum import Enum

class Colors:
    OKGREEN =   '\033[92m'
    FAILRED =   '\033[91m'
    NATURAL =   '\033[0m'

OK              = 200
BAD_REQUEST     = 400
NOT_FOUND       = 404
URI_TOO_LONG    = 414
TEAPOT          = 418
NOT_IMPLEMENTED = 501

localhost = "http://localhost:80"
EXIT_CODE = 0

print("Connecting to server...")
r = requests.get(localhost)
print("Request send!")

if r.status_code != OK:
    print(f"{Colors.FAILRED}[KO] {Colors.NATURAL} Get request on http://localhost:80")
    EXIT_CODE = 1;
else:
    print(f"{Colors.OKGREEN}[OK] {Colors.NATURAL} Get request on http://localhost:80")

# sleep so that the exit code is that of the python script and not the server
time.sleep(1)
exit(EXIT_CODE)

and my messy bash script to run everything:

#!/bin/bash
# the & runs a command/program in the background
./Webserver.out &

# Save the PID to kill the webserv
PID=$!
# sleep for 2 second to make sure the server has time to start up
sleep 2

# run the tests
# curl localhost:80
python3 acceptence_tests/TestClient.py 
# save the return val of the tests for CI
T1=$?

kill $PID
exit $T1 

So I print a few status updates when running the tests. When printing Accepted client on fd 4the accept() call was succesful but the request doesn’t seem to be coming through, possible a handshake issue? As mentioned above, running this locally and directly in the container delivers the expected results, how does local ports work within CircleCI?

I've tried changing the request to https with port 443, also 0.0.0.0 and it seems like the handshake is failing any ideas?

Thanks in advance!

Answer 1

This could be caused by trying to run as root in your container, since this could be blocked/limited by the host system (which is configured/owned by circleCI/github in this case) for security reasons. You could try explicitly creating and using a separate user in your Dockerfile, avoiding weird permission issues.

As an example, if you add this to your Dockerfile you'll use test_user instead of root:

RUN useradd -m test_user
USER test_user