'Implementation of running function in parallel using pipleline pattern C++

I am having difficulties to implement pipelines in c++.

Here is the problem:

Output = Pipe(stage1(), farm(4, stage2()), stage3(), input);

Output is the output queue. Input is input queue.

here is the example of stage function:

int stage1(int task) {
    return task * 2;
}

4 here is the number of workers. so stage2() is running in 4 instances.

I was already been able to implement a farm function: Here is my farm function:

 std::queue<R> farm(int num_workers, R (*worker)(T)) {
    std::queue<R> outputQueue;

    for(int i = 0; i < num_workers; i++) {
        // input queue instead of NULL
        workerThreads.push_back(makeThread(worker, NULL, outputQueue));
    }

    for (auto& t: workerThreads) t.join();
    return outputQueue;
}

What I need to do is get the input in queue and return output queue. What I am having difficulty is to implement the pipeline pattern. Because I need to use to output in the stage1() and give it to stage2() in farm() and so on. If there was only stage functions without wrapped into a farm() that wouldn't be a difficult. But I want to make Pipe taking variable number of arguments and farm() wrapper can be in any stage.

Any thoughts what I can do this in this case?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source