'Line-counter with and without channels - how to explain the difference in performance?
I'm trying to understand the performance characteristic of Go channels in order to figure out what use cases I can apply it to. In order to do that, I decided to write two "line counter" programs - one without channels and one with them.
Here's the version without channels (main.go):
package main
import (
"bufio"
"fmt"
"os"
)
func main() {
scanner := bufio.NewScanner(os.Stdin)
var sum int = 0
for scanner.Scan() {
sum += 1
}
fmt.Println(sum)
}
And here's the version with channels (main2.go):
package main
import (
"bufio"
"fmt"
"os"
)
func counter(lineCounter chan int, lineFeeder chan string) {
for {
_, ok := <-lineFeeder
if !ok {
break
}
lineCounter <- 1
}
close(lineCounter)
}
func feeder(scanner *bufio.Scanner, lineFeeder chan string) {
for scanner.Scan() {
lineFeeder <- scanner.Text()
}
close(lineFeeder)
}
func main() {
lineCounter := make(chan int)
lineFeeder := make(chan string)
scanner := bufio.NewScanner(os.Stdin)
go counter(lineCounter, lineFeeder)
go feeder(scanner, lineFeeder)
var sum int = 0
for {
cnt, ok := <-lineCounter
if !ok {
break
}
sum += cnt
}
fmt.Println(sum)
}
I'm testing it on a system with multiple CPU cores. First I'll measure how long it takes to generate input alone:
> time seq 1 $(( 1000 * 1000 * 1000 )) > /dev/null
seq 1 $(( 1000 * 1000 * 1000 )) > /dev/null 7.17s user 0.61s system 97% cpu 7.951 total
Now let's run the version without channels:
> time seq 1 $(( 1000 * 1000 * 1000 )) | go run main.go
1000000000
seq 1 $(( 1000 * 1000 * 1000 )) 9.91s user 3.87s system 47% cpu 29.117 total
go run main.go 22.27s user 4.05s system 90% cpu 29.118 total
Unfortunately, the main2.go didn't complete in reasonable time, so I reduced the input size by two orders of magnitude:
> time seq 1 $(( 1000 * 1000 * 10 )) | go run main2.go
10000000
seq 1 $(( 1000 * 1000 * 10 )) 0.11s user 0.11s system 2% cpu 8.087 total
go run main2.go 7.96s user 0.92s system 109% cpu 8.094 total
What is causing such a significant difference? I imagine that at least part of it is due to overhead caused by copying strings and passing them around. How to make the comparison more fair?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
