How do I pipe or redirect the output of curl -v?

Question

For some reason the output always gets printed to the terminal, regardless of whether I redirect it via 2> or > or |. Is there a way to get around this? Why is this happening?

Answer 1

Your URL probably has ampersands in it. I had this problem, too, and I realized that my URL was full of ampersands (from CGI variables being passed) and so everything was getting sent to background in a weird way and thus not redirecting properly. If you put quotes around the URL it will fix it.

Answer 2

The answer above didn't work for me, what did eventually was this syntax:

curl https://${URL} &> /dev/stdout | tee -a ${LOG}

tee puts the output on the screen, but also appends it to my log.

Answer 3

If you need the output in a file you can use a redirect:

curl https://vi.stackexchange.com/ -vs >curl-output.txt 2>&1

Please be sure not to flip the >curl-output.txt and 2>&1, which will not work due to bash's redirection behavior.

Answer 4

Just my 2 cents. The below command should do the trick, as answered earlier

curl -vs google.com 2>&1

However if need to get the output to a file,

curl -vs google.com > out.txt 2>&1

should work.

Answer 5

I found the same thing: curl by itself would print to STDOUT, but could not be piped into another program.

At first, I thought I had solved it by using xargs to echo the output first:

curl -s ... <url> | xargs -0 echo | ...

But then, as pointed out in the comments, it also works without the xargs part, so -s (silent mode) is the key to preventing extraneous progress output to STDOUT:

curl -s ... <url> | perl  -ne 'print $1 if /<sometag>([^<]+)/'

The above example grabs the simple <sometag> content (containing no embedded tags) from the XML output of the curl statement.

Answer 6

The following worked for me:

Put your curl statement in a script named abc.sh

Now run:

sh abc.sh 1>stdout_output 2>stderr_output

You will get your curl's results in stdout_output and the progress info in stderr_output.

Answer 7

This simple example shows how to capture curl output, and use it in a bash script

test.sh

function main
{
  \curl -vs 'http://google.com'  2>&1
  # note: add -o /tmp/ignore.png if you want to ignore binary output, by saving it to a file. 
}

# capture output of curl to a variable
OUT=$(main)

# search output for something using grep.
echo
echo "$OUT" | grep 302 
echo
echo "$OUT" | grep title 

Answer 8

Solution = curl -vs google.com 2>&1 | less

BUT, if you want to redirect the output to a file and the output is still on the screen, then the URL response contains a newline char \n which messed up your shell.

To avoit this put everything in a variable:

result=$(curl -v . . . . )