Shell Script to convert hexadecimal values in a file to decimal

Question

I have a file named as text containing all hexadecimal numbers. I wrote the following code to convert those values to decimal:

for line in `cat text`; do

arp=$(echo "ibase=16; $line" | bc);echo $arp
     done

But it's giving me the following error:

(standard_in) 1: syntax error

My input file contains one column of hexadecimal values, e.g.

428a2f98 
71374491 
b5c0fbcf

Answer 1

It's not bash that produces the error, it's bc which requires hexadecimal numbers to use uppercase letters while your input file uses lowercase letters.

If you use bash 4 you can use ${foo^^} to expand $foo to uppercase:

bc <<< "ibase=16; ${line^^}"

or you can use tr:

bc <<< "ibase=16; $(tr '[:lower:]' '[:upper:]' <<< "${line}")"

Answer 2

If your hex numbers are in the following form:

0x1
0x2
0x3
0x4
0x5
0x6
0x7
0x8
0x9
0xA
0xB
0xC
0xD
0xE
0xF

I mean prefixed with 0x you can use:

while read line
do
     printf '%d\n' $line

done < text

Ouput:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

Answer 3

Although this question is answered, when you don't know exactly the position of numbers, this sed oneliner can help other people looking for help, and only finding this question on the net:

$ cat file | sed 's/^/echo "/;s/(0x\(..\))/(\$(echo "ibase=16;\U\1\E"\|bc))/g;s/$/"/e'

Explaining:

• s/^/echo "/ adds echo " at start
• s/(0x\(..\))/(\$(echo "ibase=16;\U\1\E"\|bc))/g needs further explaining:
  1. find (0x..) pattern. You can change to ^........ in your case
  2. creates a subshell $() where hex pattern \1 is procesed by bc
     • \U is used to uppercase hex digits. \E stops uppercasing.
• s/$/"/e adds " at end and process line with sh

You can test it on xmodmap output, which output hexadecimal characters for modifiers, while xev shows modifiers as decimals.

Hope it becomes handy in other use cases

Answer 4

while read line
do
    arp=$(echo "ibase=16; $line" | bc)
    echo $arp
done < in.txt

For in.txt

1
2
3
4
5
6
7
8
9
A
B
C
D
E
F
A0
A1
FF

This prints:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
160
161
255

Answer 5

This answer is based on albfan's answer and posted for the same reason (other people looking for help, and only finding this question on the net).
I am expanding on it to allow for replacements in files that contain backslashes, backticks, double quotes and dollar signs.
This way it is also much safer to use on unknown inputs, preventing code injection.

#!/bin/bash
cat $1 | sed -r 's/\\/\\\\/g;s/`/\\`/g;s/"/\\"/g;s/\$/\\$/g;s/0x([0-9A-Fa-f]+)/\$(echo "ibase=16;\U\1\E" \| bc)/g;s/^/echo "/;s/$/"/e'

To answer the original question, the 0x needs to be removed:

#!/bin/bash
cat $1 | sed -r 's/\\/\\\\/g;s/`/\\`/g;s/"/\\"/g;s/\$/\\$/g;s/([0-9A-Fa-f]+)/\$(echo "ibase=16;\U\1\E" \| bc)/g;s/^/echo "/;s/$/"/e'

Simple demonstration:

Input line:

console.log("This is a number: 0xffffff")

Intermediate form:

echo "console.log(\"This is a number: $(echo "ibase=16;FFFFFF" | bc)\")"

Output line:

console.log("This is a number: 16777215")

Usage:

Save as hex-to-decimal.sh, chmod 555 hex-to-decimal.sh to make it executable and uneditable and run with ./hex-to-decimal.sh inputfilename > outputfilename

Or just run as a one-liner with cat inputfilename | sed -r 's/\\/\\\\/g;s/`/\\`/g;s/"/\\"/g;s/\$/\\$/g;s/0x([0-9A-Fa-f]+)/\$(echo "ibase=16;\U\1\E" \| bc)/g;s/^/echo "/;s/$/"/e' > outputfilename

Explanation:

cat $1

• List the contents of the input file line by line.

| sed -r

• Apply a bunch of transformations (in this case substitutions and one execution) to each line of text and print the result to stdout.

s/\\/\\\\/g;

• Place a backslash before every backslash in this line.

s/`/\\`/g; s/"/\\"/g; s/\$/\\$/g;

• Do the same for backticks, double quotes and dollar signs.

s/0x([0-9A-Fa-f]+)

• Substitute anything that starts with 0x and is followed by a hexadecimal number.
• For the original question version, 0x is omitted and it matches anything that is a valid hexadecimal number. For example causing fridge to be replaced with 15ri13g14.

/\$(echo "ibase=16;\U\1\E" \| bc)/g;

• Replace it with $(echo "ibase=16; + the first group capitalized (the number without 0x) + " | bc)
• \$(echo "ibase=16;\U\1\E" \| bc)

  • $(<command>)

    • starts a child bash instance that executes the command and resumes execution with it being replaced by the output of the child (has no effect from within single quote ('') strings)

  • \U\1\E

    • \U makes what comes after uppercase, \1 is the first group (which was [0-9A-Fa-f]+), \E makes the uppercasing not apply to what comes after it. In other words; the hexadecimal number, made uppercase.

  • echo "ibase=16;<hexadecimal number>" | bc

    • Convert a hexadecimal number (input base = 16) to decimal (default output type) with the bc command.

s/^/echo "/;

• Place echo " at the start of the line.

s/$/"/e

• Place " at the end of the line and execute the line.