Batch script remove BOM () from file

Question

I have created a batch script to copy SQL files from a folder into one big SQL script. The problem is when I run this one SQL script it comes up with the error

Incorrect syntax near ''

I copied the one SQL script into Notepad++ and set the encoding to ANSI. I see this symbol  (BOM) on the lines where the error is happening.

Is there anyway I can automatically remove this within my batch script. I don't want to keep manually remove this every time I run this task.

Below is the batch script I have currently

@echo off

set "path2work=C:\StoredProcedures"
cd /d "%path2work%"

echo. > C:\FinalScript\AllScripts.sql

for %%a in (*.sql) do (

    echo. >>"C:\FinalScript\AllScripts.sql"
    echo GO >>"C:\FinalScript\AllScripts.sql"
    type "%%a">>"C:\FinalScript\AllScripts.sql"
    echo. >>"C:\FinalScript\AllScripts.sql"
)

Answer 1

As MSalters alreadyx mentioned in his comment, according to wikipedia  is the ANSI representation of an UTF8 BOM.

PowerShell is much better suited to the task dealing with encodings than batch:

## Q:\Test\2018\09\11\SO_522772705.ps1
Set-Location 'C:\StoredProcedures'
Get-ChildItem '*.sql' | ForEach-Object {
    "`nGO"
    Get-Content $_.FullName -Encoding UTF8
    ""
} | Set-Content 'C:\FinalScript\AllScripts.sql' -Encoding UTF8

To be on topic with the tag batch-file a batch invoking powershell for the essential part:

:: Q:\Test\2018\09\11\SO_522772705..cmd
@echo off
set "path2work=C:\StoredProcedures"
cd /d "%path2work%"

powershell -NoProfile -Command "Get-ChildItem '*.sql'|ForEach-Object{\"`nGO\";Get-Content $_.FullName -Enc UTF8;\"\"}|Set-Content 'C:\FinalScript\AllScripts.sql' -Enc UTF8"

Answer 2

You just need to change the encoding to UTF-8 without BOM and save the file

Note that the menu items are a little bit different on older Notepad++ versions

Answer 3

TypeWithoutBOM.bat

@echo off
set "RemoveUTF8BOM=(pause & pause & pause)>nul"
type %1|(%RemoveUTF8BOM% & findstr "^")

This batch file works like the type command but removes the first 3 bytes of the file which is shown.
Usage: TypeWithoutBOM UTF8-file.txt > newfile.txt