Skip to content
fuck-powershell

A BOM-less .ps1 is read as ANSI — non-ASCII corrupts before execution

encoding · case

5.1silentscriptagentfirst-partyrepro: verifiedbom-sniffingdefault-encoding
Affectspowershell 51, windows, korean codepage
Fails asMOJIBAKE
Mechanismbom sniffing, default encoding
Safe fixutf8 bom

Symptom

Section titled “Symptom”

A script containing non-ASCII string literals (Korean paths, emoji, accented names) behaves wrongly under Windows PowerShell 5.1: comparisons fail, files with localized names are not found, output is mojibake. The same file runs perfectly under pwsh 7. Console output “looking garbled” proves nothing — the corruption happened at parse time.

Repro

Section titled “Repro”
Terminal window
# Save this WITHOUT a BOM as test.ps1 (UTF-8):   $s = "한글"; $s.Length
powershell.exe -File test.ps1   # 5.1 on a Korean host reads it as CP949: wrong Length
pwsh -File test.ps1             # 7: correct

The only reliable check is byte-level: inspect the literal’s length/bytes, not how the console renders it. This trap was documented as a hard rule for AI agents writing Windows scripts (see ref): “.ps1 needs a UTF-8 BOM.”

Cause

Section titled “Cause”

Windows PowerShell 5.1 decides a script file’s encoding by sniffing for a BOM. No BOM means the legacy ANSI code page (CP949 on Korean systems, CP1252 on Western ones). Every non-ASCII literal is corrupted before the script runs. PowerShell 7 assumes UTF-8 by default, which is why the bug is invisible in 7-only testing — and why agents generating .ps1 files keep shipping it.

Workaround

Section titled “Workaround”
  • Write .ps1 files as UTF-8 with BOM whenever 5.1 might run them.
  • CI-lint your repo: any .ps1 containing non-ASCII bytes must start with EF BB BF.
  • Verify with byte checks (Format-Hex, $s.Length), never with console output.

Refs

Section titled “Refs”