tree 6317f8c4f3b7b5b0bc89f02b2ad1a5d259dc3bf9
parent 81608d7e9c6863c922f599e4ff1329a685218c0d
author Marcel van Lohuizen <mpvl@golang.org> 1589710970 +0200
committer Marcel van Lohuizen <mpvl@golang.org> 1591939050 +0000

encoding/unicode: add UTF8BOM encoding

Some editors always add a BOM to UTF-8. Tradionally the BOM
has been used to detect byte order encoding, which is irrelevant
for UTF-8. These editors, however, use the UTF-8 BOM as
a signature, allowing to override a charmap encoding if such
a BOM is present. This is possible as the occurence of a BOM
in such encodings is highly unlikely.

This UTF8BOM encoding implements a simple encoding for this.
It is intended for applications that require a UTF-8 encoding, but
want to handle files written by such editors without explicit BOM
handling. It can also be used to create such files.

NOTE: there is currently no encoding that implements the fallback
encoding of such editors. The BOMOverride functlinality in this
package allows implementing such an encoder, with relative ease,
though.

Change-Id: I430851a1d93351bf6055eebe88005984dde451d9
Reviewed-on: https://go-review.googlesource.com/c/text/+/234277
Reviewed-by: Russ Cox <rsc@golang.org>