language: count '_' alongside '-' in ParseAcceptLanguage guard The BCP 47 scanner aliases '_' to '-' in scanner.init at internal/language/parse.go lines 104-111, so a payload built with '_' as the separator bypasses the 1000-dash guard at language/parse.go:169 (the CVE-2022-32149 fix landed in CL 442235) and re-triggers the O(N^2) gobble path on attacker-controlled Accept-Language input. Count both separators in the guard. Reproduction summary: 100k tokens with '_' separator: 1.35 s parse time Same 100k tokens with '-': 49.9 us (guard fires) Amplification: ~27,000x See golang/go#79684 for the full reproducer with go.mod, main.go, and measurement output. Test: TestParseAcceptLanguageUnderscoreGuard in language/parse_test.go asserts errTagListTooLarge is returned when 1001 '_' separators are present. Filed per the upstream invitation from the Go Security Officers: the original security@golang.org report was determined to be not a security issue, with an invitation to file as an upstream change. Fixes golang/go#79684 Change-Id: I4e9a582c54a3e7268072870c0b22a3a392cfa1f2 GitHub-Last-Rev: 701c4da23913b82c62e71ade7c80d178f22d7b21 GitHub-Pull-Request: golang/text#66 Reviewed-on: https://go-review.googlesource.com/c/text/+/783500 Reviewed-by: Nicholas Husin <husin@google.com> Reviewed-by: Nicholas Husin <nsh@golang.org> Auto-Submit: Damien Neil <dneil@google.com> LUCI-TryBot-Result: golang-scoped@luci-project-accounts.iam.gserviceaccount.com <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Roland Shoemaker <roland@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Auto-Submit: Roland Shoemaker <roland@golang.org>
This repository holds supplementary Go packages for text processing, many involving Unicode.
It is important that the Unicode version used in x/text matches the one used by your Go compiler. The x/text repository supports multiple versions of Unicode and will match the version of Unicode to that of the Go compiler. At the moment this is supported for Go compilers from version 1.7.
To submit changes to this repository, see http://go.dev/doc/contribute.
The git repository is https://go.googlesource.com/text.
To generate the tables in this repository (except for the encoding tables), run go generate from this directory. By default tables are generated for the Unicode version in core and the CLDR version defined in golang.org/x/text/unicode/cldr.
Running go generate will as a side effect create a DATA subdirectory in this directory, which holds all files that are used as a source for generating the tables. This directory will also serve as a cache.
Run
go test ./...
from this directory to run all tests. Add the “-tags icu” flag to also run ICU conformance tests (if available). This requires that you have the correct ICU version installed on your system.
TODO:
To generate the tables in this repository (except for the encoding tables), run go generate from this directory. By default tables are generated for the Unicode version in core and the CLDR version defined in golang.org/x/text/unicode/cldr.
Running go generate will as a side effect create a DATA subdirectory in this directory which holds all files that are used as a source for generating the tables. This directory will also serve as a cache.
To update a Unicode version run
UNICODE_VERSION=x.x.x go generate
where x.x.x must correspond to a directory in https://www.unicode.org/Public/. If this version is newer than the version in core it will also update the relevant packages there. The idna package in x/net will always be updated.
To update a CLDR version run
CLDR_VERSION=version go generate
where version must correspond to a directory in https://www.unicode.org/Public/cldr/.
Note that the code gets adapted over time to changes in the data and that backwards compatibility is not maintained. So updating to a different version may not work.
The files in DATA/{iana|icu|w3|whatwg} are currently not versioned.
The main issue tracker for the text repository is located at https://go.dev/issues. Prefix your issue with “x/text:” in the subject line, so it is easy to find.