State of golang.org/x/text

Segmentation

• High cost of bridging with C, slowing down IATN and localization
• Need to rethink language matching and language identification
• Unicode flags can be represented with two code points

Data Flow

• Need to preprocess data to make it usable for Go
• ICU and Go implementations are different, but Go is more compact
• ICU has a solution for complex linguistic features

Programming

• Go code is shorter than ICU string
• Programmer should not be forced to become a linguist
• Need to design a better API for language matching and identification

Language Support

• Angolan Portuguese is closer to European Portuguese than Brazilian Portuguese
• Formalisms support complex linguistic features, but Go lacks this support
• ICU provides an extension for complex linguistic features

Server-Specific Issues

• Need to handle server-specific requirements for language identification and matching
• Go applications are mostly servers, so need to adapt for server-centric language support

API Design

• Go’s API should be designed to be easy for linguists to add attributes
• Need to optimize for Go’s statically linked binaries
• ICU has a solution for complex linguistic features, but Go should not rely too heavily on ICU

File Formats

• Go’s file formats should be generic and not English-centric
• Need to design a better API for language matching and identification

Language Identification

• Need to rethink language identification and matching in Go
• ICU has a solution for complex linguistic features, but Go should not rely too heavily on ICU

Language Support for Text

• Need to design a better API for language matching and identification
• Go should provide support for complex linguistic features
• ICU and Go implementations are different, but Go is more compact