How to Normalize Text Input to ASCII Using Enhanced Unicode Handling in Go?

Normalizing Text Input to ASCII Using Enhanced Unicode Handling

Converting non-ASCII characters to their ASCII equivalents is a common requirement in many programming contexts. When working with user input, it becomes necessary to handle special characters like curly quotes in a consistent manner.

Utilizing Strings.Map for Rune Mapping

The Go standard library provides the strings.Map function for efficiently transforming a string's runes (Unicode code points) into a new string. This function allows for granular control over character replacements.

In the example provided, the following code converts curly quotes to straight quotes using the strings.Map function:

<code class="go">data := "Hello “Frank” or ‹François› as you like to be ‘called’"
cleanedData := strings.Map(normalize, data)</code>

The normalize function is a closure that handles the character replacements based on predefined Unicode character ranges:

<code class="go">func normalize(in rune) rune {
    switch in {
    case '“', '‹', '”', '›':
        return '"'
    case '‘', '’':
        return '\''
    }
    return in
}</code>

Output:

Original: Hello “Frank” or ‹François› as you like to be ‘called’
Cleaned: Hello "Frank" or "François" as you like to be 'called'

By utilizing strings.Map, we can normalize text input to ASCII equivalents without relying on string replacements. This provides a versatile and efficient approach for handling non-ASCII characters in various programming scenarios.

The above is the detailed content of How to Normalize Text Input to ASCII Using Enhanced Unicode Handling in Go?. For more information, please follow other related articles on the PHP Chinese website!