Home Backend Development Golang How to Efficiently Remove Diacritics from UTF-8 Strings in Go?

How to Efficiently Remove Diacritics from UTF-8 Strings in Go?

Dec 08, 2024 pm 02:03 PM

How to Efficiently Remove Diacritics from UTF-8 Strings in Go?

Removing Diacritics in Go

When working with UTF8 encoded strings, it may be necessary to remove diacritics, such as the accents from "žůžo" to get "zuzo". To handle such scenarios efficiently, there are standard libraries and techniques available in Go.

One approach involves leveraging the unicode.Is() function to identify diacritics (characters classified as "Mn" for nonspacing marks).

The following code snippet demonstrates how to remove diacritics from a given string utilizing the unicode/norm and golang.org/x/text/transform packages:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

package main

 

import (

    "fmt"

    "unicode"

 

    "golang.org/x/text/transform"

    "golang.org/x/text/unicode/norm"

)

 

func isMn(r rune) bool {

    return unicode.Is(unicode.Mn, r) // Mn: nonspacing marks

}

 

func main() {

    t := transform.Chain(norm.NFD, transform.RemoveFunc(isMn), norm.NFC)

    result, _, _ := transform.String(t, "žůžo")

    fmt.Println(result)

}

Copy after login

This code removes diacritics by applying a series of transformations:

  1. Normalized Form Decomposition (NFD): Breaks down the string into its base Unicode characters, including diacritics.
  2. RemoveFunc(isMn): Filters out characters that are nonspacing marks (diacritics).
  3. Normalization Form Composition (NFC): Recomposes the string without diacritics.

As a result, the output will be a string stripped of diacritics, as in the example: "žůžo" => "zuzo".

The above is the detailed content of How to Efficiently Remove Diacritics from UTF-8 Strings in Go?. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot Article Tags

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Go language pack import: What is the difference between underscore and without underscore? Go language pack import: What is the difference between underscore and without underscore? Mar 03, 2025 pm 05:17 PM

Go language pack import: What is the difference between underscore and without underscore?

How do I write mock objects and stubs for testing in Go? How do I write mock objects and stubs for testing in Go? Mar 10, 2025 pm 05:38 PM

How do I write mock objects and stubs for testing in Go?

How to implement short-term information transfer between pages in the Beego framework? How to implement short-term information transfer between pages in the Beego framework? Mar 03, 2025 pm 05:22 PM

How to implement short-term information transfer between pages in the Beego framework?

How can I define custom type constraints for generics in Go? How can I define custom type constraints for generics in Go? Mar 10, 2025 pm 03:20 PM

How can I define custom type constraints for generics in Go?

How can I use tracing tools to understand the execution flow of my Go applications? How can I use tracing tools to understand the execution flow of my Go applications? Mar 10, 2025 pm 05:36 PM

How can I use tracing tools to understand the execution flow of my Go applications?

How to write files in Go language conveniently? How to write files in Go language conveniently? Mar 03, 2025 pm 05:15 PM

How to write files in Go language conveniently?

How can I use linters and static analysis tools to improve the quality and maintainability of my Go code? How can I use linters and static analysis tools to improve the quality and maintainability of my Go code? Mar 10, 2025 pm 05:38 PM

How can I use linters and static analysis tools to improve the quality and maintainability of my Go code?

How to convert MySQL query result List into a custom structure slice in Go language? How to convert MySQL query result List into a custom structure slice in Go language? Mar 03, 2025 pm 05:18 PM

How to convert MySQL query result List into a custom structure slice in Go language?

See all articles