Merge two mapping hierarchies from YAML files, preserving all keys

In this article, php editor Xigua will introduce how to merge two mapping hierarchies from YAML files and preserve all keys. The process is very simple and only requires the use of appropriate functions and methods. By merging two mapping hierarchies, we can combine their keys and values ​​to create a larger, more comprehensive mapping hierarchy. Whether you are working with configuration files or data processing, merging mapping hierarchies is a very useful technique. Next, we will walk you through the process step by step so that you can easily master this technique.

Question content

I am working on a solution involving nested keys in a yaml file. The software will read the files passed in the parameters and load them in the order in which keys are updated/added.

I have 2 yaml files and I want to merge them without losing any keys. I want to stack all profiles to generate a single map without removing any keys.

So I have yaml 1

env: test1
template:
  app:
    database: 
      name: oracle

yaml2

env: test2
template:
  app:
    database: 
      version : 12

The result I want is (the order is yaml1 - yaml2)

env: test2
template:
  app:
    database: 
      name: oracle
      version: 12

I tried copying using a map but since the keys have the same name I ended up with

env: test2
template:
    app:
        database:
            version: 12

I am using

gopkg.in/yaml.v3 for reading yaml provides me with the map[string] interface{}

and map using copy

package main

import (
    "fmt"
    "log"
    "maps"
    "os"
    "path/filepath"

    "gopkg.in/yaml.v3"
)

type configuration struct {
    c  m
    fl []string
}

type m = map[string]interface{}

func (c *configuration) Update(nc m) {
    if c.c == nil {
        c.c = nc
    } else {
        maps.Copy(c.c, nc)
    }
}

func (c configuration) Print() {
    d, err := yaml.Marshal(&c.c)
    if err != nil {
        log.Fatalf("error: %v", err)
    }
    fmt.Printf("---:\n%s\n\n", string(d))
}

func (c configuration) ParseDir(path string) {

}

func (c *configuration) LoadFromFile(filename string) {

    // YAML string stored in a variable
    yf, yfErr := os.ReadFile(filename)

    if yfErr != nil {
        log.Fatal("Error reading the file ", yfErr)
    }
    // Map to store the parsed YAML data
    var data m

    // Unmarshal the YAML string into the data map
    err := yaml.Unmarshal(yf, &data)
    if err != nil {
        log.Fatal(err)
    }
    c.Update(data)
}

func listFiles(path string) []string {
    var returnLf []string
    err := filepath.Walk(path,
        func(path string, info os.FileInfo, err error) error {
            if err != nil {
                return err
            }
            if info.Mode().IsRegular() {
                returnLf = append(returnLf, path)
            }
            return nil
        })
    if err != nil {
        log.Println(err)
    }
    return returnLf

}

Workaround

Suppose you want to merge YAML maps with "template" as the key from two YAML documents, a fairly simple implementation would look like this:

package main

import (
    "fmt"

    "gopkg.in/yaml.v3"
)

const data1 = `---
env: test1
template:
  app:
    database: 
      name: oracle
      foo: whatever
`

const data2 = `---
env: test2
template:
  app:
    some_stuff: [1, 2, 3, 4]
    database: 
      version : 12
      foo: 42
`

type T struct {
    Env  string         `yaml:"env"`
    Tmpl map[string]any `yaml:"template"`
}

func mergeMapsRecursively(dst, src map[string]any) map[string]any {
    res := make(map[string]any)

    for dstKey, dstVal := range dst {
        srcVal, exists := src[dstKey]
        if !exists {
            res[dstKey] = dstVal
            continue
        }

        dstValMap, dstValIsMap := dstVal.(map[string]any)
        srcValMap, srcValIsMap := srcVal.(map[string]any)
        if dstValIsMap && srcValIsMap {
            res[dstKey] = mergeMapsRecursively(dstValMap, srcValMap)
        } else {
            res[dstKey] = srcVal
        }
    }

    for srcKey, srcVal := range src {
        if _, exists := dst[srcKey]; !exists {
            res[srcKey] = srcVal
        }
    }

    return res
}

func main() {
    var a, b T

    if err := yaml.Unmarshal([]byte(data1), &a); err != nil {
        panic(err)
    }

    if err := yaml.Unmarshal([]byte(data2), &b); err != nil {
        panic(err)
    }

    fmt.Printf("%#v\n%#v\n%#v\n", a.Tmpl, b.Tmpl, mergeMapsRecursively(a.Tmpl, b.Tmpl))
}

Playground Link.

mergeMapsRecursively Function recursively merges fields that exist in two maps (if they are both maps), or replaces the value in dst with src value in , otherwise — just like maps.Copy does.

If this isn't exactly what you asked for, I hope my example will get you on the right track anyway.

The above is the detailed content of Merge two mapping hierarchies from YAML files, preserving all keys. For more information, please follow other related articles on the PHP Chinese website!