How can I efficiently merge multiple DataTables with varying schemas into a single DataTable in C#?

Combining Multiple DataTables with Variable Schemas into a Single Table

To address the issue of combining multiple DataTables into a single table, we recommend an efficient method that effectively handles the varying column structures and row alignment.

Quick Solution

The following C# code snippet provides a straightforward approach:

public static DataTable MergeTables(IList<DataTable> tables, string primaryKeyColumn)
{
    if (!tables.Any())
        throw new ArgumentException("Tables must not be empty", "tables");
    if (primaryKeyColumn != null)
        foreach (DataTable t in tables)
            if (!t.Columns.Contains(primaryKeyColumn))
                throw new ArgumentException("All tables must have the specified primarykey column " + primaryKeyColumn, "primaryKeyColumn");

    if (tables.Count == 1)
        return tables[0];

    DataTable table = new DataTable("TblUnion");
    table.BeginLoadData(); // Turns off notifications, index maintenance, and constraints while loading data
    foreach (DataTable t in tables)
    {
        table.Merge(t); // same as table.Merge(t, false, MissingSchemaAction.Add);
    }
    table.EndLoadData();

    if (primaryKeyColumn != null)
    {
        // join rows with repeating primary key columns
        var pkGroups = table.AsEnumerable()
            .GroupBy(r => r[primaryKeyColumn]);
        var dupGroups = pkGroups.Where(g => g.Count() > 1);
        foreach (var grpDup in dupGroups)
        { 
            // use first row and modify it
            DataRow firstRow = grpDup.First();
            foreach (DataColumn c in table.Columns)
            {
                if (firstRow.IsNull(c))
                {
                    DataRow firstNotNullRow = grpDup.Skip(1).FirstOrDefault(r => !r.IsNull(c));
                    if (firstNotNullRow != null)
                        firstRow[c] = firstNotNullRow[c];
                }
            }
            // remove all but first row
            var rowsToRemove = grpDup.Skip(1);
            foreach(DataRow rowToRemove in rowsToRemove)
                table.Rows.Remove(rowToRemove);
        }
    }

    return table;
}

Usage

To utilize the MergeTables method, follow these steps:

1. Define a list of input DataTables.
2. Optionally, specify the name of the primary key column if the tables share a common primary key.
3. Call the MergeTables method with the list of tables and the primary key column, if applicable.
4. The method returns a single DataTable with the merged data, handling any discrepancies in column structures and row alignment.

Example

Consider the following sample DataTables:

DataTable1 (Columns: c1, c2, c3, c4)
| c1 | c2 | c3 | c4 |
|---|---|---|---|
| 1 | 8500 | abc | A |
| 2 | 950 | cde | B |
| 3 | 150 | efg | C |
| 4 | 850 | ghi | D |
| 5 | 50 | ijk | E |

DataTable2 (Columns: c1, c5, c6, c7)
| c1 | c5 | c6 | c7 |
|---|---|---|---|
| 1 | 7500 | klm | F |
| 2 | 900 | mno | G |
| 3 | 150 | opq | H |
| 4 | 850 | qrs | I |
| 5 | 50 | stu | J |

DataTable3 (Columns: c1, c8, c9, c10)
| c1 | c8 | c9 | c10 |
|---|---|---|---|
| 1 | 7500 | uvw | K |
| 2 | 900 | wxy | L |
| 3 | 150 | yza | M |
| 4 | 850 | ABC | N |
| 5 | 50 | CDE | O |

To merge these three tables, we can call the MergeTables method as follows:

var tables = new[] { DataTable1, DataTable2, DataTable3 };
DataTable mergedTable = MergeTables(tables, "c1");

The mergedTable will contain all the rows from the three input tables, with the columns aligned by the common primary key column "c1". Any missing values in a row will be populated from the first non-null row with the same primary key value.

The above is the detailed content of How can I efficiently merge multiple DataTables with varying schemas into a single DataTable in C#?. For more information, please follow other related articles on the PHP Chinese website!