How to Combine DataFrames Generated in a For Loop Using pd.concat?

Merging DataFrames Generated in a For Loop

When working with multiple data sources, it is often necessary to combine data into a single consolidated dataframe. This question illustrates a common issue faced when attempting to append dataframes generated within a for loop using the pd.concat function.

The initial approach presented in the question faces an error due to the incorrect invocation of pd.append. This function requires at least two arguments, the first being the dataframe to append to, while the second argument should be the data to be appended. The code tries to append data to itself, which is not valid.

The correct way to append dataframes is to store them in a list and then use pd.concat to merge them into a single dataframe. Here's an improved solution:

<code class="python">appended_data = []
for infile in glob.glob("*.xlsx"):
    data = pandas.read_excel(infile)
    appended_data.append(data)
# concatenate the list of dataframes
appended_data = pd.concat(appended_data)
# save the merged dataframe to an excel file
appended_data.to_excel('appended.xlsx')</code>

This code imports the necessary libraries, iterates over the excel files, reads data from each file and stores the dataframe in a list. Finally, it uses pd.concat to concatenate the list of dataframes and exports the merged dataframe to a new excel file. This approach allows for seamless appending of dataframes generated in a loop.

The above is the detailed content of How to Combine DataFrames Generated in a For Loop Using pd.concat?. For more information, please follow other related articles on the PHP Chinese website!