how to merge multiple dataframes with different amount of rows into one giant dataframe in python

You can use pandas.concat() method to merge multiple dataframes. First, put all the required dataframes in a list and pass it as an argument in pandas.concat() method. Use ignore_index=True as a parameter to ignore previous dataframes indexes and reset with new index. Here's an example:

main.py
import pandas as pd

df1 = pd.DataFrame({'A': ['A0', 'A1', 'A2', 'A3'],
                    'B': ['B0', 'B1', 'B2', 'B3'],
                    'C': ['C0', 'C1', 'C2', 'C3'],
                    'D': ['D0', 'D1', 'D2', 'D3']})

df2 = pd.DataFrame({'A': ['A4', 'A5', 'A6'],
                    'B': ['B4', 'B5', 'B6'],
                    'C': ['C4', 'C5', 'C6'],
                    'D': ['D4', 'D5', 'D6']})

df3 = pd.DataFrame({'A': ['A7', 'A8'],
                    'B': ['B7', 'B8'],
                    'C': ['C7', 'C8'],
                    'D': ['D7', 'D8']})

merged_df = pd.concat([df1, df2, df3], ignore_index=True)

print(merged_df)

643 chars
21 lines

Output:

main.py
    A   B   C   D
0  A0  B0  C0  D0
1  A1  B1  C1  D1
2  A2  B2  C2  D2
3  A3  B3  C3  D3
4  A4  B4  C4  D4
5  A5  B5  C5  D5
6  A6  B6  C6  D6
7  A7  B7  C7  D7
8  A8  B8  C8  D8

180 chars
11 lines

In this example, we have merged 3 dataframes df1, df2, and df3. merged_df is the final dataframe that contains all the rows of df1, df2, and df3. As you can see, ignore_index=True parameter has reset the indexes for merged_df.

similar python code snippets

how to join two dataframes of different years on the same days in python
moving average in pandas in python
update a dataframe row from another row in python
not select columns of a dataframe from a list of those columns in python
merge a dataframe row values from another row values in python
add a numpy array to dataframe in python
merge two series in a dataframe in python
merge two dataframe by a column in python
merge with the help of left join, and with the remaining dataframes - through inner. as a result of connecting dataframes in python
how to append at the end of a dataframe in python

related categories

pandas
dataframes
merge

gistlibby LogSnag