Wie entferne ich aufeinanderfolgende Duplikate in Pandas?

Removing Consecutive Duplicates in Pandas

While Pandas' drop_duplicates() method is effective for eliminating all duplicate values, it does not discern consecutive occurrences. To address this limitation, there are efficient methods to selectively drop only consecutive duplicates.

One approach employs the shift function to compare the current value to the previous one:

a.loc[a.shift() != a]

This logic returns a mask where consecutive duplicates are characterized by False values. The loc method then selects only the rows with True values, effectively removing the consecutive duplicates.

Another method utilizes the diff function to detect changes:

a.loc[a.diff() != 0]

However, this approach is less efficient for large datasets due to the overhead associated with the differentiation calculation.

Update

It's worth noting that the default shift period is 1, so shift() and shift(1) produce equivalent results:

a.loc[a.shift(1) != a]

This ensures that the first consecutive value is correctly identified as a duplicate.

Das obige ist der detaillierte Inhalt vonWie entferne ich aufeinanderfolgende Duplikate in Pandas?. Für weitere Informationen folgen Sie bitte anderen verwandten Artikeln auf der PHP chinesischen Website!