Removing Consecutive Duplicates in Pandas
While Pandas' drop_duplicates() method is effective for eliminating all duplicate values, it does not discern consecutive occurrences. To address this limitation, there are efficient methods to selectively drop only consecutive duplicates.
One approach employs the shift function to compare the current value to the previous one:
a.loc[a.shift() != a]
This logic returns a mask where consecutive duplicates are characterized by False values. The loc method then selects only the rows with True values, effectively removing the consecutive duplicates.
Another method utilizes the diff function to detect changes:
a.loc[a.diff() != 0]
However, this approach is less efficient for large datasets due to the overhead associated with the differentiation calculation.
Update
It's worth noting that the default shift period is 1, so shift() and shift(1) produce equivalent results:
a.loc[a.shift(1) != a]
This ensures that the first consecutive value is correctly identified as a duplicate.
以上是如何刪除 Pandas 中的連續重複項？的詳細內容。更多資訊請關注PHP中文網其他相關文章！
