pandas 库函数方法集合--drop_duplicates

20231230星期六：

# 知识点：drop_duplicates：

import pandas as pd

data1 = [
    {"id":"156","add_id":"add01","name_id":"420101","class":"math"},
    {"id":"157","add_id":"add02","name_id":"420102","class":"math"},
    {"id":"158","add_id":"add03","name_id":"420103","class":"eng"},
    {"id":"159","add_id":None,"name_id":"420104","class":"eng"},
    {"id":"159","add_id":None,"name_id":"420104","class":"eng"},
    {"id":"159","add_id":None,"name_id":"420105","class":"eng"},
    {"id":None,"add_id":None,"name_id":None,"class":None},
]

datadf = pd.DataFrame(data1)
print("drop_duplicates 之前\n",datadf)
# inplace=True，表示在原有数据上进行修改，如果为False(默认是False)，那么就要重新赋值给新的变量：
datadf.drop_duplicates(subset=None,inplace=True)
print("drop_duplicates 之后\n",datadf)

# subset：指定在哪些列上进行去重操作，默认为所有列。
datadf.drop_duplicates(subset=['id','add_id'],inplace=True)
print("drop_duplicates 指定列去重之后\n",datadf)

drop_duplicates 之前

id add_id name_id class

0 156 add01 420101 math

1 157 add02 420102 math

2 158 add03 420103 eng

3 159 None 420104 eng

4 159 None 420104 eng

5 159 None 420105 eng

6 None None None None

drop_duplicates 之后

id add_id name_id class

0 156 add01 420101 math

1 157 add02 420102 math

2 158 add03 420103 eng

3 159 None 420104 eng

5 159 None 420105 eng

6 None None None None

drop_duplicates 指定列去重之后

id add_id name_id class

0 156 add01 420101 math

1 157 add02 420102 math

2 158 add03 420103 eng

3 159 None 420104 eng

6 None None None None