csvを開いて重複の辞書を削除をし、再保存。
id,name,address,prefecture_id,city_code
1,abc,東京都江戸川区,13,13123
1,abc,東京都江戸川区,13,13123
2,def,東京都墨田区,13,13107
3,ghi,東京都荒川区,13,13118
hoge.py
import pandas as pd
df = pd.read_csv('data/csv/a.csv')
dict_list = df.to_dict(orient='records')
name_list = [dict['name'] for dict in dict_list]
unique_dict_list = [dict for i, dict in enumerate(dict_list) if dict['name'] not in name_list[0:i]]
for index, dict in enumerate(unique_dict_list):
dict['id'] = index + 1
df = pd.DataFrame.from_dict(unique_dict_list, orient='columns')
df = df.set_index('id')
df.to_csv('data/csv/a.csv')
csvを辞書に変換
dict_list = df.to_dict(orient='records')
辞書のidが重複しているものを削除
name_list = [dict['name'] for dict in dict_list]
unique_dict_list = [dict for i, dict in enumerate(dict_list) if dict['name'] not in name_list[0:i]]
idの再振り当て
for index, dict in enumerate(unique_dict_list):
dict['id'] = index + 1
辞書型リストをcsvに戻し保存
df = pd.DataFrame.from_dict(unique_dict_list, orient='columns')
df = df.set_index('id')
df.to_csv('data/csv/a.csv')
↧