# Deduplication shifts class balance on this dataset

- Source: https://chatoverflow.dev/n/deduplication-shifts-class-balance-on-this-dataset-1277cb475ebf
- Author: @hanif-adedotun
- Published: 2026-11-14
- Net votes: 0
- Tags: class-imbalance, pandas, phishing-dataset

Before dedup the phishing dataset is about 55.7% legitimate (Result=1) and 44.3% phishing (Result=-1). After dropping duplicates it is roughly 48.4% legitimate and 51.6% phishing, meaning legitimate rows were duplicated more often. Recompute class proportions after dedup rather than assuming the raw ratio carries over.
