'TypeError: Object of type int64 is not JSON serializable' when json.dump-ing fraud metrics (tp/fp/precision) computed with pandas/scikit-learn from Nessie transactions
Symptom: a fraud detector that pulls Nessie v2 transactions into pandas, scores them with scikit-learn's IsolationForest and writes evaluation_metrics.json crashed at the very end with 'TypeError: Object of type int64 is not JSON serializable'. The whole run (simulation, feature building, model fit) had to be repeated after the fix, so this cost several minutes per attempt.
Cause: counts like len(df[mask]) are Python ints, but values from y_true.sum(), confusion-matrix cells (tn, fp, fn, tp = confusion_matrix(...).ravel()), and df['amount_cents'].mean() are numpy.int64 / numpy.float64. The standard-library json encoder only knows the built-in types.
Fix: convert at the boundary:
metrics = {
'total_transactions': int(len(df)),
'disputed_count': int(y_true.sum()),
'true_positives': int(tp), 'false_positives': int(fp),
'precision': float(precision), 'recall': float(recall),
}
json.dump(metrics, f, indent=2)or pass a default hook: json.dump(metrics, f, default=lambda o: o.item() if hasattr(o, 'item') else str(o)). For DataFrames use df.to_json() / df.to_dict(orient='records') after astype(object), or write CSV.
Tip for long Nessie runs: write the metrics file (or at least print the dict) before any slow step that comes after, and run a tiny smoke test of the serialization with fake numpy values. Then a type error at the end doesn't throw away a whole simulator run, which can take minutes of queue time.