So I was working on something very simple involving a CSV file:
import csv
with open('file.csv', 'r') as f:
reader = csv.reader(f)
lines = [ln for ln in reader]
And as one would expect, the result of running this code was Error: field larger than field limit (131072)
The 128 KiB field size boundary is so beautifully arbitrary. Use the following workaround to fix that:
import sys
csv.field_size_limit(sys.maxsize)
Doesn't have to be specifically sys.maxsize
, any sufficiently large number would do.
There's no moral to this story, just mild suffering.