CSV new-line character seen in unquoted field error
the following code worked until today when I imported from a Windows machine and got this error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode? import csv class CSV: def __init__(self, file=None): self.file = file def read_file(self): data =  file_read = csv.reader(self.file) for row in file_read: data.append(row) return data def get_row_count(self): return len(self.read_file()) def get_column_count(self): new_data = self.read_file() return len(new_data) def get_data(self, rows=1): data = self.read_file() return data[:rows] How can I fix this issue? def upload_configurator(request, id=None): """ A view that allows the user to configurator the uploaded CSV. """ upload = Upload.objects.get(id=id) csvobject = CSV(upload.filepath) upload.num_records = csvobject.get_row_count() upload.num_columns = csvobject.get_column_count() upload.save() form = ConfiguratorForm() row_count = csvobject.get_row_count() colum_count = csvobject.get_column_count() first_row = csvobject.get_data(rows=1) first_two_rows = csvobject.get_data(rows=5)
It’ll be good to see the csv file itself, but this might work for you, give it a try, replace:
file_read = csv.reader(self.file)
file_read = csv.reader(self.file, dialect=csv.excel_tab)
Or, open a file with
universal newline mode and pass it to
reader = csv.reader(open(self.file, 'rU'), dialect=csv.excel_tab)
splitlines(), like this:
def read_file(self): with open(self.file, 'r') as f: data = [row for row in csv.reader(f.read().splitlines())] return data
I realize this is an old post, but I ran into the same problem and don’t see the correct answer so I will give it a try
_csv.Error: new-line character seen in unquoted field
Caused by trying to read Macintosh (pre OS X formatted) CSV files. These are text files that use CR for end of line. If using MS Office make sure you select either plain CSV format or CSV (MS-DOS). Do not use CSV (Macintosh) as save-as type.
My preferred EOL version would be LF (Unix/Linux/Apple), but I don’t think MS Office provides the option to save in this format.
For Mac OS X, save your CSV file in “Windows Comma Separated (.csv)” format.
If this happens to you on mac (as it did to me):
- Save the file as
CSV (MS-DOS Comma-Separated)
Run the following script
with open(csv_filename, 'rU') as csvfile: csvreader = csv.reader(csvfile) for row in csvreader: print ', '.join(row)
Try to run
dos2unix on your windows imported files first
This is an error that I faced. I had saved .csv file in MAC OSX.
While saving, save it as “Windows Comma Separated Values (.csv)” which resolved the issue.
This worked for me on OSX.
# allow variable to opened as files from io import StringIO # library to map other strange (accented) characters back into UTF-8 from unidecode import unidecode # cleanse input file with Windows formating to plain UTF-8 string with open(filename, 'rb') as fID: uncleansedBytes = fID.read() # decode the file using the correct encoding scheme # (probably this old windows one) uncleansedText = uncleansedBytes.decode('Windows-1252') # replace carriage-returns with new-lines cleansedText = uncleansedText.replace('\r', '\n') # map any other non UTF-8 characters into UTF-8 asciiText = unidecode(cleansedText) # read each line of the csv file and store as an array of dicts, # use first line as field names for each dict. reader = csv.DictReader(StringIO(cleansedText)) for line_entry in reader: # do something with your read data
I know this has been answered for quite some time but not solve my problem. I am using DictReader and StringIO for my csv reading due to some other complications. I was able to solve problem more simply by replacing delimiters explicitly:
with urllib.request.urlopen(q) as response: raw_data = response.read() encoding = response.info().get_content_charset('utf8') data = raw_data.decode(encoding) if '\r\n' not in data: # proably a windows delimited thing...try to update it data = data.replace('\r', '\r\n')
Might not be reasonable for enormous CSV files, but worked well for my use case.
Alternative and fast solution : I faced the same error. I reopened the “wierd” csv file in GNUMERIC on my lubuntu machine and exported the file as csv file. This corrected the issue.
- Database Administration Tutorials
- Programming Tutorials & IT News
- Linux & DevOps World
- Entertainment & General News
- Games & eSport