Encoding and character sets
Read a garbled CSV using its character encoding
Tools to use
The .csv extension does not determine a file's character encoding. If the creating and opening applications assume different encodings, identical bytes can display as different text. Saving that garbled display can replace the evidence, so preserve the original first.
The charset converter encodes text or decodes bytes under an explicitly selected convention. It does not automatically certify the original encoding. Start with the single character あ, then apply the same reasoning to a real file.
The bytes for あ depend on the encoding
UTF-8 e38182
UTF-16LE 4230
UTF-16BE 3042
Windows-31J 82a0Enter あ and switch the encoding to obtain these four results. The character is unchanged, while byte values and lengths differ. Hex output represents storage or transmission bytes, not a new unreadable character. Record the selected encoding alongside the hex.
Decode with the matching convention
Decode e38182 as UTF-8 and 82a0 as Windows-31J to recover あ. UTF-16 byte order is part of the convention: 4230 means different things in LE and BE. Match producer and consumer instead of treating the two UTF-16 variants as interchangeable.
A successful round trip confirms that example, not a unique diagnosis of the entire file's original encoding. ASCII-only text can look correct under several conventions. Check representative non-ASCII characters, symbols, and line endings, and consult the producer's export settings or specification.
A decoding error can preserve useful evidence
Invalid byte sequences produce an error rather than silently becoming replacement characters. For example, e381 truncates the UTF-8 sequence e38182. An error can indicate either the wrong selected encoding or missing input bytes; investigate both.
No error does not prove the interpretation is right: another encoding may also accept the bytes. Check expected words, symbols, record counts, and the producer's declared convention. Preserving the expected content matters more than merely displaying something.
Characters already replaced by ? or � may be lost
If only the display was decoded incorrectly, the original bytes may still be recoverable under the right convention. After saving replacement ? or � characters, different originals may have collapsed to the same substitute. That saved text alone cannot uniquely reveal what was lost.
Look for the original export or backup rather than repeatedly converting damaged text. Copying the garbled display is different from providing the original bytes. State which evidence you have and work on a separate copy instead of overwriting the source.
The name Shift_JIS alone is not a compatibility guarantee
Supported choices are UTF-8, UTF-16LE, UTF-16BE, and the Windows-31J mapping associated with WHATWG shift_jis. This is not a claim of complete identity with older JIS standards or every product using the same label. Test the actual problematic punctuation or extension characters in the receiving application too.
Encoding a character unavailable in Windows-31J, such as 😀, fails rather than silently inserting ?. If a recipient requires that encoding, consider an agreed character policy, a changed interface, or an explicit substitute representation. Those are semantic decisions, not choices the converter should make silently.
Correct decoding does not validate the CSV structure
Once text decodes correctly, inspect rows and columns in CSV Editor. Delimiters, quoted newlines, leading zeroes, and empty strings are separate issues. A spreadsheet may still turn an ID into a number while fixing the display. Compare representative records with the original data.
Document the encoding, BOM policy, line endings, and CSV delimiter in the interchange contract. This tool's encoded result is hex text; saving what the screen shows does not automatically create a ready-to-import business file. Verify the final file representation and import procedure against the recipient's requirements.
Readable text is only one stage of file exchange
Keep the original CSV bytes before opening and resaving in a spreadsheet. Saving after an incorrect decode can replace characters with question marks or remove leading zeroes, creating problems beyond encoding. Test candidate decodings from the same original rather than repeatedly overwriting it. Separate reopening with a different decoder from saving in a different encoding to track where information changed.
An ASCII-only row looking correct is weak evidence because several encodings agree on those bytes. Inspect non-ASCII text and symbols actually present in the file, but do not assume a few readable characters prove the whole file. Prefer the export settings or sender's specification when available, and label a guessed encoding as a guess.
Once text is readable, check row and column counts, quoted newlines and empty-field meaning. Successful decoding does not prove the same CSV data was reconstructed. Share encoding, delimiter, line endings and column types together. Where this tool displays encoded bytes as hexadecimal, that text is not itself a ready-to-use business CSV file.