Encoding and character sets
What is inside this Base64 string?
Tools to use
Have you opened an API response or configuration file and found a long string that seems meaningless? It is tempting to assume that it is encrypted and stop investigating. Sometimes, however, it is simply data represented as Base64.
We will decode a small public example and change one condition at a time. You do not need to paste real credentials to learn the workflow. Separating the original data, its encoded representation, and the chosen output makes longer strings much easier to investigate.
What is hidden in YWJj?
Open the Base64 tool and choose decode, standard Base64, and UTF-8 output. Enter only the four characters below and run it. Do not include quotation marks or the surrounding explanation.
YWJjThe result is abc. Switch to encode and enter abc: the output should return to YWJj. You have now made a round trip between the original data and its representation. No secret key or decryption password was involved.
Encoding changes a representation according to a rule; decoding recovers data from that representation. Making something look unreadable does not necessarily make it inaccessible to someone else. Keep that distinction in mind as you investigate real examples.
Why make the data harder to read?
For abc, sending the original text seems simpler. The problem changes when you need to carry an image or an arbitrary file through a text-oriented format. Not every byte can necessarily be inserted directly into that format.
Base64 represents the original bytes using a restricted alphabet. Its purpose is representation, not secrecy. When an API defines a Base64 field, think of that field as carrying bytes in a textual form, rather than as proving that those bytes are protected.
Why can two copies of abc produce different output?
While encoding, compare abc with abc followed by one LF newline. The latter produces YWJjCg== instead of YWJj. The invisible byte after the letters is part of the data too; Base64 has not randomly changed its answer.
YWJj
YWJjCg==If the decoded text looks identical, select Hex output. The bytes are 616263 without the newline and 6162630a with LF. Data from another environment may use CRLF instead. A similar-looking display is not evidence that two byte sequences are equal.
Unicode works too, but the character encoding matters
Try the three characters 日本語. SheepDev encodes text as UTF-8 bytes before applying Base64, producing 5pel5pys6Kqe. Decoding that value with UTF-8 output recovers the same three characters. The example works regardless of the language of the surrounding interface.
Base64 does not understand Japanese, Chinese, or Korean. Converting characters to bytes and converting bytes to Base64 are separate operations. If the source bytes use another encoding, such as Shift_JIS, valid Base64 may still fail to produce valid UTF-8 text.
Inspect the Hex output and use the encoding confirmed by the data producer. Do not simply cycle through encodings until an error disappears. Even a readable-looking result does not prove that you recovered the intended text; compare it with known source content.
Longer output is expected: Base64 is not compression
abc occupies three UTF-8 bytes, while YWJj is four ASCII characters. Base64 generally maps three input bytes to four characters, so longer data grows to roughly 1.33 times its original size. Encoding an image is not, by itself, a way to reduce transfer volume.
| Original | Standard Base64 |
|---|---|
| a | YQ== |
| ab | YWI= |
| abc | YWJj |
Short strings can have a larger apparent expansion because of padding. Try a, ab, and abc in turn and watch the trailing equals signs change. They are length-related padding, not corrupt characters or copying debris. Whether padding may be omitted depends on the format that consumes the data.
What changes with Base64url?
Standard Base64 uses + and /. Base64url substitutes - and _ in those positions. SheepDev also omits trailing padding when encoding Base64url. Choose the variant specified by the receiving format, not merely because your application happens to run on the web.
abc does not demonstrate this difference because some inputs look identical in both variants. Encode ???? instead: standard Base64 yields Pz8/Pw==, while Base64url yields Pz8_Pw. Use the matching setting when decoding. Check that setting before manually editing characters.
Decoded data is not always text
Base64 can carry more than prose. Images, compressed data, and ciphertext may not be valid UTF-8 after decoding. A malformed Base64 string and bytes that cannot be interpreted as UTF-8 are different problems, even if both prevent a text result.
Try /w==. Standard Base64 decoding to Hex gives ff, but a lone FF byte is not valid UTF-8 text. A UTF-8 error therefore does not prove the Base64 representation is wrong. Hex output lets you inspect bytes; it is not a feature for displaying the original image.
A practical troubleshooting order
- Confirm that the producer specifies Base64. A long alphanumeric appearance alone cannot distinguish it from an ID or a hash.
- Check that you copied the complete value, not surrounding quotes, annotations, or a data: URL header. Also check for truncation.
- Check the standard/Base64url choice. SheepDev does not automatically remove all interior whitespace or line breaks. Find out whether the source format deliberately wraps encoded lines.
- Once the representation decodes, identify the content. For text, check the character encoding; otherwise check the original file format. Do not accept meaningless output as a correct recovery.
Unreadable-looking does not mean secret
Remember that you recovered abc from YWJj without a key. Encoding a password or personal information does not change that property. Treating an encoded value as redacted information when reporting a problem can expose the original content.
Hashing serves a different purpose: you do not decode a hash like Base64 to recover a message. Likewise, reading a JWT does not establish its signature or trustworthiness. Use the dedicated hash and JWT guides to decide what evidence each operation actually provides.
Bring the experiment back to your work
A useful investigation note says more than “decoding worked.” Record the format specified by the producer, the output mode, and the difference between expected and observed content. “Standard Base64 decodes to FF in Hex but not to UTF-8” points the next investigation toward content type or encoding.
Confirm the format, isolate the value, and identify the original data. With that routine, a converter becomes a way to narrow down a problem rather than merely transform a string. Being able to read the result is never a verdict that its contents are safe to execute or publish.