Text and regular expressions
Why does text exceed a byte limit?
Tools to use
A field says '20 characters', yet visibly short text fails. Or the form accepts it and saving fails later. The person entering it may have counted correctly: a visible character, a programming-language unit, and a storage unit need not be the same.
Use the text counter to measure one input in several ways. It describes the string you supply; it does not connect to the failing service or inspect database settings. Treat it as a small experiment that narrows the possibilities before checking the actual specification and implementation.
Enter ABC, あいう, and 😀 separately
Start with only ABC: graphemes, code points, UTF-16 units, and UTF-8 bytes are all 3. Replace it with あいう: the first three stay at 3, but UTF-8 bytes becomes 9. This explains how tests containing only ASCII can miss a byte limit that Japanese input reaches much sooner.
input | graphemes | codePoints | utf16CodeUnits | utf8Bytes
ABC | 3 | 3 | 3 | 3
あいう | 3 | 3 | 3 | 9
😀 | 1 | 1 | 2 | 4Now use only 😀. It is one grapheme and one code point, but two UTF-16 units and four UTF-8 bytes. 'An emoji counts as two characters' is therefore an incomplete rule: two under which measure? Emoji assembled from several elements can produce yet another set of counts.
The same-looking é can have different contents
Compare precomposed é with e followed by a combining acute accent. The former has one code point and two UTF-8 bytes; the latter has two code points and three UTF-8 bytes. Ordinary grapheme segmentation treats both as one user-perceived character.
é
éPasting both lines also counts the newline, so test them separately. Unicode Inspector can show U+00E9 versus U+0065 and U+0301. Normalization can align these forms, but that is not permission to silently rewrite input merely to fit a limit. For identifiers or signatures, check the receiving system's rules before changing the original sequence.
What fits into a 12-byte field?
Imagine a field limited to 12 UTF-8 bytes. Twelve ASCII A characters, four あ characters, and three 😀 emoji each reach that boundary. Adding one more produces 13, 15, and 16 bytes respectively. Testing just below, exactly at, and just above a boundary explains failures better than pasting an arbitrary long paragraph.
Do not infer a general character limit by dividing bytes by three. Mixed Latin text, ideographs, emoji, and combining sequences have no fixed bytes-per-character ratio. A product may expose a user-facing character limit and a separate storage-byte limit, explaining and validating both. Which limits are appropriate is a product decision, not something the counter determines automatically.
Spaces and the final newline are also input
Appending one ordinary space to ABC makes four UTF-8 bytes. The visual change is subtle, so apparently identical pasted text can produce different counts. Trimming with line tools changes the input. When investigating a limit, measure the original first and record which transformation made it shorter.
Line count here follows newline separators: A followed by one newline has two lines, including the empty final line. Visual wrapping caused by a narrow screen does not add input newlines. LF and CRLF also have different stored bytes. Pasting into a browser field can normalize line endings, so this count need not equal the original file's byte size.
String bytes are not the entire transmitted payload
A 12-byte name placed in JSON gains a key, quotes, and delimiters. URL encoding produces a representation with a different length. Files can involve a BOM, line-ending choices, or compression. When checking a transport limit, distinguish the raw string's bytes from the size of the actual serialized payload.
Grapheme counting uses the browser's Intl.Segmenter; unsupported environments do not receive an invented substitute count. Segmentation also depends on the runtime's Unicode data, so this is not a guarantee of identical behavior in another service. If a failure is device-specific, record the device and browser while comparing exactly the same input.
Turn 'short text fails' into a reproducible report
Record the example, grapheme count, UTF-16 units, UTF-8 bytes, and the stage that rejected it. Real names or secret strings are unnecessary: a short synthetic example using the same character types is easier to share. 'Three 😀 pass; four fail during saving' narrows the condition much better than 'emoji do not work'.
The measurements identify what to inspect next: the visible counter, validation, API constraints, and storage definition. Check which units must agree before merely increasing a limit. The counter cannot fix those systems, but it helps establish a consistent contract and an error message users can understand.
Test the exact limit and the next input
For a 12-byte UTF-8 limit, あいうえ occupies 12 bytes and あいうえお 15. ABCDEFGHIJKL also occupies 12 bytes despite its very different visual length. Test both at the boundary and then add a character to distinguish character limits from byte limits. ABCDEFGHIJK😀 is 11 plus 4 bytes, totaling 15; eleven ASCII letters do not leave room for any arbitrary final character.
Truncating at an arbitrary byte offset can split a UTF-8 character. Cutting by code points can still split a combining sequence or emoji cluster. This counter measures length; it is not an automatic safe-shortening feature. An input system needs an explicit policy: report the limit and ask for correction, or define the truncation unit. Verify that the displayed count and enforced limit measure the same thing.