How Text Becomes Data: Unicode, Characters, and UTF-8

When you type a letter, the computer does not place a miniature drawing of that letter inside the file. It stores numerical data.

To display the text later, software must agree on several things:

• Which abstract character the number identifies
• How that number is stored as bytes
• Which visual shape should be drawn

These jobs belong to related but different systems.

Unicode identifies characters. An encoding such as UTF-8 turns those character identifiers into bytes. A font supplies visible shapes.

Separating these layers makes text portable across devices, programs, languages, and countries.

Characters are abstract ideas

A character is an abstract text element, such as:

• The Latin capital letter A
• The Tamil letter அ
• The digit 7
• A comma
• A mathematical symbol
• An emoji

The character is not identical to its appearance. The letter A can be printed in a serif font, drawn by hand, shown in bold, or displayed at many sizes while remaining the same character.

This distinction resembles music. A written note identifies a musical idea, while different instruments produce different sounds from it.

Computers need stable identifiers for the abstract characters so that appearance can change without changing the underlying text.

The problem with early character sets

Early computer systems often served limited communities and assigned small numerical values to the characters they needed.

ASCII became an influential standard. It defines 128 values for English letters, digits, punctuation, and control characters. For example:

65 → A
66 → B
97 → a

ASCII was useful but could not represent the world's writing systems. Different organizations and regions developed additional character sets. The same byte value might mean one character under one encoding and a different character under another.

Text exchanged without the correct encoding information could become unreadable. A document that worked on its creator's computer might display nonsense elsewhere.

The larger problem was not merely storage. Systems lacked one shared map of human writing.

Unicode creates a universal map

Unicode is a standard that assigns a unique number called a code point to a vast collection of characters.

Code points are commonly written with U+ followed by hexadecimal digits:

U+0041 → A
U+0061 → a
U+20AC → €
U+1F600 → 😀

Unicode covers modern and historical scripts, punctuation, mathematical notation, technical symbols, and emoji. The standard continues to evolve as characters are added and clarified.

A code point answers:

Which character is this?

It does not by itself specify the exact bytes used in a file. That is the job of a Unicode encoding.

UTF-8 turns code points into bytes

UTF-8 is the most widely used Unicode encoding on the web. It represents a code point using one to four bytes.

Its design has several useful properties:

• ASCII characters keep their familiar one-byte values.
• Other characters use additional bytes.
• Byte patterns indicate where a character begins and how long it is.
• The encoding covers every Unicode code point.

Examples:

This means one character does not always equal one byte.

It also means string length can be more complicated than it appears. Software may count bytes, Unicode code points, or user-perceived symbols, and those counts are not always equal.

From a key press to stored text

Suppose you press the A key:

1. The keyboard sends information identifying the physical key event.
2. The operating system considers the active keyboard layout and modifier keys.
3. The application receives text input representing A.
4. The application stores the Unicode character internally.
5. When saving a UTF-8 file, it writes the byte 01000001.
6. When reopening the file, software decodes that byte as U+0041.
7. A font renderer draws the appropriate shape on screen.

The physical key does not permanently mean A. Changing the keyboard layout can make the same key position produce another character. Input method editors can also combine multiple key presses to create one character.

Text input is interpretation, not a direct wire from a labeled key to a stored letter.

Fonts turn characters into glyphs

A glyph is a visual representation of a character or character sequence.

Unicode may identify a character correctly while the system still cannot draw it. If no installed font contains a suitable glyph, you may see an empty rectangle, replacement symbol, or "tofu" box.

Fonts also shape text according to language rules. Some scripts require characters to join or change appearance based on neighboring characters. A sequence may be rendered as one combined visual form.

Therefore:

stored character ≠ fixed picture

The rendering engine, font, style, language, and surrounding text all influence appearance.

Why some visible symbols use several code points

A user-perceived character can be built from multiple Unicode code points.

For example, an accented letter may be represented as:

• One precomposed code point, or
• A base letter followed by a combining accent

Both can look identical.

Emoji can be even more complex. Skin-tone modifiers, gender variations, family combinations, and flags may use sequences of code points joined according to Unicode rules.

This creates practical software challenges:

• A cursor should not split a visible symbol incorrectly.
• Deleting one "character" may require removing several code points.
• A username length rule must specify what it counts.
• Visually identical strings may use different underlying sequences.

Unicode normalization provides standard ways to make certain equivalent sequences comparable.

What causes garbled text?

Garbled text often appears when bytes are decoded using the wrong character encoding. The phenomenon is sometimes called mojibake.

Imagine UTF-8 bytes representing é. If a program incorrectly interprets each byte using an older single-byte encoding, it may display two unrelated characters.

The data was not necessarily destroyed. The reader used the wrong rules.

A robust text workflow should:

• Prefer a well-defined encoding such as UTF-8.
• Declare the encoding when a format or protocol requires it.
• Avoid repeatedly converting through incompatible legacy encodings.
• Validate input rather than silently guessing.

Web pages typically declare UTF-8 in metadata, and modern protocols often standardize encoding behavior.

Text files contain more than visible letters

Text data can include invisible control characters:

• Spaces and tabs
• Line breaks
• Directional formatting controls
• Zero-width joiners
• Combining marks

Even line endings have variants. Unix-like systems commonly use line feed (LF), while Windows text traditionally uses carriage return plus line feed (CRLF).

Most editors hide these details, but version-control diffs and cross-platform tools may reveal them.

Text also does not inherently know its meaning. The characters:

2026-07-16

could be interpreted as a date because a person or program expects that format. At the encoding layer, they are simply a sequence of characters.

Text and security

Unicode enables global communication, but visual flexibility can be abused.

Some characters from different scripts look similar. An attacker might register a deceptive domain name using a character that resembles a familiar Latin letter. Invisible or direction-changing characters can also make source code or messages confusing.

Security-sensitive software may normalize input, restrict allowed character sets in certain identifiers, and display ambiguous values carefully.

The lesson is not that Unicode is unsafe. The lesson is that visual equality and data equality are different questions.

Common misunderstandings

"Every letter takes one byte"

That may hold for basic ASCII characters in UTF-8, but many Unicode characters use two, three, or four bytes.

"Unicode is a font"

Unicode identifies characters. Fonts provide glyphs used to draw them.

"One visible symbol always equals one code point"

Combining marks and emoji sequences can produce one user-perceived symbol from multiple code points.

"Garbled text means the file is permanently corrupt"

Sometimes the original bytes are intact and only the decoding assumption is wrong. Recovering the correct encoding can restore the text.

Knowledge check

The one idea to remember

Text becomes data through shared layers: characters receive Unicode identifiers, encodings such as UTF-8 turn them into bytes, and fonts render them as visible glyphs.

When text breaks, ask which layer is responsible: input, character identity, byte encoding, decoding, or visual rendering.

Next, we will use a similar measurement-and-encoding model to understand digital images.