Multitasking and Scheduling: How Programs Share the CPU

Music continues playing while you type, files download in the background, notifications arrive, and the browser redraws a page. The computer may have far more active threads than CPU cores.

The operating-system scheduler manages the mismatch.

Scheduling is the process of deciding which runnable thread receives which CPU core and for how long.

Fast, repeated decisions create responsive multitasking while trying to use hardware efficiently.

Task states

Not every thread is ready to run.

A thread may be:

• Running on a core
• Runnable and waiting for a turn
• Sleeping until a timer
• Blocked on file, network, or user input
• Waiting for a lock
• Stopped or terminated

The scheduler focuses primarily on runnable work.

A thread waiting for network data should not consume CPU cycles repeatedly asking whether data arrived. The kernel blocks it and lets another thread run. A completion event later makes it runnable.

Efficient waiting is central to multitasking.

Preemptive scheduling

Modern general-purpose operating systems use preemptive scheduling.

The OS can interrupt a running thread after a time interval or when a higher-priority event occurs. It saves the thread's state and selects another.

Without preemption, one badly behaved application could refuse to yield and freeze the machine.

Timer interrupts ensure the kernel regains control. The scheduling algorithm then applies policy.

Time slices

A runnable thread may receive a time slice, also called a quantum.

Short slices:

• Improve interactive response
• Let more tasks receive frequent turns
• Increase context-switch overhead

Long slices:

• Reduce switching overhead
• Improve sustained CPU efficiency
• Can make other tasks wait longer

Modern schedulers use dynamic behavior rather than one simple fixed slice for every situation.

The user generally experiences the result, not the individual turns, because switches happen in milliseconds or less.

Context switching

During a context switch, the OS:

1. Stops the current thread at a safe point.
2. Saves register and scheduling state.
3. Updates accounting.
4. Selects another runnable thread.
5. Restores that thread's state.
6. Returns to user execution.

The switch itself does not advance application logic. It also can reduce cache efficiency.

Schedulers avoid needless switches while still meeting responsiveness and fairness goals.

Priority

Some work is more urgent than other work.

An audio thread must provide the next samples before the playback buffer empties. A background search index can wait.

Schedulers use priorities or weights to influence access to CPU time.

Priorities require care. If high-priority work runs continuously, low-priority tasks can experience starvation. Systems may adjust priorities over time or reserve resources to preserve progress.

Users should be cautious when manually assigning extreme priorities. A compute-heavy process at an inappropriate real-time priority can make the system difficult to control.

Interactive and background work

Interactive workloads often run briefly and then wait:

• Process a key press
• Redraw a small area
• Handle a mouse event
• Play the next audio buffer

Background workloads may be continuously runnable:

• Video encoding
• Compilation
• Scientific computation

Schedulers try to give interactive tasks low latency while allowing background tasks high throughput when the CPU would otherwise be idle.

This is why a user interface can remain responsive during a long calculation, provided the system has enough resources and software cooperates.

A concrete example: music while compiling

Suppose a developer compiles a large project while listening to music:

1. Compilation creates many CPU-intensive worker tasks.
2. The audio application periodically wakes to prepare samples.
3. Its work is short but time-sensitive.
4. The scheduler gives it prompt CPU access.
5. It refills the audio buffer and sleeps again.
6. Compilation uses the remaining processor time.

If the audio task waits too long, playback crackles. If it receives an entire core unnecessarily, compilation slows more than needed.

Good scheduling meets the audio deadline with a small amount of well-timed work.

Multiple cores and load balancing

On a multi-core CPU, the scheduler assigns threads across cores.

It tries to balance:

• Keeping cores busy
• Preserving cache locality
• Avoiding migration overhead
• Respecting power and thermal policy
• Matching tasks to performance or efficiency cores

Moving a thread to another core may leave useful cached data behind. Keeping it on one core improves locality, but an overloaded core may need relief.

This is a continuous trade-off.

CPU-bound and I/O-bound tasks

A CPU-bound task spends most of its time calculating. More CPU time or parallel cores may speed it up.

An I/O-bound task spends much of its time waiting for storage, network, or devices. Giving it more CPU time may not help.

Many applications alternate between these phases.

Schedulers observe behavior rather than understanding application meaning. Threads that wake briefly and block again often receive treatment suitable for interactive or I/O-driven work.

Real-time scheduling

Real-time systems prioritize meeting timing deadlines.

Examples include:

• Industrial control
• Audio processing
• Robotics
• Medical equipment

Real-time does not necessarily mean "very fast." It means behavior has bounded timing requirements.

A hard real-time deadline cannot be missed without system failure. A soft real-time workload tolerates occasional misses with degraded quality.

General desktop operating systems offer real-time features but also run many unpredictable services, so they are not automatically suitable for every strict control requirement.

Priority inversion

Suppose a high-priority thread needs a lock held by a low-priority thread. A medium-priority thread keeps running, preventing the low-priority holder from releasing the lock.

The high-priority thread is indirectly delayed by lower-priority work. This is priority inversion.

Systems can use priority inheritance: temporarily raise the lock holder's priority so it can finish and release the resource.

This example shows that scheduling interacts with synchronization. Priority alone cannot resolve every dependency.

Measuring scheduler behavior

Useful signs include:

• CPU utilization per core
• Run-queue length
• Context-switch rate
• Thread wait time
• Scheduling latency
• Deadline misses

One hundred percent CPU use can be healthy during intentional computation. It becomes a problem when interactive work cannot receive timely access or heat and power exceed system goals.

Performance analysis should identify whether threads are running, runnable, or blocked.

Common misunderstandings

"Every open application constantly uses the CPU"

Many applications spend most of their time sleeping or waiting and use little CPU.

"Multitasking means every program runs simultaneously"

Some tasks run in parallel on separate cores; many others take rapidly interleaved turns.

"Higher priority makes a task finish sooner without cost"

It can reduce the task's waiting time by increasing waiting elsewhere.

"A blocked thread is wasting a CPU core"

A properly blocked thread is not running. The scheduler uses the core for other work.

Knowledge check

1. What is a runnable thread?

It is ready to execute but may be waiting for a CPU core or time slice.

2. Why does preemption matter?

It lets the operating system regain control and prevent one task from monopolizing a core.

3. What is the cost of a context switch?

The OS saves and restores state, performs scheduling work, and may disturb processor caches without advancing application logic.

4. What is priority inversion?

A high-priority task waits on a resource held by lower-priority work that cannot run promptly.

The one idea to remember

Multitasking is controlled sharing of CPU cores among runnable threads.

The scheduler continually balances latency, throughput, fairness, priority, locality, power, and deadlines.

Next, we will examine virtual memory, which gives processes private address spaces and lets the OS manage RAM flexibly.