What is an Operating System?

Strip a computer back and you have three things: a CPU that executes whatever instructions it is pointed at, memory that holds those instructions while they run, and a disk that keeps them when the power is off. None of it knows what a file is, what a program is, or that you exist.

In principle a program could drive all of that itself. In practice three problems make it impossible:

• Complexity. Talking to a disk means knowing that exact controller's registers and timings. Every program would have to ship a driver for every device it might ever meet.
• Protection. Nothing would stop a buggy program from writing over another program's memory, or over the disk sectors holding your files. One bad pointer takes down the machine.
• Sharing. There is one CPU and dozens of programs that all want it now. Someone has to arbitrate, and it cannot be one of the competitors.

So we insert one trusted layer that owns the hardware, and make everyone else ask it for things. That layer is the operating system.

Every textbook definition boils down to these two, and it is worth being able to name both.

1. An abstraction (the “extended machine”)

It replaces the hardware you actually have with a much nicer imaginary one. Instead of disk sectors you get files; instead of raw RAM addresses you get an address space that starts at zero and belongs only to you; instead of a CPU you must share you get a process that appears to have one to itself. Same program, same source code, runs on a laptop or a server, because it never touched either.

2. A resource manager

It multiplexes limited hardware among competing programs — in time (each takes a turn on the CPU) and in space (each gets a slice of memory and disk). Deciding who, when, and for how long, without letting anyone break anyone else, is the job that produces every topic later in this course.

The CPU itself enforces the split. It runs in one of two modes, decided by a bit in a hardware register:

• User mode — where your program runs. Privileged instructions (talk to a device, change the memory map, disable interrupts) are simply not allowed, and attempting one traps.
• Kernel mode — where the OS runs. Everything is permitted.

A program cannot call into the kernel like a normal function, because that would just be jumping to an address in user mode. It executes a special trap instruction instead, and the hardware does the mode switch for it:

1. 1Your program puts the request in registers — “read 4 KB from this file” — and executes a trap.
2. 2The CPU switches to kernel mode and jumps to a fixed entry point the OS installed at boot. Your program cannot choose where it lands.
3. 3The kernel checks you are allowed to do this, then issues the real device commands.
4. 4The result is copied back and the CPU returns to user mode, on the instruction after the trap.

That round trip is a system call. read(), write(), open(), fork(), exit() — the whole interface a program has to the machine is about a few hundred of these. A library call like printf() is ordinary user-mode code that eventually makes one.

Process management

Creating, scheduling and terminating processes; letting them communicate and synchronise without corrupting each other.

Memory management

Giving each process its own address space, tracking what is in RAM, and moving pages to and from disk when RAM runs out.

File systems

Turning a flat array of numbered blocks into directories, names, permissions and files that survive a reboot.

I/O and device drivers

One uniform interface over wildly different hardware, plus buffering and caching to hide how slow most of it is.

Protection and security

Deciding which user and which process may touch which resource, and enforcing it with hardware help.

The kernel is the part that runs in kernel mode. “Operating system” is often used more loosely to include the shell, utilities and libraries shipped alongside it.