The following is an excerpt from my post: How Linux GUIs Work: An Introduction to X11, Display Managers, and Desktop Environments. This has been separated into its own post for easier reading.
X, or commonly known as Xorg is the display server used in most GNU/Linux systems. It’s rather similar to VNC. It makes a local “desktop” and connects to it. X11 is called such as it is the 11th major revision of the X system.
X has support for “screens”, “graphics cards”, mice and keyboards. An X ‘screen’ is not simply a monitor, as if you have 2 or 3 ‘screens’, you can’t actually move stuff around! They are independent sessions. (I’ll go into some more detail in another post). To get around this limitation, we just make a single screen, with a resolution that spans all of your monitors. (Currently, this is done through xrandr in most computers, previously, Xinerama was pretty common).
If you simply start xorg by itself, you’ll just get an empty screen (checker pattern or black) with a crosspoint as a cursor. This is because there isn’t really anything running on that server. You can actually use the computer this way, manually launching programs from the terminal, its not very common (a common use of this is digital signage, where you don’t want any excess bloat – just the one application that needs to run in fullscreen) as you can’t resize or maximize programs. Instead, we use a layer of abstraction, the desktop manager.