Free lesson · Operating Systems

Demonstrate Unix domain socket (server side)

Easy11 min readoperating-systemsosconcurrency

The IPC Mechanism That Powers the Internet

Every time you open a web page, query a database, call an API, or send a message, you are using sockets. Sockets are the most versatile IPC mechanism - they work between processes on the same machine, between processes on different machines, across local networks, and across the global internet. They are the foundation of networked computing.

A socket is an endpoint for communication. When two processes want to communicate, each creates a socket. The OS connects the sockets - either through the network stack (for TCP/UDP) or through an optimized local path (for Unix domain sockets). Once connected, the processes read and write through their sockets using the same read() and write() system calls they use for files and pipes.

Understanding sockets at the OS level matters for interviews because socket-based communication is the default IPC mechanism for microservices, databases, caches, and distributed systems. When an interviewer asks "how does your web server communicate with Redis?", the answer involves sockets. When they ask "what happens when you make an HTTP request?", the answer involves sockets. When they ask about the C10K problem or high-performance networking, the answer involves the OS's socket implementation and I/O multiplexing (epoll).

The Socket Lifecycle

Creating and using a socket follows a well-defined lifecycle. The steps differ slightly between the server (which waits for connections) and the client (which initiates connections).

Server Side

socket() - Create a socket. Specify the communication domain (AF_INET for IPv4, AF_INET6 for IPv6, AF_UNIX for same-machine) and the type (SOCK_STREAM for TCP, SOCK_DGRAM for UDP).

bind() - Associate the socket with a specific address and port. For a web server, this might be port 80 or 8080. For a Unix domain socket, this is a file path.

listen() - Mark the socket as a passive socket that accepts connections. The argument specifies the backlog - the maximum number of pending connections waiting to be accepted.

accept() - Block until a client connects. When a connection arrives, accept returns a new socket file descriptor for that specific connection. The original socket continues listening for more connections. This is how a server handles multiple clients - it gets a new file descriptor for each connection.

Client Side

socket() - Create a socket (same as server).

connect() - Connect to the server's address and port. This initiates the TCP handshake (for TCP sockets) or establishes the connection path (for Unix domain sockets).

After the connection is established, both sides use read() / write() (or send() / recv()) to transfer data, and close() to terminate the connection.

TCP Socket Semantics: Byte Stream, Not Messages

A critical detail about TCP sockets that catches many developers off guard: TCP is a byte stream, not a message protocol. When you call write() with 1000 bytes, the receiver might get all 1000 bytes in one read(), or 500 bytes in one read and 500 in the next, or 100 bytes in ten reads. TCP does not preserve message boundaries - it guarantees that all bytes arrive in order and without corruption, but not that they arrive in the same chunks they were sent in.

This is why application protocols (HTTP, PostgreSQL's wire protocol, Redis's RESP protocol) all implement their own message framing on top of TCP. The three most common approaches are length-prefixed messages (send the message length as a 4-byte header before the body, so the receiver knows exactly how many bytes to read), delimiter-based framing (end each message with a special byte or sequence, like \r\n in HTTP headers), and fixed-size messages (every message is exactly N bytes, padded if necessary). Knowing about message framing over TCP demonstrates practical networking knowledge in interviews.

UDP sockets, by contrast, preserve message boundaries. Each sendto() delivers exactly one datagram, and each recvfrom() retrieves exactly one datagram. But UDP provides no ordering, no reliability, and no flow control - datagrams can arrive out of order, be duplicated, or be lost entirely. This tradeoff makes UDP suitable for real-time applications (video streaming, gaming, DNS) where low latency matters more than perfect delivery.

Socket Buffers and Flow Control

Every TCP socket has two kernel-managed buffers - a send buffer and a receive buffer. When your process calls write(), the data goes into the send buffer, and the kernel transmits it over the network as fast as the TCP protocol allows. When data arrives from the network, the kernel puts it in the receive buffer, where it waits until your process calls read().

These buffers are typically 128 KB to several MB each (configurable via setsockopt() with SO_SNDBUF and SO_RCVBUF). The buffers provide decoupling between the application and the network - your application can write at its own pace, and the kernel handles the actual network transmission asynchronously.

If the send buffer is full (the network is slower than the application), write() blocks until space is available. If the receive buffer is full (the application is slower than the network), the TCP protocol tells the sender to slow down - this is TCP flow control (the receive window). This flow control prevents fast senders from overwhelming slow receivers, but it also means that a slow reader can cause backpressure all the way back to the sender.

Understanding socket buffers is useful for diagnosing production networking issues. If your server's receive buffers are consistently full, the application is not reading fast enough - you need to optimize your request processing or add more worker threads. If send buffers are full, the network or the client is slow - you might need to increase buffer sizes or investigate network congestion.

Tip: For production debugging: On Linux, you can see socket buffer usage for every connection with ss -i or by reading /proc/net/tcp. The Send-Q and Recv-Q columns show how much data is waiting in the send and receive buffers. High values indicate backpressure. This is a practical diagnostic skill that interviewers at infrastructure-focused companies value.

sequenceDiagram
    participant Server as Server Process
    participant OS as Operating System
    participant Client as Client Process
    
    Server->>OS: socket()
    Server->>OS: bind(address, port)
    Server->>OS: listen(backlog)
    Note over Server: Waiting for connections...
    
    Client->>OS: socket()
    Client->>OS: connect(server_addr, port)
    
    OS->>OS: TCP Handshake<br/>(SYN, SYN-ACK, ACK)
    
    OS-->>Server: accept() returns<br/>new connection fd
    OS-->>Client: connect() returns<br/>success
    
    Client->>Server: write("GET / HTTP/1.1")
    Server->>Client: write("<html>...</html>")
    
    Client->>OS: close()
    Server->>OS: close()

This sequence diagram shows the socket lifecycle. The server creates a socket, binds to an address and port, and listens for connections. The client creates a socket and connects to the server. The OS performs the TCP handshake. The server's accept call returns a new file descriptor for the connection. The client sends an HTTP request through the socket, the server sends a response, and both sides close their sockets.

Unix Domain Sockets: Same-Machine Sockets

When two processes on the same machine need to communicate via sockets, they can use Unix domain sockets instead of TCP sockets. Unix domain sockets use the file system for addressing (the socket has a file path like /var/run/myapp.sock) instead of IP addresses and ports.

Unix domain sockets are significantly faster than TCP sockets for same-machine communication because they bypass the entire network stack - no TCP/IP header processing, no checksumming, no routing decisions. The kernel provides a direct, memory-to-memory data path between the two processes. Measurements typically show Unix domain sockets achieving 30-50% higher throughput and 30-50% lower latency than TCP localhost connections.

This is why performance-sensitive same-machine communication uses Unix domain sockets:

PostgreSQL and MySQL listen on Unix domain sockets by default for local connections. When your web server and database are on the same machine, connecting via /var/run/postgresql/.s.PGSQL.5432 (a Unix domain socket) is faster than connecting via localhost:5432 (a TCP socket).

Docker uses a Unix domain socket (/var/run/docker.sock) for communication between the Docker CLI and the Docker daemon. When you type docker ps, the CLI sends a request through this socket to the daemon, which returns the list of containers.

Nginx can use Unix domain sockets to communicate with upstream application servers (like a Python/Gunicorn or Ruby/Puma process) on the same machine. The proxy_pass http://unix:/tmp/app.sock; directive routes requests through a Unix domain socket instead of a TCP connection.

Redis supports Unix domain sockets for clients on the same machine. Connecting via /tmp/redis.sock instead of localhost:6379 reduces latency by eliminating TCP overhead - meaningful for applications that make thousands of Redis calls per second.

import socket
import os
import tempfile

# Demonstrate Unix domain socket (server side)
sock_path = os.path.join(tempfile.gettempdir(), "demo.sock")

# Clean up any existing socket file
if os.path.exists(sock_path):
    os.unlink(sock_path)

# Create a Unix domain socket
server = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
server.bind(sock_path)
server.listen(1)

print(f"Unix domain socket created at: {sock_path}")
print(f"Socket type: AF_UNIX (same-machine, no TCP overhead)")
print(f"File exists: {os.path.exists(sock_path)}")

# In a real app, we would accept() connections here
# For demo, just show the socket is created

server.close()
os.unlink(sock_path)
print("Socket cleaned up")

TCP vs Unix Domain Sockets

A Unique Feature: Passing File Descriptors

Unix domain sockets have a capability that no other IPC mechanism provides: passing file descriptors between processes. Using the SCM_RIGHTS control message, one process can send an open file descriptor through a Unix domain socket to another process. The receiving process gets a new file descriptor that refers to the same underlying file, socket, or device.

This is how Nginx's master process passes accepted client connections to worker processes. The master process accepts a TCP connection (getting a file descriptor), then sends that file descriptor through a Unix domain socket to a worker. The worker receives the file descriptor and handles the client directly. The master process never reads or writes the client's data - it just distributes connection file descriptors to workers.

This file-descriptor-passing capability is also used by container runtimes, process managers, and any system where a privileged process needs to hand off resources to less-privileged worker processes.

The Connection Lifecycle: What Happens at Each Stage

Understanding the full TCP connection lifecycle helps you answer system design questions about connection management, timeouts, and resource exhaustion.

Connection establishment (three-way handshake). When the client calls connect(), the OS sends a SYN packet to the server. The server's OS responds with SYN-ACK. The client's OS sends ACK, and the connection is established. This handshake takes one round-trip time (RTT) - typically 0.1-100 ms depending on network distance. For same-machine connections via localhost, the RTT is negligible (microseconds).

Data transfer. Both sides send and receive data through the socket. TCP guarantees in-order delivery, retransmission of lost packets, and flow control. The application sees a reliable byte stream.

Connection termination (four-way handshake). When one side calls close(), the OS sends a FIN packet. The other side acknowledges with ACK, then sends its own FIN, which is acknowledged with ACK. After closing, the socket that initiated the close enters the TIME_WAIT state for 2× the Maximum Segment Lifetime (typically 60 seconds). During TIME_WAIT, the port cannot be reused for new connections.

TIME_WAIT is a common production issue. A web server that opens and closes many short-lived connections to a backend service can exhaust its ephemeral port range because thousands of ports are stuck in TIME_WAIT. Solutions include connection pooling (reusing connections instead of opening new ones), setting SO_REUSEADDR (allowing port reuse), and using keep-alive connections (HTTP/1.1 and HTTP/2 keep connections open across multiple requests).

This is a practical detail that demonstrates production awareness in interviews. If the interviewer asks "what happens when your service runs out of ports?", explaining TIME_WAIT and connection pooling shows real operational experience.

Interview: "How do processes communicate over the network? What about on the same machine?"

"Over the network, processes use TCP or UDP sockets. The server creates a socket, binds to an address and port, and listens for connections. The client connects by specifying the server's address and port. Once connected, they exchange data using read() and write() - the OS handles the TCP/IP protocol, checksumming, retransmission, and flow control.

On the same machine, processes can use Unix domain sockets instead of TCP. Unix domain sockets use file system paths for addressing and bypass the network stack entirely - no TCP headers, no checksumming, no routing. This makes them 30-50% faster than TCP for same-machine communication. PostgreSQL, MySQL, Redis, and Docker all use Unix domain sockets for local communication."

Sockets and I/O Multiplexing: The Connection to Scalability

A key challenge with sockets is handling many simultaneous connections. If a server uses one thread per connection, it creates thousands of threads for thousands of clients, which causes context switch overhead and memory pressure (as we discussed in Module 02). The solution is I/O multiplexing - a single thread monitors many sockets simultaneously and handles whichever ones have data ready.

Linux's epoll, BSD's kqueue, and the older select()/poll() system calls all provide I/O multiplexing. The idea is the same: the process registers interest in multiple file descriptors (sockets) and blocks until at least one of them is ready for reading or writing. When the call returns, the process handles the ready sockets and goes back to waiting.

This is the foundation of event-driven servers like Node.js, Nginx, and Redis. A single thread can handle tens of thousands of concurrent connections because it only processes sockets that actually have data - it never blocks waiting for a single slow client.

We cover I/O multiplexing in depth in Module 10, but the connection to sockets is worth establishing here: sockets are the data plane (how data moves between processes), and I/O multiplexing is the control plane (how the server decides which sockets to process next). Together, they enable the high-concurrency networking that modern web infrastructure depends on.

graph TD
    subgraph COMPARISON["Socket Types for IPC"]
        TCP["TCP Socket<br/>────<br/>Cross-machine<br/>Full TCP/IP stack<br/>Slower but universal"]
        UNIX["Unix Domain Socket<br/>────<br/>Same-machine only<br/>Bypasses network stack<br/>Faster, file permissions"]
        UDP["UDP Socket<br/>────<br/>Cross-machine<br/>No connection setup<br/>No delivery guarantee"]
    end
    
    TCP -->|"Use for"| NET["Network<br/>communication"]
    UNIX -->|"Use for"| LOCAL["Same-machine<br/>IPC"]
    UDP -->|"Use for"| REALTIME["Real-time data<br/>(gaming, video, DNS)"]

This diagram shows three socket types for IPC. TCP sockets are for cross-machine communication with full reliability guarantees. Unix domain sockets are for same-machine IPC with faster performance by bypassing the network stack. UDP sockets are for cross-machine communication where speed matters more than reliability, like gaming, video streaming, and DNS queries.

Tip: For system design interviews: Always specify whether you mean TCP sockets or Unix domain sockets when discussing same-machine communication. Saying "the web server talks to Redis via a Unix domain socket" is more specific and shows deeper understanding than "the web server talks to Redis." If the components are on different machines, TCP sockets are the only option. If they are on the same machine, Unix domain sockets are the faster choice.

Exercise: Starter: Write a simple TCP echo server in Python that listens on localhost:8888, accepts one connection, reads a message, echoes it back, and closes. Then write a client that connects, sends "Hello", reads the echo, and prints it. Run both and verify the round-trip works.

Intermediate: Modify the echo server to use a Unix domain socket instead of TCP. Measure the round-trip latency for 10,000 small messages using both TCP localhost and Unix domain socket. What is the performance difference? Is it consistent with the 30-50% improvement claim?

Advanced: Build a multi-client echo server using selectors (Python's I/O multiplexing wrapper). The server should handle 100 concurrent clients on a single thread using non-blocking sockets and the selector to detect which sockets have data ready. This is the foundation of event-driven server architecture - understanding it prepares you for Module 10's treatment of epoll and the C10K problem.

Back to Operating Systems