Linux network administrator's guide: addressing to troubleshooting
The jobs of a Linux network administrator are the same as they were in the dial-up days: give machines addresses, route packets, resolve names, hand out leases, filter traffic, run services, and find out why something does not connect. The commands are new. ifconfig, route and netstat gave way to ip and ss, and a network manager now keeps the settings. This page is the overview, with links to the detailed HOWTOs.
The classic Linux Network Administrator's Guide is kept by the Linux Documentation Project at tldp.org/LDP/nag2. Its chapters on UUCP, SLIP, IPX and ipfwadm are history now.
Old commands and their replacements
| Old command | Today | Shows or does |
|---|---|---|
ifconfig | ip addr, ip link | Interfaces and addresses |
route | ip route | Routing table |
arp | ip neigh | Neighbor (ARP/NDP) cache |
netstat -tlnp | ss -tlnp | Listening sockets and their programs |
traceroute | tracepath, mtr | The path packets take |
ipfwadm, ipchains, iptables | nft, firewalld, ufw | Packet filtering and NAT |
The net-tools package can still be installed, but it does not understand everything the kernel can do (several addresses per interface, policy routing), so learn the ip forms. ip -br addr and ip -br link give a compact one-line-per-interface view.
Addressing
An IPv4 address with its prefix length, such as 192.0.2.10/24, says both "this host" and "these neighbors are on the same link": the first 24 bits are the network, the last 8 the host, so the network holds 256 addresses (254 usable hosts). Private networks use the RFC 1918 ranges 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16, which are not routed on the Internet. The examples here use the documentation ranges 192.0.2.0/24 and 2001:db8::/32, which are never assigned to real networks.
IPv6 is no longer optional: most hosts get an IPv6 address automatically through router advertisements, and an ordinary LAN uses a /64. For subnet arithmetic with worked examples, see the IP subnetting mini-HOWTO.
ip -br addr # every interface, its state and addresses
ip addr show dev enp1s0 # one interface in detail
ip -6 addr # IPv6 only
Who keeps the network settings
Settings made with ip addr add are lost at reboot. Permanent settings belong to one of these, and you should use only one per interface:
- NetworkManager (
nmcli,nmtui): the default on desktops, and on Fedora and RHEL 9 and 10 servers. Profiles are stored in keyfile format. - systemd-networkd: small and declarative, configured with
.networkfiles in/etc/systemd/network/. Debian's wiki describes it as the default for non-graphical systems, but the Debian 13 installation guide still points to/etc/network/interfacesfor changing the network settings made during installation, so check which one is active (systemctl is-active systemd-networkd networking NetworkManager). - netplan: Ubuntu's YAML layer in
/etc/netplan/, which hands the settings to a back end (a renderer): systemd-networkd on Ubuntu Server, or NetworkManager. - ifupdown (
/etc/network/interfaces): the classic Debian method, still found on older and upgraded systems.
A static address with NetworkManager (the profile name comes from nmcli connection show):
sudo nmcli connection modify "Wired connection 1" ipv4.method manual \
ipv4.addresses 192.0.2.10/24 ipv4.gateway 192.0.2.1 ipv4.dns 192.0.2.53
sudo nmcli connection up "Wired connection 1"
The same with systemd-networkd:
# /etc/systemd/network/10-lan.network
[Match]
Name=enp1s0
[Network]
Address=192.0.2.10/24
Gateway=192.0.2.1
DNS=192.0.2.53
Then sudo systemctl restart systemd-networkd and check with networkctl status enp1s0. On Ubuntu, edit a file in /etc/netplan/ and run sudo netplan try, which rolls the change back after 120 seconds unless you confirm it, so a mistake over SSH fixes itself. Drivers, link speed, bonding and VLANs are in the Ethernet HOWTO; bridges for virtual machines in the bridge HOWTO.
Routing
ip route # the routing table; look for the "default via" line
ip -6 route
ip route get 198.51.100.7 # which route, interface and source address the kernel would use
ip route get answers "why does this traffic leave through the wrong interface?" in one line. A Linux machine forwards packets between interfaces only when you allow it; the kernel default for net.ipv4.ip_forward is 0. To make a router, set it in a file under /etc/sysctl.d/:
echo 'net.ipv4.ip_forward = 1' | sudo tee /etc/sysctl.d/90-router.conf
sudo sysctl --system
Sharing one Internet connection with NAT is covered in the IP masquerade HOWTO; several uplinks, policy routing and traffic shaping in the advanced routing HOWTO.
Name resolution (DNS)
Programs ask the C library, which follows /etc/nsswitch.conf (usually /etc/hosts first, then DNS). Ubuntu and Fedora run systemd-resolved, a local caching stub: /etc/resolv.conf then points to 127.0.0.53 and the real servers are shown by resolvectl. On Debian and RHEL systemd-resolved is usually not in use, and /etc/resolv.conf lists the servers directly.
getent hosts www.example.com # what ordinary programs will get (uses nsswitch)
resolvectl status # servers per interface (systemd-resolved only)
resolvectl query www.example.com
dig www.example.com AAAA +short # ask DNS directly (package bind9-dnsutils or bind-utils)
dig @192.0.2.53 www.example.com # ask one particular server
If getent fails but dig works, the problem is local configuration, not DNS. Caching resolvers and authoritative servers are in the DNS HOWTO.
DHCP
Clients need nothing extra: NetworkManager and systemd-networkd have DHCP built in (DHCP=yes in a .network file). For a server, ISC's classic dhcpd reached end of life in 2022; use its successor Kea for larger networks, or dnsmasq, which serves DHCP and DNS together for a small LAN. Configuration and fixed reservations are in the DHCP mini-HOWTO.
Firewall
The kernel's packet filter is nftables. Write rules with nft, or let firewalld (Fedora, RHEL) or ufw (Ubuntu) manage them; use one tool, not several. See the Firewall HOWTO for a safe default ruleset, and the ipchains page for translating old ipchains and iptables rules.
Services
Before you open a port in the firewall, see what is actually listening:
sudo ss -tulnp # TCP and UDP, listening, numeric, with process names
ss -tn state established # current TCP connections
A service bound to 127.0.0.1 is reachable only from the machine itself; 0.0.0.0 or [::] means every interface. The old inetd super-server has been replaced by systemd socket units. For the common services, see NFS, Samba, LDAP, printing, VPNs and TLS certificates. Dial-up and DSL links are in the PPP HOWTO.
Troubleshooting from the bottom up
Most "the network is down" reports are one broken layer. Test them in order, and stop at the first one that fails.
ip -br link # 1: is the interface UP with a carrier?
sudo ethtool enp1s0 # "Link detected: yes", speed and duplex
journalctl -k -b | grep -i enp1s0
ip -br addr # 2: is there an address, and the right one?
ip route # 3: is there a default route?
ping -c 3 192.0.2.1 # can you reach the gateway?
ping -c 3 198.51.100.7 # and an outside address?
getent hosts www.example.com # 4: does the name resolve?
nc -zv www.example.com 443 # 5: does the port answer?
curl -sSI https://www.example.com/
When packets seem to vanish, watch them. tcpdump shows what really arrives at and leaves an interface; -n skips name lookups so the capture does not create DNS traffic of its own:
sudo tcpdump -ni enp1s0 port 53 # DNS queries and answers
sudo tcpdump -ni enp1s0 host 192.0.2.20 and port 22
sudo tcpdump -ni any -c 200 -w /tmp/cap.pcap # save 200 packets for Wireshark
For problems further away (slow sites, packet loss somewhere upstream), mtr combines ping and traceroute and shows loss at each hop. A report run is easy to paste into a ticket for your provider:
mtr -rwc 100 www.example.com # report mode, wide hostnames, 100 cycles
mtr -rwc 100 -T -P 443 www.example.com # use TCP to port 443 where ICMP is filtered
tracepath www.example.com # no root needed; also reports path MTU
Loss shown at one middle hop but not at the hops after it usually means that router just limits replies to probes; real loss continues to the destination.
Mistakes to avoid
- Two tools managing one interface (NetworkManager and ifupdown, or netplan and a hand-written networkd file).
- Editing
/etc/resolv.confby hand on a system where systemd-resolved or NetworkManager rewrites it. - Changing a remote machine's address or firewall without a console or an automatic rollback (
netplan try, a timed rollback for nftables). - Opening a port in the firewall for a service that only listens on
127.0.0.1. - Forgetting IPv6: a service filtered on IPv4 may be wide open on IPv6.
For the rest of the administrator's routine (users, disks, services, logs, backups), see the system administrator's guide. To understand the protocols underneath, read the Unix and Internet fundamentals HOWTO.
Sources
- ip(8), ip-address(8), ip-route(8) and ip-neighbour(8), man7.org
- ss(8), man7.org
- tcpdump(8), man7.org
- mtr(8), Debian 13 manual pages and tracepath(8), man7.org
- ethtool(8), man7.org and nc(1), Debian 13 manual pages
- nmcli(1) and nm-settings-nmcli(5), Debian 13 manual pages
- systemd.network(5), man7.org and networkctl(1), Debian 13 manual pages
- Netplan tutorial, Netplan documentation
- NetworkConfiguration, Debian Wiki and Debian 13 installation guide: configuring the network
- Configuring and managing networking, RHEL 10, Red Hat documentation
- IP sysctl (ip_forward), Linux kernel documentation
- resolvectl(1), nsswitch.conf(5) and getent(1), man7.org
- Changes/systemd-resolved (Fedora 33), Fedora Project Wiki
- dig(1), Debian 13 manual pages
- ISC DHCP (end of life), Internet Systems Consortium
- RFC 1918: Address Allocation for Private Internets, RFC 5737: IPv4 documentation addresses, RFC 3849: IPv6 documentation prefix