Suggest an edit

Improve this article

Refine the answer for “How to troubleshoot network problems between Docker containers?”. Your changes go to moderation before they’re published.

Approval required

Content

What you’re changing

Title (EN)

Short answer (EN)

Shown above the full answer for quick recall.

Answer (EN)

**Network troubleshooting** in Docker means walking from DNS down to packet inspection until you find where the connection breaks. Most issues are in the top three layers (wrong network, missing DNS, blocked port). Reach for tcpdump and iptables only when those fail.

## Theory

### TL;DR

- Containers reach each other only if **on the same network**.
- The **default `bridge` network has no DNS** between containers; user-defined bridges do.
- Host firewall (UFW, firewalld, custom iptables) often interferes with the docker0 bridge.
- `--network=host` shares the host's network namespace; no isolation.
- Overlay networks (Swarm) need UDP 4789 (VXLAN) and an MTU lower than the host's.
- Diagnostic order: **layer-by-layer** from name resolution down to packets.

### Diagnostic checklist

1. **What network is each container on?**
2. **Is DNS resolving the peer's name?**
3. **Is the peer's IP reachable (ping)?**
4. **Is the target port open (nc, telnet)?**
5. **Is the target service actually listening on that port and binding to the right interface?**
6. **Is a firewall (host or in-container) dropping packets?**
7. **Is anything wrong at layer 2/3 (MTU, MAC address conflict)?**

### Network types and their gotchas

**Default `bridge` (no name)**
- Auto-assigned to containers without `--network`.
- **No DNS between containers.** Names do not resolve. You must use `--link` (deprecated) or IPs.
- Avoid for anything beyond toy use.

**User-defined bridge** (`docker network create mynet`)
- DNS works: containers can reach each other by container name.
- Isolated from the default bridge.
- Default for Compose projects (one bridge per project).

**`host` network** (`--network=host`)
- Container shares host's network namespace.
- No port mapping needed; no isolation.
- Useful for performance-sensitive cases or system tools.
- Linux-only behavior; on Docker Desktop (Mac/Windows) it does **not** mean host networking, it means host of the VM.

**Overlay** (Swarm)
- Spans multiple Docker hosts via VXLAN.
- Needs UDP 4789 between hosts for VXLAN traffic, TCP/UDP 7946 for control plane.
- MTU mismatch is the classic overlay bug: VXLAN encapsulation adds 50 bytes; if host MTU is 1500, container MTU should be ≤ 1450.

**`none`** (`--network=none`)
- Loopback only. Used for sandboxing.

## Examples

### Step 1: confirm both containers are on the same network

```bash
docker inspect app --format '{{json .NetworkSettings.Networks}}' | jq
# {
#   "my-app-default": {
#     "IPAddress": "172.18.0.3",
#     "Aliases": ["app", "abc123"]
#   }
# }

docker inspect db --format '{{json .NetworkSettings.Networks}}' | jq
# {
#   "some-other-network": {
#     "IPAddress": "172.19.0.5"
#   }
# }
```

Different networks: connect them, or move one container.

```bash
docker network connect my-app-default db
# now db is on both networks
```

### Step 2: DNS

```bash
docker exec app sh -c 'getent hosts db'
# 172.18.0.5      db
```

If nothing resolves: you are probably on the default `bridge`. Move to a user-defined network.

```bash
docker exec app cat /etc/resolv.conf
# nameserver 127.0.0.11        ← embedded Docker DNS
```

`127.0.0.11` is the embedded resolver Docker injects into user-defined networks. Missing means default-bridge issue.

### Step 3: IP-level reachability

```bash
docker exec app ping -c 3 db
# PING db (172.18.0.5): 56 data bytes
# 64 bytes from 172.18.0.5: seq=0 ttl=64 time=0.080 ms
```

No response: networking layer is broken. Check `iptables`:

```bash
sudo iptables -L DOCKER-USER -nv
sudo iptables -L DOCKER -nv
```

UFW often resets these chains; add explicit allows or disable UFW interference.

### Step 4: port-level reachability

```bash
docker exec app nc -vz db 5432
# Connection to db 5432 port [tcp/postgresql] succeeded!
```

`Connection refused`: the service is not listening on that port.
`Operation timed out`: a firewall (host or in-container) is dropping the packet.

```bash
# Inside the db container
docker exec db ss -tlnp
# State    Recv-Q   Send-Q   Local Address:Port
# LISTEN   0        128      127.0.0.1:5432   ← bound to localhost, not container interface!
```

Classic mistake: Postgres bound to `127.0.0.1` instead of `0.0.0.0` is unreachable from peer containers. Fix the service config: `listen_addresses = '*'` for Postgres.

### Step 5: packet capture (when nothing else helps)

Find the bridge name:

```bash
docker network inspect my-app-default --format '{{.Id}}'
# 4f7c9... (first 12 chars become the bridge interface name suffix)
ip link show | grep br-
# br-4f7c9...
```

Capture:

```bash
sudo tcpdump -i br-4f7c9 -nn 'host 172.18.0.5 and port 5432'
```

Run a request from app, watch the packet flow. SYN with no SYN-ACK = firewall drop or service not listening. Reset = service refused.

### Step 6: container-level firewall

Some images ship with iptables rules (security-hardened distros):

```bash
docker exec db iptables -L -nv
# Chain INPUT (policy DROP 0 packets, 0 bytes)
```

`policy DROP` blocks everything not explicitly allowed. Either fix the in-container rules or use a base image without them.

### Common scenarios and fixes

#### "Cannot connect to db from app" — Compose project

```yaml
services:
  app:
    image: myorg/app
  db:
    image: postgres:16
```

Compose auto-creates `<project>_default` network. Both services on it. `app` reaches `db` by name. Should just work.

If broken: confirm both came up (`docker compose ps`), then:

```bash
docker compose exec app ping db
docker compose exec app nc -vz db 5432
```

#### "Host cannot reach container's published port"

```bash
docker run -d -p 8080:80 nginx
curl localhost:8080  # works
curl 192.168.1.5:8080  # fails from another machine
```

Check the host firewall: `ufw status`, `firewall-cmd --list-all`, or `iptables -L`. Open port 8080.

#### "Container can hit external internet but not other container"

Likely on the default `bridge` (no DNS) or different networks. Move to a user-defined bridge:

```bash
docker network create mynet
docker run --network=mynet --name=app myorg/app
docker run --network=mynet --name=db postgres:16
```

#### "Random connection drops or slow Swarm overlay"

MTU mismatch. Check host MTU:

```bash
ip link show eth0 | grep mtu
# mtu 1500
```

VXLAN overhead = 50 bytes. Containers in overlay should run at MTU 1450 (or 1400 to be safe across cloud NAT). Set per-network:

```bash
docker network create -d overlay \
    --opt com.docker.network.driver.mtu=1450 \
    swarm-net
```

#### "`--network=host` works on Linux, fails on Mac"

Docker Desktop runs Linux in a VM. `host` networking targets the VM, not your Mac. To reach `localhost` on the Mac from a container, use `host.docker.internal`.

### Useful tools

- `nicolaka/netshoot`: a debug image with `dig`, `nc`, `tcpdump`, `iperf`, `mtr`, `tshark`. Run inside the same network as the misbehaving container:

```bash
docker run --rm -it --network=container:app nicolaka/netshoot
# Now you have full diagnostic toolkit in app's network namespace
```

- `docker logs <container>` for the service's own log; often the failure is on the listening side.
- `docker exec <container> cat /etc/hosts` to see what the container sees as its own name.

## Real-world usage

- **"App cannot reach DB"**: 90% of the time, networks misaligned or DNS is missing. Apply steps 1-2.
- **Intermittent failures**: usually MTU on overlay, or DNS TTL with restarting containers.
- **Production outages**: tcpdump on the bridge, then iptables. Get a packet capture before restarting anything.

### Common mistakes

**Putting services on the default bridge**

No DNS. Always create a user-defined network or rely on Compose's auto-network.

**Binding a service to `127.0.0.1` inside a container**

Reachable only from within that container. Bind to `0.0.0.0` so other containers can reach it on the bridge.

**Modifying host iptables manually**

Docker rewrites iptables on every restart. Custom rules belong in the `DOCKER-USER` chain (Docker leaves it alone).

**Using `--link`**

Deprecated. Sets up `/etc/hosts` entries, no DNS. Use user-defined networks.

### Follow-up questions

**Q:** Why does the default bridge not have DNS?

**A:** Historical. The default bridge predates Docker's embedded DNS server. User-defined bridges added DNS but old behavior was kept for compatibility.

**Q:** What is `127.0.0.11`?

**A:** Docker's embedded DNS resolver. Each container in a user-defined network has it as its `nameserver` and queries are resolved to other containers' IPs.

**Q:** How do I see what packets are dropped at the host firewall?

**A:** Add a logging rule before the DROP rule: `iptables -I DOCKER-USER -j LOG --log-prefix 'docker-drop: '`. Then `dmesg -w` shows the drops in real time.

**Q:** (Senior) How do you debug an overlay network that works locally but loses packets across hosts?

**A:** Confirm UDP 4789 (VXLAN) is allowed between hosts (cloud security groups often block it). Run `tcpdump -i any port 4789` on both hosts during a failing request. Check MTU; lower it to 1400 if VXLAN is going through any tunnel that adds more overhead. Inspect `docker network inspect <overlay>` for peer counts; mismatched peers means Swarm gossip is unhealthy.

**Q:** (Senior) How do you detect MAC address conflicts on a custom bridge?

**A:** `docker network inspect <net>` lists each container's MAC. Conflicts come from manually setting `--mac-address`. The kernel logs `bridge: received packet on br0 with own address as source address` on conflicts. Fix by removing manual MAC assignment.

Markdown · drag & drop images · ⌘B / ⌘I shortcuts1454 words

For the reviewer

Note to the moderator (optional)

Visible only to the moderator. Helps review go faster.