Networking

Overview

PodWarden tracks network topology across your fleet. Hosts, storage connections, and stacks are tagged with network types that describe how they're reachable. Before every deployment, PodWarden checks whether the target cluster can actually reach the required storage and services — and warns you if something won't work.

This prevents the most common distributed infrastructure failure: deploying a workload that references storage it can't reach, then debugging silent mount failures at runtime.

Network Types

Three product-agnostic network types describe connectivity:

These types are deliberately product-agnostic. mesh covers any overlay VPN — Tailscale today, Nebula or WireGuard tomorrow. The system doesn't care which product provides the connectivity, only what's reachable.

A host, storage connection, or workload can have multiple network types. A server with both a public IP and a Tailscale address would be tagged ["public", "mesh"].

Hosts

Viewing network types

Host network badges appear on the hosts list page and host detail page. Badges are color-coded: blue for public, purple for mesh, amber for LAN.

Auto-detection

During probing, PodWarden discovers all network interfaces on the host and automatically classifies them:

A host can have multiple types. For example, a homelab node with both a LAN IP and a Tailscale IP gets ["lan", "mesh"]. A remote node with only Tailscale gets ["mesh"].

Hosts with a tailscale_id (discovered via Tailscale) also get the mesh type inferred automatically.

Editing network types

On the host detail page, click Edit next to the network types section. Toggle the types that apply and save. The auto-inferred mesh type from Tailscale is always included for hosts with a tailscale_id.

Storage Connections

Tagging storage

When creating or editing a storage connection, use the Reachable via selector to tag which networks can reach this storage:

• An NFS share on a LAN server: lan
• An NFS share on a Tailscale host (e.g. storage.h): mesh
• An S3 bucket on the public internet: public
• A MinIO instance on your VPN: mesh

Auto-suggest

PodWarden auto-suggests network types based on the storage address:

Suggestions only apply when the network types field is empty. You can always override them.

Stacks

Required connectivity

stacks have a Required connectivity field that declares what network types the target cluster must support:

• A WordPress site that needs to be publicly accessible: ["public"]
• An internal API that only talks to mesh peers: ["mesh"]
• A workload that needs both internet access and LAN storage: ["public", "lan"]

Set this in the create or edit form under Required connectivity.

Clusters

Cluster network types are computed automatically as the intersection of all member hosts' network types. This means:

• If all hosts in a cluster have public and mesh, the cluster supports ["mesh", "public"]
• If one host only has mesh, the cluster supports ["mesh"] — because Kubernetes can schedule pods on any node

The intersection approach ensures every node in the cluster can reach the required resources, since K8s may place the pod on any node.

Deploy-Time Checks

How it works

When you click Deploy on a deployment, PodWarden runs a pre-flight network check:

1. Computes the cluster's reachable networks (intersection of member hosts' types)
2. Checks the stack's required_network_types against cluster networks
3. Checks each volume mount's storage connection network_types against cluster networks
4. If any network type is required but not available, generates a warning

Warning behavior

• Warnings are advisory only — they never block deployment
• If warnings are found, an amber modal shows the specific issues
• Click Deploy anyway to proceed, or Cancel to fix the configuration first
• If the check endpoint is unavailable, deployment proceeds normally

Example warnings

• "Workload requires 'public' but cluster only supports: mesh, lan"
• "Storage connection 'nas-media' requires 'lan' but cluster only supports: mesh"

Skipped checks

No warnings are generated when:

• Network types are empty/unspecified on any entity (no false positives)
• The cluster has no hosts yet
• The stack has no required network types and no storage connections with network types

Provisioning Networking

When PodWarden provisions a host and joins it to a K3s cluster, it makes several networking decisions automatically. You don't need to configure any of this — PodWarden handles it based on the host's detected network types and the control plane's configuration.

How PodWarden Picks the Connection Path

The most important decision during provisioning is which address the new node uses to reach the cluster's control plane:

• LAN nodes (hosts with lan in their network types) connect via the control plane's LAN IP — faster, no tunnel overhead
• Mesh-only nodes (hosts with only mesh) connect via the control plane's Tailscale IP — since the LAN is unreachable from remote sites

If a host has both lan and mesh, PodWarden prefers the LAN path for better performance.

Flannel Interface Selection

K3s uses flannel for pod-to-pod networking. The overlay must use the same network path as the API connection — otherwise pods on different nodes can't communicate.

PodWarden determines the correct flannel interface automatically based on the route to the control plane:

• If connecting via Tailscale → flannel uses tailscale0
• If connecting via LAN → flannel uses the LAN interface (e.g. eth0, ens18)

This replaces the need to manually configure --flannel-iface when joining nodes.

Tailscale Pre-Warm

Before connecting to a remote control plane, PodWarden "pre-warms" the Tailscale tunnel by sending pings. This forces the WireGuard tunnel to establish, avoiding connection timeouts caused by idle tunnel expiry or NAT traversal delays.

This happens automatically — you may see "Pre-warming Tailscale tunnel" in the provisioning logs.

Mixed Networks: The NAT Proxy

The most complex scenario is a mixed-network cluster — some nodes on LAN, some on mesh only. This is common in homelabs with remote nodes:

Home LAN:                       Remote site:
┌──────────────────┐            ┌──────────────┐
│ k3s-1 (CP)       │            │ bonus        │
│ LAN: 10.10.0.183 │───mesh────│ TS: 100.x.x  │
│ TS: 100.x.x      │            │ (no LAN)     │
│                   │            └──────────────┘
│ k3s-2 (worker)   │
│ LAN: 10.10.0.175 │
└──────────────────┘

The problem: K3s servers have a single --advertise-address (typically the LAN IP). This address is used for the agent tunnel — the websocket connection that enables kubectl logs, kubectl exec, and port-forwarding. Mesh-only nodes can't reach the LAN address, so these commands fail with 502 Bad Gateway or connection timeout errors, even though the node is otherwise healthy and running pods normally.

The solution: PodWarden automatically sets up iptables NAT rules on mesh nodes that redirect the unreachable LAN address to the control plane's Tailscale IP. This is transparent — the K3s agent thinks it's connecting to the advertised LAN address, but the traffic is silently redirected through the mesh tunnel.

This is set up as a systemd service (k3s-proxy-nat) that starts before the K3s agent and persists across reboots. It is removed automatically when you wipe the host.

You don't need to do anything — PodWarden detects the mismatch during provisioning and configures the NAT proxy automatically. You'll see "NAT proxy" messages in the provisioning logs when this happens.

VXLAN Overlay in Mixed Networks

In addition to the control plane tunnel (handled by the NAT proxy above), mixed-network clusters have a second connectivity challenge: the pod overlay network.

K3s uses flannel with VXLAN to create a virtual network that connects pods across nodes. Each node announces its IP address for this overlay traffic. In a mixed cluster, LAN nodes announce their LAN IP (e.g. 10.10.0.183) — but mesh-only nodes can't reach LAN addresses. Without a fix, pod-to-pod traffic between LAN nodes and mesh nodes is silently dropped.

This affects everything that relies on the pod network: inter-pod communication, DNS resolution (CoreDNS runs as pods), Kubernetes Services, and commands like kubectl exec and kubectl logs that route through the overlay.

What PodWarden does:

PodWarden applies two complementary fixes during provisioning, one on each side of the mixed network:

1. On LAN nodes — PodWarden installs a lightweight systemd service (k3s-vxlan-mesh-fix) that ensures VXLAN packets sent to mesh nodes use the correct Tailscale source IP and valid checksums. This service is transparent and automatically covers all mesh peers in the cluster.

2. On mesh-only nodes — PodWarden installs a service (k3s-vxlan-fdb-fix) that redirects pod overlay traffic from unreachable LAN addresses to the correct Tailscale addresses. A mapping file is maintained and updated automatically when new nodes join the cluster.

The result: With both fixes in place, the full Kubernetes networking stack works seamlessly across mixed networks — pod-to-pod traffic, DNS resolution via CoreDNS, Service routing, and kubectl exec/kubectl logs all function identically whether pods are on LAN nodes or mesh nodes.

You don't need to configure anything — PodWarden detects the mixed-network topology during provisioning and sets up both fixes automatically. You'll see "VXLAN mesh fix" messages in the provisioning logs.

When Is the NAT Proxy Needed?

When Is the VXLAN Fix Needed?

Verifying It Works

After provisioning a mesh node, verify the full stack works:

1. The node appears as Ready in the cluster's node list
2. Workloads can be deployed to the node
3. kubectl logs works for pods on the node (this specifically tests the agent tunnel)
4. kubectl exec works for pods on the node
5. Pod DNS works for pods on mesh nodes (kubectl exec <pod> -- nslookup google.com)

If kubectl logs returns a 502 error for pods on a specific node but the node is otherwise healthy, the NAT proxy may not have been set up correctly. Try wiping and re-provisioning the host.

DNS Policy

By default, Kubernetes pods use dnsPolicy: ClusterFirst, which configures /etc/resolv.conf with the cluster's CoreDNS server and K8s search domains (with ndots:5). This works well for most workloads, but can cause problems for Docker-in-Docker or nested container runtimes.

The problem with Docker-in-Docker

Workloads like Kasm Workspaces run Docker inside a Kubernetes pod. Docker's embedded DNS server inherits the pod's /etc/resolv.conf and tries to forward unknown queries to CoreDNS at 10.43.0.10. However, CoreDNS is only reachable from the pod network — not from Docker's internal bridge network (172.17.0.x). When DNS forwarding fails, all name resolution inside Docker containers breaks, causing services like nginx to crash-loop.

The fix: dnsPolicy: Default

Setting dns_policy to Default on the stack makes the pod use the node's DNS configuration instead of CoreDNS. The node's /etc/resolv.conf typically points to a DNS server reachable from any network — including Docker bridge networks.

Setting dns_policy

Set the dns_policy field on the stack definition:

• UI: Edit the stack and set the DNS Policy field
• MCP: create_stack(name="kasm", ..., dns_policy="Default") or update_stack(id="...", dns_policy="Default")
• API: Include "dns_policy": "Default" in the POST/PUT body to /stacks

The dns_policy is applied to all pods in the stack, including compose stack services.

When to use each policy

• ClusterFirst (default): Use for all standard workloads. Pods can resolve Kubernetes service names (e.g. my-svc.default.svc.cluster.local).
• Default: Use for Docker-in-Docker workloads (Kasm, Gitea with container builds, CI runners). Trade-off: pods lose the ability to resolve K8s service names, but gain DNS that works inside nested containers.
• ClusterFirstWithHostNet: Automatically set by PodWarden when host_network: true. You rarely need to set this manually.
• None: Advanced use only. Requires providing a custom dnsConfig in the pod spec.

Best Practices

1. Tag hosts as you add them. Set network types during host creation or after discovery. Auto-detected mesh from Tailscale helps, but explicitly tag public and lan.

2. Tag storage connections when you create them. The auto-suggest helps, but verify it — a .h hostname might not always mean mesh-only.

3. Set required connectivity on workloads that need specific access. A public-facing web app should require public. An internal batch processor probably doesn't need any specific type.

4. Don't over-tag. Leave network types empty when connectivity isn't a concern. Empty means "no restrictions" — PodWarden won't generate false warnings.

5. Use the check before deploying to new clusters. When moving a workload to a different cluster, the pre-flight check catches network mismatches immediately.

6. Prefer LAN when available. Nodes with both LAN and mesh connectivity will automatically use the LAN path for cluster traffic. This gives lower latency and higher throughput than mesh tunnels.

7. Keep one CP advertise-address. If your cluster has both LAN and mesh nodes, keep the control plane's --advertise-address as the LAN IP. PodWarden handles the NAT proxy for mesh nodes automatically. Don't change it to a Tailscale IP — that would break LAN nodes.

Public Access: Gateway Nodes, Ingress & DDNS

For exposing workloads to the internet — gateway nodes, ingress rules, HTTPS, and dynamic DNS — see the dedicated guide: Ingress, Gateway Nodes & DDNS.