CKNE for Kindergarteners

The whole exam in one picture: the cluster is a city, and you are the traffic engineer.

Somebody else built the buildings (CKA) and decided who may enter them (CKS). You own the roads: how a car (packet) gets from one building (pod) to another, how it finds the address (DNS), which roads are one-way (policy), where the city gates are (gateways), and the cameras that show you what actually drove by (Hubble).

• Roads are the CNI. No CNI = no roads = every building is an island (nodes NotReady).
• Street numbers are pod IPs, handed out by IPAM from each node's block (PodCIDR).
• The phone book is CoreDNS.
• A shop's front desk is a Service: one number, many clerks (endpoints) behind it.
• The concierge who forwards calls to the right clerk used to be kube-proxy. In this city Cilium does it, in eBPF — faster, and the phone-book-of-forwardings lives in cilium-dbg service list, not iptables.
• The city gate is a Gateway; the signposts at the gate are HTTPRoutes.
• The armoured trucks are WireGuard: same roads, nobody can read the cargo.
• The cameras are Hubble. Every task ends with "check the cameras."

The exam's favourite trick: the road exists, but the sign is wrong. The Service is there, the pod is there, and one label, one port, or one policy is off. Find which with the ladder, prove it with a flow.

1. Service Networking and DNS (25%) — "the front desk and the phone book"

The front desk (Services)

Analogy: a Service is a front-desk phone number. Endpoints are the clerks who pick up. If nobody has the right badge (label), the phone rings forever.

Look for: endpointslices empty → selector vs labels. port vs targetPort → the clerk's extension. <pending> on a LoadBalancer → nobody is assigning outside numbers (need a CiliumLoadBalancerIPPool).

k -n NS get svc web -o jsonpath='{.spec.selector}'; k -n NS get pods --show-labels
k -n NS get endpointslices -l kubernetes.io/service-name=web

The concierge that isn't there (kube-proxy replacement)

Analogy: in most cities the concierge (kube-proxy) rewrites every address with sticky notes (iptables). Here the concierge was fired and the roads themselves know the way (eBPF). Looking for sticky notes proves nothing.

Look for: k -n kube-system get ds kube-proxy → NotFound. Then:

cdbg service list          # the real forwarding table
cdbg status | grep KubeProxy

The phone book (CoreDNS)

Analogy: the Corefile is the phone book's editing rules. hosts = a sticky note with one number. rewrite = "when someone asks for A, look up B." A second zone:53 { forward . X } block = "for that suburb, call this other phone book."

Look for: the sticky notes must come before the kubernetes and forward lines or nobody reads them. A typo = CoreDNS CrashLoop = the whole city can't dial anything. Restart after editing.

k -n kube-system edit cm coredns; k -n kube-system rollout restart deploy/coredns
k run dq --rm -it --restart=Never --image=busybox:1.36 -- nslookup NAME

Readiness (endpoint availability)

Analogy: a clerk who is still putting on their uniform should not be on the phone list. That's the readiness probe. A StatefulSet peer who needs to find its colleagues before it is ready needs a special list that includes the not-yet-ready: publishNotReadyAddresses: true on a headless Service.

The city gate (Gateway API)

Analogy: GatewayClass = which company runs gates (cilium). Gateway = one gate, with a listener (port 80 / 443, which hostname). HTTPRoute = the signposts: /checkout → this building, /catalog 90/10 → those two, anything else → 404.

Look for: the gate needs an address (from the LB pool); the signpost must be Accepted by the gate (namespace allowed? hostname matches?) and ResolvedRefs (the building exists). Most specific signpost wins, not the first one.

k -n NS get gateway; k -n NS get httproute R -o yaml | yq .status
curl -H 'Host: shop.example.com' http://$GWIP/checkout

2. Network Security and Policy (25%) — "one-way streets and armoured trucks"

One-way streets (NetworkPolicy / CiliumNetworkPolicy)

Analogy: no policy = every street two-way. The moment you put one sign on a building, every other direction into it is closed. You don't write "deny"; you write the list of who may come in.

Look for: "only GET /api" or "only api.stripe.com" → that's beyond plain street signs; you need Cilium's (CiliumNetworkPolicy with rules.http or toFQDNs). FQDN rules need the DNS street open and watched (rules.dns.matchPattern: "*") or they allow nothing.

Two different "no"s: a timeout is a drop (L4); a 403 is the gate guard reading the request and saying no (L7). The exam may ask you to tell them apart.

hubble observe -n NS --verdict DROPPED

Armoured trucks (encryption)

Analogy: WireGuard puts every cross-node packet in a locked truck. Same roads, same speed (almost), unreadable cargo. Trucks only run between nodes — two pods on one node never leave the building.

Look for: cdbg encrypt status → Number of peers = nodes − 1. On the wire (tcpdump -i eth0) you see UDP 51871, never the HTTP. "Node level" too? encryption.nodeEncryption=true.

helm upgrade cilium cilium/cilium --version $CV -n kube-system --reuse-values \
  --set encryption.enabled=true --set encryption.type=wireguard

The gate's ID badge (TLS on the Gateway)

Analogy: HTTPS at the gate means the gate shows a badge (certificate) with the city's name on it. The badge lives in a Secret; if the Secret is in another namespace, the owner must sign a permission slip (ReferenceGrant).

k create secret tls shop-tls --cert=tls.crt --key=tls.key
# listener: protocol HTTPS, port 443, tls.mode Terminate, certificateRefs → shop-tls
curl -k --resolve shop.example.com:443:$GWIP https://shop.example.com/

Who are you? (pod identity)

Analogy: Cilium gives every pod an ID number derived from its labels; policies talk about ID numbers, never IPs. A mesh (Istio) gives each pod a passport (mTLS certificate) instead. Same question — who is calling — two different ID systems.

3. Advanced Traffic Management (20%) — "the highways out of town"

Telling the outside world where you are (BGP)

Analogy: your LoadBalancer IPs are new streets. Nobody outside knows they exist until you tell the highway authority (the router). BGP is the formal notice: "AS 65001 here; I own 172.18.250.0/28."

Look for: three papers — who we peer with (CiliumBGPClusterConfig), how we talk (CiliumBGPPeerConfig), what we announce (CiliumBGPAdvertisement, matched by labels). cilium bgp peers must say Established.

The one exit everyone must use (egress gateway)

Analogy: the partner's guard only lets in cars with a known plate. So all finance cars leave the city through one gate, and get that gate's plate on the way out. CiliumEgressGatewayPolicy: which cars (selectors), which destinations (destinationCIDRs), which gate (nodeSelector), which plate (egressIP).

cdbg bpf egress list        # pod → dest → egress IP → gateway node

Policy says whether a car may go; egress gateway says which gate it uses.

Two cities, one phone book (ClusterMesh) — read only

Cilium can join two clusters so a Service with the same name in both is one global front desk (service.cilium.io/global: "true"). Not in this lab; know the annotation and cilium clustermesh status.

LLM traffic — read only

A special signpost (InferencePool) whose clerks are model servers and whose picker chooses by queue depth and cache, not round-robin. Know the nouns; see strategies.md.

4. Core Infrastructure and CNI (15%) — "building the roads"

No roads yet (installing a CNI)

Analogy: nodes NotReady with "cni plugin not initialized" = the city has buildings and no roads. Install Cilium with Helm; tell it where the town hall is (k8sServiceHost), because with no concierge the ClusterIP for the API server doesn't work yet.

helm install cilium cilium/cilium --version 1.18.2 -n kube-system \
  --set kubeProxyReplacement=true --set k8sServiceHost=<API-IP> --set k8sServicePort=6443
cilium status --wait

Look for on the node: /etc/cni/net.d/ (the road plan) and /opt/cni/bin/ (the road crew). Both must exist.

Street numbers (IPAM)

Analogy: every node gets a block of numbers (PodCIDR); pods get one from their node's block. A pod with a number from the wrong block means two different offices are handing out numbers (ipam mode mismatch).

k get nodes -o custom-columns=N:.metadata.name,CIDR:.spec.podCIDR
k get ciliumnodes -o jsonpath='{range .items[*]}{.metadata.name} {.spec.ipam.podCIDRs}{"\n"}{end}'

Looking at the actual cars (tcpdump / ip / iptables)

Analogy: Hubble is the camera network; tcpdump is standing on the corner watching. On this city's roads, cross-node cars travel inside armoured trucks inside tunnels — stand on eth0 and you see only trucks (UDP 51871). Stand on cilium_vxlan or use -i any host <podIP> to see the cars.

A drop looks like: the same SYN three times, no SYN-ACK. Nothing listening looks like: SYN, then RST.

A second driveway (Multus)

Analogy: some buildings need a private back road (storage, telco). Multus adds a second driveway (net1) beside the normal one (eth0). The back road is outside the city's traffic rules — no policy, no cameras.

k get network-attachment-definitions -A
k -n NS exec P -- ip addr           # eth0 and net1

5. Observability (15%) — "the cameras"

Watching live (Hubble flows)

hubble observe -n NS --last 20
hubble observe -n NS --verdict DROPPED -o json | jq -r .flow.drop_reason_desc
hubble observe -t l7 --protocol http          # only if an L7 rule exists on that port

Analogy: a live camera feed with a short memory (a ring buffer). Not history.

Counting (metrics)

hubble_drop_total{reason=…}, hubble_flows_processed_total, hubble_dns_queries_total, hubble_http_requests_total — on each agent's hubble-metrics port (9965). Cilium's own health is cilium_* on 9962.

Keeping the tapes (flow export)

hubble.export.static.enabled=true → every flow written as JSON to a file on the node for a log shipper. That is "auditing traffic with logs".

Tracing — read only

Cameras show that a car passed; tracing shows the whole trip of one passenger across many buildings. Needs OpenTelemetry/Jaeger and app cooperation; not in this lab.

Kindergarten rules for the whole exam

1. Prove it from inside a pod. k exec … curl / nslookup. Node-side success is not the task.
2. Check the cameras before and after. hubble observe --verdict DROPPED tells you in ten seconds what an hour of guessing won't.
3. Iptables is empty here on purpose. Don't panic; read cilium-dbg service list instead.
4. Helm upgrades: --version + --reuse-values, always. Or you just turned off five other features.
5. Every "deny" is really an allow-list. Write who may, not who may not.
6. Timeout = drop. 403 = L7 said no. RST = nothing listening. Read the symptom before you touch anything.
7. Put the exact file where the task said, with the exact keys. The grader is a script.