ICA for Kindergarteners

The whole exam in one picture: Istio is the traffic system for a city of pods.

Without a mesh, cars (requests) drive wherever they like and nobody knows who went where. Istio adds:

• A crossing guard on every corner — in sidecar mode that's a guard standing next to every single pod; in ambient mode it's a neighbourhood watch on every street (ztunnel) that handles the simple stuff, plus a proper checkpoint (waypoint) you set up only where you need the clever stuff.
• Road signs — the VirtualService says where cars go: split them 80/20, send the ones with a red sticker left, wait 2 seconds then give up.
• Parking-lot rules — the DestinationRule says what happens when they arrive: which lot is "v1," how many cars fit, kick out a lot that keeps crashing.
• Badges and whispering — mTLS. Every pod wears a badge; they talk in code. AuthorizationPolicy is the guest list at the door.

ICA is hands-on with some multiple-choice (the official page: "performance-based and multiple-choice"). You'll be building the signs and lots yourself in a live mesh, and then also asked what happens — and the only way to know is to have made it happen. The two sentences that answer most of both:

VirtualService = the road. DestinationRule = the parking lot. Ambient: the watch (ztunnel) does simple things. Clever things need a checkpoint (waypoint). No checkpoint, no clever things.

1. Traffic Management (35%) — "road signs and parking lots"

Send 8 out of 10 cars left (traffic split)

Analogy: paint two parking lots, "v1" and "v2" (DestinationRule), then put up a sign that says 80% this way, 20% that way (VirtualService).

What to look for: "canary," "shift traffic," "weight," "subset."

kind: DestinationRule            # the lots
spec:
  host: reviews
  subsets:
    - { name: v1, labels: { version: v1 } }     # a lot is a set of POD labels
    - { name: v2, labels: { version: v2 } }
---
kind: VirtualService             # the sign
spec:
  hosts: [reviews]
  http:
    - route:
        - { destination: { host: reviews, subset: v1 }, weight: 80 }
        - { destination: { host: reviews, subset: v2 }, weight: 20 }

istioctl waypoint apply -n mesh --enroll-namespace --wait     # ambient: build the checkpoint!
istioctl analyze -n mesh                                      # spell-check the signs

Kindergarten rules:

• A sign pointing at a lot that was never painted → 503 NC.
• A lot painted with labels no pod has → 503 UH.
• Weights that don't add to 100 are not refused — Istio quietly shares them out anyway (70/40 becomes ~64/36). Do the sum yourself.
• Ambient with no checkpoint: the sign is ignored and cars split 50/50.

Red-sticker cars go left (match routing)

http:
  - match: [{ headers: { end-user: { exact: jason } } }]
    route: [{ destination: { host: reviews, subset: v2 } }]
  - route: [{ destination: { host: reviews, subset: v1 } }]     # everyone else — LAST

First sign that matches wins, so the "everyone else" sign goes at the bottom.

Wait, then give up; try again (timeouts and retries)

http:
  - timeout: 2s
    retries: { attempts: 3, perTryTimeout: 1s, retryOn: "5xx,reset" }

Kindergarten rule: timeout is the whole trip's clock. 3 tries × 1s wants 3s; the 2s clock wins, so the third try never happens. And Istio already retries twice even if you say nothing.

Close the lot that keeps crashing (outlier detection / circuit breaking)

kind: DestinationRule
spec:
  trafficPolicy:
    connectionPool: { tcp: { maxConnections: 10 } }          # lot capacity
    outlierDetection:                                         # kick out a bad lot
      consecutive5xxErrors: 3
      interval: 5s
      baseEjectionTime: 30s
      maxEjectionPercent: 100

Kindergarten rule: maxEjectionPercent defaults to 10%. With 2 lots, 10% of 2 rounds to zero — nothing can ever be kicked out. Set it.

Pretend the road is closed (fault injection)

Analogy: a fire drill. Make half the cars hit a fake roadblock so you can see if the drivers (the callers) cope.

http:
  - fault:
      delay: { fixedDelay: 2s, percentage: { value: 100 } }
      abort: { httpStatus: 503, percentage: { value: 50 } }

percentage is { value: N }, not a bare number. Delay happens before abort, so even the aborted cars sit at the roadblock first.

The city gate (ingress Gateway)

Gateway API style (what current Istio prefers):

kind: Gateway
spec:
  gatewayClassName: istio
  listeners: [{ name: http, port: 80, protocol: HTTP, allowedRoutes: { namespaces: { from: Same } } }]
---
kind: HTTPRoute
spec:
  parentRefs: [{ name: gw }]
  rules: [{ matches: [{ path: { type: PathPrefix, value: / } }], backendRefs: [{ name: shop, port: 80 }] }]

Is the gate actually built? Programmed=True. If it says AddressNotAssigned, nobody gave it a street address (LoadBalancer IP).

Roads out of town (egress / ServiceEntry)

Analogy: by default cars may drive to any city. Flip the mesh to REGISTRY_ONLY and they may only go to cities on the map — a ServiceEntry adds a city to the map.

kind: ServiceEntry
spec: { hosts: [api.example.com], location: MESH_EXTERNAL, resolution: DNS,
        ports: [{ number: 443, name: https, protocol: HTTPS }] }

2. Securing Workloads (25%) — "badges and guest lists"

Everyone wears a badge (mTLS / PeerAuthentication)

Analogy: PERMISSIVE = badges optional (visitors and residents both get in). STRICT = badge or you don't get through the door at all. The sign goes on a street (namespace), a house (workload), or the whole city (istio-system).

What to look for: "require mTLS," "reject plaintext," "this namespace only."

kind: PeerAuthentication
metadata: { name: default, namespace: secure }      # street-level: no selector
spec:
  mtls: { mode: STRICT }

Check it: a resident (in-mesh pod) gets 200; a visitor from outside gets the door slammed — a connection reset, not a 403. STRICT happens before HTTP exists.

Kindergarten rule: "this namespace only" means put it in that namespace, never in istio-system — that's the whole city. In ambient the watch (ztunnel) does badges; no checkpoint needed.

The guest list (AuthorizationPolicy)

Analogy: a bouncer with a list: who (which badge), doing what (GET), where (/api/*).

kind: AuthorizationPolicy
spec:
  targetRefs: [{ kind: Service, group: "", name: api }]     # AMBIENT: hand the list to the CHECKPOINT
  action: ALLOW
  rules:
    - from: [{ source: { principals: ["cluster.local/ns/authz/sa/frontend"] } }]
      to:   [{ operation: { methods: [GET], paths: ["/api/*"] } }]

Check it: right badge + GET → 200. Wrong badge → 403. Right badge but POST → 403.

Kindergarten rules:

• In ambient, giving a list with paths and methods to the watch (selector) instead of the checkpoint (targetRefs) → the watch can't read it → 503 for everyone, even the allowed ones.
• Badges are the long name: cluster.local/ns/X/sa/Y. Short names match nobody.
• The moment there's any ALLOW list on a door, everyone not on it is out. rules: [] = nobody. No rules at all = everybody.

A ticket from another town (JWT)

RequestAuthentication checks the ticket is genuine — but a person with no ticket still walks in. The AuthorizationPolicy with requestPrincipals is what says "ticket required."

A lock on the city gate (edge TLS)

Gateway listener with tls: { mode: Terminate, certificateRefs: [{ name: cert }] } on a kubernetes.io/tls Secret. The name on the cert must be a SAN, or modern browsers won't trust it. Passthrough = don't open the letter, just forward the sealed envelope (needs TLSRoute).

3. Installation, Upgrades, Configuration (20%) — "hiring the traffic department"

Hire them (install)

istioctl install --set profile=ambient -y      # neighbourhood-watch style (needs Gateway API CRDs)
istioctl install --set profile=default -y      # a guard beside every pod
k label ns shop istio.io/dataplane-mode=ambient      # this street joins the watch
k label ns shop istio-injection=enabled              # or: every new pod gets a guard
k rollout restart deploy -n shop                     # guards are assigned at BIRTH — restart to get one

Train the new department beside the old one (canary upgrade)

Analogy: don't fire all the guards at once. Hire the new team (revision), move one street to them, watch, then move the rest.

istioctl install --set revision=canary -y            # second department; NOTHING moves yet
k label ns staging istio.io/rev=canary --overwrite   # one street switches
k rollout restart deploy -n staging
istioctl proxy-status                                # who reports to whom
istioctl tag set default --revision canary --overwrite     # promote
istioctl uninstall --revision=old -y                 # old team goes LAST

Kindergarten rules: installing the new team moves nobody — that's the point. Don't dismantle a team by deleting its pods by hand; istioctl uninstall it, or the next hire wedges at 503 with a permissions error.

4. Troubleshooting (20%) — "why are the cars stuck"

Walk in this order, every time:

istioctl analyze -n NS                              # 1. are the signs spelled right?
istioctl proxy-status                               # 2. are the guards awake? (SYNCED vs STALE)
istioctl proxy-config cluster deploy/waypoint.NS | grep svc   # 3. do the lots exist?
istioctl proxy-config route   deploy/waypoint.NS -o json      # 4. what do the signs really say?
k -n NS logs deploy/waypoint | grep -o 'response_flags=[A-Z]*' # 5. WHY 503?

The 503 decoder ring:

And the ambient special: "my clever sign does nothing" → there's no checkpoint. istioctl waypoint apply.

Kindergarten rules for the whole exam

1. Road or lot? VirtualService or DestinationRule — decide first.
2. Simple or clever? Badges and ports = the watch. Paths, methods, weights, retries = needs a checkpoint.
3. Where does the sign go? House / street / city — and city means istio-system.
4. What kind of "no"? 503 = a config mistake. 403 = the guest list. A slammed door with no number = mTLS.
5. Guards are assigned at birth. Label the street, then restart the pods.