LFCS for Kindergarteners
The whole exam in one picture: a Linux box is a house, and you're the caretaker.
- systemd is the butler. Services are the appliances the butler switches on. Timers are the alarm clocks.
- Packages are the groceries; the package manager is the pantry.
- Kernel parameters are the thermostat dials in the basement.
- Disks and LVM are shelves. fstab is the list taped to the fridge of what goes on which shelf. Swap is the spare room for overflow.
- Users and groups are who lives here and which family they belong to. ACLs are extra name tags on specific doors.
- nftables is the front gate. SSH is the intercom. chrony sets all the clocks.
- AppArmor / SELinux is a leash — even the owner can't go past the end of it.
You have 2 hours, ~17–20 chores, and only man pages to read. Every chore
ends the same way: check it the way an inspector would.
1. Operations Deployment (25%) — "the butler and the pantry"
Teach the butler a new appliance (systemd service)
Analogy: write a card for the butler: what to run, when, and whether to turn it back on if it trips.
# /etc/systemd/system/thing.service
[Unit]
Description=My thing
After=network.target
[Service]
ExecStart=/usr/local/bin/thing # the FULL path — the butler has no $PATH
Restart=on-failure
[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload # "butler, re-read your cards" — after EVERY edit
sudo systemctl enable --now thing # on now, AND on at every boot
systemctl status thing
sudo journalctl -u thing -n 50 # what the appliance said before it tripped
What to look for when one's broken: status=203/EXEC = the card has the
wrong path. "Failed to locate executable" — same. systemctl cat thing shows
the card plus any sticky-note overrides.
Kindergarten rule: start = on today. enable = on every day. "Must
survive reboot" means enable.
Alarm clocks (timers and cron)
# thing.timer
[Timer]
OnBootSec=30s
OnUnitActiveSec=1min
[Install]
WantedBy=timers.target # NOT multi-user — a common slip
sudo systemctl enable --now thing.timer # enable the CLOCK, not the appliance
systemctl list-timers
Or cron: crontab -e. In /etc/cron.d/ you must say whose alarm it is:
*/5 * * * * root /path/cmd.
The thermostat (kernel parameters)
sudo sysctl -w net.ipv4.ip_forward=1 # turn the dial NOW (forgotten at reboot)
echo 'net.ipv4.ip_forward = 1' | sudo tee /etc/sysctl.d/60-x.conf # write it on the wall
sudo sysctl -p /etc/sysctl.d/60-x.conf # read the wall now
Kindergarten rule: "now and after reboot" = both. The wall alone changes nothing until you read it.
The pantry (packages)
| chore | Ubuntu | Rocky |
|---|---|---|
| buy it | apt-get install X | dnf install X |
| which bag did this come in? | dpkg -S /path | rpm -qf /path |
| which bag would have it? | apt-file search X | dnf provides */X |
| what's in the bag? | dpkg -L X | rpm -ql X |
| don't ever swap this bag | apt-mark hold X | dnf versionlock add X |
| has the bag been tampered with? | debsums -c X | rpm -V X |
dpkg -S only knows bags already in the pantry. For something you don't
have yet, apt-file / dnf provides.
A drawer that won't open (filesystem recovery)
Analogy: the label fell off a drawer, so the house doesn't know what's in it. There are spare labels hidden inside — find one.
sudo mke2fs -n /dev/X # -n !!! "where WOULD the spare labels be" — builds NOTHING
sudo e2fsck -y -b 32768 /dev/X # repair using spare label #32768
sudo e2fsck -fy /dev/X; echo $? # while UNMOUNTED; 0 or 1 = fixed
sudo mount /dev/X /mnt/rescue
Kindergarten rules: never check a drawer while it's open (mounted).
mke2fs without -n empties the drawer. A tiny drawer (128 MiB) has no
spare labels at all.
Boxes inside the house (containers)
sudo podman pull docker.io/library/nginx:1.27-alpine # say the WHOLE address
sudo podman volume create webdata
sudo podman run -d --name web -p 8081:80 -v webdata:/usr/share/nginx/html:ro --restart always nginx…
sudo podman generate systemd --name web --files --new # hand the box to the butler
sudo systemctl enable container-web.service
Kindergarten rule: --restart always means "podman restarts it if
podman is awake." Podman has no butler of its own — for reboot, systemd
must own the box.
A whole other house in the garage (libvirt)
sudo virsh net-autostart default; sudo virsh net-start default
sudo virsh pool-define-as pool dir --target /var/lib/libvirt/pool; pool-start; pool-autostart
sudo virsh vol-create-as pool disk.qcow2 1G --format qcow2
sudo virsh define vm.xml; sudo virsh autostart vm; sudo virsh dominfo vm
define = write it in the book. create = start it once, forget it.
destroy = pull the plug (not delete). --config = next time; --live =
right now.
The leash (AppArmor / SELinux)
Analogy: even the owner, on a leash, can't reach the top shelf.
AppArmor (Ubuntu):
aa-status
sudo aa-complain /path/prog # loose leash — just note where they'd have been stopped
sudo aa-enforce /path/prog # tight leash
sudo dmesg | grep DENIED # where the leash pulled
SELinux (Rocky):
getenforce; sudo setenforce 1
ls -Z /path; sudo semanage fcontext -a -t httpd_sys_content_t "/web(/.*)?"; sudo restorecon -Rv /web
sudo setsebool -P httpd_can_network_connect on
sudo ausearch -m AVC -ts recent; sudo sealert -a /var/log/audit/audit.log
Kindergarten rule: chcon is a sticky note that blows away;
semanage + restorecon is written in the book.
2. Networking (25%) — "the gate, the intercom, the clocks"
Give a room an address (IP, route, names)
sudo ip addr add 10.0.0.5/24 dev eth1; sudo ip link set eth1 up
sudo ip route add 10.9.0.0/24 via 10.0.0.1
ip route get 10.9.0.7 # "which door would you leave by?" — ask, don't guess
Write it on the wall (Ubuntu netplan, /etc/netplan/90-x.yaml):
network: { version: 2, ethernets: { eth1: { addresses: [10.0.0.5/24], routes: [{ to: 10.9.0.0/24, via: 10.0.0.1 }] } } }
sudo netplan generate; sudo netplan try; sudo netplan apply
Names: /etc/hosts, then getent hosts NAME — that's how a program
asks, so that's how you check.
Kindergarten rule: never rewire the door you came in through (eth0 / your
SSH). netplan try puts it back if you locked yourself out.
The front gate (nftables)
Analogy: the gate starts wide open. You want: nobody in, except people already talking to us, except the postman (22), except visitors to the shop (80). Write the whole list then shut the gate — never shut it first.
table inet filter {
chain input { type filter hook input priority 0; policy drop;
iif "lo" accept # you may walk around inside
ct state established,related accept # replies to things WE started
tcp dport { 22, 80 } ct state new accept # postman, shop visitors
}
}
table ip nat {
chain prerouting { type nat hook prerouting priority -100;
tcp dport 8080 redirect to :80 } # side door → front door
}
sudo nft -c -f /etc/nftables.conf && sudo nft -f /etc/nftables.conf # check, then apply in ONE go
sudo systemctl enable nftables # the gate remembers at boot
Kindergarten rule: forget established,related and your own outgoing
calls get no answers — it looks like the internet broke.
The intercom (SSH)
sudo tee /etc/ssh/sshd_config.d/90-x.conf <<'EOF'
PermitRootLogin no
PasswordAuthentication no
EOF
sudo mkdir -p /run/sshd; sudo sshd -t; sudo systemctl reload ssh
sudo sshd -T | grep -i permitrootlogin # what the intercom REALLY thinks, all notes merged
Kindergarten rule (Ubuntu): the butler owns the intercom's doorbell
(ssh.socket). Writing Port 2222 in sshd's card does nothing — you add
the bell to the butler's card:
# /etc/systemd/system/ssh.socket.d/90-x.conf
[Socket]
ListenStream=2222
Client side: ~/.ssh/config with Host / HostName / Port / User;
check with ssh -G name.
The clocks (chrony)
echo 'server time.cloudflare.com iburst' | sudo tee /etc/chrony/conf.d/90-x.conf
sudo systemctl restart chrony; chronyc sources -v # ^* = the clock we trust
Two doors as one (bond) and a hallway (bridge)
sudo ip link add bond0 type bond mode active-backup miimon 100
sudo ip link set eth1 down; sudo ip link set eth1 master bond0; sudo ip link set eth1 up
cat /proc/net/bonding/bond0
sudo ip link add br0 type bridge; sudo ip link set eth2 master br0
A door must be closed (down) before you join it to the pair. The house number goes on the bond, not on either door.
The receptionist (reverse proxy / load balancer)
nginx: location /a { proxy_pass http://127.0.0.1:9001/; proxy_set_header Host $host; }
— the trailing / means "drop the /a before passing it on."
HAProxy: backend pool balance roundrobin server a 127.0.0.1:9001 check —
without check, the receptionist keeps sending guests to an empty desk.
3. Storage (20%) — "shelves"
Shelves you can extend (LVM)
Analogy: a plank (PV) → a bookcase made of planks (VG) → a shelf on the bookcase (LV) → a drawer on the shelf (filesystem). To make a bigger shelf, extend the shelf then the drawer.
sudo pvcreate /dev/vdb /dev/vdc
sudo vgcreate vgdata /dev/vdb /dev/vdc
sudo lvcreate -L 2G -n lvapp vgdata
sudo mkfs.ext4 -L APPDATA /dev/vgdata/lvapp
sudo lvextend -r -L 3G /dev/vgdata/lvapp # -r = also stretch the drawer
pvs / vgs / lvs to look. -L = size, -l = pieces (-l +100%FREE).
XFS drawers can grow but never shrink.
The list on the fridge (fstab)
echo "UUID=$(sudo blkid -s UUID -o value /dev/vgdata/lvapp) /srv/app ext4 defaults 0 2" | sudo tee -a /etc/fstab
sudo mount -a # try the list NOW, not at reboot
findmnt --verify --verbose
Kindergarten rule: use the shelf's serial number (UUID), not its
position (/dev/vdb) — positions change when you move furniture.
The spare room (swap)
sudo fallocate -l 256M /swapfile; sudo chmod 600 /swapfile # 600 or it's refused
sudo mkswap /swapfile; sudo swapon -p 10 /swapfile; swapon --show
echo '/swapfile none swap sw,pri=10 0 0' | sudo tee -a /etc/fstab
The neighbour's garage (NFS) and the door that opens as you walk up (autofs)
echo '/srv/share 127.0.0.1(rw,sync,no_root_squash,no_subtree_check)' | sudo tee /etc/exports
sudo exportfs -ra # re-read the list, no restart
sudo mount -t nfs 127.0.0.1:/srv/share /mnt/x
# autofs
echo '/autofs /etc/auto.x --timeout=30' | sudo tee /etc/auto.master.d/x.autofs
echo 'share -fstype=nfs,rw 127.0.0.1:/srv/share' | sudo tee /etc/auto.x
sudo systemctl enable --now autofs; ls /autofs/share # the LOOK opens the door
Kindergarten rule: ls /autofs looks empty until you walk up to a
specific door. That's not broken.
Is the shelf slow? (iostat)
iostat -x 1 3 — take at least two readings; the first is "since forever."
await is the number that matters.
4. Essential Commands (20%) — "everyday chores"
"The bin is full but there's nothing in it" (disk space)
Analogy: someone's still holding a bag you already threw out — it's not
in the bin (du can't see it) but it still takes up room (df counts it).
df -h; du -sh /srv/x # they disagree → a deleted-but-open file
sudo lsof +L1 | grep /srv/x # who's holding the bag
sudo systemctl restart THAT # make them let go (or truncate /proc/PID/fd/N)
df -i # the OTHER "full": out of labels (inodes), not space
An ID card (certificates)
openssl req -x509 -nodes -newkey rsa:2048 -days 365 -keyout site.key -out site.crt \
-subj "/CN=site.local" -addext "subjectAltName=DNS:site.local" # the NAME must be on the card (SAN)
openssl x509 -in site.crt -noout -text | grep -A1 'Subject Alternative'
sudo chmod 600 site.key
Does the card match the key? Compare openssl x509 -modulus with
openssl rsa -modulus. Then wire nginx: listen 443 ssl; ssl_certificate …; ssl_certificate_key …; → nginx -t → reload.
The diary with an undo button (git)
git clone URL dir; git config user.name "Me" # no --global = this diary only
git add f; git commit -m "note"; git push origin main
git checkout -b staging; …; git merge --no-ff staging # keep the "I had a branch" page
git tag -a v1.0 -m "release"; git push origin v1.0 # tags do NOT push by themselves
git log --oneline --graph --all; git revert SHA; git reflog
Is the house healthy? (performance)
top, free -h, vmstat 1, iostat, systemctl --failed,
journalctl -p err -b. Limits for people: /etc/security/limits.d/. Limits
for appliances: LimitNOFILE= on the butler's card.
5. Users and Groups (10%) — "who lives here"
Move someone in (users, groups)
sudo groupadd -g 4200 webops
sudo useradd -u 4201 -m -s /bin/bash -G webops -e 2030-12-31 deploy
sudo useradd -r -s /usr/sbin/nologin -M robot # a resident who can't use the front door
sudo usermod -aG webops alice # -a! or you REPLACE her families
id deploy; getent passwd deploy; sudo chage -l deploy
Name tags on doors (ACLs)
sudo setfacl -m u:deploy:rwx /srv/releases # this door, this person
sudo setfacl -d -m u:deploy:rwx /srv/releases # AND every new door built inside — "default"
getfacl /srv/releases; ls -ld /srv/releases # a + at the end = has name tags
sudo chmod 2775 /srv/releases # setgid: new files join the room's family
Kindergarten rule: "and future files too" = the -d default tag.
House rules (profiles, limits)
/etc/profile.d/x.sh — a note everyone reads at the door
(export VAR=…). Test as they would: bash -lc 'echo $VAR'.
/etc/security/limits.d/90-x.conf — deploy soft nofile 8192 and hard.
Residents listed at the town hall (LDAP)
ldapsearch -x -H ldap://127.0.0.1 -b dc=nodomain '(uid=ldapuser)' # is the town hall listing them?
# /etc/nslcd.conf: uri + base → systemctl enable --now nslcd
# /etc/nsswitch.conf: passwd: files systemd ldap (files FIRST)
getent passwd ldapuser; id ldapuser # does the HOUSE recognise them? — this is the real test
Kindergarten rule: the town hall knowing someone (ldapsearch) is not
the same as the house letting them in (getent). Both must be true.
Kindergarten rules for the whole exam
- Exactly the names, numbers, and paths in the chore — no improving.
- "After reboot" means written on a wall: fstab,
enable, sysctl.d, netplan, nftables.conf, limits.d. - Check it the inspector's way:
getent,findmnt,is-enabled,nft list ruleset,getfacl,sshd -T,curl. - Don't lock yourself out — the gate, the intercom, and the door you came in by.
- Ubuntu or Rocky?
apt/AppArmor/netplan/ufwvsdnf/SELinux/nmcli/firewalld. Check before you type.