Reading order
What to read first, and what it unlocks
These 26 concepts assume nothing — any of them is a place to start. Pick a system for the full path through it.
- 1
Start here — nothing else assumed
26 concepts- The Delta Transaction Log
That _delta_log is an audit journal beside the real table, so a Parquet file dropped into the directory becomes queryable and a deleted one disappears. The log is the table: a file with no add action is invisible no matter how much data it holds, and a removed file sits on storage until VACUUM. The second half of the misconception is that checkpoints make reading the log free — they only shorten the tail of JSON commits, while the checkpoint itself carries one row per live file, so a table with millions of small files pays seconds of metadata time before any data is read.
- Entity Resolution in a Knowledge Graph
That the pipeline deduplicates entities, because the step after the merge summarises a list of descriptions into one. It does not compare anything: membership was already decided by exact string equality. The fragments split an entity's edges, which breaks multi-hop paths and pushes whole subgraphs outside the largest connected component, where cluster_graph discards them and no community report ever mentions them.
- Kafka Partitions
That the partition count is a throughput dial you can turn up when you need more consumers and back down when you do not. It is neither reversible nor free. Kafka refuses to reduce it — the controller answers "would not be an increase" — so every expansion is permanent. And because the producer picks a partition with murmur2(key) & 0x7fffffff % numPartitions, changing the divisor remaps most keys: a key's old records stay in the partition they were written to while its new records go somewhere else, and the two are consumed independently. The per-key ordering guarantee people rely on is not weakened by adding a partition, it is broken for every key that moved.
- cgroup v2: The Limit That Kills You Is Four Levels Above Your Container
That migrating a node to cgroup v2 renames files and changes nothing — memory.limit_in_bytes becomes memory.max, cpu.shares becomes cpu.weight, same behaviour. Two structural changes bite. First, the single hierarchy means your container's memory limit is only the innermost of three or four limits it is charged against, and when the failing one is kubepods.slice — which the kubelet limits by default — the OOM domain becomes every pod on the node, the badness denominator becomes node allocatable rather than your limit, and oom_score_adj stops cancelling out and starts outweighing resident memory by three orders of magnitude, so the pod that dies is the Burstable pod with the smallest memory request rather than the pod that allocated. Second, since Kubernetes 1.28 the kubelet sets memory.oom.group=1 on container cgroups, so the kernel kills every process in the victim's container instead of the single fattest one.
- How kube-scheduler Places a Pod
That a pod goes Pending because the cluster is out of capacity, so the fix is a bigger or extra node. The scheduler never reads utilisation: it compares the pod's requests against allocatable minus the sum of every already-placed pod's requests, so a node running at 12% CPU can be 100% requested and refuse everything. Worse, because placement is greedy and per-pod with no backtracking, the default LeastAllocated scoring spreads small pods evenly and leaves every node with a hole too small for the next big one — the cluster has the CPU free, just never in one place.
- maxUnavailable Counts Available Pods, Not Working Ones
Not written yet — 3 published lessons already depend on it.
- K8s Cpu Manager Static
Not written yet — 2 published lessons already depend on it.
- The eBPF Verifier Rejects Correct Programs
Not written yet — 1 published lesson already depends on it.
- Arithmetic Intensity and the Roofline
That memory-bound and compute-bound are labels attached to operations, so attention is 'the memory-bound one' and matmuls are 'the compute-bound ones'. Intensity is a property of a specific execution, not of an operation: the same weight matmul runs at 1 FLOP per byte with one token in the batch and 700 with a thousand, which is the entire reason batching works. Two consequences people miss. Batching moves the weight matmuls along the roofline but cannot move attention over the KV cache at all, because each sequence's cache is read by exactly its own query, so a big batch at long context is still memory-bound and the throughput gain stops arriving. And the ridge point has been rising with each GPU generation - 153 on an A100, 295 on an H100 - so upgrading hardware makes more of your kernels memory-bound, not fewer.
- Raising max_position_embeddings Does Not Extend Context
That a model's context length is the value of max_position_embeddings, so raising it extends the window. RoPE has no per-position parameters to run out of; the limit is that roughly a quarter to a half of the dimension pairs never complete a single full rotation inside the training window, so at any longer position they are rotated to angles the model has never been asked to interpret. For Llama 2 7B that is 18 of the 64 pairs. The scaling methods are not conveniences — position interpolation removes every unseen angle but shrinks the separation between adjacent positions to 0.1% of what it was at a scale factor of 32, and YaRN exists because those two failures live in different halves of the frequency ladder.
- Llm Linear Attention
Not written yet — 4 published lessons already depend on it.
- Your bandwidth-delay product used the wrong bandwidth and the wrong delay
That the bandwidth-delay product is arithmetic you do once — the speed your link is sold at, times the number your ping prints — and that setting the socket buffer to the answer fills the pipe. Both inputs are usually wrong. The bandwidth is the narrowest hop on the path, not the link you pay for, and sizing to your own link rate on a path with a slower hop is worse than leaving the default alone. The delay has to be the unloaded minimum RTT, because throughput measured during a transfer multiplied by the RTT measured during the same transfer always returns the window you already had — so the rule certifies whatever queue you are already carrying instead of correcting it.
- Linux does not implement RFC 896, and TCP_NODELAY is not what fixed your latency
That Nagle's algorithm delays any small write while data is unacknowledged, that the resulting 40 ms is a flat tax on request-response traffic, and that TCP_NODELAY is the fix. Linux does not implement RFC 896: tcp_nagle_check applies Minshall's variant and blocks a partial segment only when the last sub-MSS segment sent is still unacknowledged, so a write of exactly two full segments followed by a small write never stalls, while a 4-byte header followed by anything always does. The 40 ms is not paid on the opening exchange either, because Linux quickacks the first rcv_wnd/(2*rcv_mss) segments up to a cap of 16 and only begins delaying once one reply-within-ato has pushed the socket into pingpong mode, which net.ipv4.tcp_pingpong_thresh sets at 1. And TCP_NODELAY removes only the sender's half of the interaction: the receiver still delays its acknowledgement by TCP_DELACK_MIN, which is a hard 40 ms floor that does not shrink with the round-trip time, so on a 0.2 ms path the delay is 200 round trips. The write pattern is the defect; one writev fixes both halves and puts fewer packets on the wire than TCP_NODELAY does.
- Tcp Ecn
Not written yet — 2 published lessons already depend on it.
- Tcp Cubic
Not written yet — 1 published lesson already depends on it.
- MVCC, Dead Tuples and What VACUUM Does Not Do
That VACUUM reclaims disk space and that running it more often fixes bloat. Plain VACUUM never shrinks the file except by truncating a wholly empty tail, and a single old transaction, replication slot or prepared transaction pins the removable cutoff so that no dead tuple newer than it can be removed at any frequency — the table bloats while every dashboard shows autovacuum succeeding.
- Partitioning Only Helps the Queries That Name the Key
Not written yet — 2 published lessons already depend on it.
- Cosine Similarity Is Not Relevance
That the top result by cosine similarity is the chunk most likely to contain the answer, and that a similarity score is a calibrated confidence you can threshold. Cosine measures paraphrase-style likeness in an anisotropic space with no absolute scale, and mean pooling gives the answering sentence a weight of exactly 1/n in its own chunk's vector. Teams raise top_k, tune a threshold, and swap encoders while the actual failure — the answer's chunk ranking sixth because nine sentences of boilerplate were averaged in with it — goes unmeasured, because retrieval recall was never logged.
- chunk_size Is Measured in the Wrong Units
That chunk_size controls how much text ends up in a chunk's vector. It controls how much text ends up in the chunk; whether the encoder reads all of it is a separate limit measured in a different unit. all-MiniLM-L6-v2 stops at 256 word pieces — 254 once [CLS] and [SEP] are counted — and its model card describes this in one sentence with no warning, no exception and no truncated flag. The lost text is still in your vector store, still returned verbatim once the chunk is retrieved, so it reads correctly in every debugging session; it simply had no influence on the vector that decides whether the chunk is ever retrieved. And the token-aware fix fails: setting chunk_size to 256 in LlamaIndex's cl100k tokens produces chunks of 278, 292 and 313 word pieces on three real documents, all of them over the 254 the encoder will read.
- "The Certificate Is Valid" Is Four Separate Checks
That certificate validation is one boolean the TLS library returns. It is four checks, they fail for different reasons, and the fourth — matching the hostname you asked for against the names in the certificate — is not part of the path validation algorithm in RFC 5280 and is not performed by every library unless the application supplies the expected name. A client that builds a perfect chain to a trusted root and never compares the hostname has proved that some real certificate authority vouched for some name, which is not the same as proving you are talking to the host you dialled.
- Lax by Default Is One Browser's Policy, Not the Web's
That leaving SameSite off is safe because browsers now default to Lax. Chrome does. Firefox ships network.cookie.sameSite.laxByDefault=false and Safari never implemented the default at all, so on those browsers an unlabelled session cookie is attached to a cross-site top-level POST exactly as it was before SameSite existed. Even in Chrome the default is the weaker Lax-allowing-unsafe mode, which sends an unlabelled cookie on a cross-site POST for the first 120 seconds of the cookie's life — and Lax of either kind always sends the cookie on a cross-site top-level GET.
- The Slowness Is the Feature
That salting a fast cryptographic hash such as SHA-256 makes it suitable for passwords, because the salt is what stops the attack. The salt stops precomputation and stops one computation covering many accounts; it does not make a single guess cost one cycle more. What makes guessing uneconomic is the work factor — the deliberate cost of each evaluation — and a general-purpose hash is fast by design, which is precisely the property you do not want here.
- The Timing Leak That Matters Skips Work, Not Bytes
That timing attacks are about comparison loops, so replacing == with a constant-time compare closes the channel. The difference a byte comparison makes is about 2.4 nanoseconds, which no filter recovers across a network at any practical sample count. The differences that are trivially readable are branches that skip work — an early return before the password hash is about 240 milliseconds, a hundred million times larger, and two requests find it. Teams add hmac.compare_digest and leave the account-enumeration oracle in place.
- Sec Credential Stuffing
Not written yet — 2 published lessons already depend on it.
- Spark Partitioning
That repartition(n) controls how work is distributed for the rest of the query, so raising n rebalances a skewed job. Round-robin repartitioning is discarded by the very next shuffle, which re-hashes rows by the operator's own key, and a key that maps to one partition id cannot be divided by any value of n.
- Spark Driver Memory
Not written yet — 2 published lessons already depend on it.
- The Delta Transaction Log