Concept

Block size — where it appears

How many elements a single transfer between memory and storage moves. It is the parameter the external-memory model is built around, and it decides where methods cross over without appearing in any statement of the problem.

Named by 3 essays across one field — each of them below, with the objects they name alongside it.

50%60%70%80%90%100%0%1%2%5%10%25%50%share of keys arriving latemean leaf filleven splitsrightmost-split rulesibling first, two into threeln 2131,072 keys, leaves of 64late keys arrive at a random later point

The sibling a full leaf asks first

The rule databases use to fix ascending inserts fills their leaves completely and collapses to 53.4% when one key in a hundred arrives late. A leaf that offers a key to a sibling before it splits, and splits two full leaves into three when neither will take one, holds 84.2% on the same stream — and is better with a trickle of late keys than without one, because a perfectly ascending stream has no sibling with room.

applied · Transfer
234681016110size ratioblock transfers · levelsa range of 100 keyslevelsan absent point lookup1,048,576 keys, 5 bits a keyfilters answer one of these two

The read a filter has no key for

A Bloom filter on every run of a log-structured store turns a lookup for a missing key from a read of every level into a fraction of one — 0.72 transfers across eight levels at five bits a key, and 0.00027 at twenty. A range query over the same store reads nine transfers at five bits and nine at twenty, because a filter answers whether one named key is in a run and a range has no key to name.

applied · Transfer
0%25%50%75%100%8101214161820222426283032bits of key prefix the filter is built onshare of the empty levels the filter skips83%72%64%46%ranges of 0.1 keysranges of 1 keyranges of 10 keysranges of 100 keys5 bits a key, 4 levels, 2,000 ranges eachdigits: the best length's share

The runs a range query can skip

A Bloom filter over key prefixes can tell a range query that a run holds nothing it wants, which a filter over keys cannot. On a four-level store of a million keys, a range expected to hold one key costs 4.76 transfers with no such filter and 2.41 with one built on 20-bit prefixes. The best length moves two bits for every tenfold longer range. By a hundred keys the most any range filter could save is a fifth of a transfer, because only the smallest level is ever empty. And the same memory spent on 20-bit prefixes makes a lookup for an absent key three times dearer.

applied · Transfer

Named alongside it

The objects these essays reach for when they reach for this one.

Design parameterSpace time tradeBloom filterLsm treeRange queryRead amplificationSorted runWorkloadAmortisationB-treeBlock transferBulk loading

All concepts