JOSHUA WONG
Menu

Database Engineering

Learn how to build a database engine, from the first row to recovery and concurrency.

Build toward a small database engine by following one shop’s products and orders. Begin with what a row means, trace a lookup in memory, then learn why files, pages, indexes, transactions, and recovery need their own machinery.

Start with the foundations

The opening lessons assume that you can follow a short program. They introduce SQL and systems terminology as it becomes useful. Examples include diagrams, a prediction question with a worked answer, and small code traces you can read without running anything.

The complete path now uses focused explanations: follow a concrete problem, trace its mechanism through actual values, and predict what changes in a second example. Foundations explain rows, bytes, pages, sorted search, and consistent reads before later lessons reuse them. Each rewritten chapter links to extended references for the original technical depth; read those when you want the additional mechanisms and qualifications.

Where the path leads

The first nine stages work toward a single-machine engine with clear behavior and evidence for its reliability. The final stage studies engines across machines. Reading alone does not establish production readiness; later implementation work must test failures, recovery, concurrency, and resource limits.

Use the learning path and source guide for the prerequisite map, university references, and implementation milestones. Roadmap.sh helps check topic coverage; original explanations connect the concepts here.

Reference material

Before you start

  • Follow a short program with variables, loops, and functions
  • No SQL or database-internals knowledge required for the opening lessons
Start learning →

Curriculum

10 modules / 59 lessons

  1. What a database promises

    1. What are we trying to build?
    2. Give each product a row and an identity
    3. Ask for a row, then change it
    4. Connect an order to its product
    5. Make a purchase succeed or fail together

  2. A tiny engine in memory

    1. Build the first lookup in memory

  3. Rows become bytes

    1. Turn a record into bytes
    2. Save a row, close the program, read it back
    3. Move a record without losing its address

  4. Data exceeds memory

    1. Bring a piece of the file into memory
    2. Follow a page read through the computer
    3. Keep useful pages through a large scan

  5. Finding rows efficiently

    1. Why not just keep every key in one sorted list?
    2. Finding one row without scanning the table
    3. Write new values now, merge them later
    4. Find products containing two words

  6. Executing queries

    1. Decide which row owns a fact
    2. Turn a SQL request into a meaningful plan
    3. Join two tables without comparing every pair
    4. Pass rows from one operation to the next
    5. Move a filter only when the answer stays the same
    6. Choose a plan using an estimate, then check it
    7. Decide where the purchase logic runs

  7. Surviving interruption

    1. Record the decision before changing the saved rows
    2. Restart recovery without applying a change twice

  8. Overlapping transactions

    1. Let two operations overlap without confusing their steps
    2. One ticket, two buyers
    3. Two transfers wait for each other
    4. Two correct reads can make an incorrect report
    5. What does a reader see while a value changes?
    6. Try an update, then check whether its input changed
    7. Keep a page alive while a reader uses it

  9. Making the engine operable

    1. Separate time working from time waiting
    2. A queue cannot absorb an endless surplus
    3. Find the resource that fills up first
    4. Commit the order and remember to notify shipping
    5. Recover yesterday’s data after today’s mistake
    6. A valid user can still request the wrong row
    7. Change a stored field while old code still runs
    8. Find out where a slow commit waits
    9. Choose who owns the database process
    10. A purchase and a yearly report need different work

  10. Engines across machines

    1. Send a request across a stream of bytes
    2. Retry a purchase without buying twice
    3. A silent server may still be running
    4. Which value may a later reader see?
    5. What reached the second copy before success?
    6. Keep one agreed history when a leader fails
    7. Find the owner of a key
    8. Agree whether both halves of a purchase commit
    9. Send partial totals instead of every row
    10. An old cache fill can arrive after an update
    11. Do not lose the update between loading and listening
    12. A new worker starts with an empty cache
    13. Decide which region may sell the last item
    14. A storage library does not choose a cluster leader
    15. What does a missing in-memory key mean?
    16. Follow a transaction through three system designs
    17. Trace one purchase through the engine you can build