Overview¶
Welcome to ssd.py, yet another wonderful simulator provided to you,
for free, by the authors of OSTEP, which is also free. Pretty soon,
you’re going to think that everything important in life is free! And,
it turns out, it kind of is: the air you breathe, the love you give
and receive, and a book about operating systems. What else do you
need?
To run the simulator, you just do the usual:
prompt> ./ssd.py
The simulator models a few different types of SSDs. The first is what we’ll call an “ideal” SSD, which actually isn’t much an SSD at all; it’s more like a perfect memory. To simulate this SSD, type:
prompt> ./ssd.py -T ideal
To see how this one works, let’s create a little workload. A workload, for an SSD, is just a series of low-level I/O operations issued to the device. There are three operations supported by ssd.py: read (which takes an address to read, and returns the data), write (which takes an address and a piece of data to write, in this case, a single letter), and trim (which takes an address). The trim operation is used to indicate a previously written block is no longer live (i.e., the file it was in was deleted); this is particular useful for a log-based SSD, which can reclaim the block’s space during garbage collection and free up space in the FTL. Let’s run a simple workload consisting of just one write:
prompt> ./ssd.py -T ideal -L w10:a -l 30 -B 3 -p 10
The -L flag allows us to specify a comma-separated list of commands. Here, to
write to logical page 10, we include the command “w10:a” which means “write”
to logical page “10” the data of “a”. We also include a few other specifics
about the size of the SSD with the flags -l 30 -B 3 -p 10, but let’s
ignore those for now.
What you should see on the screen, after running the above:
FTL (empty)
Block 0 1 2
Page 0000000000 1111111111 2222222222
0123456789 0123456789 0123456789
State iiiiiiiiii iiiiiiiiii iiiiiiiiii
Data
Live
FTL 10: 10
Block 0 1 2
Page 0000000000 1111111111 2222222222
0123456789 0123456789 0123456789
State viiiiiiiii iiiiiiiiii iiiiiiiiii
Data a
Live +
The first chunk of information shows the initial state of the SSD, and the second chunk shows its final state. Let’s walk through each piece to make sure you understand what they mean.
The first line of each chunk of output shows the contents of the FTL. This simulator only models a simple page-mapped FTL; thus, each entry within it shows the logical-to-physical page mapping for any live data items.
In the initial state, the FTL is empty:
FTL (empty)
However, in the final state, you can see that the FTL maps logical page 10 to physical page 10:
FTL 10: 10
The reason for this simple mapping is that we are running the “ideal” SSD, which really just acts like a memory; if you write to logical page X, this SSD will just (magically) write the data to physical page X (indeed, you don’t even really need the FTL for this; we’ll just use the ideal SSD to show how much extra work a real SSD does, in terms of erases and data copying, as compared to an ideal memory).
The next lines of output just label the blocks and physical pages of the underlying Flash the simulator is modeling:
Block 0 1 2
Page 0000000000 1111111111 2222222222
0123456789 0123456789 0123456789
In this simulation, you can see that the SSD has 3 physical Flash blocks, and that each block has 10 physical pages. Each block is numbered (0, 1, and 2), as is each page (from 00 to 29); to keep the display compact (width-wise), the page numbering is shown across two lines. Thus, physical page “10” is labeled with a “1” on the first line, and a “0” on the second.
The next line shows the state of each page, i.e., whether it is INVALID (i), ERASED (E), or VALID (v), as per the chapter:
State viiiiiiiii iiiiiiiiii iiiiiiiiii
The states for the “ideal” SSD are a bit weird, in that you can have “v” and “i” mixed in a block, and that the block is never “E” for erased. Below, with the more realistic “direct” and “log” SSDs, you’ll see “E” too.
The final two lines show the “contents” of any written-to pages (on the “Data” row) and whether that data is currently live (that is, referred to in the FTL) in the “Live” row:
Block 0 1 2
Page 0000000000 1111111111 2222222222
0123456789 0123456789 0123456789
Data a
Live +
Here, we can see that on Block 2 (i.e., Page 10), there is the data “a”, and it is indeed live (shown by the “+” symbol).
Let’s expand our workload a little bit, before getting to the more realistic types of SSDs. After writing the data, let’s read it, and then let’s use trim to delete it:
prompt> ./ssd.py -T ideal -L w10:a,r10,t10 -l 30 -B 3 -p 10
If you run this, you’ll see two identical states: the initial (empty) state, and the final (also empty!) state. Not too exciting! To see more of what is going on, you’ll have to use some more flags. Yes, this SSD simulator uses a lot of flags; sorry, all lovers of parsimony! But alas, there is some complexity here we must explore.
One useful flag is -C, which just shows every command that was issued, and
whether is succeeded or not.
prompt> ./ssd.py -T ideal -L w10:a,r10,t10 -l 30 -B 3 -p 10 -C
FTL (empty)
Block 0 1 2
Page 0000000000 1111111111 2222222222
0123456789 0123456789 0123456789
State iiiiiiiiii iiiiiiiiii iiiiiiiiii
Data
Live
cmd 0:: write(10, a) -> success
cmd 1:: read(10) -> a
cmd 2:: trim(10) -> success
FTL (empty)
Block 0 1 2
Page 0000000000 1111111111 2222222222
0123456789 0123456789 0123456789
State iiiiiiiiii viiiiiiiii iiiiiiiiii
Data a
Live
prompt>
Here, you can see the write, read, and trim, and you can also see what each command returned: success, the data read, and success, respectively. This will be more interesting later, when the simulator generates the operations randomly.
Similarly, the -F flag shows the state of the Flash between each operation,
instead of just at the end. Note the subtle changes at each step:
prompt> ./ssd.py -T ideal -L w10:a,r10,t10 -l 30 -B 3 -p 10 -F
FTL (empty)
Block 0 1 2
Page 0000000000 1111111111 2222222222
0123456789 0123456789 0123456789
State iiiiiiiiii iiiiiiiiii iiiiiiiiii
Data
Live
FTL 10: 10
Block 0 1 2
Page 0000000000 1111111111 2222222222
0123456789 0123456789 0123456789
State iiiiiiiiii viiiiiiiii iiiiiiiiii
Data a
Live +
FTL 10: 10
Block 0 1 2
Page 0000000000 1111111111 2222222222
0123456789 0123456789 0123456789
State iiiiiiiiii viiiiiiiii iiiiiiiiii
Data a
Live +
FTL (empty)
Block 0 1 2
Page 0000000000 1111111111 2222222222
0123456789 0123456789 0123456789
State iiiiiiiiii viiiiiiiii iiiiiiiiii
Data a
Live
prompt>
Of course, you can use -C and -F in concert to show everything (an exercise
left to the reader).
The simulator also lets you generate random workloads, instead of specifying operations yourself. Use the “-n” flag for this, with an associated number (we also specify a random seed with “-s” to get a particular workload):
prompt> ./ssd.py -T ideal -l 30 -B 3 -p 10 -n 5 -s 10
If you run this with -C, -F, or both, you’ll see either the exact commands,
the intermediate states of the Flash, or both. However, you can also use the
“-q” flag to quiz yourself on what you think the commands are. Thus, run the
following:
prompt> ./ssd.py -T ideal -l 30 -B 3 -p 10 -n 5 -s 10 -q
(output omitted for brevity)
Now, by examining the intermediate states, see if you can discern what the commands must have been (writes and trims are left completely unspecified, whereas reads just ask you to figure out which data was returned).
You can then either manually use -C -F to show everything, or just add the
-c flag to “solve” the problem for you, to check your answers.
Let’s now do the same thing (a random workload of five operations) but use different more realistic SSDs. The first is the “direct” SSD mentioned in the chapter. This too isn’t particularly realistic, but at least uses erases and programs to update the Flash. Specifically, when a logical page is written, it is mapped directly to the physical page of the same number. This mapping necessitates first a read of all the live data in that block, then an erase of the block, and then a series of programs to restore all previously live data as well as write the new data to Flash. Let’s run it, show the commands (-C) but not the intermediate states (no -F):
prompt> ./ssd.py -T direct -l 30 -B 3 -p 10 -n 5 -s 10 -C
FTL (empty)
Block 0 1 2
Page 0000000000 1111111111 2222222222
0123456789 0123456789 0123456789
State iiiiiiiiii iiiiiiiiii iiiiiiiiii
Data
Live
cmd 0:: write(12, z) -> success
cmd 1:: write(19, 9) -> success
cmd 2:: write(9, f) -> success
cmd 3:: trim(9) -> success
cmd 4:: read(19) -> 9
FTL 12: 12 19: 19
Block 0 1 2
Page 0000000000 1111111111 2222222222
0123456789 0123456789 0123456789
State EEEEEEEEEv EEvEEEEEEv iiiiiiiiii
Data f z 9
Live + +
prompt>
As you can see from the final state, the FTL contains two live mappings: logical page 12 refers to physical page 12, and 19 to 19 (remember, this is the direct mapping). You can also see three data pages with information within them: physical page 9 contains “f”, 12 contains “z”, and 19 contains “9” (data can be letters or numbers or really any single character). However, also note that “9” has been trimmed; this removes its entry from the FTL, but the data lies their dormant (for now). If you then tried to read logical page 9, it no longer would succeed:
prompt> ./ssd.py -T direct -l 30 -B 3 -p 10 -C -L w9:f,t9,r9
FTL (empty)
Block 0 1 2
Page 0000000000 1111111111 2222222222
0123456789 0123456789 0123456789
State iiiiiiiiii iiiiiiiiii iiiiiiiiii
Data
Live
cmd 0:: write(9, f) -> success
cmd 1:: trim(9) -> success
cmd 2:: read(9) -> fail: uninitialized read
FTL (empty)
Block 0 1 2
Page 0000000000 1111111111 2222222222
0123456789 0123456789 0123456789
State EEEEEEEEEv iiiiiiiiii iiiiiiiiii
Data f
Live
prompt>
One last SSD we should pay attention to is the actual most realistic one, which uses log-structuring (as do most real SSDs). To use it, just change the SSD type to “log” (we’ll again turn on -C so we can just know which operations took place, instead of quizzing ourselves):
prompt> ./ssd.py -T log -l 30 -B 3 -p 10 -s 10 -n 5 -C
FTL (empty)
Block 0 1 2
Page 0000000000 1111111111 2222222222
0123456789 0123456789 0123456789
State iiiiiiiiii iiiiiiiiii iiiiiiiiii
Data
Live
cmd 0:: write(12, z) -> success
cmd 1:: write(19, 9) -> success
cmd 2:: write(9, f) -> success
cmd 3:: trim(9) -> success
cmd 4:: read(19) -> 9
FTL 12: 0 19: 1
Block 0 1 2
Page 0000000000 1111111111 2222222222
0123456789 0123456789 0123456789
State vvvEEEEEEE iiiiiiiiii iiiiiiiiii
Data z9f
Live ++
prompt>
Note how the log-structured SSD writes data to the Flash. First, the current log block (Block 0, in this case) is erased. Then, the pages are programmed in order. Use the -F flag to see each step for more detail.
The simulator can also show more statistics, including operation counts and the estimated time that the modeled SSD would take to complete the given workload. To see these, use the -S flag:
prompt> ./ssd.py -T log -l 30 -B 3 -p 10 -s 10 -n 5 -S
(stuff omitted)
Physical Operations Per Block
Erases 1 0 0 Sum: 1
Writes 3 0 0 Sum: 3
Reads 1 0 0 Sum: 1
Logical Operation Sums
Write count 3 (0 failed)
Read count 1 (0 failed)
Trim count 1 (0 failed)
Times
Erase time 1000.00
Write time 120.00
Read time 10.00
Total time 1130.00
Here you can see the physical erases, writes, and reads per block as well as a sum of each, then the number of logical writes, reads, and trims issued to the device, and finally the estimated times. You can change the costs of low-level operations such as read, program, and erase, with the -R, -W, and -E flags, respectively.
Finally, with the SSD in log-structured mode, there is a garbage collector (GC) that can be configured to run periodically. This behavior is controlled by the -G and -g flags, which set the high and low watermarks for determining whether the garbage collector should run. Setting the high watermark to a value N (i.e., -G N) means that when the GC notices that N blocks are in use, it should run. Setting the low watermark to M (i.e., -G M) means that the GC should run until only M blocks are in use.
The -J flag is also useful here: it shows which low-level commands the GC issues (reads and writes of live data, followed by erases of reclaimed blocks). The following issues 60 operations, and sets the high and low watermarks to 3 and 2, respectively.
prompt> ./ssd.py -T log -l 30 -B 3 -p 10 -s 10 -n 60 -G 3 -g 2 -C -F -J
Using -C, -F, and -J lets you really see what is happening, step
by step, inside the log-structured simulation.
There are a few other flags worth knowing. The entire time, we’ve been using the following three flags to control the size of the simulated SSD:
-l NUM_LOGICAL_PAGES, --logical_pages=NUM_LOGICAL_PAGES number of logical pages in interface
-B NUM_BLOCKS, --num_blocks=NUM_BLOCKS number of physical blocks in SSD
-p PAGES_PER_BLOCK, --pages_per_block=PAGES_PER_BLOCK pages per physical block
You can change these values to simulate larger or smaller SSDs than the simple one we’ve been simulating so far.
One other set of controls lets you control randomly generated
workloads a bit more precisely. The -P flag lets you control how
many reads/writes/trims show up (probabilistically). For example,
using -P 30/35/35 means that roughly 30% of operations will be
reads, 35% writes, and 35% trims.
The -r flag allows reads to be issued to non-live addresses (the default only
issues reads to live data). Thus, -r 10 means roughly 10% of reads will
fail.
Finally, the -K and -k flags let you add some “skew” to a workload. A skew is
first specified by -K, e.g., -K 80/20 makes 80% of writes target 20% of
the logical space (a hot/cold kind of workload). Skew is common in real
workloads, and has different effects on garbage collection, etc., so it is
good to be able to model. The related -k flag lets you specify when the skew
starts; specifically, -k 50 means that after 50 writes, start doing the skew
(before then, the writes will be chosen at random from all possible pages in
the logical space).
Wow, have you gotten this far? You are some impressive person! We suspect you will go far in life. Or, we suspect that you typed “cat README” and not “more README” or “less README”, in which case we suspect you are just learning about “more” or “less”, more or less.