Tyler Bugbee

Operating Systems Final Review

Wed, 06 Dec 2017 22:30:00 +0000

Chapters 8 - Onwards

Memory Management (Chapter 8 and 9)

Memory Allocation -

Internal and External Fragmentation

Refers to available memory rather than the size of files

Internal fragmentation: allocated memory may be slightly larger than the requested memory and not being used
- For example, consider an OS that can only allocate blocks of size 5KB. If a process requires 29KB of memory, the smallest allocation the OS can make that will satisfy this need is 30KB
- This means there is 1KB of internal fragmentation after the memory is allocated. “Room for Growth”
External fragmentation: total memory space exists to satisfy the request, but it is not contiguous
- Consider contiguous blocks of 50KB, 30KB, 10KB, and 100KB. The 50KB blocks and 100KB blocks are free space
- A process requires 125KB of free space to run. There is enough space between the two free blocks, but they are not immediately contiguous
- External fragmentation can be reduced by using compaction:
  - Shuffle memory contents to place all free memory together in one large block
  - Compaction is only possible if relocation is dynamic and is done at execution time

Segmentation and Paging

Segmentation:
Memory-management scheme which organizes a program into a collection of segments, which are contiguous regions of virtual memory

Each process has a segment table in hardware. Each entry is a segment
A segment can be located anywhere in physical memory
- Each segment has a(n): start address, length, and access permission (R/W/RW)
Processes can share segments
- Same start address, length, same/different access permissions
A logical address consists of a two-tuple:

<segment-number, offset>

Segment table - maps two-dimensional physical addresses; each table entry has:
- base - contains the starting physical address where the segments reside in memory
- limit - specifies the length of the segment
Because segments vary in length, memory allocation is a dynamic storage-allocation problem
For example, consider the following table:

Base	Bound	Access Permissions
1000	200	R
4000	1521	RW
9000	400	RW

If we have the following requests from a program:

Read (seg 0, offset 150) = legal read of segment 0 at address 1150 (Base = 1000 + offset (150))
- Offset is less than bound (200), so this is within the segment
Write (seg 1, offset 51) = legal read of segment 1 at address 4051 (Base = 4000 + offset (51))
Write (seg 2, offset 20) = legal write of segment 2 at address 9020 (Base = 9000 + offset (20))
Read (seg 2, offset 501) = illegal read of segment 2 at address 9501 (Base = 9000 + offset 501)
- Results in a segmentation fault, offset is more than bound
Write (seg 0, offset 10) = illegal write of segment 0 at address 1010 (Base = 1000 + offset 10)
- Results in an illegal write exception. Address is within the segment, but the permissions are only read for this segment Paging:
- Divide physical memory into fixed-sized blocks called frames
  - Avoids external fragmentation
  - Avoids problem of varying sized memory chunks
- Divides logical memory into blocks of the same size called Pages
- To run a program of size N pages, we need to find N free frames and load the program
- We set up a page table to translate logical to physical addresses
- For additional virtual memory notes, see notes on virtual memory

Kernel Memory Allocation: Buddy system, Slab

Buddy System

Allocates memory from fixed-size segments consisting of physically-contiguous pages
Memory allocated using power-of-2 allocator
- Satisfies requests in units sized as power of 2
- Request rounded up to the next highest power of 2
- When smaller allocation is needed than is available, the current chunk splits into buddies of next-lower power of 2
  - Continue this process until appropriately sized chunk is available
For example, assume 256KB chunk available, kernel requests 21KB
- Split into A1 and A2 of 128KB each
  - One further divided into B1 and B2 of 64KB
    - One further into C1 and C2 of 32 KB each, one is used to satisfy the request
Advantage - quickly coalesce unused chunks into a larger chunk
Disadvantage - fragmentation Slab Allocator
A slab is one or more physically contiguous pages
Cache consists of one or more slabs
Single cache for each unique kernel data structure
- Each cache filled with objects - instantiations of the data structure
When cache is created, it is filled with objects marked as free
When structures are stored, objects are marked as used
If the slab is full of used objects, the next object is allocated from the empty slab
- If there are no empty slabs, a new slab is allocated
Benefits include no fragmentation, fast memory request satisfaction

Storage and File System (Ch. 10, 11, 12)

RAID System: Parity and Hamming

Operating Systems Midterm Review

Wed, 06 Dec 2017 21:00:00 +0000

Review of Chapters 2 - 7

Midterm Review

Small Sat Lab - Day in the Life Testing Overview

Thu, 16 Nov 2017 21:30:00 +0000

Section 1: Testing

prove that we’re converging towards final goal using flatsat
“maturing the design” - ground segment
- why do we think we’re capable of doing this?
showing ambition
begin from a narrative - go to
states - show how current state infers next state
- ex: rPi represents obc, tx1 represents tx2,
introduction: thought behind what we need to do: short, reference some documents, the fact that we want to show evidence

Notes on Virtual Memory

Sat, 11 Nov 2017 12:00:00 +0000

Virtual memory notes

Paging

Frames are fixed-sized blocks of physical memory
Pages are fixed-sized blocks of logical memory

Process

When a process is to be executed, its pages are loaded into any available memory frames from their source (a file system or the backing store).
The logical address space is now totally separate from the physical address space, so a process can have a logical 64-bit address space even though the system has less than 2^64 bytes of physical memory

Hardware Support

Every address generated by the CPU is divided into two parts: a page number (p) and a page offset (d). The page number is used as an index into a page table. The page table contains the base address of each page in physical memory. This base address is combined with the page offset to define the physical memory address that is sent to the memory unit.
The page size (like the frame size) is defined by the hardware. The size of a page is a power of 2, varying between 512 bytes and 1 GB per page. If the size of the logical address space is 2^m, and a page size is 2^n bytes, then the high-order m-n bits of a logical address designate the page number and the n low-order bits designate the page offset.

Every logical address is bound by the paging hardware to some physical address. Using paging is similar to using a table of base registers, one for each frame of memory
When a process arrives in the system to be executed, its size, expressed in pages, is examined. Each page of the process needs at least one frame
- thus, if the process requires n pages, at least n frames must be available in memory. If n frames are available, they are allocated to this arriving process
- The first page of the process is loaded into one of the allocated frames, and the frame number is put in the page table for this process. The next page is loaded into another frame, its frame number is put onto the page table, and so on

frame table

data structure containing one entry for each physical page frame with the following information:

which frames are allocated of physical memory
- and to which page of which process
which frames are available
how many total frames there are
etc

translation look-aside buffer (TLB)

associative, high-speed memory. Each entry in the TLB consists of two parts:

a key (or tag)
- when the associative memory is presented with an item, the item is compared with all keys simultaneously
a value
- If the item is found, the corresponding value field is returned
The TLB only contains a few of the page-table entries. When a logical address id generated by the CPU, its page number is presented to the TLB
- If the page number is found, its frame number is immediately available and is used to access memory
- If the page number is not in the TLB (known as a TLB miss), a memory reference to the page table must be made
  - When the frame number is obtained, we can use it to access memory
  - In addition, we add the page number and frame number to the TLB, so that they will be found quickly on the next reference
- If the TLB is already full of entries, an existing entry must be selected for replacement.

Protection (valid/invalid bits)

these bits are kept in the page table - one can define a page to be read-write or read-only

Every reference to memory goes through the page table to find the correct frame number
Protection bits can be checked to ensure that no writes are being made to a read-only page
one such bit is the valid-invalid bit
- when this bit is set to valid, the associated page is in the process’s logical address space and is thus a legal (or valid) page
- when the bit is set to invalid, the page is not in the process’s logical address space
- the OS sets this bit for each page to allow or disallow access to the page

Replacement

If you overallocate memory, the OS must use a page replacement algorithm (process is requesting more than the available memory)

if no frame is free, we find one that is not currently being used and free it
we do this by changing the page table (and all other tables) to indicate that the page is no longer in memory
we can now use the freed frame to hold the page for which the process faulted

Find the location of the desired page on the disk
Find a free frame:
a. if there is a free frame, use it b. if there is no free frame, use a page-replacement algorithm to select a victim frame c. write the victim frame to the disk; change the page and frame tables accordingly
read the desired page into the newly freed frame; change the page and frame tables
continue the user process from where the page fault occurred

This process slows down when there are no free frames and two page transfers are required: so we can reduce this overhead by using a modify bit (or dirty bit)

each page or frame has a modify bit associated with it in the hardware
the modify bit for a page is set by the hardware whenever any byte in the page is written into, indicating that the page has been modified
when we select a page for replacement, we examine its modify bit
if the bit is set, we know that the page has been modified since it was read in from the disk
- in this case, we must write the page to the disk
if the bit is not set, the page has not been modified since it was read into memory.
- in this case, we need not write the memory page to the disk: it is already there

FIFO

associates the time when that page was brought into memory with each page
when a page must be replaced, the oldest page is chosen
implement: replace the page at the head of the queue each time. When a page is brought into memory, we insert it at the tail of the queue

LRU

last recently used algorithm - replace the page that has not been used for the longest period of time
associates with each page the time of that page’s last use
when a page must be replaced, LRU chooses the page that has not been used for the longest period of time
implement:
- associate with each page-table entry a time-of-use counter and increment for every memory reference. We replace the page with the smallest time value
- keep a stack of page numbers. Whenever a page is referenced, it is removed from the stack and put on the top. In this way, the most recently used page is always at the top of the stack and the least recently used page is always at the bottom -> modify 6 pointers at the worst case scenario

Second-chance (clock)

Basic algorithm is a FIFO replacement algorithm. When a page has been selected, however, we inspect its reference bit

if the bit is 0, we proceed to replace this page
if the bit is 1, we give the page a second chance and move on to select the next FIFO page
- when a page gets a second chance, its reference bit is cleared, and its arrival time is reset to the current time
- thus, a page that is given a second chance will not be replaced until all other pages have been replaced (or given second chances)
implement: a pointer indicates which page is to be replaced next. When a frame is needed, the pointer advances until it finds a page with a 0 reference bit. As it advances, it clears the reference bits
- once a victim page is found, the page is replaced, and the new page is inserted in the circular queue in that position
- if all bits are set, second-chance replacement degenerates to FIFO replacement

Page fault handler

When a process tries to access a page that was not brought into memory - access to a page marked invalid causes a page fault. The paging hardware will notice that the invalid bit is set, causing a trap in the OS. The procedure for handling this page fault is as follows:

We check an internal table for this process to determine whether the reference was a valid or an invalid memory access
If the reference was invalid, we terminate the process. If it was valid but we have not yet brought in the page, we now page it in
We find a free frame (by taking one from the free-frame list, for example)
We schedule a disk operation to read the desired page into the newly allocated frame
When the disk read is complete, we modify the internal table kept with the process and the page table to indicate that the page is now in memory

Welcome to my Blog!

Wed, 27 Jul 2016 07:33:00 +0000

Hello, and welcome to my first post. My name is Tyler Bugbee and I’m a third-year Computer Science student at the University of Georgia. I hope to use this website as an outlet for tech-talk, information on what I’m working on in school and otherwise, and other curiosities as they relate to computer science.

My current interests are machine learning, predictive analytics, large-scale data problems, and parallel and distributed computing.

You can find me at GitHub, LinkedIn, and Gmail. Feel free to contact me directly!