Make life easy for the hardware

Resources:

1. Mike Acton's talk about Data Oriented Design
2. Scott Meyer's talk about CPU Caches
3. Carl Cook at CPPCon

CPU access patterns aren't as simple as "get the address from the program counter, then look at the memory address for data". It turns out CPUs are a lot smarter than that. A CPU will have various mechanisms to make accessing data as fast as possible, including

1. various levels of caches before hitting main memory
2. access patterns akin to grabbing blobs of data (called a cache line) and stuffing it into cache
3. attempting to make predictions on your access patterns and grab data greedily to improve future access

A cache line is effectively a contiguous blob of memory that the CPU will grab when you access an address. An analogy might be - you're making apple pie. You want to find the 10 best apples at the grocery store. Rather than pick out one by one, you scoop an arm full into your art. Then you start looking at the apples in your cart. If they're all good, yay! If not, you throw away the ones you don't want and grab another armful of apples.

If you treat your cart as L1 cache and going back to the apple bin as accessing main memory, it turns out the analogy means that you take 200x as long (see Peter Norvig's numbers)

It turns out people who really care about performance - game programmers - figured this out a long time ago. Bob Nystrom's written a great chapter in Game Programming Patterns about cache locality.

So what can we do?

1. Be judicious with OOP patterns. Making everything an instance of a class and having relationships between them is great, but referencing those instances effectively is chasing a pointer which can nearly guarantee a cache miss
2. Use monomorphic arrays when it makes sense. If you know you're going to access all the elements of some list often (eg all the values of a map), just use an array. Make it cache friendly.

Big O isn't everything - benchmark!

1. Eric Lippert's blog post on a very neat assembly instruction
2. The chapter of Crafting Interpreters on writing a hash table

I expect this is particularly emphasized by the interview process for the FAANG companies, but Big O analysis is more or less second nature to me. But one thing that's worth noting is that sometimes your Big O assumptions miss some truths about realworld numbers.

The idea that a single assembly instruction can do work much faster than the "clever" code I'd write is still kind of wild. Then when thinking about hash tables - of course access is O(1)! But with a caveat.

In the case of a hash collision, your hash table implementation matters a LOT. Are your values a linked list of pointers referencing the next value? If so, you're now chasing pointers again. Which means cache misses. Which means ~200x as long to execute. Could you have done a simple linear scan on an array of data if it were stored in a cache line and held in L1 faster than you could resolve a hash collision? Well, maybe.

In practice, I'm not sure just HOW useful this is. But a key takeaway for me would be when measuring performance, don't assume your "optimal" algorithm is as fast as it could be

Higher Order Functions aren't all that fancy

1. Let's learn about monomorphism from this V8 engineer
2. Some thoughts from Ben Lesh

I think this is fallout from the ES6 excitement that led to a ton of "functional-lite" code in the JS ecosystem, but I'm starting to feel a little bit of burnout from it. Consider the following:

const arrayOfNums = [1, 2, 3, 4, 5];
arrayOfNums.map(x => x * 2)
	.filter(x => x % 3 == 0)
	.reduce((obj, num) => obj[num] = String(num), {})

Comfortably we can call this linear in time and space. But in practice:

1. we have a function allocation for each element in each callback
2. We create a ton of arrays in the middle that are thrown away
3. We end up creating a ton of "shapes" in our reduce unless the function gets inlined away by the compiler.

I fully get the code is arbitrary, but at the same time...does this really convey intent better? Maybe - map tells us there's a new array created potentially of a new type, filter tells us we'll get an array of the same type but fewer elements, and reduce tells us...well, not much. But maybe, particularly given that we're running this in a browser where a user might have a dozen or more tabs open (on a phone!) we should be more judicious with our use of higher order functions.

In particular, I've poked around the codebase for React and Rome and have noticed both Sebastian Markbåge (and team) and Sebastian McKenzie both seem to keep it simple. While loops, for...of iterators, basic data structures (queues, stacks, heaps) and some well-typed classes.

In @sebmarkbage's case in particular, despite his OCaml influence, he's not writing functional-lite JS. Which makes sense! No tail call optimization, different engine under the language runtime, he's playing to the strength of the language. That's something I can take away.

Coming into RC, I had a ton of goals. I wanted to write some C. I wanted to write some Rust. I wanted to write a small compiler from scratch. I wanted to contibute to a big open source compiler. I wanted to read a big text book. I wanted to write a ton of blog posts. Looking back, it's kinda silly just how much I thought I could do, or how much I even wanted to get done in 6 weeks.

When I reflect on what I HAVE done in the last 3 weeks, I'm actually very proud of it. However, it's pretty clear to me that what I value (reflected in what I do) is very different than what my goals were (reflected in what I said). My goals were diverse, unfocused, and really about greedily touching as much as I could to sate this need for intellectual stimulation. But it turns out, I didn't really want that.

To date I've

1. Written a ton of Rust (a new language for me) by following Crafting Interpreters
2. Learned more depth about parsing algorithms from Engineering a Compiler and some videos from Dmitry Soshnikov, an engineer / educator I respect a ton
3. Worked through some Nand2Tetris
4. Contributed to Rome, an open source Javascript toolchain

...and that's it. No distributed systems papers, no Game Boy emulator, just letting my interests and passion guide my focus. That's not to say I'll never switch focus - that Game Boy emulator might be fun in Zig! Maybe I'll spent my last two weeks working towards that.

Two quotes that I like, that might sound a bit too adversarial, are "No plan survives first contact with the enemy" or phrased better by Mike Tyson - "everybody has a plan until they get punched in the mouth". I had a plan coming into RC. I looked at the calendar, chatted with some folk, and fairly promptly threw that all away for serendipity. I just focused on the few things I've wanted to work on for a while...and just did it. For me, for my needs and where I am in my life, this is so much better.

To reflect on values:

Working in web dev for years, and doing a lot of front-end work on business software in particular didn't really allow me the intellectual stimulation or more me towards my goals of depth over breadth. RC gave me the space to both realize that this was even a goal of mine - that I preferred a sense of expertise over loosely touching many topics - and the time to just dig deep into topics. I'm grateful for the opportunity and the financial privilege, and I think the largest growth opportunity I've even had is this realization of my values.

These notes are supplemented by Dmitry Soshnikov's videos on parsing

Introduction

Compiler parsers serve as the second stage of the "front end", after scanning. A scanner takes a set of source text and effectively annotates them into tokens, or small structs with a classification. For example if (true) {print("hello")} might yield [{type: IF}, {type: LEFT_PAREN}, {type: TRUE}, {type: RIGHT_PAREN}, {type: LEFT_BRACE}, {type: IDENTIFIER, value: "print"}, {type: LEFT_PAREN}, {type: STRING, value: "hello"}, {type: RIGHT_PAREN}, {type: RIGHT_BRACE}, {type: EOF}]. A parser receives these input tokens as a stream and tries to determine syntactic structure based on some grammar, or rules of a valid language. Parsers typically use a context-free grammar, which Cooper/Torczon define as "a set of rules that describe how to form sentences."

Formally, a CFG is defined as the tuple (T, NT, S, P) where T is the set of terminal values (that is, as you follow the rules, you cannot substitution any deeper rule), NT is the set of non-terminal values (variables, a set of abstractions), S is the goal symbol or start symbol, and P is the set of productions or rules for rewrites / substitutions. That is, any non-terminal should be rewriteable as a string of other symbols in the grammar.

We use Backer-Naur Form traditionally to describe our grammar. For example, an incomplete grammar for math might look like

1) Expr -> Expr + Expr
2)       | Expr * Expr
3)       | number

When we create a derivation, a sequence of rewriting steps following the rules of the grammar, we generate sentential forms which are sentences that can occur in one step of a derivation. As we do so, we generate a parse tree. For example, if we used our above rules and tried to parse 2 + 5 * 10 we might get:

In doing this, we also followed leftmost derivation where we rewrote the leftmost non-terminal in each step. There also exists rightmost derivation.

If it is possible that following leftmost (or rightmost) derivation can result in different parse trees, we call the grammar ambiguous. One way to remove ambiguity is to add a new nonterminal that embodies the repeated logic. The book uses the example of

Statement → if Expr then Statement else Statement 
			| if Expr then Statement
			| Assignment

Which is ambiguous for "if expr1 then if expr2 then assign1 else assign2" (which arm does the else belong to?) and corrects it to

Statement → if Expr then Statement
			| if Expr then WithElse else Statement
			| Assignment

WithElse → if Expr then WithElse else WithElse
			| Assignment

This forces the else clause to always be associated with the correct arm. We also assert that the closer to the start symbol a rule is, the lower its precedence. That is, more important / more binding power elements will appear lower in the parse tree usually.

Top Down Parsing

Top-Down parsers start with the root of a parse tree and build downwards, continuing until all leaves of the tree are terminal values and the input stream of tokens is exhausted (or, in case of error). In the case of an error, a parser will assume it applied some wrong production and will backtrack and unwind the set of changes it made to its last branching choice, try the next one and continue. Backtracking isn't mandatory for a grammar, however. A good set of pseudocode for a backtracking algorithm

let node = root
let word = scanner.nextWord()
let testingStack = []
while true
	if node is a variable
		// look at all rules for node
		for 1 to node.rules.length
			testingStack.push(node.rules[idx])
		node = node.rules[0]
	else if word == node
		word = scanner.nextWord()
		node = testingStack.pop()
	else if word == EOF and we have no more nodes to look at
		return root
	else
		backtrack
		

What happens in the case of a left-recursive grammar with a top-down parser? Think of our above math

1) Expr -> Expr + Expr
2)       | Expr * Expr
3)       | number

We would end up with an infinite cycle of testing the leftmost Expr with the first substitution rule. So, in order to change this, we would need a right-recursive grammar by rewriting the productions.

Expr -> Term Expr'

Expr' -> + Expr'
       | * Expr'
	   | Term
	   
Term -> number

We need to watch out for direct left recursion as well as indirect left recursion.

Once we've done this, we can introduce a lookahead symbol which allows us to peek at the first symbol in each rule and match it to the upcoming token in order to predict the next applicable rule. We call the set of valid lookahead symbols the FIRST set, which is defined as "the set of terminals can that appear at the start of a sentence derived from some rule a'" .

That is, for Expr', the FIRST set includes Term, + and *

We can also define a FOLLOW set as all the words that can come after a nonterminal. That is, for Expr', that would be eof

We also introduce left-factoring that rewrites ambiguity in left terms by isolating common prefixes into their own set of productions. That is

Expr -> T + F
     | T - F
	 | T

We see a common prefix of T, isolate and move the rest

Expr -> T Expr'

Expr' -> + F
      |  - F
	  |  ε

The pseudocode to use Expr' looks like

function exprPrime() {
	if word == + or word == -
		word = scanner.nextWord()
		if isF(word)
			return true
		else return false
	else if word == "" or word == EOF // epsilon case
		return true
	else
		reportSyntaxError()
		return false
}

A topdown parser that uses FIRST, FOLLOW, and FIRST+ sets to avoid backtracking is called LL(1). That is, we scan left to right on input, working on leftmost derivation, with a lookahead of 1 token. As long as the grammar is right-recursive and backtrack free, LL(1) is an option.

Bottom-Up Parsing

Like the name suggests, a bottom-up parser finds the leaves (terminals) of the parse tree and builds non-terminals on top. The parser "extends the frontier upwards" by looking for some production a' -> b' - that is, if b' exists at some rightside position k, we can replace b' with a'. If that production yields a valid derivation, we call this the handle, which is the pair (a' -> b', k). The process of replacing b' with a' in the frontier is called a reduction.

The scanner still runs with left to right input, but our parser works on rightmost derivation. However, we reverse the order to make sure we handle the leftmost leaf first. As a result, these parsers are LR parsers - left to right input, reverse rightmost derivation.

The LR(1) parsing algorithm uses a single lookahead symbol. It uses two tables, an Action table and a Goto table to find the next handle for the derivation. It does this by holding a stack of the current upper frontier (the layer of the parse tree it's building) interleaves with states from the handle-finding steps to figure out how to make a reduction.

A shift is just an advancement of the scanner stream - the equivalent of consuming a token. The shifted symbol becomes the new parse tree state.

Pseudocode

push $
push starting state s0
word = scanner.nextWord()

while true
	let state = stack.pop()
	if Action[state.word] == reduction of a' to b'
		cardinality = b'.length
		pop cardinality times
		state = pop()
		push(a')
		push Goto[state, A]
	else if Action[state.word] == "shift state si"
		push word
		push si
		word = scanner.nextWord()
	else if Action[state.word] == accept
		break
	else
		failure()

return success

Action tables are a mapping of states and words to steps - either shifts or reduces. Goto tables are a mapping of handles (functions to replace some b' with a') and state transitions.

In order to build these tables, we need to be able to represent handles and potential handles alongside the lookahead symbols. Each potential handle is presented by an LR(1) item which looks like [a' -> b' • y. a] where a' -> b' y is a grammar production, • represents the position of the stacktop, and a is a terminal symbol. The set of items is called the canonical collection. Where we place • means different things.

1. [a' -> • b'y, a] = a' would be valid, if we find a b' next then we can discover a'. This item is called a possibility - it represents a possible completion for an input seen.
2. [a' -> b'• y, a] = the parser has progressed from the state before and has recognized b'. This item is partially complete
3. ['a -> b'y• , a] = the parser has found some b'y such that a' followed by a would be valid. If the lookahead symbol is a, then the item is a handle and the parser can reduce b'y to a'. This is a complete item.

To build the canonical collection, a parser states with an initial state [Goal -> • SomeExpr, eof] that is, the first handle that can be reduced such that the lookahead symbol is the end of the file. We then find every potential state change as a set. The two operations that can occur are taking a closure and computing a transition.

1. A closure completes a state. It adds to the set of items any related implications. That is, anywhere Goal -> List is valid, we can also add the productions of ways to derive List. So if we see [Goal ->•List, eof], we can say that both [List -> • List Pair, eof] and [List -> • Pair, eof] are valid.
2. To compute a transition on some symbol x, the algorithm should find the item of items that has • before x, and pick the set of items that would transition • AFTER x. This is the goto procedure.

Pseudocode for closure

function closure(s)
	while s is different from lastSeenS
		for each item in s where [a' -> b' • nonTerminalC, a]
			for each production for nonTerminalC -> y in ProductionList
				for each lookAhead in FIRST for someState
					s = union(s, ([nonTerminalC ->• y, lookAhead]))
	
	return s

So if we had some grammar for math, we might start with [Goal -> • Expr, eof] and the closure operator might add to the set

[Expr -> • Num, eof], [Expr -> • Num, +], [Expr -> + Expr, eof]

The goto procedure takes some set CCi in the canonical collection, a grammar symbol c, and computes the states that would be made if x was accepted in state i.

goto(set, symbol)
	moved = null
	for item in set
		if item matches some [a' -> b'• xAndExtra, a]
			moved = union(moved, [a->b'x• theExtra, a])
	return closure(moved)
		

It tests to see if it can move the acceptance past the lookahead symbol , and if it does adds the moved state into a total collection, then finds the closure over that.

Conclusion

This is a very high-level overview, ignoring a lot of error management. There are many other parsing algorithms, such as SLR and LALR. I handwaved over a couple of details in the parsing pseudocode for brevity, but I have found some notes / books that can help.

1. https://tomassetti.me/guide-parsing-algorithms-terminology/#parsingAlgorithms
2. https://www.amazon.com/Parsing-Techniques-Practical-Monographs-Computer/dp/1441919015

I'm really excited with this. The steps are:

1. Read into a String, which is a heap-allocated type in Rust.
2. The entry point hands the string over to the virtual machine which allocates a "chunk" and gives the chunk to the compiler.
3. The compiler loads that into a scanner and starts to parse token by token
4. When the compiler calls advance, the scanner grabs a token, hands the token back to the compiler and the compiler parses it - if it's a number, it creates a constant, if it's a parenthesis, it assumes grouping, etc.
5. The compiler writes these opcodes to the chunk
6. When the compiler is done grabbing tokens and turning into opcodes, it finishes and returns a compiler InterpretResult
7. When the VM sees the compiler is done, the chunk is full of instructions
8. The VM then walks the instructions and works with an internal stack and instruction pointer which is the index of the next instruction to run

Overall we are successful. Next step in the book is to add more types of values (bools, for example).

I also got Nand2Tetris week 2 done, which is really neat. Making an ALU from basic chips like Mux was great. I only got stumped when I forgot that a chip can have multiple outputs. Excited to try and get all the hardware stuff done during RC.

Dmitry Soshnikov's class on finite automata is also going well - I need to try and finish it this week, so I might make that my goal for tomorrow. Additionally, I need to finish the parsing chapter from Engineering a Compiler. Whew, lots of stuff to do and such little time!.

A pointer is the memory address of a variable. If we declare int x = 6; a few things happen. The number 6 is created somewhere in memory (stack), then we assign a name to identify it. That name in a human way to read the location of that value. Calling a variable is the same as telling the machine "please go to this location and tell me what is stored there".

A pointer can be stored directly in a variable, known as a pointer variable. We declare these with the star sign, such as int *p - which is, p is the memory address of an integer. When we assign values to a pointer variable we do so by getting the address of another variable using the address-of operator.

// declare a pointer variable for an int

int &p;

// declare a variable 
int x = 5;

// tell p to point  to the location of x
p = &x;

Pointers would be useless if all they could do is point to a location. We'd like to tell the computer "please go to this location and get me the variable there". We call this dereferencing and it is called with the * operator again.

int x = 0;
int *p = &x;

*p = 42; // now we told the computer "go to the location at p and change the value to 42

cout << *p << endl; // this is now 42
cout << x << endl; // this is now 42

Additionally, pointers can point to pointers which follow the memory address. That is

int *p1, *p2;
int v1 = 0;

p1 = &v1;
p2 = p1; // p2 now points to v1's address

We don't actually need to even create a variable to point to in order for a pointer to be valid. We can use the new keyword to assign a pointer to a location holding an int. When we use the new keyword, we are creating a dynamic variable

int *p = new *int;

Dynamic variables need to be held somewhere. That portion of unallocated memory is called the heap or freestore. It is possible to fill the heap with too many variables, which will cause the next call to new to crash the program. In order to prevent this crash, we use the delete keyword to free memory used by a variable.

int *p = new int;

*p = 10;

cout << *p << endl; //prints 10

delete p; // now the value of p is nothing

Note that we delete the pointer, not the dereferencing. Once a pointer is deleted, it is called a dangling pointer - don't dereference these!

The opposite of a dynamic program, one that has its lifetime handled by the compiler, is an automatic variable or ordinary variable.

We can use typedef to alias types to apply reader-semantics or domain-information to a type. That is typedef double Kilometer would allow us to then say Kilometer distance = 123.45;

This also works for pointers. If we wanted to say that, for example, this pointer is pointing to an int on the heap, we could say typedef int* IntPtr; This also lets you declare multiple variables of pointer type easily, such as IntPtr p1, p2;

If you want to pass a pointer into a function by reference, you need the & sign to determine reference and * to annotate pointer type.

void functionthing(&*int ptr);

// or
typedef *int IntPtr;

void otherfn(&IntPtr ptr);

Problem Solving with C++, 9.2

A dynamic array is one that is heap-allocated and can grow and shrink in size according to the number of elements in the array.

We can think of the variable of an array as a pointer since it refers to the address of the first element in the array, but it CANNOT be reassigned to a pointer (unlike a real pointer). That is

int a[10];
typedef *int IntPtr;
IntPtr p;

p = a; // valid! P can point to a

a = p; // invalid!! a is not of type *int

To allocate an array on the heap, we use the new keyword like

double *p = new double [10];

// do stuff with array

delete [] p; // array notation to delete an array pointer

Note that deleting a pointer to an array must be delete [] ptr; - this tells the system to delete the entirety of the length of the array, rather than the first element.

We can use pointer arithmetic to index an array, that is

double *p = new double [10];

for (int i = 0; i <10; i++ ) {
	// go to location p, adds i * sizeof(double) and get value
	cout << *(p + i) << endl; 
} 

My Rust VM is going well. I'm up to the tricky part, the Pratt parser. If I understand correctly, the way a Pratt parser works (more or less) is:

Given tokens = a list of tokens
startingPrecedence = 0
rules = table mapping tokentype to prefixFunction, infixFunction, precedenceValue


function parsePrecedence(precedence) =
  while token in tokens
    kick off parsing with precedence argument
      get first token
      Look up the token's prefixFunction in the rules
        that is, it must be prefix because it's the first thing in the list
        if the first token is a + sign, that would be an error
        if that token is a number, for example, that will be valid since numbers can be considered prefix values
      run that prefixFunction (eg, if token is a number, run the compileNumberToken function, if the token is a minus, run the compileUnaryToken function)

      as long as our function precedence is <= the next token's precedence
        consume next token
        get that token's infixFunction from our lookup table
        run infixFunction()

parsePrecedence(0)

note that prefixFunction, infixFunction, etc will ALSO call parsePrecedence with slightly higher precedences - that is, the infixFunction for multiplying might call parsePrecedence(20)

This means too that when when call parsePrecedence(20) for our multiplier it will say something like

1. go get tokens, and keep generating compiler output as long as those are MORE important than me, since it's a recursive call
2. when you're done, eg when we see an addition, exit and let the calling function continue its lower precedence work

So the workflow for

1 + 2 * 3 + 4

looks like

get token 1
call makeNumber(1)
get precedence 0

look at next token, addition
  call addition infix which enters higher precedence
    get token 2
    call makeNumber(2)
    next token is *, which is higher predecence
      get token 3
      call makeNumber(3)
      next token is +, which is lower precendece - exit
    emit *
  emit +
look at next token, which is +
  call addition infix with which is higher precedence
    capture 4
    call makeNumber(4)
    done
    pop call stack
  emit +
done

If we were emitting opcodes for a stack VM, that might end up looking like

[1, 2, 3, *, +, 4, +]

And that is, in an ugly nutshell, notes on a Pratt Parser.

The basic properties of arrays

1. Arrays are stored continuously in memory (one uninterrupted section)
2. Elements are all of the same type
3. We access elements via 0-based index

The compiler uses the index as an offset, so the math is

addressOfElementAtIdxI = addressOfStart + i * sizeOfEachElement

This downsize of this math is that we can access arbitrary memory by passing any index in - if i is out of bounds, it'll basically say "go to the start of the array, go to i * size and get what's there" EVEN if i is not in the array. Security flaw in business logic.

These arrays are technically static arrays (we'll cover dynamic arrays later) - static arrays are stack allocated. As a result

Array size MUST be a compile-time constant and given at declaration

So how would we calculate the average of a list of grades AND the number of grades above the average?

#include <iostream>
using namespace std;

const int MAX_SIZE = 60;
int main() {
	int numberOfStudents;
	cout << "Enter the number of students in the grade (max " << MAX_SIZE << "): ";

	cin >> numberOfStudents;

	if (numberOfStudents > MAX_SIZE) {
		//error
	}

	int grades[MAX_SIZE];
	int sum;

	for (int i = 0; i < numberOfStudents; i++) {
		int grade;
		cin >> grade;
		sum += grade;
		grades[i] = grade;
	}

	double average = (double)sum / numberOfStudents;
	cout << "the class average is " << average;
	cout << " and the grades above that are ";

	for (auto grade: grades) {
		if (grade > average) {
			cout << grade << " ";
		}
	}

	return 0;
}

Problem Solving with C++, 7.1

An array provides an interface over a finite collection, or list of elements that are all of the same type T. For example, if we want to see a list of all the receipts I have from Ample Hills (oh no), we might declare float receipts[12];

Our syntax declares the TYPE of the element, the name of the collection, and the size of the array. Arrays are fixed size in C / C++ due to stack allocation, and needing the size to be known at compile time.

We can access variables by their index, or position by order, starting from 0 as the first element. That is, the first expense I have at Ample Hills would be receipts[0] and the last receipts[11]

We can declare arrays alongsize other variables with something like int height, sandwiches[5], weight. Array index accessors also allow us to write to an array, which looks like sandwiches[5] = 100;

We can read and write from arrays if we know the length ahead of time. That might look like

const int NUMBER_OF_ICE_CREAM_FLAVORS = 10;
float iceCreamCosts[NUMBER_OF_ICE_CREAM_FLAVORS];

for (int i = 0; i < NUMBER_OF_ICE_CREAM_FLAVORS; i++) {
	cout << "The ice cream at " << i << " is $" << iceCreamCosts[i];
}

How does array access really work?

Imagine the following declaration

int arrayThing[5];

This tells the C++ compiler "reserve space in memory large enough for 5 integers and tell me where the first element is is memory". That is, the array value is really a memory address; a location in memory where the array starts. Thus, saying arrayThing[0] is the same as "to go the location at arrayThing and tell me what's there". But how does arrayThing[3] work? This is the same as telling the compiler "Go to the location of arrayThing, find out how many bytes is the size of 3 of the base type int, then add that value"

That is, if arrayThing is located at 100, and an int is 4 bytes large, this tells the compiler "Go to 100 + 3 * 4 and get me that value".

Out of range accessing values is illegal and can return garbage values, or allow bug exploitation.

In order to be safe, if we ask a user to tell us where in the array their element is we must validated, to avoid giving them access to arbitrary memory.

We can declare arrays with size and elements by using braces and commas like int children[3] = {2, 12, 1}

C++ introduces a range based for loop with

for (int child: children) {
	cout << child;
}

That is, we declare the data type and the name of the variable to identify the element in the array. Note that the variable is pass-by-value - you cannot change the array. You CAN declare it with a pass-by-reference style such as

for (int &child: children) {
	cout << child;
}

C++ also allows the auto keyword to infer type.

for (auto child: children) {
	cout << child;
}

Problem Solving with C++, 7.2

Sending an array-index-accessor into a function is valid. That is

bool isOne(int num) {
	return num == 1;
}

int arr[5] = {1, 2, 3, 4, 5};

isOne(arr[0]);

Is syntactically correct. The index accessor will be evaluated prior to function call.

We can also pass in entire arrays as a function argument. They end up behaving similar to pass-by-reference (that is, functions can write to them)

A function parameter can be an array of some type T, and the function can also take array size for iteration

void writeToArray(int arr[], int size) {
	for (int i = 0; i < size; i++) {
		arr[i] = 100;
	}
}

Due to the way arrays are stored in memory, you need to tell a function how many elements are in the array. Specifically, when you pass an array into a function technically all you're doing is telling the function where in memory the array starts, and how large each element is in bytes. You don't pass the third piece of array-information, the size. So you must do that explicitly.

Since functions can modify arrays passed as arguments, we might want to prevent the code from doing so for compiler-level safety. To do that, we use the const keyword to declare a constant array parameter. That might look like

void readArray(const int arr[], int size) {
	for (int i = 0; i < size; i++) std::cout << arr[i];
}

readArray(someArray, 10);

Problem Solving with C++, 7.3

Sometimes we don't know at compile time exactly how much memory we need. So we allocate a massive space, one that would cover more than the use case. In this case, we end up with partially filled arrays. When this happens, the unused space (that is, the end range that we don't populate, the excess) is garbage. We must be careful to NOT access that, by tracking and only using the real value that the user put in.

Module 10 - Strings

C++ has its own string type, which we can use with the directive #include <string>. Strings are double quoted and respond to concatenation with the plus operator. We can read strings with std::cin and write to stdout with std::cout. cin by default only reads up to a space, so keep that in mind.

If we want to capture an entire line, we use

int main() {
	string str;
	cout << "enter your name: ";
	getline(cin, str);
	cout << "Your name is: " << str;
	return 0;
}

Strings can be read with index-accessors like arrays. A string is effectively a sequence of characters, so each element is a char.

We can also slice (find substrings) with a start (inclusive) and length.

str.substr(startIdx, len);

Strings also have a length method on the class, such as str.length()

How can we print a string backwards?

#include <iostream>
#include <string>
using namespace std;

int main() {
	string str;
	cout << "Enter your name: ";
	getline(cin, str);
	int len = str.length();

	for (int i = len-1; i >= 0; i--) {
		cout << str[i];
	}
	return 0;
}

Strings have lexicographic order based on ASCII code from index 0 to length. That is

// pseudocode

for i in min(str1.len, str2.len)
	if str1[i] < str2[i]
		return str1 is less
	else if str1[i] > str2[i]
		return str2 is less
	else
		continue loop

if not yet returns
	shorter string is less, return that

Strings also have a find method to see if the substring of a string exists and where the starting index is (first appearance, in case of duplicates).

	string test = "bananananna"
	int startingIndex = test.find("anna");
	bool notFound = test.find("notInside") == string::npos;

Find can also take an optional starting index as the second param, which will start looking AFTER the second param such as str.find(sample, idxToStart);

Problem Solving with C++, 8.2

Strings are arrays of characters terminated with the null byte '\0'. We include strings via the string directive, which is in the std library so we'll need to add the namespace std, or access via std::string

We can read all input from a user into a string with the getline function, which stops reading when it hits a newline '\n' We can change that to read until some OTHER character, such as getline(std::cin, stringVar, '?')

String accessors can be protected at runtime with str.at(idx) which throws a runtime error rather than reading garbage values with bracket notation. It also allows you to write with str.at(idx) = 'a';

Discrete Math, 6.1

Language:

Experiment - a procedure that results in one out of a number of possible outcomes

The set of all outcomes is the sample space.

A subset of the sample space is an event.

That is, an experiment could be flipping two coins. The set of opttions, the sample space, is {HH, HT, TH, TT}. The event of at least one head is {HH, HT, TH}.

Discrete probability is concerned with experiments where the sample space is countably finite or countability infinite. What does the latter mean?

A set is countably infinite if there's a one-to-one correspondence between the elements of the sets and the integers. That is, it's possible to assign every element in a set to an integer and eventually reach some value by counting, no matter how long it takes. Think about binary strings. We can assign 0=0, 1=1=, 10=2, 11=3, ... so on and so forth and, as long as we progress in the set, can reach some integer N.

Probability is concerned with likelihood. Every event has a probability, and the sum of all event probabilities is 1.

A uniform distribution occurs when the probability of every outcome is equal (a dice roll, for example). In a uniform distribution, if every outcome is equally weighted, then the probability of an event is the number of outcomes in the event divided by the total number of events in the sample space.

That is, the probably of 2 or lower when rolling a die is

|{1, 2}| / |{1, 2, 3, 4, 5, 6}| = 2/6 = 1/3

6.1.1

b) {hhtt, thht, tthh} = 3 outcomes / 16 sample = 3/16

c) {thhh, thht, tthh} = 3 outcomes = 3/16

6.1.2

b) There are n! total ways to order the group. If Ceiia is first and Felicia is last, there are (n-2)! possible middle-orderings. Thus, 1 / (n(n-1))

6.1.3

a) There are 2n total socks. We need to choose two. thus 2n choose 2, or n(2n - 1)

b) The ways to pick 2 white socks is n choose 2. The ways to pick 2 black socks is n choose 2. Thus 2(n choose 2) or n (n-1)

c) The total number of ways to pick socks is n(2n -1). The ways to pick matching pairs is n(n-1). Thus, the probability of matching pairs is (n-1) / (2n-1)

Discrete Math, 6.2

If two events have no overlap, we call them mutually disjoint. That is, with two coin flips, the events where we have exactly one head and exactly two heads. Then we can add these probabilities to find the probability of at least one head over two coin flips. This ONLY works when the intersection of events is zero. That is

The probability of exactly one head over two coin flips is .5 {TH, HT}. The probability of exactly two heads is .25 {HH}. The sum is .75

We can formalize as

P(E1 or E2) = P(E1) + P(E2)

If two events are NOT mutually exclusive we can use the Inclusion-Exclusion principle as

P(E1 or E2) = P(E1) + P(E2) - P(E1 and E2)

(this is the same as the above, except in a mutually disjoint case, P(E1 and E2) = 0)

Often times, we can use the complement of an event to determine our probability - that is, it's easier to reason about the probability an event DOESN'T happen. Then we can calculate

P(E') = 1 - P(E)

6.2.1

a) Probability at least n-1 heads in n coin flips? Same as odds of no tails + odds of exactly one tails. Odds of no tails = 1/(2^N) and odds of exactly one tails = n / (2^n) -> sum = (n+1) / (2^n)

b) Odds at least two consecutive coin flips are the same? What are odds no consecutive coin flips are the same? 2 / 2^n (HTHTHTH... or THTHTH....). So complement is 1 - (2 / 2^n)

6.2.2

b) Prob of Celia first = 1/n. Prob of Felicity last = 1/n. Prob of Celia first AND Felicity last = (n-2)! / n! = 1 / n(n-1). Make Celia and Felicity n-1 / n(n-1), sum to 2(n-1)/n(n-1) - 1 = 2n-3 / n(n-1)

Discrete Match, 6.3

Sometimes we want to discuss the probability of a thing happening predicated on another event. That is, two coin flips - what are the odds of 2 heads? 1/4. What are the odds of 2 heads, given we flipped one coin and got heads already? 1/2.

We use conditional probability to denote the probability of event F happening given some event E happening, that is p(E|F) - or given E, F.

The formula is

p(E |F) = p(E and F) / p(F)

Which can be simplified to

|E and F| / |F|

For our coin flips, the event E = two coin flips are {HH}. F = first coin is heads. |E and F| = 1 ({HH}. |F| = 2 {HH, HF}. Therefore, 1/2

Another example, let E and F = two dice rolls, the first is 5 and the sum is at least 11. F = event that the first die is 5.

|F| = all the possible two-die rolls where the first is 5. That is, (5,1), (5,2)... = |F| = 6

|E and F| = all the possible ways to sum to 11 or higher, given one die is 5 => 1

1/6

Essentially, we are narrowing down probabilities given some event F happening. Like...the probability I get 100 on an exam AND get accepted into a masters / probability I get 100 on an exam

Two events are independent if conditioning on one event does not change the probability of the other. Think about two dice rolls. Probabiliy both are the same? 6/36 = 1/6. Probability both are same, given first is 5? Well if first is 5 the second must be 5 -> 1, 1/6. These rolls are independent events. Specifically, the odds of the die rolling a 5 on the first roll does not impact the odds of a 5 on the second roll.

If two events are independent then the probabiliy of both happening is the product of the probabiliyt of each happening.

p(X and Y) = p(X) * p(Y)

If EVERY event in a set are indepedent from each other, we say they are mutually independent and calculate

p(A1 and A2 and ... and An) = p(A1) * p(A2) * ... * p(An)

6.3.1

a)

p(A) = 1/2

p(B) = {46, 55, 56, 64, 65, 66} = 6/36 = 1/6

p(C) = 1/6

b) |A and C| = |{55, 53, 31}| = 3. |C| = 6. 3/6 = 1/2

c) |B and C| = |{55, 56}| = 2. |C| = 6. 2/6 = 1/3.

d) |A and B| = |{46, 55, 64, 66}|. |B| = |{46, 55, 56, 64, 65, 66}|. 4/6 = 2/3

6.3.3

a) There are 8! total ways to line up the party. To have bride and groom next to each other, we have 7! total variations, times 2 for bride before groom or vice versa. = 2 / 8 = 1/4

b) There are 7! ways to arrange a group with MoH on the left. There are 8! total arrangements. 1/8

c) p(A) * p(B) = 1/32. P(A and B) = 2 * 6! total arrangements of MoH on left and BG together / 8! total. 2 / (8 * 7) = 1 / 28. Not independent.

6.3.5

Total number of hands is 52 choose 5.

A - Total number of ways to pick a 4-of-a-kind is equal to (13 choose 1) * 48 - picking one rank and 1 of the leftover cards.

B - Ways to pick from no aces = 48 choose 5, at least one ace = 1 - 48 choose 5.

Discrete Math, 6.4

Bayes' Theorem is a way of reasoning about probability based on the evidence of outcomes.

Specifically, Bayes's theorem lets us theorize about p(F | X) if we know p(X | F), p(X | F'), and p(F).

Assuming F = a fair die is selected, and F' = an unfair die loaded with 2/7 probability of rolling a 6, and X = a die rolls 6.

What is the probability of selected a fair die and rolling a 6 given rolling a 6?

Well, if we know the probability of rolling a 6 and selecting a fair die given a fair die (1/6), the probability of rolling a 6 and selecting an unfair die given an unfair die (2/7), and the probability of a fair die (1/2).

we can calculate

the numerator = 1/6 * 1/2 = 1/12

The denominator = 1/12 + (2/7) * (1/2)

The probability then = ~0.37

5/72 / (5/72 + 10/98) = 49/121

6.4.1

a)

Let F = biased coin

p(B) = 1/2. P(X | B) = (3/4)^7 * (1/4) ^ 3.. P(X | B') = (1/2) ^ 10

( ((3/4)^7 * (1/4) ^ 3) * 1/2) / ((3/4)^7 * (1/4) ^ 3 * 1/2) + 1/2 * (1/2) ^ 10)

0.00104 / (0.00104 + 0.00048)

0.6842

6.4.2

Let F = fair die

p(F) = 1/2

p(X | F) = (1/6)^6

p(X | F') = 0.25^2 * 0.15^4

(1/6)^6 * 1/2 / ((1/6)^6 * 1/2 + 0.25^2 * 0.15^4 * 1/2)

0.4038

Discrete Math, 6.5

A random variable X is a function from the sample space S to an experiment of real numbers, where X(S) is the range of the function X.

If two dice are rolled, the random variable D is the sum of the rolls, so D(x, y) = x + y

If X is a random variable and r is a real number, then X = r is an event. That is, for rolling two dice, if D(x+y) = 5, then that's an event with the subset of {(1, 4), (3, 2), (4, 1)}

The distribution of a random variable is the set of all pairs (r, p(X = r)) for all r in X(S)

6.5.1

a) {1, 2, 3, 4, 5, 6, 4, 8, 10, 12, 9, 15, 18, 16, 20, 24, 25, 30, 36}

b) 4 / 36 = 1/9

Discrete Math, 6.6

The expected value of a random variable is defined as

E[X] = the sum of all X(s) * p(x) for all s in the sample space.

That is, the expectation of a random variable is equal to the sum of all elements in the range * the probability we will see them.

If X = winning 100_000_000 dollars in the lottery, and only one ticket can win, and the number of possible tickets are 50^6

then E[X] = 100_000_000 * (1 / 50^6) + 0 * (1 - 1 / 50^6) = ~0.0064

This can also be calculated as

E[X}] = r * p(X = r) for all r in X(S).

6.6.1

Let |S| = 10 choose 2 = 45

Number of ways to pick two girls = 7 choose 2 = 21

Number of ways to pick 1 girl = 7 choose 1 * 3 choose 1 = 21

E[G] = 2 * (21/45) + 1 * (21/45) = 63/45 = 21/15 = 7/5

6.6.2

E[D] = 2 * 1/6 + 1 * 1/6 + -1 * 4/6 = -0.17

Discrete Math, 6.7

Linearity of expectations says the expectation of the sum of two random variables is equal to the sum of the expectations.

If X and Y are two random variables in S, and c is some real number constant

E[X + Y] = E[X] + E[Y]

E[cX] = cE[X]

From the book

Two friends run a lemonade stand over the summer. It takes 10 lemons to make a batch of lemonade. Usually, lemons are 50 cents apiece, but with probability 1/4, they are on sale for 40 cents apiece. If there is a baseball game at the park where they put up their stand, they will sell $20 worth of lemonade. If there is no baseball game, they will sell $10 worth of lemonade. On a given day, there is a baseball game with probability 1/2. What is the expected profit?

E[P] is the same as E[R - C] where R is revenue and C is cost

E[R - C] = E[R] - E[C]

E[R] = 0.5 * 20 + 0.5 * 10 = 15

E[C] = 3/4 * 5 + 1/4 * 4 = 4.75

15 - 4.75 = 10.25

Discrete Math, 6.8

A Bernoulli trial is an experiment wiht two outcomes - success and failure.

In a sequence of trials, called a Bernoulli process, the outcomes of the experiments are assumed to be mutually independent and have the same probability of success and failure. Success denoted by p and failure as 1-p

The probability of exactly k successes in a sequence of n independent Bernoulli trials, with probability of success p and probability of failure q = 1 - p is

The distribution over the random variable defined by the number of successes in a sequence of independent Bernoulli trials is called the binomial distribution. The probability that the number of successes is k in a sequence of length n with probability of success p is denoted by b(k; n, p).

Spent the morning just reading lightly. Nothing too major. There was the 11:00am check in, then a 11:30 compilers study group planning session. We're going to read Engineering a Compiler as two chapters a week, and there will also be a 1x a week meeting on Tuesdays just to discuss open source projects to contribute to. LLVM sounds really interesting, I may try to get my hands dirty in a week or two.

During the first Hack n Tell sprint, I managed to write some more Rust and finish chapter 15 of Crafting Interpreters. We have a very small stack based virtual machine. I'm hesitant to call it bytecode, as I'm using Rust's typesystem to capture constants as a type rather than pushing two ops into an array. But it's the spirit of the law, I guess. Overall I'm happy with the work. Rust is slowly starting to become syntactically natural with practice. Admittedly I'm sure it's not all idiomatic, but that's okay. Diff here.

For the second Hack n Tell, I hit a wave of inspiration from the compiler study group meeting (inspiration from community is the best part of RC, it seems). I remembered Rome - I've got a lot of respect for @sebmck, and figured if I want to get my hands in a big project that's a developer toolchain application, why not Rome? It's early, they'll need help. Luckily there's a LOT of missing tests, so after hopping in the Discord I opened up a pull request. I'm hoping to make 4-5 contributions during RC. Ideally not all tests, but it at least lets me navigate the code base a bit.

Tomorrow I'll probably chill a bit - I did the reading for Nand2Tetris week 1 today and want to watch the videos. The calendar looks less hectic so I'll probably just consume some learning, MAYBE another test case.

I also began working on Crafting Interpreters, specifically the bytecode VM, in Rust. Repo here. This will give me something to fall back to just to make sure I'm writing code regularly. I'd like to finish chapter 15 this week (introducing the virtual machine), and then also kick off Nand2Tetris. I DO have an exam Thursday, so I may take a bit easier just to study.

If I can go pretty deep into compiler stuff in the first 3-4 weeks, I can use the last couple of weeks to read some papers and MAYBE learn some Go along the way.

Having fun though!

I'm interested in the idea of "developer infrastructure" organizations and teams, so I'll be digging into projects and lectures related. Specifically, I'd like to

• Implement most of Modern Compiler Implementation in ML in OCaml
• Read a ton of distributed systems papers from this MIT syllabus
• Keep working through the CS-537 videos and learn more about operating systems
• Maybe implement some of those scheduler algorithms

I've got 6 weeks to get through this, PLUS embrace all the serendipity of being around new people who are also curious. I'll be nice to myself if I don't make it through it all, but I think just the act of learning and being out of my comfort zone will do a lot for me. Plus - there's a lot of RC alum I respect a ton like Dan Luu and Julia Evans. I'm a pretty big believer that if you want to be where someone is, you should do what they did when they were in your shoes. So, here's hoping we learn something.

We introduce the k-combinations problem, that is, for some non-negative numbers n and k where n is the size of the collection (number of cards in a deck, number of dishes in a buffet line) and k is the size of the combination and n >= k, we define the number of possible unordered ( aka "red blue green" and "blue green red" are unique) combinations - n choose k - as (n! / (k! * (n-k)!)),

For example, if we have a collection of n = 5 objects and want to know all the number of k = 3 possible combinations, we calculate (5! / (3! * 2!)) or 120 / 12 = 10 possible combinations.

The mathematical notation looks like:

If we wanted to do this with while loops, we can formulate as

int main() {
	int n, k;

	cout << "enter n and k"
	cin >> n >> k
	int nFact = 1;
	for (int i = 1; i <= n; i++) nFact *= i;

	int kFact = 1;
	for (int i = 1; i <= k; i++) kFact *= i;

	int nkFact = 1;
	for (int i = 1; i <= (n-k); i++) nkFact *= i;

	cout << "the answer is " << (nFact / (kFact * nkFact);
	return 0;
}

There's repeated logic here that distracts us and doesn't add expressivity in the factorial logic. We now learn how to make our own functions.

We need a function we'll call factorial that takes an int that represents the number we want the factorial of, and return an int as the answer. So we define a header which looks like

/*
	RETURN_TYPE FUNCTION_NAME(ARG_TYPE ARG_NAME, ...) {
		FUNCTION BODY
		RETURN
	}
*/

int factorial (int num) {
	int result = 1;
	for (int i = 1; i <= num; i++) {
		result *= i;
	}
	return result;
}

Using these functions, we can clean up our main function as

int main() {
	int n, k;

	cout << "enter n and k"
	cin >> n >> k
	int nFact = factorial(n)

	int kFact = factorial(k)

	int nkFact = factorial(n - k)

	cout << "the answer is " << (nFact / (kFact * nkFact);
	return 0;
}

Function calls then can be considered their own control flow, by jumping to a position we've been in (the function block) and return later.

Using the runtime stack model, we can understand the execution of this function. Our main program will exist in a frame on the stack with all local variables (nFact, kFact, etc). So n=5 and k=3 would get stored in the frame, then the calls to factorial need to be evaluated before nFact can be assigned a value. So we create another frame for factorial with local variables, along with the return address of where we go to when the factorial function is done. Then we associate the arguments of the function with the parameters - that is, factorial(5) should tell the frame that num = 5. During evaluation, the frame only has access to its own variables, it CANNOT access variables in another frame. After evaluation, we pop the frame off the stack and jump back to the return address with the value of the function.

This chapter of Crafting Interpreters is a great but detailed explanation of an implementation.

Note: C++ functions must be declared before usage. So you cannot write the main function at the top of a file and have functions below. Either write in another file and include, or just define it above the main function. We can also just declare the header inline ahead, something like

int factorial(int num); // no body, only header

int main() {
	// using factorial
}

int factorial(int num) {
	// actual implementation
}

Aside: a function that does not return (only does I/O) has a return type of void.

When we pass variables into functions, it is default pass by value (sometimes called call by value). That is, a function that receives variables cannot mutate those variables. It can only act with the values.

If we want to pass parameters into functions, we can also use pass by reference or call by reference with an address-of operator in the function, eg void func(int &x) . So, if we wanted a function that swapped the values of two parameters:

void swap(int &a, int &b) {
	int temp = a;
	a = b;
	b = temp;
}
int main() {
	int a = 1;
	int b = 2;
	swap(a, b);
	return 0;
}

In this example, the swap stack frame hold references to a and b rather than the values. This means that when swap makes changes, it changes the variables in the main stack frame rather than its local stack frame.

Okay, new program. Take a positive integer num and write the number of digits and sum of digits. We can't just have one function that returns TWO values (number of digits and sum of digits). So instead we can use call-by-reference to pass in the values and update in the function

void analyzeDigits(int num, int &count, int &sum);

int main() {
	int num, count, sum;
	cout << "enter a positive integer: ";
	cin >> num;

	analyzeDigits(num, count, sum);
	cout << "the number of digits in " << num << " is ";
	cout << count;
	cout << " and the sum is ";
	cout << sum;

	return 0;
}

void analyzeDigits(int num, int &count, int &sum) {
	count = 0;
	sum = 0;
	while (num > 0) {
		int digit = num % 10;
		num /= 10;
		count++;
		sum += num;
	}
}

Problem Solving with C++, Chapter 4

The top-down design or stepwise refinement of a problem it to divide it into smaller and smaller subproblems that become trivial to implement, and then stitch those steps together. In C++, we can separate these subproblems into functions.

C++ has built in libraries with predefined functions that we've used before, like sqrt from cmath. A function has arguments attached to parameters and also (often) has a return value. When we use a function, we call or invoke it, such as int root = sqrt(3.0)

When we bring a function into a file, we can add it via include directives a la #include <cmath>. The value in the angled brackets is the name of a file called the header file which dictates the structure of a function - return type, function name, parameter types and names. Header files do NOT include implementation.

Include directives are handled by the preprocessor in a step that looks like copy-pasting the code into the file including the library. You can see what it looks like with clang++ -E FILE_TO_COMPILE

Another function is type casting via static_cast - for example

static_cast<double>(9);

We can also declare and define our own functions. Function declaration (or the function prototype) describes how a function is CALLED (the header). Function definition describes what a function DOES. Declarations must happen before a function is invoked, but definition can occur later. In a declaration, we have formal parameters to stand in for the values that a function will actually be called with. Note: formal parameters are optional but types are mandatory, yet we will use parameter names for clarity.

When a function is invoked, by default we plug in values for the parameters in a call by value mechanism (see above notes).

Arguments are invoked by order! Cannot mix and match the order between declaration and invocation.

We use functions as black boxes to hide information about implementation. A programmer may need to know what a function does without how it does it. Implementation may change but function APIs may not - think about a more efficient search algorithm. Writing and using functions as if they were black boxes is also called procedural abstraction.

When variables are declared inside a function they are locally scoped and do not leak outside. For a value that is needed by many functions, we might use a globally scoped constant by defined it outside all functions.

Technically, everything exists in some block scope where blocks are defined within curly braces, and global scope is the global block.

Note: Our using directives can be block scoped as well if we want to open a namespace locally to a function. This can help avoid clobbering.

C++ offers function overloading where we can define functions with different arity or parameter types and the same function name. We've seen examples with the division operator that can work on doubles, floats and ints in different ways. For example:

int add(int a, int b);
int add(int a, int b, int c);
double add(double a, double b);

When you overload a function name, the function declarations for the two different definitions must differ in their formal parameters. You cannot overload a function name by giving two definitions that differ only in the type of the value returned.

Problem Solving with C++, Chapter 5

Functions can return void. For void functions, we can omit the final return. This is useful for I/O like writing to terminal, or call by reference mutation. Defining a function that calls by reference just adds an ampersand before the reference variable, eg void example(int &x).

Since program variables are implemented as memory locations, a reference is actually a variable whose value is the memory location. This allows us to circumvent scope limitations. We are also allowed to mix call-by-value and call-by-reference in function parameters.

When we write function declarations, we can add comment blocks to describe preconditions (what we assume to be true) and postconditions (the effect of the function).

We can write driver programs to test our functions. These a small, temporary tools (often loops) to test various inputs

int main() {
	do {
		functionStuff()
	} while (inputIsValidRunner)

}

We can also add stubs that just replace a function dependency with an output value so we don't test more than the one function we care about. Example:

// stub
int taxableIncome(int salary) {
	return 100;
}

int totalSavings(int salary) {
	// our function we want to test
}

int main() {
	int income = taxableIncome();
	return totalSavings(income);
}

We can use the assert macro with a boolean expression as a precondition

#include <cassert>
int function() {
	assert(booleanExpression);
	// do stuff
}

To disable assertions, add #define NDEBUG above the include directive.

Data structures and Algorithms in C++, Chapter 2

We're now into algorithm stuff. Some rules:

Definition 2.1: T(N) = O(f(N)) if there are positive constants c and n0 such that T(N) ≤ cf(N) when N ≥ n0. My note - this is the upper bound

Definition 2.2: T(N) = Ω(g(N)) if there are positive constants c and n0 such that T(N) ≥ cg(N) when N ≥ n0. My note - omega is the lower bound

Definition 2.3: T(N) = θ(h(N)) if and only if T(N) = O(h(N)) and T(N) = Ω(h(N)). - Theta is the actual value, so time is equal to lower bound and upper bound

Definition 2.4: T(N) = o(p(N)) if, for all positive constants c, there exists an n0 such that T(N) < cp(N) when N > n0. Less formally, T(N) = o(p(N)) if T(N) = O(p(N)) and T(N)=/= θ(p(N)). 51

We are concerned with relative rates of growth that help us determine which functions take more time than others. We throw away constants and note that this is about eventuality so: 1000N vs N^2 - N^2 for some value of N will eventually overtake 1000N, thus we say that N^2 is the limiting factor.

We call the first definition Big-O notation.

We ignore constants and lower order - eg we don't write O(N^2 + N) becaues N^2 is the dominant element.

Note: These are algorithmic analyses. We cannot, in the real world, ignore caching, memory locality, disk I/O time, etc. However, we do work with massively large numbers of inputs in programming so these analyses become useful tools.

Rules

1. For loops - the running time is at most the number of elements of the loop
2. Nested loops - analyze inside out. The running time is the inner loop's runtime multiplied by the number of times it is executed, the number of elements in the outside loop
3. Consecutive work - it's additive. So two sequential for loops become O(N) + O(N) = O(2N), and we throw away constants so it's O(N)
4. If/else - the running time is never more than the running time of the condition plus the larger of the work in the if/else branches

Logarithmic algorithms might be confusing but it is defined as:

O(logn) if it is constant time to cut the problem size by a fraction (like divide and conquer algorithms like binary search)

Module 8 – Algorithm Analysis

We'll look at the problem of seeing if a number is prime or not, and learn how to talk about runtime complexity.

bool isPrime (int num) {
	// count the number of integers that divides cleanly into num
	// aka, num % i == 0
	// only correct answer is 1 and num for a prime
	int countDivs;

	for (int i = 1; i <= num; i++) {
		if (num % i == 0) countDivs++;
	}

	return countDivs == 2;
}

We know we can cut the problem space in half: let k be some int between num/2 and num - there is no complementary divider that is an integer. That is, d = num / k and if k > num/2 then d < 2

bool isPrime (int num) {
	// count the number of integers that divides cleanly into num
	// aka, num % i == 0
	// make sure only number in first half is 1
	int countDivs;

	for (int i = 1; i <= num/2; i++) {
		if (num % i == 0) countDivs++;
	}

	return countDivs == 1;
}

There's a third version! Instead we can look up to sqrt(num). Think about 100 - the largest unique divisor is up to sqrt(100) = 10. 20 divides into 100, but 5 times which we checked up to sqrt. That is, every number after the sqrt has a complementary divider LESS than sqrt.

Proof by contradiction: let k and d be complementary dividers of num (k * d == num). Assume BOTH are greaterr than sqrt(num). we can then say k * d > sqrt(num) * sqrt(num) or num > num. Contradiction, implying that both k and d cannot be greater than sqrt. Thus at least one must be greater.

#include <cmath>

bool isPrime (int num) {
	// count the number of integers that divides cleanly into num
	// aka, num % i == 0
	// make sure only number in first half is 1
	int countDivs;

	for (int i = 1; i <= sqrt(num); i++) {
		if (num % i == 0) countDivs++;
	}

	return countDivs == 1;
}

Runtime analysis:

Time 1 (T1), Time 2 (T2), Time 3 (T3). We know the running time depends on the size of the integer. Let n be the size of the input. Let us parameterize the running time over the size of the input. Running time depends on operators, so we ignore machine-dependent constants and count all primitive operators (addition, multiplication) as 1. We also only care about asymptotic analysis - we care about asymptotic order of the input. So drop lower order terms and constants.

T1(n): O(n) = O(n)

T2: O(n/2) = O(n)

T3: O(sqrt(n))

Formal definition

Asssume two functions f(n) and g(n) mapping positive integers to positive g(n)

We say f(n) = O(g(n)) if there exists to real constants c1 and c2 and a positive integer n0 such that c2 * g(n) <= f(n) <= c1 * g(n) for all n >= n0.

Proof: 3n^2 + 6n - 15 = O(n^2)

Find upper bound:

Drop leading constants

3n^2 + 6n - 15 <= 3n^2 + 6n

Find an upper bound by adding a 6n

3n^2 + 6n - 15 <= 3n^2 + 6n <= 3n^2 + 6n^2 = 9n^2

we can then say c1 = 9

find lower bound:

3n^2 <= 3n^2 + 6n - 15 <= 3n^2 + 6n <= 3n^2 + 6n^2 = 9n^2

if 3n^2 <= 3n^2 + 6n - 15

then 6n - 15 >= 0

n >= 2.5

Then we can say n0 = 3

Then we can drop lower order constant 6n

3n^2 <= 3n^2 + 6n - 15 <= 9n^2

Then we can say c2 = 3

Therefore 3n^2 + 6n - 15 = O(n^2)

Discrete Math Section 5.1

How do we count combinations of numbers? We have two rules we'll start with.

Product rule - The product rule says that for N number of sets, the cardinality of the Cartesian product is the product of the cardinality of each set. That is, |A1 x A2 x .. x An| = |A1| * |A2 * ... * |An|. For example, if we wanted to know all combination of breakfasts given:

main: {eggs, pancakes, waffles}

sides: {toast, bacon}

drinks: {coffee, water, orange juice}

The cardinality of set of all combinations is 3 * 2 * 3 = 18

This works for strings too - for example, If Σ is a set of characters (called an alphabet) then Σ^n is the set of all strings of length n whose characters come from the set Σ. For binary, for binary strings of length 5, we know there are 2^5 = 32 possible combinations.

Sum rule - The sum rule says that, for N number of sets, if they are mutually disjoint then the union of |A1 u A2 u ... u An| = |A1| + |A2| + .. + |An|

That is, if you know all options and only pick one element from the sets, then the number of options is equal to the sum of the cardinalities. Given our above menu of mains, sides and drinks, and user can only pick one, the sum rule states that the number of options is 8.

The book uses passwords as an example to merge these two.

Assume that L is the letters of an alphabet (all lower case) and D are the digits. |L| = 26, |D| = 10. The set of all characters is C = L union D, so the size of the set C is |C| = |L| + |D| = 36. Assume we must have passwords between 6 and 8 characters. The sum rule, since each letter combination is disjoint, says |C6 union C7 union C8| = |C6| + |C7| + |C8|. For each, we can apply the product rule and get 36^6 + 36^7 + 36^8.

Discrete Math Section 5.2

The bijection rule says that if there is a bijection from one set to another then the two sets have the same cardinality. This allows us to take a set of unknown cardinality, find a bijection to a set of known (or easily counted) quantity, then assert the count of the first set.

For example, if you have a movie theater and want to count the number of people who attended in a day, you can define a bijection as the number of tickets at the end of the day. You then have a well defined inverse.

We can also use this to count the number of elements in a power set of a finite set X. If |X| = n, then we can say that f is a bijection:|{0, 1} ^n| = |P(x)| where a bit being 0 is the absense of an element in the set and 1 is the presence.

The k-to-1 correspondence defines a relationship of K elements in set X for every element in set Y - that is, if each guest has two shoes then the number of shoes should be 2x the number of guests, thus k = 2.

Formally - Let X and Y be finite sets. The function f:X→Y is a k-to-1 correspondence if for every y ∈ Y, there are exactly k different x ∈ X such that f(x) = y.

Discrete Math Section 5.3

The generalized product rule says that in selecting an item from a set, if the number of choices at each step does not depend on previous choices made, then the number of items in the set is the product of the number of choices in each step.

An example - I go to the same restaurant every day for lunch. They have 10 lunch specials. I want to try something new every day this work week. On Monday I have 10 options. On Tuesday I must eliminate yesterday's choice, giving me 9 options. On Wednesday I lose another. The number of total combinations is 10 * 9 * 8 * 7 * 6.

Discrete Math Section 5.4

Building off the generalized product rule, an r-permutation is a sequence of r items with no repetitions, all taken from the same set. In this case, order does matter so the sequences ()"red", "blue", "green") and ("blue", "green", "red") are different permutations.

The number of r-permutations in a set of length n where r <= n, called P(n, r), is n! / (n-r)!

That is, if n = 5 and r = 3 then (5 * 4 * 3 * 2 * 1) / (2 * 1) = 5 * 4 * 3

A permutation (without the parameter r) is a sequence that contains each element of a finite set exactly once. You can think of it as an r-permutation where r = the number of elements in the set. The number of options is n! (n-factorial)

Example - imagine I have 4 people at a dinner party (Alice, Bob, Carol, Dan). I want to seat them at a bench, but I want Carol and Dan to sit next to each other. How many options do I have?

First, Carol and Dan - I have two options to sit them next to each other (Carol then Dan, or Dan then Carol). Then I need to figure out the number of ways to seat Alice, Bob and the Carol/Dan pair. That's 3! (three discrete elements) times 2 for the number of ways I can organize the pair, for 12 options.

Question: A wedding party consisting of a bride, a groom, two bridesmaids, and two groomsmen line up for a photo. How many ways are there for the wedding party to line up so that the bride is next to the groom?

Answer: 240. First decide whether the bride is to the left or right of the groom (2 choices). Then glue the bride and groom together and there are 5! ways to permute the five items. By the product rule, the number of line-ups with the bride next to the groom is:

Discrete Math Section 5.5

An r-combination, sometimes called an r-subset, is a way of counting order-independent combinations of elements from a set. If you care about which three people from an election got the most votes and win representative seats, order doesn't matter so {Alice, Bob, Carol} and {Carol, Bob, Alice} are the same - note the set notation.

To calculate the number of r-combinations in a set, you start with the number of permutations and divide by r!.

The notation is the same as the above n-choose-k example, but we can also say C(n, r) for n-choose-r

That is

• P(n, r) / r!
• n! / ((n-r)! * r!)

An equation is called an identity if the equation holds for all values for which the expressions in the equation are well defined. We can say C(n, r) = C(n, n - r) is an identity because it holds true for all non-negative n and any r from 0 to n.

Discrete Math Section 5.6

One of the examples here specifies words / phrases to look for that hint at combinations vs permutations. For example, given 10 people and 4 prizes, how many ways can you allocate them if

• the prizes are all the same? Order doesn't matter so it's combination, C(n, r) or (10! / (6! * 4!))
• The prizes are all different? Order does matter, so it's permutation, or P(n, r) or 10! / 6!

Review book for more examples of non-mathematical terms. eg if a waiter puts dishes in the center of a table, the number of ways to order from the menu is the combination (subsets) - if everyone gets a dish it becomes permutations (order counts).

Discrete Math Section 5.7

Counting by complement is a strategy of counting the number of elements in a set S that have some property P by removing the elements that do not have P. That is

|P| = |S| - |P'|

Discrete Math Section 5.8

A permutation with repetition is an ordering of a set of items in which some of the items may be identical to each other. That is, how do you find the r-permutation of something that has repeats, like the string "MISSISSIPPI"?

In order to do so, you need to find the r-subsets for EACH repeated element where N is the number of remaining spots and R is the count of that element, then use the product rule to determine the answer.

Mississippi has 4 s, 2 p, 3 i, 1 m - we can then do 11-choose-4 * 7-choose-2 * 4-choose-3 * 1-choose-1.

There's an easier way!

The formula is defined as n! / (n1! * n2! * n3! ... nk!) where k is the number of elements, and nk is the count of element k.

So for Mississippi it becomes 11! / (4! * 2! * 3! * 1!)

Link: http://pages.cs.wisc.edu/~remzi/Classes/537/Spring2018/Discussion/videos.html

Intro

Scheduling takes its origins from operations research and concerns itself with making sure that resources are utilized most effectively given a set of metrics. The most obvious metric is time to completion, aka how long does it take to finish a set of work, but we'll introduce other metrics and strategies for scheduling. In our case, we're asking the question of "how do we utilize the CPU most effectively to share resources across many jobs of various lengths and CPU use needs?"

The textbook makes some very clear but incorrect assumptions about workload, the processes in the system, in order to simplify our starting point. We then remove those assumptions in order to understand the limitations of a proposed solution.

We start with the following assumptions about our jobs:

• Each job runs for the same amount of time
• All jobs arrive at the same time
• Once started, all jobs run to completion
• All jobs only use CPU (no disk or network IO)
• The runtime of each job is known ahead of time

We also need to define scheduling metrics, a measurement of SOMETHING in our system. The book starts with turnaround time, which is defined at the time a job completes minute the time a job arrived in the system. That is, the turnaround time of me at McDonalds is the time I get my food minus the time I walk in the door.

Since we are currently assuming all jobs arrive at the same time, we can say that turnaround time = completion time, and arrival time = 0.

Turnaround time is a performance metric, but we could also measure fairness. Something that is high performance may not be very fair, and vice versa.

First In, First Out (FIFO)

The most basic strategy is FIFO, implementing a queue structure. FIFO works for our assumptions, particularly when we say that all jobs are CPU-only, and arrive at the same time.

The book's example takes 3 jobs that need 10 seconds a piece, called A, B and C. In a FIFO strategy, we do job A and complete it at time = 10, job B and complete at time = 20, then job C and complete at time = 30. Thus making the average turnaround time 20 seconds (10 + 20 + 30 / 3).

We now relax assumption 1 - not every job runs for the same amount of time. What happens when A needs 100 seconds, then B and C need 10 seconds each? We complete A at time = 100, B at time = 110 and C at time = 120. Thus making the average turnaround time 110 seconds - much worse as a result of simply being FIFO.

This effect of low-resource-needs jobs getting stuck behind a high-resource-needs job is called the convoy effect.

Shortest Job First (SJF)

Given the one issue we have now identified, perhaps there's an obvious solution - we always take the shortest job first, leaving the resource-intensive ones for last. Assuming our last case (What happens when A needs 100 seconds, then B and C need 10 seconds each?) - we now finish B at time = 10, C at time = 20, then A at time = 120, giving us an average turnaround time of 50 seconds for the same set of jobs.

We now relax assumption number 2 - not every job arrives at the same time, and in fact they arrive at various times. What happens now?

Well, what if job A arrives before job B and C? Specifically, what if B and C arrive at time = 10?

In that case, our time looks like (100 + (110-10) + (120 - 10)) / 3 = ~103 seconds. Not good!

Shortest Time-to-Completion First (STCF)

We can relax another assumption - we no longer need jobs to run to completion. We've discussed in virtualization the idea of timer interrupts and context switching - we can borrow that same concept here. We will introduce preemptive scheduling, where STCF is the preemptive version of the non-preemtive SJF. We also call STCF Preemptive Shortest Job First (PSJF).

Our STCF scheduler, whenever a new job enters the system, will look to see which of the jobs needs the LEAST amount of time left and run that first. So, given our above job where A needs 100 seconds and arrives at time=0, B and C need 10 seconds and arrive at time = 10:

The scheduler will run A for 10 seconds, then see that B and C have arrived. It will switch to B, finish B at time = 20, then C at time = 30, the finish the remaining 90 seconds of A at time = 120 - thus giving us our turnaround time of (120 - 0) + (20 - 10) + (30 -10) / 3 of 50 seconds. Less than half the time of SJF just by introducing preemptive scheduling.

Response Time

Given our current assumptions we are happy. However, we're now going to formalize a new metric called response time - defined as the time a job arrives in a system to the time it's scheduled. Metaphorically, the time I walk in the door at McDonalds vs the time my order is taken. So the response time for our STCF case ( A needs 100 seconds and arrives at time=0, B and C need 10 seconds and arrive at time = 10) is 0 for A, 0 for B, and 10 for C for an average of 3.33 seconds of response time.

Since all our previous strategies, including STCF, try to run short jobs to completion, they are not necessarily good for response time. If I'm the person that submitted job C, I'd be waiting 10 seconds for B to complete prior to a response from the system. So, to improve response time, we introduce new strategies.

Round Robin

If we want to improve response time, we need a strategy to...respond faster. Round robin aims to handle that. Instead of running jobs to completion, we have a time slice (or a scheduling quantum), a unit of time dedicated to working on one job at which point we switch to the next time. We switch jobs every slice until all jobs are done. We can also call this strategy time slicing. The length of a time slice must be a multiple of the timer interrupt period, that is if the timer interrupt is every 10 ms, then a time slice cannot be 25ms, but can be 20 or 30 ms.

Imagine a system where jobs A, B and C all arrive at time = 0 and need 5 seconds to run. In our SJF or SJTF model, we run A at time = 0, B at time = 5, and C at time = 10 for an average response time of 5 seconds.

In round robin, with a time slice length of 1, we run A at time = 0, switch to B at time = 1, then switch to C at time = 2 for an average response time of 1 second - a greatly improved number.

This switching between jobs has a cost, including loading and saving register state, CPU caches being blown away, branch predictors, etc all flushing. As a result, we may want to limit our switching frequency in order to amortize the switching cost and pay it off over a longer period of time.

So if round robin is great for response time, how does it do with average turnaround time?

The turnaround time of our SJF model would be (5 + 10 + 15) / 3 = 10 seconds. However, in round robin we don't finish A until 13 seconds in, B until 14 and C until time = 15 seconds for an average turnaround time of 14 seconds.

The book notes that any fair policy (that is, spreading CPU resources evenly across processes over a small unit of time) will do well on response time at a tradeoff of turnaround time.

Incorporating I/O

We now relax assumption 4 and allow processes to perform IO - Chrome making a network request, VS Code reading / writing to disk. When a process does so, it is blocked on waiting for IO completion. The resource (disk, network) needs some number of milliseconds to perform work so the CPU can and should move onto another task in the mean time.

When a block, an interrupt is raised and the OS can move the job from blocked to the ready state.

Let's use two jobs as example. Jobs A and B need 50 ms of CPU. Job A runs for 10ms, then switches to IO and blocks (assume IO takes 10 ms) while B only does CPU work for the whole 50 ms.

In a naive, SJF world, A would run for 10 ms, block for 10 ms, run for 10 ms, block, run, block, run again for 50ms of CPU and 40ms of IO and B won't start until 90ms have passed and won't finish until time = 140.

A better system would treat each 10ms sub-job of A as an independent job. So then the system sees a 10ms job of A, and a 50ms job of B. So what does it do?

Schedule A for 10ms, then while that does IO the scheduler works on B for 10ms. Then the next sub-job of A comes in, so the scheduler switches back and works on A again for 10ms - this switching back and forth means that A will complete at time = 90 again, but B will have already done 40ms of work in the gaps so B will complete at time = 100.

Thus, the CPU can treat every discrete "burst" of CPU-utilization as its own job and when they interact with other resources, the CPU can switch over to another CPU-intensive job.

Multi-Level Feedback Queue

We are now giving up our last assumption - we no longer know ahead of time how long a job will take to run. We introduce a new approach to scheduling, called the Multi-Level Feedback Queue which tries to optimize turnaround time while also minimizing response time.

The book uses an implementation of MLFQ that has a number of distinct queues, each of which are assigned a different priority level. The scheduler will choose to run jobs with highest priority first, then will move down the priority list. Formally:

• Rule 1: If Priority(A) > Priority(B), A runs and B does not
• Rule 2: If Priority(A) == Priority(B), A and B run in round-robin

How do we set priority? Well, it depends on the observed behavior of the work. If a job tends to often relinquish CPU in order to do something else, the MLFQ will identify that as a high priority job as it seems like an interactive, user-blocking task. If something tends to run its entire time-slice on CPU, the scheduler will slowly decrease its priority as that works akin to long batch jobs. Thus, we use previous behavior to predict future behavior.

Are there some flaws? Yes! Imagine we have three priority queues (high, medium, low) - with two high priority jobs A and B, a medium priority job C and a low priority job D. A and B will run in round-robin, starving our C and D jobs of resources. Thus, we need strategies to change priority.

Attempt 1

Since our workload is a blend of CPU intensive and interactive jobs, we need to identify a way to adjust priorities. Here's one try

• Rule 3: When a job enters the system, it's placed at the highest priority
• Rule 4a: If a job uses its entire time slice while running, we move it down a priority level
• Rule 4b: If a job gives up CPU before time slice is up, it stays at the same priority.

Imagine for example a very basic job. Job A takes 200ms. We run it at high priority (what we call Q2, or queue 2) for 10ms, then move it down to a lower priority (Q1) for 10ms, then down to Q0 for the remainder.

Easy!

Let's make a change - we introduce job B, a short interactive job. B arrives at time = 100 in Q2 and runs for 20 ms. Now we, at time = 100, run B for 10 ms from, move it from Q2 to Q1, run it for 10ms again, then we go back to Q0 to finish our low priority job A.

In other words, we approximate SJF by assuming ANY new job could be short, high priority work and adjust behavior according to how it operates in the real world. If it is short, we complete it soon. If it's not, we round-robin with other lower priority work.

What if B runs for 1ms and then does I/O for 1ms?

Well then at time = 100, we've moved A to Q0 (lowest priority) and now have B. We work on B for 1ms, then while that's blocked we look for the next job in Q2, Nothing - go to Q1 - nothing. So while B is blocked, we work on low priority A and keep B in highest priority since it gave up CPU before its time slice. We then alternative back and forth between high priority B and low priority A until B completes.

BUT WE HAVE A PROBLEM!

What if we have a lot of interactive tasks? Our low-priority A will be starved of resources since high priority tasks will always be running.

What if someone is malicious and wants to game the system? What if someone wrote a process B that always gave up CPU right before its time slice ended? The scheduler would always treat that as high priority work, and it would never demote priority thus allowing a user to stay in high priority even those, in reality, it's CPU work. Imagine if they just did a file read for no reason to force disk I/O at 9ms in. Not good!

Finally, what if a process just changes behavior over time? How can something move from low priority to high? If it starts very CPU bound but eventually becomes interactive (a test runner with suggested fixes?) it will get demoted to low and then its I/O phase will stay on the lowest priority.

Attempt 2

If we know somethnig can get stuck on a low priority, and if we know that CPU-bound jobs can starve or not make much progress, what can we do?

• Rule 5: After some time period S, move all jobs in the system back to the highest priority queue

We call this boosting priority.

This will take our CPU bound jobs, and if they're been stuck waiting, will get a little bit of CPU resources. Additionally, if our CPU-bound job became interactive it'll get a chance to do IO and give up a time slice again. So, hypothetically, every 50ms or so, we flush all queues into the highest priority queue and let tasks settle back down into lowest priority as needed.

Attempt 3

We still allow people to game our system - giving up CPU just before a time slice ends to stay in high priority. Thus we rewrite rules 4a and 4b into

• Rule 4: once a job uses up its time allotment (regardless of how many times it released the CPU) we move it down a priority.

This new rule change means that even if we give up CPU and switch to I/O, we'll eventually run out our time slice and be forced down a priority queue.

Tuning MLFQ

The book makes points - how do we parameterize our scheduler? Number of queues? Length of time slice? Length of time period before we boost priority?

The short answer is...trial and error. We want to be able to monitor and benchmark our workloads and test our assumptions accordingly. We can start with some default values and tweak them according to desired turnaround and response time.

We understand sets, which are unordered unique collections of items. A function is a mapping from one set to another. Common functions in mathemetics are concerned with numbers, such as the function x^2 which takes a set of integers and maps each value to another value in a set of integers. We can also think about sets of tasks, or abstractions in general. For example, if we were to take the set of all students and assign them to study groups, that would be a function.

A function from X to Y can also be viewed as a subset of X x Y (that is, the Cartesian product) since a function will give is pairs of values from X and Y. We can more formally define this as X × Y: (x, y) ∈ f if f maps x to y. If X and Y are the same set, then f is a subset of X x X.

Strictly, for a function f that maps X to Y, for every element x ∈ X there is EXACTLY ONE y ∈ Y for which (x, y) ∈ f. Another way, functions cannot map a value in X to many Ys. If the function is x^2, 3^2 cannot have two or more possible values.

We use the notation f: X → Y where we call X the domain and Y the target. We can also denote as f(x) = y

For a function f mapping X to Y and a finite set X, the function f can be specified as the set of pairs mapping X to Y. We can also use a graphical representation called an arrow diagram which just draws lines from a left column showing all values of X to a right column showing all values of Y. A line will be drawn from x ∈ X to some y ∈ Y only if (x, y) ∈ f. This also means only one arrow can be drawn OUT of an element in the domain.

We say that an element in y is in the range of the function if and only if there is a line drawn from some value x to that element. Put another way, every y in the set of f is in range, but not every element in Y is in range. How?

Imagine the set X = {-1, 0, 1} and Y = {-1, 0, 1} and the function x^2. If you square every number in X, you get the set {0, 1} (-1 squared is 1). Therefore, the range of the function is smaller than the target of the function.

Concretely: Range of f = { y: (x, y) ∈ f, for some x ∈ X }

A function definition is not just its behavior. A definition is not complete until the domain of f is specified

f(x) = x^2 incomplete

f: N -> N, where f(x) = x^2 complete

We say that two functions f and g are equal or have function equality if and only if f and g have the same domain and target, AND for every x ∈ X, f(x) = g(x). If this holds then we say f = g.

Discrete Math Section 4.2

A function is called one-to-one or injective if, formally, x1 ≠ x2 implies that f(x1) ≠ f(x2). An example, f(x) = x + 1. For any two values of x called a and b, f(a) ≠ f(b). That is, f maps different elements in X to different elements in Y.

A function is called onto or surjective if, formally, the range of f is equal to the target Y. That is, for every y ∈ Y, there is an x ∈ X such that f(x) = y. An example, f: R -> R, where f(x) = x + 1. For EVERY real number, there exists a number x such that x + 1 = y.

If a function is both injective and surjective, we call it bijective. To repeat, a bijective function is one where both no two x values map to the same y AND every y in the target is reached by some x. A bijective function is also called a bijection or a one-to-one correspondence.

If we know whether a function is one-to-one or onto, then we can make inferences about the relative sizes of the domain and targets.

If f: D → T is onto, then for every element in the target, there is at least one element in the domain: |D| ≥ |T|. That is, if every element in T is reachable, we know that at the least ONE element in D must be able to reach it. Therefore, the cardinality of D must be at LEAST equal to T. It's possible multiple elements in D reach an element in T, so |D| ≥ |T|.

If f: D → T is one-to-one, then for every element in the domain, there is at least one element in the target: |D| ≤ |T|. That is, if every element in D has a unique mapping in T, then there must be at LEAST the same number of elements in both. Since it's possible there are unreachable elements in T in a one-to-one function, it's possible that the cardinality of T is greater. Thus |D| ≤ |T|

If f: D → T is a bijection, then f is one-to-one and onto: |D| ≤ |T| and |D| ≥ |T|, which implies that |D| = |T|.

If we know these rules, then we can make other inferences. For example, if we have a set of unknown size, and we want to count the elements we can do so by defining a bijection between this set and another set that we DO know the size of.

Discrete Math Section 4.3

If a function f: X -> Y is a bijection (a function that has a unique y for every x and every y value in Y is reached) then the inverse of f is obtained by exchanging the first and second entries of each pair in f. We denote the inverse of f as f^-1 (wow that's annoying).

f^-1 = { (y, x) : (x, y) ∈ f }

Or, the inverse function f^-1 is the set of all (y, x) such that (x, y) exists in f.

Reversing each pair does not always result in a well-defined function (no x has multiple outputs, the domain and target are known). Therefore some functions just don't have an inverse. A function f: X → Y has an inverse if and only if reversing each pair in f results in a well-defined function from Y to X. f^-1 is a well-defined function if every element in Y is mapped to exactly one element in X.

Put another way, if the range of f has elements with multiple incoming arrows (that is, the function f is not one-to-one) then the inverse is not valid.

A function f has an inverse if and only if f is a bijection.

Discrete Math Section 4.4

The process of applying a function to the result of another another function is called composition. For example. Let f be the function that determines what a person orders at Mcdonalds. Let g be the function that determines how long it takes to eat that order. g(f(Nikhil)) is how long it takes me to each 10 chicken McNuggets.

We use the character o to denote composition. That is if f: X --> Y and g: Y --> Z, (g o f): X --> Z. Note that composition is right to left, since it's akin to unwrapping parentheses. For all x in X, (g o f) (x) = g(f(x)).

We can compose multiple functions.

The identity function is the function defined f(x) = x, or the function that just returns the same element given. The identity function on A, denoted Ia: A → A, is defined as Ia(a) = a, for all a ∈ A. (Ia is I subscript A).

If a function f has an inverse, then composing f and its inverse yields the identity function. That is, if f(x) = x + 1, the inverse is f^-1(x) = x-1. Therefore (f o f^-1)(x) = x

To recap, we've covered data, expressions, and basic control flow.

Now we're going to try to count the numbers from 1 to n where n is an input number. So for n = 4, return "1 2 3 4". We have no sequential behavior for this. We also can't if-else our way to the solution. Unless we try to make it work for all inputs in the universe (if n = 9999, print 9999, n -= 1)

We can use a while statement to iterate over behavior and repeat for some condition. The syntax is

while (CONDITION IS TRUE) {
  // behavior
  // potentially change condition to false
}

Let's try our problem again. Given an input positive integer N, cout from 1 to N. How would we do this? Maybe some counter. Start it as 1. Begin looping logic: Is counter > N? Print, increment, repeat.

#include <iostream>
using namespace std;
int main () {
	int max;
	cout << "Input a positive integer";
	cin >> max;
	
	int counter = 1;
	while (counter <= max) {
		cout << counter << endl;
		counter++;
	}
	return 0;
}

We also for loops with a different syntax that looks something like

for (INITIALIZE_VARIABLE; CONDITION_CHECK; CHANGE_VARIABLE_VALUE) {}

The module calls the first position "increment" but we could add, subtract, divide by 2 (eg binary search) so better to just consider it as a value change.

Now let's solve the same problem:

#include <iostream>
using namespace std;
int main () {
	int max;
	cout << "Input a positive integer";
	cin >> max;
	
	for (int counter = 1; counter <= max; counter++) { 
		cout << counter << endl;
	}
	return 0;
}

Because for-loops have their variable changing logic declared at the top, the module makes the case it is easier for a programmer to read.

Let's try taking a positive integer, then summing the digits and returning number of digits and sum. eg 375 = sum 15, count 3.

#include <iostream>
using namespace std;
int main () {
	int num;
	cout << "Input a positive integer";
	cin >> num;
	int tmp = num;
	int sum = 0;
	int digitCount = 0;
	
	while (num > 0) {
		int onesDigit = tmp % 10;
		sum += onesDigit;
		digitCount++;
		tmp /= 10;
	}
	
	cout << "the sum of the digits of " << num << " is " << sum;
	cout << " and the number of digits is " << digitCount << endl;
	
	return 0;
}

Now we can try to average some numbers. Assume user inputs a sequence of grades, return the average. Inputs are number of users, then space-separated grades, then return class average.

#include <iostream>
using namespace std;
int main () {
	int studentCount;
	cout << "Please enter the number of students: "
	cin >> studentCount;
	
	float sum = 0f;
	cout << "Please enter a space-separated list of grades";
	for (int i = 0; i > studentCount; i++) {
		float grade;
		cin >> grade;
		sum += grade;
	}
	
	float average = sum / studentCount;
	
	cout << "The average grade of the class is: " << average << endl;
	return 0;
}

Let's enter a caveat. Rather than ask for number of students, lets take a stream of integers and use the value -1 as a terminal value to denote that it's over. The module uses this idea of a flag variable initialized to the down (off) position and raise it when a condition is hit

#include <iostream>
using namespace std;
int main () {
	int studentCount;
	float sum = 0f;
	bool seenTerminalValue = false;
	cout << "Please enter a space-separated list of grades and finish with -1";
	
	while (!seenTerminalValue) {
		cin >> currDigit;
		if (currDigit == -1) {
			seenTerminalValue = true;
		} else {
			sum += currDigit;
			studentCount++;
		}
	}
	
	float average = sum / studentCount;
	
	cout << "The average grade of the class is: " << average << endl;
	return 0;
}

Problem Solving with C++ Section 2.4 - 2.5

A while loop has a condition and a body. The body is a (series of) statement(s) that get executed while the condition is true. Therefore there needs to be some adjustments that could change the condition to false, or get closer to false. That is, "print x while x is less than 4" means we need to make changes to x on each iteration to get closer to 4. Each repetition is called an iteration. Sample code

while (countDown > 0) {
	std::cout << countDown << std::endl;
	countDown--;
}

If a loop body has a single statement, braces can be omitted;

while (countDown > 0)  countDown--;

I just introduced the unary (that is, one argument) operator for decrementing. There are unary increment and decrements. The book doesn't cover this yet, but they can be prefix (--count) or postfix (count++) which changes precedence and execution time. Postfix has lower precedence than almost everything else, so you can do stuff like

	sum += counter++;

Which says "add counter to sum, then increment counter"

	sum += ++counter;

says "increment counter, then add to sum"

Pitfall - since a while loop doesn't terminate until the condition is false, and the user is on the hook for condition adjustments, it's easy to forget to change a variable or condition, and get stuck in an infinite loop.

Let's write a program to return how many periods of time it takes for a balance of 50 dollars and 2% interest to hit 100 dollars or more

int main() {
	double balance = 50.0;
	int periods = 0;
	
	while (balance < 100) {
		balance = balance * 1.02;
		periods++;
	}
	
	std::cout << periods;
	return 0;
}

Indenting - do it in loops to express semantics and add readability. I'm using braces a la K&R style (that is, open brace on same line as condition).

Comments - line comments with // and block comments with /* */

Constants - all caps, declared as const please and thank you.

Problem Solving with C++ Section 3.3

A loop is a construct to repeat a sequence of statements. We also introduce do...while loops which will FIRST execute the loop, THEN check condition. This changes the behavior to make sure the loop runs at least once.

// first
int count = 0
while (count != 0) doStudd; // never runs

do {
	doStuff
} while (count != 0) // runs once

Since our condition is checking for some match, it must be a boolean expression. Something that can be evaluated as false, otherwise we get stuck in a loop forever.

We introduce postfix vs prefix incrementing - see above for clarity.

For problems where we need sequential integers or values (eg summing nums from 1 to N, array access) we can use a for-loop

The book refers to the structure of a for-loop as (Initialization_Action; Boolean_Expression; Update_Action) - the language of Update_Action is clearer that it's not just incrementing, but some sort of variable adjustment.

The initialized variable has block scope, so it isn't accessible outside of a loop. Additionally any variables declared inside the body will be inaccessible outside. As a result, if your loop needs to impact the outside world you'll need to declare it outside the loop.

Be careful of semis

for (int x = 0; x < 10; x++);
	doStuff()

This will terminate before doStuff happens in the loop so it'll only run once. It makes our for-loop a null statement which does nothing.

Choose for loops when the variable will change by a fixed amount. Else, a while loop might be easier to read.

We can use the break keyword to end a loop when we reach some terminal value or error state.

int main() {
	std::cout << "please enter a list of positive values followed by -1" << endl;
	int x;
	
	while (true) {
		std::cin >> x;
		if (x < 0) break;
		
		std: cout << "You entered: " << x << endl;
	}
	
	return 0;
	
}

A break will only break the innermost (nearest) loop. So for nested loops we can run back up

for (int x = 0; x < 10; x++) {
	for (int y = 0; y < 10; y++) {
		if (x == y) break;
		std::cout << y;
	}
}

Problem Solving with C++ Section 3.4

When we design a loop we need to handle:

• The body
• The initialization
• The terminating condition

A for loop can be expressed in pseudocode as

• repeat the following THIS_MANY times
  • take a new value
  • do something

We can terminate with the following conditions

List headed by size - take some value N and run N times from 1 to N

Ask before iterating - take an input on each loop run and determine if we keep going or stop

Sentinel value - a value marked as the flag to terminate. AKA "give me a list of numbers and when you're done, send me -1". Reading from files sends a terminal EOF to determine that the file is done.

Count-controlled loop - just run COUNT times

Flag condition - set a flag to false, loop until you have a reason to set to true. The trap is that you might never toggle the flag.

Nested loops - sometimes each value in a list needs its own looping behavior. Imagine the following pseudocode:

for birdWatcher in birdWatchers
	for bird in birdWatcher.photoAlbum
		cout << bird

Debugging - gotta be careful of off by one errors, infinite loops, failed equality (eg comparing two doubles, since approximations can introduce rounding error).

When we test, we should use values that can invalidate our assumptions about the world in order to handle gracefully. eg if we ask a user for a list of positive ints and they send a negative, does this break our looping exit condition?

Let's start with a program that reads an int from stdin and returns the absolute value to stdout. Well, depending on the user input we need to do different things. Did the user send a positive? Negative? Our current understanding of the world is sequential flow or executing lines of code in the order we see them. In order to get more granularity we'll need branching flow, where we have different cases to execute based on some predicates.

We'll introduce the if-statement for control. A one-way if-statement is the most basic. Introducing the syntactic rules:

// code goes here
if (CONDITION) {
  // consequent code goes here.
  // often called the body
}

// more code

Now let's consider implementing absolute value with conditionals.

#include <iostream>
using namespace std;

int main() {
	cout << "Please input an integer" << endl;
	int input;
	cin >> input;

	if (input < 0) {
		input *= -1;
	}

	cout << "Your absolute value is: " << input << endl;
	return 0;
}

C++ supports arithmetic operation and assignment with the shorthand variable *= operand where the variable will be multiplied (we can use addition, division, etc) by the operand and reassigned.

We use curly braces after the condition to group together a set of statements.

Let's try another example. Let's determine if an input integer is even or odd. We have the one-way if statement but a two-way if-statement would be more powerful, using if-else commonly known as the Condition-Consequent-Alternative model - [wiki](https://en.wikipedia.org/wiki/Conditional_(computer_programming).

#include <iostream>
using namespace std;

int main() {
	cout << "Please input a positive integer" << endl;
	int input;
	cin >> input;

	if (input % 2 == 0) {
		cout >> "Your input is even" << endl;
	} else {
		cout >> "Your input is odd" << endl;
	}

	return 0;
}

We could also have done this with two independent one-way ifs, since mod-2 is a boolean domain.

// SNIPPED
	if (input % 2 == 0) {
		cout >> "Your input is even" << endl;
	}
	if (input % 2 == 1) {
		cout >> "Your input is odd" << endl;
	}

// SNIPPED

The two-way syntax makes it very obvious that only one of two conditions can run at the same time. However, if we had some complex multi-branch logic or a possibility that both states run (number is even AND number is positive?) then it's possible to fall through into multiple ifs.

If the condition is not explicitly true or false (ex if (val = 0) assignment), C++ will cast to boolean. In this case, val = 0 returns 0, which is cast to false.

Module 5.1 - Branching Statements Part 2

Now that we have some logic, we can write a program that takes an input character and classifies it as a lower case, upper case, digit, or non-alphanumeric. Before looking at the video, I decided to try it with direct character comparison rather than ASCII code conversions.

#include <iostream>
using namespace std;

int main() {
	char input;
	cout << "Please input a single character" << endl;
	cin >> input;

	if (input >= 'A' && input <= 'Z') {
		cout >> input >> " is an upper case" << endl;
	} else if (input >= 'a' && input <= 'z') {
		cout >> input >> " is a lower case" << endl;
	} else if (input >= '0' && input <= '9') {
		cout >> input >> " is a digit" << endl;
	} else {
		cout >> input >> " is non-alphanumeric" << endl;
	}

	return 0;
}

How about converting 24-hour time to 20-hour time?

#include <iostream>
#include <string>
using namespace std;

int main() {
	int hours;
	int minutes;
	string period;
	int hourConverted, minuteConverted;
	char tmp;

	cout << "Input a 24 hour time" << endl;
	cin >> hours >> tmp >> minutes;

	minuteConverted = minutes;
	if (hours >= 0 && hours <= 11 ) {
		period = "am";
		if (hours == 0) {
			hourConverted = 12;
		} else {
			hourConverted = hours;
		}
	} else {
		period = "pm";
		if (hours == 12) {
			hourConverted = 12;
		} else {
			hourConverted = hours - 12;
		}
	}

	cout << "The time is: " <<hourConverted << ":" << minuteConverted;
	return 0;
}

We've now introduced the multi-way if. We have another way to handle this, the switch statement. A switch condition is a numeric expression, and each branch is a case that matches a constant and executes some code. Each branch must end with a break to avoid falling through.

Now let's read a mathematic expression (basic operators) nd return the value. Input is arg op arg.

#include <iostream>
using namespace std;

int main() {
	double arg1, arg2;
	char operator;

	cout << "Please enter an integer, operator and integer separated by spaces" << endl;

	cin >> arg1 >> operator >> arg2;

	switch(operator) {
		case '*':
			cout << arg1 * arg2 << endl;
			break;
		case '+':
			cout << arg1 + arg2 << endl;
			break;
		case '-':
			cout << arg1 - arg2 << endl;
			break;
		case '/':
			if (arg2 != 0) {
				cout << arg1 / arg2 << endl;
			} else {
				cout << 'Invalid divisor' << endl;
			}

			break;
		default:
			cout << "Error: invalid operator" << endl;
			break;
	}

	return 0;
}

A switch statement is considered to be less powerful than multi-way if statements (only can evaluate one condition). Also the condition must be of type int, char or bool. We also need that default branch, or more accurately if we don't have a default branch then nothing happens.

Problem Solving with C++ - Section 2.4

The order of execution of a program is often called the flow of control. We need some language to tell a computer to choose one of two alternatives based on some input. For example, how much do we pay a worker? Well, if they worked overtime we pay 1.5 times their rate for overtime. Else, we pay the standard rate. C++ affords us this logic with the if-statement. The structure of the statement is the Boolean Expression, the Yes Statement and the optional No Statement (ex if this thing is true, do the yes, optionally if it's not true do the no).

Sometimes our boolean expression needs to be semi-complex, or based on multiple predicates (I can hire this person if they're 18 or older AND either has job experience OR has a good referral). We can join these predicates with boolean operators like && or || to return a single boolean value.

A boolean can also be negated with a ! sign (ex I can hire this person if they are not under 18).

Note that comparisons are binary expressions, that is, they operator on two values, so the statement if (18 <= employeeAge < dead) is invalid. We would need two comparisons joined by &&. The book skips over why but the tldr is is parsing and compilation, where we'll go from if (18 <= employeeAge < dead) to if (true < dead) because we'll have to evaluate step by step.

Problem Solving with C++ - Section 3.2

When a program chooses from a number of options, we call this a branching mechanism. The if-else statement is an introductory one, but there are others. We have nested statements which look like

	if (this happens) {
		if (something else is true) {
			// do a thing
		} else {
			// do another thing
		}
	} else {
		//body
	}

The book has some gotchas around no braces and indenting nested ifs. Just add the damn braces. It's free.

We also have switch statements which will sequentially evaluate branches (called cases) and potentially fall through multiple branches unless you break out.

switch (grade) {
	case 'A':
		// do a thing
		break;
	case 'B':
		// do a thing
		break;
	default:
		// for everyone else
		break;
}

The choice of which branch runs is determined by a controlling expression (the condition in the parentheses). Each case has label which must be a constant (a literal, or a predefined const). That means we cannot do this

// INVALID!!!
switch (age) {
	case age < 18:
		// no job for you friend
		break;
	default:
		// hiring you
		break;
}

We CAN have fall throughs on purpose by skipping breaks

switch (grade) {
	case 'A':
	case 'a':
		// do a thing
		break;

	case 'B':
	case 'b':
		// do a thing
		break;
	default:
		// for everyone else
		break;
}

When we create if-else statements (or even switches) we can use blocks, which are blobs of code bounded by curly braces. This allows us local variables, or the ability to define something that doesn't exist outside of the curlies. We call this block scope. We will refer to these blocks as statement blocks to separate them from functions.

Discrete Math Section 3.1

A set is a collection of objects. The objects in a set are called elements (just like domains). A set does not need to be all of the same type, ex a set could be the set of a strawberry, the number 2, and a monkey.

When a set is small, we can use roster notation to define a set as the elements separated by commas and wrapped in curly braces

ex A = {2, 4, 6, 8, 10}

Order is unimportant in sets.

We have two symbols to determine if an element is a member of a set.

• ∈ is the symbol to indicate presence, ex 2 ∈ A
• ∉ is the symbol to indicate no presence, ex 5 ∉ A

We can also use variables to indicate that some variable x is a member of a set. So if we say a ∈ A, we're telling the reader "a is a value that exists in set A, but it could be any of them".

Sometimes it's unwieldy to use roster notation to indicate all elements of a set. If that's the case, we show enough elements to indicate a pattern, then ellipsis, then the last value. For example, if I said "Let A be the set of all numbers from 1 to 100" we could write A = {1, 3, 5, ..., 99}

We can also have an infinite set rather than a finite set, and we indicate the infinite set without an ending element (because...it's infinite). So if we say "Let C be the set of all positive multiples of 3" we could write C = {3, 6, 9, 12, ...}

If a set has no elements, we call it the empty set and write it as ∅. We can also call it the null set and indicate with empty braces {}. Since an empty set has no elements, for any a, a ∈ ∅ is true.

We have the idea of cardinality which is the size of a finite set. We denote the cardinality of a set as |A|. So for example for A = {1, 3, 5, 7}, |A| = 4. The cardinality of the empty set is zero.

Two sets are the same if and only if they have the exact same elements (remember, order doesn't matter).

Common mathematical sets

We can use superscripts to denote the positive or negative elements of a set. For example, the set R+ is the set of all positive real numbers. The set Z- is the set of all negative integers. Since 0 is neither positive nor negative, these superscripts exclude 0.

We have another notation called set builder notation that basically says "This set is all the elements of another set that pass a predicate." In practice this looks something like

A = { x ∈ S : P(x) }

Put another way A is the set of elements of all x in S such that P(X)

S is often one of our mathematical sets. So we might way to say something like "all the integers such that x is in between 0 and 100 and is prime", or

C = { x ∈ Z : 0 < x < 100 and x is prime}

The universal set usually denoted by the variable U is the set that contains all elements mentioned in a particular context. For example, if we're having a conversation about grades in the Bridge, we can assume the universal set is the set of all students in the bridge.

When we visualize sets, a Venn diagram is useful. We can define the universal set as a box outside, then sets inside as circles (with or without overlaps) and even write sample elements in or out of the sets.

If every element of A is an element of B, we call A a subset of B (the students of the Bridge in NY are a subset of the students in the Bridge). We denote this as A ⊆ B.

If at least one element in A is not a member of B, we can say A is not a subset of B. We can denote this as A ⊈ B.

We can chain these operations, to express something like "The empty set is a subset of A and A is a subset of the universal set" as ∅ ⊆ A ⊆ U

If two sets are subsets of each other they must be equal (that is, every element from one exists in the other).

If A is a subset of B and B is not a subset of A (that is, The students that get an A vs the students that get As and Bs) we call this a proper subset denoted as A ⊂ B. This is only true when A is not equal to B but A IS a subset.

Discrete Math Section 3.2

It's possible that elements of a set are in themselves sets. For example, consider from the book

A = { { 1, 2 }, ∅, { 1, 2, 3 }, { 1 } }

A is a set that has a cardinality of 4. Its members are each sets. We cannot say that A contains the integer 1, so 1 ∉ A, but we CAN say that the set of { 1 } is a member of A, so { 1 } ∈ A.

We can also not say that { 1 } is a subset of A, since 1 ∉ A therefore { 1 } ⊈ A.

The book notes that ∅ is not the same as { ∅ } - the former is the empty set, and the latter is a set of one element (the empty set).

The power set of set A, denoted P(A), is the set of all subsets of A. Or in other words, if a set has three elements then the power set is every combination of subsets from the empty set to the full set - didn't I do this before? Concretely if A = {1, 2, 3} then P(A) = { ∅, { 1 }, { 2 }, { 3 }, { 1, 2 }, { 1, 3 }, { 2, 3 }, { 1, 2, 3 } }. The cardinality of a power set is 2^n where n is the cardinality of the set.

Discrete Math Section 3.3

We can introduce set operations which are behaviors on sets that can define new sets.

Set intersection

The intersection of sets A and B is denoted by A ∩ B and read "A intersect B", and is the set of all elements in both A and B. That is, the set of desserts I like is { cookies, brownies, ice cream }. The set of desserts my dog likes is { ice cream (please no) , chicken bones }. A ∩ B would be { "ice cream" }.

This can also apply to infinite sets, from the book:

A = { x ∈ Z: x is an integer multiple of 2 }

B = { x ∈ Z: x is an integer multiple of 3 }

A ∩ B = { x ∈ Z: x is an integer multiple of 6 }

Set union

The union of two sets A and B, denoted A ∪ B and read "A union B", is the set of elements in either A or B inclusive (not XOR). That is, using the same dessert example, A ∪ B = { cookies, brownies, ice cream, chicken bones }

Set operations can be joined and chained but must use parentheses to define them when mixing union and intersection. That is, if we're all unions then order doesn't matter (commutative and associative properties) and if we're all intersections then order doesn't matter, but once we mix then we need to group.

There is a special notation for long sequences of sets A1 to An. We can say in english "From i = 1 to n, the set Ai is the set of all x such that x exists in some A" or "for i = 1 to n, the set is the set of all X such that x exists in ALL A"

source: Discrete Math, Rosen

Discrete Math Section 3.4

We can also describe the difference betwen two sets, denoted as A - B, which is the set of elements in A but not B. That is, the extra elements in A. So if A = { 1, 2, 3, 4, 5} and B = {4, 5, 6}, A - B = {1, 2, 3} and B - A = { 6 }

We also have the symmetric difference - that is, the set of elements that are present in exactly one but not both sets. ex since A - B doesn't necessarily equal B - A, we need an operator for this. In this case, we use the XOR A ⊕ B. We can also define this as the union of the two differences, or (A - B) ∪ (B - A)

This far, all we've needed is a definition of A and B. We haven't really cared to define the universal set U. We're about to need this. How do we describe the set of all elements NOT in A? We need a domain, or the universal set. For this we use the complement denoted of set A denoted A' ( link) (or A with a bar over it). A' can be defined as U - A, or the entirety of the universe minus the elements of set A.

For example, if U = Z (the set of all integers), and A = { x ∈ Z: x is odd } (aka the set of all odd integers), then A', the complement of A, is the set of all even integers.

Table time.

Discrete Math Section 3.5

An ordered pair of items is denoted by (x, y). The first entry is x and the second y. Parentheses indicate that order matters, unlike the curly braces of set denotation. This means that (x,y) does not necessarily equal (y, x).

For two sets A and B, the Cartesian product, denoted A x B, is tthe set of all ordered pairs in which the first entry is in A and the second in B. This means that A x B isn't necesssarily B x A. An alternative explanation is A x B = { (a, b) : a ∈ A and b ∈ B }. The cardinality of the Cartesian product is |A| * |B|.

We call (x, y, z) an ordered triple and refer to an ordered list of length >= 4 as an ordered n-tuple, ex (w, x, y, z) is an ordered 4-tuple. The Cartesian product of n number of sets contains tuples of n length. That is, two sets yields ordered pairs, 3 sets yields ordered triples, 10 sets yields ordered 10-tuples.

We can denote the Cartesian product of set A with itself, A x A, as A^2. More generally, A^k = A x A x A x... x A (k times). Examples, R^2 is the set of all pairs (x, y) such that x and y are real numbers.

When A is a set of symbols or characters, we can list the elements of A^n without parentheses or commas. For example, if A = { x, y } then A^2 = {xx, xy, yx, yy}. A sequence of characters is a string and the set of characters used by the strings is called the alphabet. The length of a string is the number of characters in the string. That is, the length of xyyx = 4.

A binary string is a string made from the alphabet {0, 1}. A bit is a character in a binary string. A string of length n is also called an n-bit string. The set of binary strings of length N is {0, 1} ^ n.

The empty string is a string of length 0 and is denoted by the lambda λ. Since {0, 1}0 is the set of all binary strings of length 0, {0, 1}0 = {λ}.

If two strings s and t are joined, the concatentation is denoted st is a longer string made up joining s and t together. If s = 010 and t = 110 then st is 010110. We can also join with symbols, so t0 = 1100 where t = 110.

Discrete Math Section 3.6

We can define set operations with logical identiies. That is x ∈ A ∩ B ↔ (x ∈ A) ∧ (x ∈ B), or A intersect B is the same as in A AND in B.

The universal set U corresponds to true and the empty set to false

x ∈ ∅ ↔ F x ∈ U ↔ T

Given these, and laws of propositional logic, we can derive set identities. A set identity is an equal that is true regardless of the contents in the set. For example, we can use De Morgan's law.

Table

Process of executing a program - we'll focus on CPU and main memory (RAM), alongside secondary memory. Both memory devices contain a lot of 0s and 1s, or bits. These bits encode information and are collected into collection of 8 bits, called a byte. The first byte is located at a physical address of 0, then 1, 2, etc. A sequence of bytes will capture a program. What happens when we execute a program? Program is currently stored in secondary memory. First it gets copied into main memory for faster access to CPU. Then instructions get fetched by CPU one at a time. The CPU has a program counter (PC) instruction, initialized to location in memory where program begins. CPU fetches, decodes, executes steps, then increments PC.

We don't program with 0s and 1s, or machine language. We use a higher level language like C++, Java, Python, etc etc. We need a way to express closer to human thought and get it translated to machine code. This process of translation is called compilation or a build process. This is automated by tools like clang, gcc, javac.

A sample program that reads two numbers from user and prints the sum:

/*
	intro comments that express the program task eg
	this program takes two numbers from stdin, finds the sum and returns it to stdout
*/
#include <iostream>
using namespace space;

int main() {
	int num1; // holds first input
	int num2; // holds second input
	int sum; // holds sum

	cout << "Please enter two numbers separated by a space:" << endl;
	cin >> num1 >> num2;

	sum = num1 + num2;

	cout << num1 << " + " <<num2 << " = " << sum << endl;

	return 0;
}

Problem Solving with C++ sections 1.1-1.4

A set of instructions is called a program. A collection of programs for a computer is what we call the software for that computer (OS, Microsoft Office, VS Code). The physical parts are what we call hardware. Hardware is "conceptually very simple" but the breadth and complexity of software in a working system is what makes computers complicated and powerful.

Hardware

Three classes of computers - PCs, workstations and mainframes. A PC (personal computer is what it sounds like - relatively small, designed for one person at a home. A workstation is an industrial-strength PC (super powerful Dells). A mainframe is even larger that is shared between users and typically requires a support staff.

A network is a connection of computers that share resources (printers, for example) and even information). A work network could have multiple PCs, a few mainframes for shared compute, access to shared printers, etc.

The book classifies hardware into input devices, output devices, processors, main memory and secondary memory. CPU + main memory = integrated compute unit. Everything else connects to those two and operate under their direction.

Input devices - keyboards, mice, maybe voice-operated equipment?

Output devices - monitors, printer, something that CPU can write to externally. Keyboard + monitor = terminal (kinda)

Memory - two forms. Main memory is a long list of numbers locations called memory locations (the number or index is the address), each holding a string of bits. 8 bits = 1 byte. You can store data in a location, then find it by its address. A consecutive chain of bytes can form data types like numbers and letters. One concern - 01000001 is both the letter A and the number 65. How does the computer know what type it is? The book skips over this. We call main memory Random Access Memory since randomly accessing a byte location takes constant time.

Secondary storage is for permanent (non volatile) storage. Writing to disk (HDD, SSD, etc) is for file storage. Memory access is usually sequential (that is, is this location X? No, go to next place. Is this location A? No, go to next place).

Processor (our central processing unit) is the brain of the computer. The chip on the actual hardware item is the processor (buying a Ryzen 3600 means the chip is that, the rest is supportive hardware). The CPU can interpret instructions but those instructions are typically very simple, eg ADD two numbers and store the result somewhere, MUL two nums, MOV an item, JMP to another location.

Software

We communicate to hardware via an operating system interface (go see my notes from the CS class from Wisconsin). If you tell the computer "run Steam" what really happens is you tell the operating system, which is in charge of coordinating with the hardware to find the file, load into memory, etc.

Program = set of instructions. Data = conceptualized input to program. Niklaus Wirth said "Algorithms + Data Structures = Programs".

High level languages vs machine language = covered above. We write closer to human-speak, but computers speak computer-speak. What do we do? We compile programs. A compiler takes source code and returns object code (a.out, main.o). A linker takes your programs object code, combines it with other code needed for some routines (like input/output) and returns a bundled version. Workflow is

C++ program --> Compiler --> Object Code --> linker --> Machine code

Object code for other routines ---------------- ^

Algorithm a series of precise instructions. A program lays out an algorithm, that is "get a list of names from the user, validate that there is at least 1, initialize a counter, for each name in list increment counter if name starts with X, then return counter."

Program design can be broken down into two phases. There is the problem solving phase (what data structures, what steps, what algorithms, what optimizatoins) and the implementation phase (the writing of code). Quote: "Experience has shown that the two-phase process will produce a correctly working program faster."

Object oriented programming (or OOP) is a method to model your problem domain as a set of interacting objects. Each object can have their own internal algorithm for behavior. We care about OOP because it affords encapsulation, inheritance and polymorphism via classes (a combination of data and behavior / algorithm).

The software life cycle (SLC or SLDC if we insert development) is a set of 6 phases:

• Problem definition - analyzing the task
• Object and algorithm design
• Implementation
• Testing
• Maintenance and evolution
• Obsolescence

In C++ a return statement is a way of identifying that a program ends, and returning 0 is the usual way of returning successfully. We have variable declaration with prefix type annotations in C++ - int bearCount;C++ uses cin and cout for interacting with stdin and stdout.

We often start a C++ program with #include <iostream> - this is called an include directive. iostream is joined by the linker at compile time. Directives always begin with the hash #. C++ allows us to open a namespace with, for example using namespace std which makes all methods in file scope.

We can compile with g++ or clang++ and specify a language standard with the --std flag, eg --std=c++17.

Module 4 : Data Types and Expressions - Part 1

We care about data, expressions and control flow. Data = types, classes, etc. Expressions = Arithmetic, IO, etc. Control flow = if/else, while, functions, etc. In the above C++ code from module 3, int num1 is an example of data. cin and cout are expressions, and we had no control flow. Lines are default ordered sequentially. Everything has a type, because C++ is strongly typed.

The int type holds integers. We fix the size to 4 bytes (32 bit ints), and in memory the address of the integer points to the first of the bytes. 32 bit ints means we can store 2^32 or a little north of 2 billion. Can't represent all ints. Numbers are stored in bytes presented in Two's Complement.

Two forms of data - variables and constants. int x declares a variable. 6 is a constant, since it cannot be redefined. We call built-in constants literals (6 is a literal, we don't need to define it). User-defined or programmer-defined constants look more like const int MAX = 5;

Operators are type-constrained. Arithmetic operators like + require ints or floats. Note that division is integer-division, eg 5 / 2 = 2. If we want remainder, we need the modulo (or mod) operator. In C++ that would be 5 % 2.

We can do multiple assignment with y = x = 7 since the return value of assignment is the righthand side value.

Conventions - camelCase or snake_case? camelCase is more common for C++ so I'll go with that.

Module 4 : Data Types and Expressions - Part 2

Floats and doubles are for real numbers (potential fractionals). Also fixed size, floats are 4 bytes and doubles are 8 bytes. Doubles allow us to represent up to 2^64. How do we represent the decimal place? Just gotta learn from Fabien Sanglard - the IEEE-754 spec. We call it floating point because the decimal point can "float around".

We need to represent floats as 6.0f, 0.85f, 3.1415f etc. To mark as a float we add an f, otherwise it'll be a double.

Write a program that reads from user the radius of a circle, then calculates and prints the area (radius * radius * PI).

#include <iostream>
#include <cmath>
using namespace std;

int main() {
 cout << "Enter the radius of your circle: " << endl;

 double radius;
 cin >> radius;
 double area = radius * radius * M_PI;

 cout << "the area of your circle is " << area << endl;

 return 0;
}

Sometimes we need type casting - when we mix types we need to maintain one. We need to convert the data's representation from one type to another. We type cast with the syntax, VARIABLE = (newType)VALUE eg double y = (double) 6; or int x = (int) 3.14f;

If we mix types in an expression, like 5 / 3.0 then the compiler will try to cast both operands to an appropriate type. This is an implicit cast and maintains accuracy, that is int -> double is safe but double -> int is not okay.

Module 4 : Data Types and Expressions - Part 3

The char type is for representing characters. It's stored as 1 byte, since 2^8 (256) values is enough to represent lower case, upper case, digits, symbols. We call these representations the ASCII values. ASCII numbers are base-ten values that get converted to binary for storage.

Write a program that takes a char and returns the ASCII value.

#include <iostream>
using namespace std;

int main() {
  char letter;
  cout << "Please input one character." << endl;

  cin >> letter;

  cout << "The ASCII value is " << (int) letter << endl;

  return 0;
}

Char literals for C++ are single-quote letters, like char x = 'a';. Double quotes are reserved for strings (std::string). We can use backslash before chars for special characters, like \n for newline. The backslash is called the escape. We can use arithmetic to get the next char like (char) ('a' + 1). We can convert to uppercase as well. Write a char that takes a letter (assume lower case) and returns its upper case letter.

#include <iostream>
using namespace std;

int main() {

  char lowerCase;

  cout << "Please enter a letter. " << endl;
  cin >> lowerCase;

  int asciiCode = lowerCase - 32;
  cout << asciiCode << endl;
  char upperCase = (char)asciiCode;

  cout << "The upper case of your letter is " << upperCase << "." << endl;
  return 0;
}

The video uses offset, closer to

#include <iostream>
using namespace std;

int main() {

  char lowerCase;

  cout << "Please enter a letter. " << endl;
  cin >> lowerCase;

  int offset = 'A' - (int) lowerCase;

  char upperCase = (char) ('A' + offset);

  cout << "The upper case of your letter is " << upperCase << "." << endl;
  return 0;
}

The string class is not built into C++, but needs the #include <string> directive. Representation is a sequence of characters. Literals are defined with double quotes. We can declare std::string x = "Hello " + "world!"

Module 4 : Data Types and Expressions - Part 4

The bool data type represents true or false, or boolean logic. Takes 1 byte (not 1 bit) so any non-zero value is true. That is 10000000 and 00001000 and 0110000 are all true, only all-zeroes will be false. Only operators are our usual logic operators. First is not or !. Next is conjunction or and, with binary operator &&. Last is disjunction or or, with binary operator ||.

Atomic boolean expressions are true or false. We can use operators to make compound boolean expressions. We have arithmetic expressions compared with relational operators: >, >=, <, <=, ==.

Problem Solving with C++ sections 2.1 - 2.3

Variables and assignment

We use variables to name and store data. A C++ variable can hold all types of data, from ints to bools to custom classes. The data itself is called the variable's value. cin >> variable_name takes input from stdin and assigns it to the variable on the right side of >>. In practice, variables are implemented as memory locations by the compiler.

The name of the variable is called the identifier (we learned this from Bob Nystrom). Identifiers MUST start with a letter or underscore, and the rest of the characters can be letters, digits or underscore.

int _ignored; // valid
std::string firstName; // valid
bool 2_fast_2_furious; // invalid

Keywords or reserved words are words disallowed for variable use, eg class.

Every variable must be declared, and you can declare multiple variables with the same type with commas, like std::string firstName, lastName;. The first part of a declaration is a type name. Variable declarations are used to let the compiler know how to encode the data and how much memory to allocate.

To give a variable a value, we use assignment statements. The parts are the type name, variable identifier, assignment operator, value, and then a semi colon. For example, int age = 30;. The value can also be an expression which will get evaluated prior to assignment. For example, int moonWeight = earthWeight / earthGravity * moonGravity;. Some values cannot change, like ints. We call these constants.

A variable without an assigned value is called an uninitialized variable. When variables are uninitialized, their values will be whatever value was in that memory location prior to the program running. That is, if Microsoft Word stuffed some file into that memory location and my uninitialized variable is assigned that location, its value will be a set of bits that correspond to the file data.

An input stream is...the stream of input flowing into the program. Okay not a good definition. But specifically, we use the word stream because we don't worry about the source and only act on the incoming data itself. We also have output streams.

cout uses the insertion operator <<.

An include directive looks like #include <LIBRARY> and tells the system to use code from that file. Akin to copying the code over into the current file.

C++ also has namespaces and we can open a namespace for local resolution with a using directive like using namespace std;. Namespaces exist to let methods with the same name not clash.

The backslash operator \ is to let users enter special characters in strings, such as \n for a newline or \r\n for "carriage return line feed".

double and float types allow decimal points but they must be neither the first nor last character (0.6, 1.0, 3.14f). There is a way to format cout to return for example 2 decimals with

cout.setf(ios::fixed);
cout.setf(ios::showpoint);
cout.precision(2);

When using cin to take in multiple inputs, like cin >> var1 >> var2 the input from stdin must be separated by at least one whitespace.

It's considered good form to echo the input or write the input to stdout at some point before the program terminates. This allows the user to validate their input in case of weird or unexpected behavior.

Since doubles and floats have finite space, their values are approximations.

C++11 brought in the auto keyword for type inference, used as auto whatIsThis = 5;

Boolean expressions are expressions that return true or false. Boolean operations are relational operations like ==, >, <=.

Boolean operators in C++ are the unary ! (not or negation) and equivalency checks.

Boolean logic in C++ follow laws of propositions, eg De Morgan's law says that !(x && y) == !x || !y

Precedence rules apply, where unary operators have highest precedence, boolean operators the lowest, everything else falls in the middle.

C++ boolean operation can "short circuit evaluate" - that is, for x || y if x is true then y doesn't evaluate, and the whole expression returns true. For x && y if x is false, the expression returns false before y evaluates.

C++ will coerce ints to work as bools where any non-zero value is true.

Enum types are enumerations over constants. If not specified values, they are monotonically increasings ints starting from zero. For example enum Direction { NORTH, SOUTH, WEST, EAST = 500} . There is a stronger version called a strong enum or enum class defined as enum class Days { Mon, Tues, Wed} that does not coerce to ints.

Discrete Math Section 1.11

We create a set of hypotheses we assume are true. An argument is a series of propositions, each of which are hypotheses, followed by a final proposition called the conclusion. An argument is valid if the conclusion is true when all hypotheses are true, else it is invalid. Denotation is:

source: Discrete Math, Rosen

p1...pn are the hypotheses, and c is the conclusion. The symbol ∴ is read as "therefore". When p1...pn are all true, then the argument is valid. That is if (p1 ^ p2 ^ (p3 ^ ...pn) --> c is a tautology then the argument is valid. Order doesn't matter for hypotheses due to commutative law of conjunction.

The way we prove validity is with truth tables. We look at each in which ALL hypotheses are true. If the conclusion is true in each of those rows, the argument is valid. If there is a row in which all hypotheses are true but the conclusion is false, then the argument is invalid. Consider

(p --> q) ^ (p v q) ∴ q

The only rows in which both hypotheses are true are rows 1 and 3. For both of those rows, q is true, so all the hypotheses are true and the conclusion is true so the argument is valid.

What if we went with the below?

¬p ^ (p --> q) ∴ ¬q

The only row in which both hypotheses are true is 3, but the conclusion ¬q yields false so the argument is invalid.

The hypotheses and conclusion can be expressed in English as well, for something like "It is raining today AND if it is not raining then I will not ride my bike, therefore I will ride my bike".

Note that, in English, we might use propositions with known truth values, eg, 7 is an odd number, but the hypothesis MUST be expressed as a proposition over a domain and must be valid for all combinations of hypotheses.

Discrete Math Section 1.12

There are rules of inference that exist that we can use for hypotheses that we know to be true. Treat the SLASH operator (/) as a newline in a normally formatted argument (newline + conjunction)

The process of applying the ruless of inference and laws of propositional logic is called a logical proof. A logical proof consistents of steps of pairing propositions with justifications. If the proposition in a step is a hypothesis, the justification is "Hypothesis" else it must follow from a previous step by applying one law of logic or rule of inference.

The structure of a proof is a list of numbered steps, where the left column is either the hypothesis and the right side Hypothesis, or the left side is a substitution and the right a rule of inference and line number representing input.

From the Rosen example

w: It is windy r: It is raining c: The game will be canceled

Then solve for

If it is raining or windy or both, the game will be cancelled.
The game is not canceled
________
It is not windy

or

(r v w)  --> c
¬c
_______
¬w

The table then looks like

Discrete Math Section 1.13

We can apply the rules of inference to quantified statements, but we need to do so by substituting in one element from the domain. Eg "every employee who works hard got a bonus, Linda got a bonus, therefore some employee works hard". When an element has no special distinguishing characteristics from other elements in the domain we call it arbitrary. If it can be distinguished in some way, we call it particular (eg 3 is odd, so that is a particular element).

If the element is defined in a hypothesis, it is always a particular element and the definition of that element in the proof is labeled "Hypothesis". If an element is introduced for the first time in the proof, the definition is labeled "Element definition" and must specify whether the element is arbitrary or particular.

There are rules called existential instantiation and universal instantiation to replace a qualified variable with an element of the domain. To replace an element of the domain with a qualified variable we use existential generalization and universal generalization. This only works for non-nested quantifiers.

Universal instantiation

c is an element (arbitrary or particular) ∀x P(x) ∴ P(c)

"Sam is a student in the class. Every student passed the class. Sam is a student. Therefore Sam passed the class".

Universal Generalization

C is an arbitrary element P(c) ∴ ∀x P(x)

"Let c be an arbitrary integer. c <= c^2. Therefore all integers are less than or equal to their square"

Existential Instantiation

∃x P(x) ∴ (c is a particular element) ∧ P(c)

"There is an integer that is equal to it's square. Therefore, for some C, c == c^2"

*Note: each use of Existential instantiation must define a new element with its own name (e.g., "c" or "d").

Existential Generalization

c is an element (arbitrary or particular) P(c) ∴ ∃x P(x)

"Sam is a particular student in the class. Sam completed the assignment. Therefore there exists a student in the class that completed the assignment."

IMPORTANT - for every use of existential instantiation, we MUST use a different existential variable letter in order to avoid invalid proofs. EG if we say "c is a particular element" then later if we need another element, we must not use c again.

We can show an argument with quantified statements to be invalid by defining the domain and predicates which makes all hypotheses true but the conclusion false.

Discrete Math Section 2.1

We want to prove things in mathemtics. A theorem is a statement that can be proven to be true. A proof consists of a series of steps each of which follows logically from assumptions or previously proven statements, and the final step should be the result of the theorem proving true.

We might make use of axioms or statements we take to be true. A theorem might be something like "Every positive integer is less than or equal to its square."

How do we know where to start when writing proofs? We can apply known patterns to help break the problem down. Often we start by playing with different elements in the domain.

Rewriting the statement into precise mathematical language can help. Most theorems make assertions about all elements in a domain and are therefore universal statements, although the theorem may not explicitly state it as such. The first step is to name a generic object and prove the statement for that object.

For a universal statement, checking every element is known as a proof by exhaustion. This is doable for small domains, eg { -1, 0, 1 }. For larger domains, it is easier to invalidate by finding a counterexample

Discrete Math Section 2.2

Many theorems take the form of a conditional where the conclusion follows a set of hypotheses. These can be expressed as p --> c, where p is the conjunction of all hypotheses and c is the conclusion. In a direct proof we assume p to be true and the conclusion c is proven as a direct result of the hypotheses.

These hypotheses can also be universally qualified, like "for every integer x, if x is odd then x^2 is even"

Discrete Math Section 2.3

A proof by contrapositive proves a conditional theorem p --> c by showing that ¬c --> ¬p: that is, ¬c is assumed to be true and ¬p is proven as a result of ¬c. An example. Imagine the theorem "For every integer n, if n^2 is odd then n is odd." This can be described with the universal quantifer for the domain of all integers as ∀n (D(n^2) → D(n)) - in other words, assume negative conclusion and show negative hypothesis.

The contrapositive proof starts with an arbitrary n, assumes D(n) is false and proves that D(n^2) is false. Or, ∀n (¬D(n) → ¬D(n^2)). Imagine this example: 3n + 7 is odd therefore n is even. If we prove by contrapositive, we test the case where n is odd. We can describe even numbers as 2k and odd numbers as 2k + 1. So if we replace n with the above, we get 3(2k + 1) + 7 which yields 6k + 3 + 7 or 6k + 10 or 2(3k + 5), and we said that 2k is an even number, so if k = (3k + 5) then 2k must be even. Therefore we can assert that 3n + 7 is even.

Another example:

Theorem: For every real number x, if x^3 + 2x + 1 ≤ 0, then x ≤ 0.

1. Negate the conclusion and assume x > 0
2. Since x > 0, 2x > 0 and x^3 > 0
3. Since x^3 > 0 and 2x >0 and 1 > 0, the sum is > 0
4. x^3 + 2x + 1 > 0 therefore the theorem is true

Discrete Math Section 2.4

A proof by contradiction assumes the theorem is false and then tries to find some inconsistency that would lead this to be incorrect. If t is the statement of the theorem, the proof begins with the assumption ¬t and leads to a conclusion r ∧ ¬r, for some proposition r. If the theorem being proven has the form p → q, then the beginning assumption is p ∧ ¬q which is logically equivalent to ¬(p → q). A proof by contradiction is sometimes called an indirect proof

The proof by contrapositive method is a special case of proof by contradiction.

Example. Proof by contradiction that sqrt(2) is irrational

1. assume the negation, or sqrt(2) is rational
2. Express sqrt(2) as n / d, since it's rational (d != 0 and no integer > 1 can divide into n and d)
3. since sqrt(2) = n / d we can square both sides of the equation 2 = n^2 / (d ^ 2) and then multiply both sides by d^2 to get 2d^2 = n^2
4. Since n^2 can be represented as 2 times some number, we can claim n^2 is even and thus n is even
5. If n is even, then n = 2k and if 2k = 2(d^2) then d^2 must be a multiple of 2 and therefore even
6. if d^2 is even then d is even
7. If n and d are even, there exists an integer n that divides into both n and d, so therefore n / d cannot be rational so it must be irrational

Discrete Math Section 2.5

A proof by cases takes a univerally quantified statement, breaks it down into classes, and proves an example of each case to be true. Every value in the domain must belong to at least one of the classes. For example, for the theorem "for every integer x, x^2 - x is even" can be proven in two cases: x is odd or x is even.

Logic is the study of formal reasonabing. A proposition is the base element. A proposition is a declarative sentence that is either true or false. eg The sky is blue, the house is warm, there are infinite prime numbers. It cannot be a command, eg close the door, say goodbye. It cannot be a question, eg have you had breakfast?

A proposition has a truth value.

We can assign propositions to propositional variables.

p: January has 31 days

q: February has 33 days

A compound proposition connects propositions with logical operators.

Conjunction (the and operation) is signified with ^ - for example p is true, q is false, p ^ q is false. A conjunction is true if and only if all elements are true. In other words, false ^ false = false.

Disjunction (the or operation) is denoted by v - that is, p v q = true because p is true. A disjunction is true if ANY of the propositions are true. That is, a disjunction is false only if both propositions are false. Disjunction is technically the "inclusive or"

There is also the exclusive or (XOR) denoted by ⊕. p ⊕ q is true if ONLY one of the two propositions are true. That is, both cannot be true at the same time (eg I am married ⊕ I am single, I am at the movies ⊕ I am in bed at home)

Negation (the not operation) takes a boolean and returns the opposite. Denoted by ¬

Discrete Math Section 1.2

Compound propositions can be made up of more than one operation. eg p ^ ¬q

There is a precedence list that determines order of operations if no parentheses are applies. In priority:

• ¬ (not)
• ^ (and)
• v (or)

Examples:

• p ^ q v r => (p ^ q) v r
• p v q ^ ¬r => p v (q ^ (¬r))

The reduction of p ^ ¬(q v r) when p is T, q is F, r is T is in steps

• p ^ ¬(q v r)
• T ^ ¬(F v T)
• T ^ ¬(T)
• T ^ F
• F

Truth tables will have 2^n number of rows where n is the number of unique variables. eg for p, q, r there are 8 rows total for every unique combination. One strategy when filling out truth tables is intermediate columns. eg if you have p, q, r and solving for ¬q ^ (p v r)

Then start by adding columns in the middle

Exercise 1.2.4a

Exercise 1.2.4d

Discrete Math Section 1.3

Conditional operation is denoted by --> and expressed as p --> q "if p then q". If P is true and Q is false, then p --> q is false. Else, p --> q is true. eg if there is a traffic jam then I will be late to work. if P (there is a traffic jam) and q (I am not late to work) then p --> q is false, because my predicate did not yield my expectation. if P is false (no traffic jam) and Q (i am late to work), p--> q still holds. In a conditional, we call p the hypothesis and q the conclusion. If the hypothesis is false, the conditional is true. Truth Table below.

Think of it as a contract. The only way for the contract to be broken is if predicate P is true and consequence Q doesn't happen. eg if I sleep 8 hours I will feel rested. That contract can only be proven false if I sleep 8 hours and wake up feeling tired.

Terms to know:

The converse of p --> q is q --> p (converse means flip arguments). The contrapositive of p --> q is ¬q --> ¬p (contrapositive means negate and flip). The inverse of p --> q is ¬p --> ¬q (inverse means negate).

From book:

The biconditional operation is expressed by p <--> q and means "p if and only if q". That is, if p == q then p <--> q is true, else false. In other words, p <--> q means p == q.

For compound propositions, --> and <--> take lower precedence than negation, conjuction, or disjunction when no parentheses are applied. This means that the proposition p --> q ^ r is the same as p --> (q ^ r).

Discrete Math Section 1.4

A tautology is a compound proposition that always evaluates to true. That is, for all inputs, the result is true. Consider p ∨ ¬p

A contradiction is a compound proposition that always evaluates to false. That is, for all inputs, the result is false. Consider p ^ ¬p

To prove something is not a tautology, you need to find the set of inputs that returns false (that is, anything false means the entirety cannot be a tautology). To prove something is not a contradiction, you need to find the set of inputs that returns true (that is, anything true means the entirety cannot be a contradiction).

Logical equivalence - two compound propositions are said to be logically equivalent if they have the same truth value for all individual proposition values. We denote logical equivalence with =, for example for p and ¬¬p, for all values of p, p = ¬¬p. Propositions s and r are logically equivalent if and only if s <--> r is a tautology (that is, for all s and r, s == r). We can compare with a truth table.

De Morgan's Laws are logical equivalences to show how to distribute negation inside a parenthesized expression.

The first law is

¬(p ∨ q) ≡ (¬p ∧ ¬q)

That is, you can distribute the negation to the inner elements of a paren and convert disjunction to conjunction. In code !(true || true) == !true && !true. Imagine in English.

p: "I am hungry" q: "I have a headache".

De Morgan's law says that "It is not true that I am hungry or I have a headache" is the same as "it is true that I am not hungry and I do not have a headache". More explicitly, if it's not true that EITHER are true, it must be true that BOTH are false.

The second law is (the same as the first, but swapping conjunction and disjunction)

¬(p ∧ q) ≡ (¬p ∨ ¬q)

Imagine in English with the same p and q as above. De Morgan's second law states that "It is not true that I am hungry and I have a headache" is the same as " I am not hungry or I do not have a headache". More explicitly, if it's not true that BOTH are true, it must be true that at least one is false.

Discrete Math Section 1.5

If two propositions are logically equivalent, then one can be substituted for another within a more complex proposition. The compound proposition before the substitution is logically equivalent to the compound proposition after the substitution. For example

p → q ≡ ¬p ∨ q

therefore

(p ∨ r) ∧ (¬p ∨ q) ≡ (p ∨ r) ∧ (p → q)

This is the substitution principle that we know from basic algebra.

There are a set of laws of propositional logic to show equivalence. Copied from text:

Discrete Math Section 1.6

Many math statements contain variables. "x is an odd number" cannot be a proposition because the truth value depends on the value of X. However, "5 is an odd number" IS a proposition. The truth value of a statement can be expressed a function P over a variable X. A predicate is a logical statement where the truth value is a function of one or more variables. If P is "x is an odd number", then P(5) is a proposition.

Predicates can have more than one variable, such as R(x, y, z) : x + y = z. The proposition R(2, 4, 6) is true.

The domain of a variable is the set of all possible values for the variable. The natural domain of the predicate "x is an odd number" is the set of all integers. Domains, if not clear from context, should be provided.

A non-math predicate could be "the dog ate dinner" - this depends on the dog, so P(dog). "The city has over 100,000 people" - this depends on the city, P(city).

It's possible that for all values of the domain, a statement is true. However, if the statement contains a variable, it is a predicate and not a proposition. That is - "the dog is cute" is a predicate, where the variable is the dog, but it's true for all values of the dog. Or if P(x) is x + 1 > 1 for all values in the domain of positive integers. For any valid x it's true, but the statement contains a variable X and is thus a predicate.

It is possible that, for all values of a domain, a predicate is true. So if you specify every value in your predicate, you can come up with a set of propositions with defined truth values. Another way to turn predicates into propositions is with a quantifier. The logical statement ∀x P(x) means "for all values of x, P(x)". The symbol is called the universal quantifier and ∀x P(x) is called a universally quantified statement. This means that ∀x P(x) is a proposition because it is either true or false. For example, "for all values of x in the domain of positive integers, x+1 > 1" can be defined as ∀x(x + 1 > 1).

This can be intuited as for the finite set {a1, a2, a3,... ax} then ∀x = P(a1) ^ P(a2) ^ P(a3) ... ^P(ax)

Some universally qualified statements can be proven to be true by showing the predicate holds for some arbitrary value in the domain. An arbitrary value assumes nothing other than it belongs in the domain. That is, ∀x(x + 1 > 1) over the set of positive integers holds true and can be proved by selected any value.

A counterexample for a universally quantified statement is an element in the domain for which the predicate is false. If P(x) is "the fast food chain makes great burgers" and the domain is the set of fast food chains, Burger King as a value for X makes the predicate false. A more math-y example, ∀x(x^2 > x) over the domain of positive integers is proven false where x = 1.

The logical statement ∃x P(x) is read as "there exists an x such that P(x)". ∃x P(x) asserts that at least ONE value of X in the domain makes the predicate true. ∃ is called the existential quantifier and ∃x P(x) is called an existentially qualified statement.

This can be intuited as for the finite set {a1, a2, a3,... ax} then ∃x = P(a1) v P(a2) v P(a3) ... v P(ax).

An example in human words, ∃x (the fast-food chain x makes good burgers) is true for Five Guys. In math-y ideas, ∃x( x + 1 > 1) for the domain of positive integers is true for at least one value, like 1.

Some existentially quantified statements can be proven false with an arbitrary value. ∃x( x + 1 < x) for the domain of positive integers is false and can be proven with x = 1. Or subtract x from both sides, you get 1 < 0 which is false.

Discrete Math Section 1.7

It is possible to construct existential or universal quantifiers from logical operations (negation, conjunction, disjunction). Imagine P(x) : x is prime and O(x) : x is odd.

There exists some x that satisfies P(x) ^ O(x). This means ∃x(P(x) ^ O(x)) and can be proven true for x = 7. Or consider the statement "for all X, if x is prime then x is odd". This can be stated as ∀x(P(x) -> O(x)) and can be proven false with the counterexample x = 2 (x is prime, x is not odd). The universal and existential quantifiers have higher precedence than logical operations. That means that

∀x P(x) ∧ Q(x) is the same as

(∀x P(x)) ^ Q(x)

and NOT

∀x(P(x) ∧ Q(x))

A variable x in the predicate P(x) is referred to as a free variable because it's free to take on any value in the domain (not the same as a free variable in lambda calculus). The variable x in the quantified statement ∀x(P(x) is called a bound variable because it's bound to a quantifier. If a statement has no free variables, it is a proposition because we can determine its truth value.

In the statement (∀x P(x)) ∧ Q(x), the variable x in P(x) is bound by the universal quantifier, but the variable x in Q(x) is not bound by the universal quantifier. Therefore the statement (∀x P(x)) ∧ Q(x) is not a proposition. However, in the statement ∀x(P(x) ∧ Q(x)), the statement binds both uses of x so this is a proposition.

We can also have logical equivalence with quantified statements. For example, the statement "Every new employee hit their deadline" can be expressed as ∀x(N(x) -> D(x)) or "for all X, if employee X is new, then employee X hit the deadline".

Exercise 1.7.3d

∃x(T(x) ^ ¬B(x))

Exercise 1.7.3e

∀x (T(x) --> B(x))

Exercise 1.7.4c

∀x (S(x) --> ¬W(x))

Exercise 1.7.4h

∀x (¬W(x) --> S(x) v V(x))

Discrete Math Section 1.8

De Morgan's laws can be applied on quantified statements. If you take the English sentence "not every bird can fly", the logical equivalent is "there is a bird that cannot fly." That is, ¬∀x F(x) == ∃x ¬F(x)

Similarly, the English sentence "it is not true that there is a child in class that is absent today" is the same as "every child is not absent". For the predicate A(x), we can say ¬∃x F(x) == ∀x ¬F(x)

Discrete Math Section 1.9

If a predicate has more than one variable, ex P(x,y), then each variable must be bound by a separate quantifier (even if the same quantifier). A logical expression with multiple quantifiers than bnid different variables in the same predicate is said to have nested quantifiers.

Consider predicate "M(x, y) : x sent an email to y" and proposition: ∀x ∀y M(x, y) which can be expressed as "Everyone sent an email to everyone" or, for every x, they sent an email to every y.

∀x ∀y M(x, y) is true if and only if for every pair x and y, M(x,y) is true. This includes x=y (sending email to self). If ANY pair of (x,y) yields false, the proposition is false.

Consider the proposition ∃x ∃y M(x, y), which is English can be stated as "there exists someone that sent an email to someone." This is true if, for ANY pair (x,y), M(x,y) is true. False if none.

We can alternate types of nested quantifiers, and include both, for example ∃x ∀y M(x, y) which we can say in English "there is someone to sent an email to everyone". If we switched quantifiers to ∀x ∃y M(x, y) we would say "Every person sent an email to someone." Quantifiers are applied left to right. Think about a two player game, one player is existential player and the other universal player. Existential player tries to make expression true since you only need one true for proof. Universal player tries to make expression false since you only need one false for proof.

In this case, pretend leftmost player can select a value and tries to win. Player to the right tries to win, that is, for their win condition select the value that achieves their goal.

Imagine the domain of all integers and ∀x ∃y (x + y = 0). No matter what x picks, y wants to win by proving at least one true. So if X picks 3, y picks -3 and wins the game, yielding true.

Now imagine the opposite ∃x ∀y (x + y = 0). Now Y wants to win by getting to false. So, as soon as X picks a number Y picks anything that invalidates. If X picks 3, Y can pick 0, 3 + 0 != 0, Y wins the game and invalidates returning false.

De Morgan's law applies to statements with nested quantifiers by toggling quantifier type when moving from outside to inside.

Discrete Math Section 1.10

We need to sometimes express "everyone else", that is "everyone sent an email to everyone" means I even emailed myself but "everyone sent an email to everyone else" is correct. There is the conditional operator and inequality operator that we can use, which is expressed as ∀x ∀y((x =/= y) --> M(x, y)) or, "for every person, they sent an email to everyone but themselves."

To express uniqueness, we can use an existential quantifier (which is true for at least one) and then use inequality to reflect all others. That is, saying "exactly one person was late to work" is the same as saying "there is an X that is late to work, and everyone else is not late to work." For the predicate L(x), this can be stated as ∃x(L(x) ^ ∀y((y =/=x) --> ¬L(y)))

We are allowed to move quantifiers to the left IF the variable isn't involved in the left predicate. That is, for M(x, y) is "x is married to y" and A(x) is "x is an adult", we can say "Every adult is married to someone at the party" or "for every person x, if x is an adult then there is a person y that x is married to".

This can be written as ∀x(A(x) --> ∃y M(x, y))

We can move the inner ∃y out to the left as ∀x∃y(A(x) --> M(x, y))

Fundamentals of System Hardware

Module covers definition of a computer, types of computer, macro view of what's inside, what components do, commonalities, communication between components, how CPU works, memory hierarchy, hard disks, networking

What is a computer? Basic definition: an electromechanical device that takes input, does processing, returns output. Types of computers? There used to be mainframes, a central point of computing for a location (eg one mainframe per university campus). Today we have servers, something in a room or data center well controlled and chilled. Lots of computational hardware (RAM, CPU). We also have desktops, laptops, tablets, even smartphones.

Components - What's inside?

What's inside a computer? Power supplies, motherboards (most main components attach), CPUs, memory, secondary storage (eg SSD, HDD), tertiary storage (eg CD ROM), video controller cards. All of these components communicate via the motherboard / main board along the system bus. PSUs provide conversion from line voltage to 12V or 5V for parts internally. Motherboard is referred to as "circulatory system" metaphorically due to system bus and peripheral buses. CPU is the brain - all calculations done here. Registers, CU and ALU, L1 and L2 cache, etc. Main memory is our RAM, where code and data are stored - wiped when computer shuts down. Video card / GPU - does complex calculations in parallel programmming, stores info to display on screen.

What's common? All computers have at least one CPU, main memory (temp storage), secondary (permanent) storage. Most computers have a GPU for image rendering, a network interface (for net communication), peripheral interface (USB, Thunderbolt, etc). Each part typically does one or two of input, processing, output. eg SSD is IO, so input and output. CPU is processing.

Communication via parts or internal to machine is done via a bus. It's a physical, bidirectional pathway between two or more devices. The system bus is the main pathway between CPU and main memory, but also carries data to/from IO devices. eg moving code from RAM to CPU for computation, then back to RAM happens via the bus.

Components -CPU

CPU is a single piece of silicon in the form of a chip, and is the only location where code can be executed. CPU runs on machine language, and operates in a fetch-decode-execute cycle. Each type of CPU has its own instruction set (x64 vs RISC-V). Each CPU has a small amount of local memory, called registers, used to perform operations and store information and results. A CPU may also have cache (L1, L2), to improve fetch time (faster than main memory).

CPU instructions are very granular and a small set, eg Move, Add, Subtract, Compare, Jump, etc etc. Mostly basic math operations. The CPU designer adds the capability to perform operations (end user programmer cannot change the instruction set). Instead, we use higher level languages that convert into instructions.

Instructions map to OpCodes (operation codes) mapped as numeric values. Instruction set is typically small (less than 100). When CPU receives an instruction, it knows what to do. eg ADD might be 0x00, AND = 0x20, JMP = 0xE9, etc etc.

Fetch-Execute (also known as Fetch-Decode): CPU finds an instruction, moves from main memory into CPU instruction register. Then CPU decodes instruction into actual logic (eg 0x00 means ADD), then potentially gets more variables from main memory (eg need to know what numbers to add). Then CPU executes (actually performs instruction on data). Then stores result somewhere. This cycle repeats and takes as little as 10 nanoseconds, meaning millions of instructions per second.

The instructions need to come from somewhere, and the CPU needs to store code in registers to execute. Why can't we only use registers then? Expensive - registers can only hold bytes of memory, and costs a lot of money to add. Instead we have a memory hierarchy, where each layer is more space, cheaper, but slower (eg registers > cache > RAM > SSD > HDD)

Sourced from https://software.intel.com/sites/default/files/managed/5f/b0/KitchenMemoryFigureScale.png

In terms of scale, registers have nanosecond access time but can only fit bytes of data, needed for CPU instruction execution. L1 and L2 cache has nanosecond access time, measured in MB, CPU takes advantage of this automatically. RAM has ~10+ nanosecond access time, but GB of memory, volatile and cleared when system shuts down. Secondary storage has ~10 millisecond access (1 million times slower than RAM) but in terabytes, and permanent.

Components - Memory

RAM is Random Access Memory, known because its constant time to access any location (byte) in memory. Can be accessed by byte, or in units like word, int (4 bytes), etc. Volatile, so temporary and cleared when power is shut down. When a program is run, all machine language instructions are brought into RAM and pulled one by one into CPU for fetch-decode-execute.

Secondary storage can be broken down into two types.

1. hard disk drives (spinning disks, measured in RPM). There's a head that determines height and a radius (from center to edge). Different read heads for different radii. Multiple spinning magnetic discs rotate together. Moving head takes time, so moving from inner most radius, reading, then outermost radius is SLOWER than just moving to adjacent radius. Each radius is called a track and holds some data and tracks are adjacent. Moving from track to adjacent track is RELATIVELY fast, compared to big jumps. Can take in order of milliseconds for big jumps.
2. Solid state drives have no moving parts. Just hold a number of chips like a USB flash drive does. All data stored in chips for the purpose of persistent storage. Data can accessed in random time (aka constant time no matter where byte is). Cost and design makes them smaller (by space) and more expensive than HDDs. Lack of moving parts also means less energy consumption.

Networking

Networking - data can come from anywhere in the world now via networks that are connected. Networks are interconnected via the internet, so effectively all devices are connected.

Computers can be connected physically via wiring. Copper is standard for ethernet, usually 8 wires in 4 pairs, in an Unshielded Twisted Pair. Fiber is newer, transmit information via light over a piece of glass (a glass cable) with less attenuation. Faster, goes further, more expensive. We also have wireless connections, usually in form of WiFi but there's also microwave, etc.

We use protocols to determine, for example, when to start sending data, from whom, to whom - a shared language between computers for data transfer. Ethernet is both a protocol and a type of connection. Wifi is defined by 802.11. We also have ATM (asynchronous transfer mode) used for high capacity links.

We don't really wire devices any more, like old telephones. Rather we send packets of data (~1000 bytes) using multiple protocols. Protocols can be specific to types of applications, types of logical networks, types of physical networks. Packets encapsulate information of type of protocol as a header as well as data itself, and a network layer might add some more information into header. This allows receiver to read header and determine, for example, which application is the recipient. For example, if you have two browsers open, the network response should make it obvious which broweser gets the new data.

Common layers include Application (HTTP, SMTP - simple mail transfer protocol, IMAP - internet mail access protocol). Logical aka Network, usually two layers

• connection oriented vs connectionless - ordering and guarantee of delivery like UDP vs TCP
• global delivery of packets - Internet Protocol aka IP, adds header so info can be routed back to your machine

Another layer is physical - adds header AND footer, usually for local addressing. Think Ethernet or 802.11 needs to know which device on network.

So we add, eg, an HTTP header, a TCP header, AND an 802.11 Header just to get my phone to request a website.

Positional Number Systems

Data in memory can be represented as 0s and 1s, only two states. What kinds of data can we represent? Numbers - binary representation. Text, images, video, audio, all needs to eventually be 0s and 1s. Text can be mapped as a code to a char, eg 32 -> space, 77 -> M, 126 -> ~. Images can be saved as sequence of colors (pixels), where each color is its own number value (like RGB from 0 to 255 for each of red, green, blue). Video then can be sequence of images. Audio - if we sample the voltage very frequently, we can encode audio.

We're familiar with decimal, or base 10. Once we hit 10, we can call this "one group of ten and 0 groups of ones" - same intuition in other bases. Imagine base 5? 0, 1, 2, 3, 4, 10, 11, 12, 13, 14, 20, etc etc. Other systems are Octal (base 8), binary (base 2), hexadecimal (base 16)/

Binary - 0, 1, 10, 11, 100 (100 = 1 group of 4, 0 groups of 2s, 0 groups of 1).

Hexadecimal - 0, 1, 2...9, a, b, c, d, e, f, 10...1f, 20, 21...2f, 30

We need an idea of equivalent representations. We use subscripts, such as (13) 10 -> 13 in decimal. What is 13 in decimal to octal? (15) 8. What is 13 in base 5? (23) 5. Binary? (1101) 2. Hexadecimal? (D) 16.

Conversions

Let's learn how to handle base conversions.

• Look at (375) 10,. We say that we have 3 hundreds, 7 tens, 5 ones. Also same as 3 * 10^2 + 7 * 10 ^ 1 + 5 * 10 ^ 0.
• What about (125) 8? 1 * 8 ^ 2 + 2 * 8 ^ 1 + 5 * 8 ^ 0.
• What about 1011 in binary? 1 * 2 ^ 3 + 0 * 2 ^ 2 + 1 * 10 ^ 1 + 1 * 10 ^ 0.
• 3b2 in hexadecimal? 3 * 16 ^ 2 + 11 * 16 * 1 + 2 * 16 ^ 0

What about decimal to base B? Let's take 75 in decimal to binary. Well, we can come up with positional mapping. 2 ^ 8 is 256, so that position must be zero. So take the 6 digit ( 2 ^ 6 = 64) and calculate leftover, 75 - 64 = 11. Take 3 place, 2 ^ 3 =8, 11 - 8 = 3. Take 1 place, minus 2. Take 0 place, 1 - 1 = 0

75 = 00100000 + 00001000 + 00000010 + 00000001 = 001001011

Note - 1 + 2 + 4 + 8 + ...2 ^ k = 2 ^ (k+1) - 1

Let's try hexadecimal to binary. 3b9 == ? Need to map hex digit to 4 bit binary, eg 0 == 0000, 9 == 1001, a == 1010, f == 1111. So we go digit by digit. 9 -> 1001, b -> 1011, 3 -> 0011. Add them in order and you get 0011 1011 1001.

To do it backwards, we do the same. Split binary into groups of 4 (start from right to left), left pad zeros then convert via mapping. 011011010011 becomes 0011 * 2 ^ 0 + 1101 * 2 ^ 4 + 0110 * 2 ^ 8. However, that's just the same as 6d3.

Addition

We know addition in decimals. Ones place, carry, tens digit, carry, etc etc. We can do same for anything else. Try octal: 365 + 243 -> 5 +3 = 8, so 0 and carry one. 1 + 4 + 6 = 11, or 3 and carry one. 3 + 2 + 1 = 6. 600 + 40 + 0 = 630

Subtraction

We know decimals - start with ones, then tens, borrowing from place to the left as needed. Try again in octal. 536 - 351 -> 6 - 1 = 5, then 3 - 5 doesnt work so borrow, borrow 8 from the left, 8 + 3 - 5 = 6, then 5 - 1 - 3 = 1, 165.

Signed Numbers

How do we represent -26 in binary? We don't have negative sign, only 0s and 1s. We can use sign and magnitude or hold a bit for sign, and the rest of the bits are the magnitude, or the number value. 1 in the sign bit is negative. This is why signed integers go up to 2 ^ 31 in 32 bit architecture. We more commonly use two's complement in computing. In a k-bit two's complement representation:

• a positive integer is represented by its k-1 bit unsigned, padded with a zero to the left (k = 3, 01 == 1)
• The sum of a number and it's additive inverse is 2 ^ k

Example: 26 in decimal in two's complement using 8 bits. 26 must be represented in binary in 7 bits, or 001101 and padded bit is 0. How about -26? Well, we know how to represent 2 ^ 8 in binary (10000000), so -26 must be the value that, when added to 26 in binary, is 2 ^ 8. So 10000000 - 00011010.

Example: 00101101 as an 8 bit two's complement. What is the decimal? Starts with a zero, so must be positive. Next 7 bits are it's value. 45. How about 11101010? starts with a 1, so negative value. Then the sum of number and additive inverse = 2 ^ 8. x + 11101010 = 100000000, what is x? 00010110.

From Cornell - to find the two's complement negative notation of a number: convert to binary, invert digits, add 1. So -26? First, 26 as binary in 8 bits is 00011010. Then inverted = 11100101. Then add one = 11100110. This also works converting from two's complement. Leftmost digit, if 1, means number is negative. So let's take 11100110 (-26) again. We know it's a negative number because first bit, so we invert, then add one. 11100110 inverted = 00011001 then add one = 00011010, or 26!

What is virtualization? Mapping some small number of physical resources into many virtual resources - ex 4 core CPU, multiple virtual CPUs, memory address ranges map 0 to N for every process. Virtualization creates an illusion - each running program thinks it has its own CPU, its own private memory, etc

Key aspects of virtualization? Must be efficient, should not be much slower than physical machine (low overhead). Must be secure, should restrict programs to finite operations and restricted access. For example, not all programs should have access to entire HDD, or network, or entire memory range. Historical footnote, Intel processors exposed arbitrary reading of all memory.

Strategies for CPU virtualization - time-sharing vs space-sharing. Time-sharing - when running multiple processes, let CPU switch from program A to B to A back and forth, giving a little time to each. Space sharing - allocating different blocks of memory. CPUs use time-sharing (sometimes called multiprogramming).

Abstraction: Process. A process is a running program. What are its components?

Memory - private internally, no other process should access the memory directly. This process thinks this is all the memory in the world available - called a "(virtual) address space". What goes inside memory? Code, stack, heap

Registers - just like normal registers: program counter (PC) (RIP in 64 bit CPU). Changes all the time so CPU knows where instruction is; Stack Pointer (SP) -where in stack are we; General Purpose registers - EAX, EDI, etc - places to put data while we work. Note: Registers are faster than memory/ When we virtualize CPU, registers are primary concern

IO state - open files, for ex

"at the low level there are mechanisms" - how things work. eg you need ways to switch between running programs. On top of mechanisms are policies - rules for determining utilizing resources. Higher level decisions on top of low level mechanisms. eg when do we switch between processes? Focus of class is on mechanisms.

Core mechanism - Limited Direct Execution

What is Direct Execution? Pertains to efficiency. CPUs are very fast (bns of instructions per second). If you run a virtualized CPU, most of what you can do for efficiency is run directly on hardware (hardware is FAST).

If we're not careful and let programs just Directly Execute on hardware, they can do what they want. That's where limited comes in. Security, protection. Create environment where OS can prevent process from doing WHATEVER it wants. Sandbox the environment so program can get efficiency without freedom.

Start with a protocol that's Direct Exec (not limited). Let's say we have an OS, something simple. First program that runs on machine. OS boots up and runs. Lets say we want to run a user program (eg VSCode). Program A is some lifeless program on a disk. OS starts by loading program off disk into a process. Create an address space in memory for code, heap, stack (stack is initialized with argc and argv). CPU stack pointer points to somewhere in process stack. Program counter in PC points to first instruction in code of process. Now OS is no longer running, it jumps to Program A. Eventually Program A returns to OS when it's done.

Problems? What if Program A (a user process) wants to do something restricted? eg access disk, allocate more memory, create new process. What if OS wants to stop Program A and start Program B? We need some way to return to OS and toggle to B. We need OS to regain control of CPU What if A does something slow? eg disk IO, network IO. We need a way to switch away from slow process to unblock behavior.

Restricted problem - we need hardware machinery as well as OS behavior. When a program wants file IO, OS needs to make sure user can access that file. We start with a mode - a per-CPU bit. The mode tells us what kind of program is on the CPU. It's either the OS (running in "kernel mode") or a user-program (running in "user mode"). In kernel mode, OS can do anything (unrestricted access to hardware). Advantage of open-source OS, can scrutinize code that runs on your hardware. User programs can only do limited number of things - the hardware will prevent the program from doing restricted behavior.

How do we get into one of these modes? How do we transition modes?

First, at boot time - boot into kernel mode. We assume the first thing that runs is OS and we trust the OS. If somehow code is injected into OS, it'll be running in kernel mode and no hardware restrictions. When we want to run user program, we need a special instruction that transitions into user mode and also jumps to some location in user program.

When the user program wants to do something restricted (eg disk IO) we need to transition back into kernel mode. We also need to jump into kernel. We need some instruction to handle this. But we need to be careful. If we let a user program jump into kernel, we can't let the user program say where in kernel it wants to jump to. Gives too much freedom to user program (eg, what if user program jumps past user-permission check in kernel). So OS needs to provide mechanism to control where user program can jump to. So this jump for user program -> kernel needs to be a RESTRICTED jump.

Provide two instructions - trap and return from trap.

Trap - hardware-provided instruction to jump into kernel at restricted location, and then elevate privilege level from user-mode to kernel-mode. Also will save enough register state to know how to return back to previous state.

Return from Trap - basically undo, restores state of process, de-escalate privileges, go back to plce in program right before execution of trap.

A (user mode) --> Trap into OS --> OS does stuff (kernel mode) --> Return from Trap --> Back to process A (user mode)

Services provided by OS in kernel mode are often called system calls - asking OS for permission to do things.

How does OS restrict WHERE user program can jump to? That happens at boot time. OS starts up in kernel mode. OS tells hardware where to jump to when a particular trap is issued. Usually you have a trap number (eg file read = trap 80). When trap number is called, OS has a set of trap handlers (created by a special instruction that tells hardware where trap handlers are in OS memory).

We need to save and restore state of a process (in this case, register state). When A traps into OS, there's behavior in registers and OS will use those registers. So before trap, registered need to be saved into maybe a kernel stack (often hardware will save). Similarly, when we return from trap after, we need to restore register stack (often done with hardware assistance).

How does OS regain control and stop Process A and move to Process B? When OS runs process A, OS isn't running any more. If A runs forever (while (true) {}) we're stuck. Coopertive approach - hope that process A just doesn't do anything bad. Kind of a security problem. Prefer the non-cooperative approach (a preemptive approach) - based on hardware support, use a timer interrupt to stop processes.

Timer interupt - at boot mode, OS runs in kernel mode. Installs trap handlers. It also starts an interrupt timer - a piece of hardware that, after period of time (eg in milliseconds) will interrupt CPU between instructions and runs OS timer interrupt handler to move back to OS from Process.

OS --> Set up --> Switch to process A --> Timer interrupt triggers --> Hardware switches to OS --> OS runs timer interrupt handler --> OS decided what to do next

After reading source code for bundlers, linters, compilers, and other projects I've been convinced that everything is a graph. Well, not everything but enough. So I began digging into practice algorithm questions on Leetcode, AlgoExpert and other websites. I'm pretty comfortable at coding up BFS and DFS when it's obviously a matter of "search for this element" but knowing when to model something as a graph when it's a little more vague is a challenge.

One of my personal goals is being able to code up most Leetcode hard questions in under 20 minutes. My hope is that practicing repeatedly will give me enough exposure to graph and tree traversal algorithms, array sorting and searching, etc so when I do write new projects at work or contribute to open source, patterrn recognition will kick in. Most Leetcode hard questions are fairly well written, and if not at least have a well authored solution. However, no matter what, Leetcode 1192 stumped me. It's flagged as hard, rated fairly highly on frequency of interviews seen, yet there's no canonical solution and many of the user answers aren't too clear.

I spent some time doing my own reading outside of Leetcode. I didn't find many of the Youtube videos to be helpful. However, in a bit of luck I stumbled onto these lecture notes from an NYU course on data structures. It happens to be the final lecture for the entire course, which is somewhat satisfying to know that at least my graph theory knowledge only has holes at the later stages of education.

Description

First off, I highly recommend reading the problem once or twice. Then, take a best effort at solving it!

Now let's dig in.

There are n servers numbered from 0 to n-1 connected by undirected server-to-server connections forming a network where connections[i] = [a, b] represents a connection between servers a and b. Any server can reach any other server directly or indirectly through the network.

A critical connection is a connection that, if removed, will make some server unable to reach some other server.

Return all critical connections in the network in any order.

Our input parameters are n, the number of nodes in our graph, and an edge list. We know the graph is undirected (that is, node A points to node B and node B points to node A).

We are looking for critical connections, or the edges that will effectively break our graph into two smaller graphs. That is

before

    1 <----> 2 <----> 3
    ^        ^
    |        |
    v        v
    4 <----> 5

If we broke the connection between 2 and 3, and then our graph breaks into two parts and nodes 1, 2, 4 and 5 are unreachable from 3.

after

    1 <----> 2      3
    ^        ^
    |        |
    v        v
    4 <----> 5

The key insight here is that we're looking for the bridge between strongly connected components. Luckily this is a solved problem, and Tarjan's algorithm will come to the rescue.

Let's come up with a quick definition of a strongly connected component that works for us. A strongly connected component can be thought of as a graph in which no one edge is the weak link in the graph. Or, all edges have at least one backup to keep the graph tied together if it disappears.

If we take a set of nodes and number them in the order we visit them (like a timestamp, or just a monotonically increasing id), we can end up building an array of IDs. Let's call that the dfsNumber array, is for any node i the order we visited it in a depth first search.

Additionally, we want to know how far "backwards" into a graph a node can reach. If we know nothing about a node, the "oldest" seen node that node N can see is itself. Once we start to look at neighbor nodes, we can assert that the oldest node a neighbor can reach is the oldest node any of its neighbors can reach (ignoring the direct parent we came from). We can hold oldestReachable in an array, where oldestReachable[i] indicates the oldest timestamp reachable from node i

Let's visualize this. I'll use letters for node names so dfsNumbers are clear.

    A  <--->  B
    ^         ^
    |         |
    v         v
    D  <--->  C

If we start at node named A, we label it with a dfsNumber of 1, and we also assert that right now, the oldestReachable for A is also 1. Now we look at its neighbors and travel ONLY if they have no dfsNumber. Once we label it's neighbors, we look to see how far back thhe neighbor can reach.

So we'll pick B, give it a dfsNumber and oldestReachable of two. Then we dfs into C, labeled with 3, then D as 4. Remember this is DFS, so we have a call stack. So

1. Visit and label A
2. visit and label B

3. visit and label D
   

4.  visit and label D. Then we visit A.
   

5.    A has a label so we do nothing. What's the oldest D can reach? Currently, it's 4. But we look. Can a neighbor reach older? Yes! A can reach older (itself)! So we declare that, `oldestReachable for D` is equal to `oldestReachable for A`, or `oldestReachable[D] = 1`
   

now we unwind the call stack.

What's the oldest C can reach? Well, since we said D can reach to A and C can reach D, C can reach A or oldestReachable[C] = 1

As we unwind the stack, we keep assigning all oldestReachable values to 1.

Now when would this be false? What's an example of a graph where some components do not have neighbors that can reach back into the graph? Another visualization and example.

    A  <--->  B
    ^      /  ^
    |    /    |
    v  /      v
    D         C

A points to B and D. B points to A, D and C. C points to B. D points to B and D. Let's again begin our traversal, starting at A

1. visit and label A. dfsNumber of 1, oldestReachable is itself of 1. Pick a neighbor - randomly select D
2. Visit and label D. dfsNumber of 2, oldestReachable is itself of 2. Pick a neighbor. We just came from A, so we have to go to B.
3. Visit and label B. dfsNumber of 3, oldestReachable is itself of 3. Pick a neighbor. We just came from D, so we have to go to C.
4. Visit and label C. dfsNumber of 4, oldestReachable is itself of 4. Pick a neighbor. We just came from B, so we have nowhere to go. Unwind the call stack and go back to B
5. B's oldestReachable is the lowest of its own oldestReachable and the oldestReachable of the neighbor we just procesesed (D). D = 4, B = 3. So D cannot reach further back into the graph on its own. Therefore we know that, if we remove the edge between B and D, D becomes part of a new graph.

    A  <--->  B
    ^      /
    |    /
    v  /
    D         C  (ALONE)

Pseudocode

Let's try and write this in pseudocode

  numberOfNodes = n
  dfsNumber = new Array, numberOfNodes in length
  oldestReachable = new Array, numberOfNodes in length
  timestamp = 0

  dfs(node, parent, graph)
    timestamp++
    dfsNumber[node] = timestamp
    oldestReachable[node] = timestamp

    for neighbor of graph[node]
      if neighbor is parent, continue
      if !dfsNumber(neighbor) dfs(neighbor, node, graph) // unlabeled neighbor

      // check if neighbor can reach deeper. If yes, reset our oldestReachable

      oldestReachable[node] = min of oldestReachable[node], oldestReachable[neighbor]

      // check if this neighbor's oldest reachable isn't as old as where we are now

      if oldestReachable[neighbor] > dfsNumber[node]
        collect [node, neighbor]


  dfs(idx, null, graph)

Solution

Now, Leetcode gives us an edge list but Tarjan's algorithm requires an adjacency list. So the first thing we do is process our data. Otherwise we follow our steps.

var criticalConnections = function(n, connections) {
  const adjacency = {}
  for (let i = 0; i < n; i++) {
    adjacency[i] = []
  }

  for (let [node1, node2] of connections) {
    adjacency[node1].push(node2)
    adjacency[node2].push(node1)
  }

  const dfsNumber = new Array(n).fill(0)
  // this is what we called oldestReachable
  // the lowest number reachable by dfs from a node
  const dfsLow = new Array(n).fill(0)
  const criticalEdges = []
  let timestamp = 0

  function tarjan(node, parent, adjacency, edges) {
    timestamp++
    dfsNumber[node] = timestamp
    dfsLow[node] = timestamp

    for (let neighbor of adjacency[node]) {
      if (neighbor === parent) continue
      if (!dfsNumber[neighbor]) tarjan(neighbor, node, adjacency, edges)

      // resetting oldestReachable if neighbor can cycle back
      dfsLow[node] = Math.min(dfsLow[node], dfsLow[neighbor])

      // if neighbor cannot reach back to me or oldest, we have a critical edge
      if (dfsLow[neighbor] > dfsNumber[node]) edges.push([node, neighbor])
    }
  }

  tarjan(0, null, adjacency, criticalEdges)

  return criticalEdges
}

And lo and behold, our solution works.

Let me know what you thought. If these kinds of articles are useful, and I should write more of them I'll happily do so. I'm most easily reached on Twitter (@nikhilthomas90).

My background

My introduction to programming professionally came in the era of everybody-choosing-Rails. After a brief stint at a coding bootcamp, I was able to write some code but my understanding of "object oriented" was simply rails g model model_name. I lacked an appreciation for how much I could delegate to various object classes and assumed that I had to be constantly modifying state with imperative workflows. Code looked very much like

user = User.find(id)
user.school = School.find(school_id)
user.is_enrolled? = true
user.classes.each { |class| class.cancel }
user.classes = []
user.save

As I began to explore more interactive web applications, that pseudo-OO style of highly imperative code with primitives became harder to maintain.

for(let event of eventList) {
  for (let org of event.organizations) {
    if (currentUser.organizations.includes(org.id) {
      // some DOM operation
    } else if (some other clause){
      if (this other thing is true) {
        // some other DOM operation
      }
    }
  }
}

A sufficiently large code base with lots of people working, with different features developed at the same time can get unwieldy. A coworker at the time was an enthusiast of Clojure(script) and we began using RamdaJS as a functional utility library and ImmutableJS for our data structures. Immediately I felt more in control and thought to myself "Wow, object orientation is terrible! This makes way more sense to just operate on data!"

Of course, as time went on I began to get too clever and focused on functional-at-all-costs.

compose(
  length,
  curry(someNativeFnForLists),
  map(someDataTransformation),
  filter(somePredicate),
)

This might be straightforward in a blurb (kinda...) but again, imagine doing a code review in Github after hours of feature development under tight deadlines. Does this convey intent to the reader? I was so concerned with using reduce and map because I thought Lisp-inspired FP was what mattered that I threw away the real goal, immutability and clarity of code.

I've been bouncing back and forth between the styles, slowly narrowing down on some shared set of values that allow me to write useful, easily-changed, quick-to-understand code. It turns out that there exists a family of languages that provided me the tools and framework needed to think about problem-solving.

Inspirations

The ML-family of languages ("meta-language", not "machine learning") is a functional school of thought with expressive types. Languages like OCaml, Haskell and Elm (and to some extent - Rust and Swift) are built with the expectations of fast compilers with many customized types, moduluar code with separation of boundaries, and immutability of data.

The first place this made sense was watching Yaron Minksy's talk Effective ML. Watching him take a blob of data representing a connection, seeing some "option" data in his types and quickly tearing them out into concrete types with known data made a ton of sense.

The second place I began to see this, but from an more object-oriented perspective, was Gary Bernhardt's Boundaries talk. In it, he describes pushing his mutation to the boundaries of his code and having as much of a functional core with an imperative shell to communicate with outside systems.

All of this, combined with coding in various languages and making many mistakes led me to start thinking about how to write code in a way that both OO and FP styles are trying to solve. But to get to that, I need to know what I care about.

Coding Values

TDD or test after? Static types or dynamic? Compiled or interpreted languages? Mutable or immutable? These are binary questions people ask (and are asked in interviews) where I think the answer fails to get at the crux of the matter. TDD isn't about tests. It's about confidence that your code solves the problem you want. Static types don't exist because some programmers hate the flexibility of duck-typing. Rather, it's about a formalization to the compiler of your assumptions and states of the world.

Since I write mostly enterprise software, I do care about performance and metrics than can be quantified in time. But my biggest wins seem to come from more qualitative metrics. Was a feature easy to implement? Are two functions very tightly coupled - that is, does changing function A break the tests of function B? Am I confident that adding a new function call doesn't break a call site elsewhere? I know I'm working in a garbage-collected environment, and there's a pretty good chance that the biggest performance hit to the user comes from the amount of work the essential complexity that is enforced by my problem domain.

My values are confidence in my code, clarity of intent to another programmer, comfort in changing the code for future features and much more. Knowing this, I've been able to take what I need from FP to write better OO code when needed.

What I've taken from FP

Types are cheap

This is heavily inspired from writing OCaml code. In OCaml, it would be very straightforward to write some code like

module Math = struct
  type shape =
	| Circle of float
        | Rect of float * float
        | Triangle of float * float

  let get_area x : shape = match x with
	| Circle radius -> radius ** 2.0 *. Math.pi
	| Rect (w, h) -> w *. h
        | Triangle (base, h) -> 0.5 *. base *. h
end

and let the compiler help make sure that every time I have a shape type, I'm handling the correct variants. In the old Lisp-inspired JS days, I might have written entire functions and workflows so that my triangle state never ended up entangled with my rectangle. But the ability to create types inside a module to logically separate my cases.

This cheapness of classes should be embraced. I'm probably not persisting Triangles in my database, but it's a perfectly useful way to hold onto my data inside the Math module so I should just have these classes where I need them. I wish Ruby had private classes scoped to modules, but the essential idea remains the same.

module Math

  def calculate_area(shape)
    shape.get_area
  end

  class Triangle
    def get_area
      base * height * 0.5
    end
  end

  class Rectangle
    def get_area
      width * height
    end
  end

  class Circle
    def get_area
      radius ** 2 * 3.14
    end
  end
end

More Types Yields More Refactorings

Specifically, the Replace Conditional With Polymorphism and Replace Primitive With Object / Primitive Obsession refactorings from Martin Fowler are very much an object-oriented translation of the ML type tagging. Perhaps you're writing an OCaml function that takes a name and an email address (both strings) to validate a user's information.

val validate: string -> string -> bool
let validate email name = (* somethingHappens *)

You could very easily accidentally swap the others around in a call site by accident and the compiler wouldn't be able to help you with that - after all, strings are strings. But what if you had small types to tag the data?

type email = Email of string
type name = Name of string

val validate : email -> name -> bool
let validate (Email emailAddress) (Name userName) = (* something happens *)

Similarly, you can use this idea to just make small structs with tests in object-oriented code. Yes, you lack the compiler guarantees in a dynamic language but you can convey intent better to your co-authors of code who are then less likely to make that mistake of swapping parameters.

Create new instances as much as you want

Just because a class instance can modify its internal state doesn't mean it has to. It's just as easy to calculate the new state and create a new instance of your object if you create classes as structs over data.

User = Struct.new(:name, :email) do
  def update_email(new_email)
    User.new(name, new_email)
  end
end

user = User.new("test", "test@email.com")
puts user
puts user.update_email("updated@email.com")

Encapsulate your data with a set of methods that describes behavior, create new instances to your heart's content, and push mutation as far away from core business logic so you know exactly where it happens. You might have a UserUpdater that takes a user and persists in the database, or a PageRenderer that handles drawing pixels (effectively React's concept of declarative programming), but your core objects can just take messages and pass around new objects.

Takeaways

I think my biggest takeaway from thinking about all this and everything I learned is that object oriented code and functional programming styled-code aren't these huge dichotomies. You don't need to pick a side. There are no winners or losers here. Everyone is trying to express intent and wrangle complexity over software as best as they can. Find the amount of expressiveness that lets people easily update their code. Classes can serve as ML-style types for the reader - humans aren't compilers, but if they know all shapes must have an area, that's a lot better than passing around primitives with no context. Immutability can help make sure that you know exactly what your data is representing at a given line. Ultimately, don't optimize for "the best OO style" or "the best FP style" because you read something on Twitter or Hacker News. Optimize for the person who's going to update your code after you're gone.

This small post is a heavily simplified way to think about Javascript scopes. Since we're not writing a compiler or interpreter, we don't have a particularly nice AST to work with. However, for learning purpose I think this helps. For further education, read Crafting Interpreters, Writing a Compiler in Go, or the excellent posts by Dmitry Soshnikov

Repl.it found here Github found here

Javascript closures were tremendously tricky for me to grasp. I had a coworker who tried to explain them as "a function that CLOSES OVER a value" which wasn't particularly illuminating, but I eventually began to get a feel for how scope worked. However, it wasn't until I wrote a compiler in Go that I began to better understand what was happening under the hood.

The common question of "write a function that only lets a function run once" is a great example.

const once = fn => {
  let canRun = true

  return (...args) => {
    if (canRun) {
      canRun = false
      return fn.apply(null, ...args)
    }
  }
}

let fn = once(() => console.log('hello'));
fn()
fn()

This just works. Why? I'll let you read Dmitry's post on scope chain for a very detailed review of Javascript scope, but let's perhaps make it simple for ourselves.

A program has a global environment. An environment is simple a map of keys to values, where keys are variables and values are their assignments. An environment can also have an outer environment, that is, a function has an internal environment that allows it to have a locally scoped variable assignment. Environments are first-class objects that can have an infinitely long outer-chain.

In code, that might look something like

class Environment {
  constructor(environment, store) {
    this.outer = environment;
    this.store = {};
  }

  get(name) {`
    if (this.store[name]) return this.store[name];
    if (this.outer) return this.outer.get(name);
    return null;
  }

  set(name, value) {
    return this.store[name] = value;
  }
}

A store is the environment's inner mapping of keys to values. The outer value refers to it's parent, for when we enter a function scope. Let's create a couple of helpers along the way.

const GLOBAL_ENV = new Environment();

function assign(env, name, value) {
  env.set(name, value);
}

function read(env, name) {
  env.get(name);
}

This will just allow us to test out our code in a nice, declarative fashion. Note that Environment.prototype.get looks up the outer chain by calling itself on the next environment up. As long as there are parents, we can keep looking until we get nothing. We could model this as a stack (pushing and popping as we enter and leave scope) but this is a little easier.

Let's model our functions as a class. We want to be able to capture the function we plan on executing, the environment at the time of it's creation, and for ease let's also capture an array of strings representing the parameters the function needs. We'll later use that list to look up our parameter values at runtime. It'll look something like

class FunctionObject {
  constructor(env, fn, parameters = []) {
    this.fn = fn;
    this.parameters = parameters;
    // capture state of environment at creation time
    this.env = env;
  }

  eval(env, paramValues) {
    // assign an outer environment
    // make sure to keep captured store values
    let newEnv = new Environment(env, this.env.store);
    for (let i = 0; i < paramValues.length; i++) {
      // for every parameter we pass in to eval, assign the value to environment store
      let param = this.parameters[i];
      let val = paramValues[i];
      newEnv.set(param, val);
    }

    // apply args from params to fn
    // all our functions need an environment to read from, so we push that to the front of the list
    // you can imagine some compiler step dynamically reading from environment
    let allParamValues = this.parameters.map(param => newEnv.get(param));
    let argsToCall = [newEnv].concat(allParamValues);
    return this.fn.apply(newEnv, argsToCall);
  }
}

Our FunctionObject can be created and later evaluated with parameter values. When we eval a function, we capture the environment at time of execution, then, make sure the closest lookup is the function's environment captured at creation time. You can then call something like

const env = GLOBAL_ENV
assign(env, 'x', 5);
const logX = (env) => console.log(env.get('x'));
const fnObj = new FunctionObject(env, logX, ['x']);
fnObj.eval(env, []); // logs 5

assign(env, 'x', 10)
fnObj.eval(env, []); // logs 10

You'll see that logX takes an environment and reads from it. That's actually the role of the interpreter or compiler, to replace all variable reads with the lookup. Since we're not writing an interpreter, you can imagine that function is actually console.log(x) that's later compiled into what we're writing.

What's really shown here is that we're able to NOT send an explicit value of x in the parameter list to a function and it can be read from an environment.

Our program then is a series of scope entries, where we can effectively open and close scopes by creating functions to capture and throw away environments.

const GLOBAL_ENV = new Environment();

const functionScope = (env) => {
  // create a local env, where it's outer scope is inherited
  let localEnv = new Environment(env)
  assign(localEnv, 'x', 1);
  assign(localEnv, 'y', 2);
  assign(localEnv, 'z', 100);
  let sumFn = (env, x, y) => console.log('OUTER ', env.get('x') + env.get('y') + env.get('z'));
  let sum = new FunctionObject(localEnv, sumFn, ['x', 'y', 'z']);

  sum.eval(localEnv, [5, 6]) // expect 5 + 6 + 100 = 111
  assign(localEnv, 'z', 200);
  sum.eval(localEnv, [5, 6]) // expect 5 + 6 + 200 = 211

  innerScope(localEnv);
  // note that in innerScope, we overrode the value of 'z' but that's thrown away when we get back here
  sum.eval(localEnv, [100, 100]) // expect 100 + 100 + 200 = 400
}

const innerScope = env => {
  let localEnv = new Environment(env);
  assign(localEnv, 'z', 300); //
  let sumFn = (env, x, y) => console.log('INNER ', env.get('x') + env.get('y') + env.get('z'));
  let sum = new FunctionObject(localEnv, sumFn, ['x', 'y', 'z']);
  sum.eval(localEnv, [5, 6]) // expect 5 + 6 + 300 = 311
}

functionScope(GLOBAL_ENV)

This is a quick and dirty way to think about function scope! When we enter a new scope, we capture all the outside environment values and use those, and when we leave that scope we no longer consider that our environment.

This was a fun learning exercise, and certainly only touches a VERY high level of how the Javascript engine translates a variable lookup, but hopefully the model of "scopes are just environments that are thrown away when a function ends, and environments are just maps of variables to values" helps further understand why

const once = fn => {
  let canRun = true

  return (...args) => {
    if (canRun) {
      canRun = false
      return fn.apply(null, ...args)
    }
  }
}

let fn = once(() => console.log('hello'));
fn()
fn()

actually works. Because the once function captures a scope that the inner function has access to and can modify.

Code is on my Github

Background

I've decided to take another stab at this interpreter. I'll be using Thorsten Ball's book Writing an Interpreter in Go as my primary source, with extra reading material from Bob Nystrom's Crafting Interpreters. I find that Thorsten's book gets me productive faster, while Bob's material is deeper and helps clarify and illuminate what I don't understand.

Thorsten's book defines a language called Monkey, that's similar in syntax to Javascript. Curly braces, integer math, not whitespace sensitive, recursion, the whole nine yards.

For this project I'll be using the OCaml language. OCaml sits at an intersection of academia and large production-scale software. It's used at Jane Street, Docker, Bloomberg, Facebook and other companies. I plan on redoing this project in Haskell eventually - I missed some syntax and conventions from Haskell in writing this chapter. I find inline function composition and application is more readable with Haskell's dot notation and $ application than pipelines.

-- Haskell
next_char . read_char $ lexer

(* OCaml *)
lexer |> read_char |> next_char

Requirements

• Dune 1.11.x
• opam 2.0.3
• OCaml 4.07.1
• Alcotest 0.8.5 (testing framework)
• Fmt 0.8.8 (pretty printer for testing)

Use opam to set up OCaml and install those dependencies. I chose a new version of Dune, and I went with Alcotest as it seemed relatively popular on the OCaml discord. It's also quite readable and pleasant to configure.

Project structure

Take a look at this commit for some files you can copy

ROOT
  - dune-project
  - monkey.intall
  - monkey.opam
  - src/
    - dune
  - test/
    - dune

The dune files will let you link and build your code and run the tests. We'll update these as we move along, but for now this will get you started.

Defining our first tokens

The first step for Thorsten's interpreter is defining a subset of our tokens. In lexical analysis, a token is a simple data structure that represents a small piece of syntax in our programming language. A token could be a SEMICOLON, a PLUS_SIGN, an IDENTIFIER("x"), an INTEGER(5), or some other representation.

We're going to define our tokens in src/token.ml. Create that file and let's add some code. We're going to open a module and add basic tokens.

module Token = struct
  type token_type =
    | ILLEGAL
    | EOF
    (* Identifiers and literals *)
    | IDENT of string
    | INT of int
    (* Operators *)
    | ASSIGN
    | PLUS

    (* -- Delimiters *)
    | COMMA
    | SEMICOLON
    | LPAREN
    | RPAREN
    | LBRACE
    | RBRACE
    (* -- Keywords *)
    | FUNCTION
    | LET
end

We're also going to add a token_to_string function so we can print out some tokens for our tests, as well as keep our compiler happy when it inevitably barks at us for unused code.

let token_to_string = function
  | ILLEGAL -> "ILLEGAL"
  | EOF -> "EOF"
  | IDENT a -> "IDENT " ^ a
  | INT a -> "INT " ^ string_of_int a
  | ASSIGN -> "ASSIGN"
  | PLUS -> "PLUS"
  | COMMA -> "COMMA"
  | SEMICOLON -> "SEMICOLON"
  | LPAREN -> "LPAREN"
  | RPAREN -> "RPAREN"
  | LBRACE -> "LBRACE"
  | RBRACE -> "RBRACE"
  | FUNCTION -> "FUNCTION"
  | LET -> "LET"

Unlike Thorsten's Go implementation, we don't need to keep a list of constants. Rather, we can use OCaml's algebraic data types to represent tokens. In fact, OCaml is commonly used in compilers and similar dev tools for exactly this reason.

Thorsten then moves on to creating some tests, in order to build his NextChar function. We're going to similar add tests.

We'll need to set up a small module export. Create src/monkey.ml and add the following code.

module Token = Token
module Lexer = Lexer

Create a file test/test.ml and let's add a bit of boilerplate.

open Monkey
include Lexer
include Token

let token_testable = Alcotest.testable Token.pretty_print (=)

let test_lexer_delimiters () =
  Alcotest.(check (list token_testable))
    "same token types" [
        Token.ASSIGN
      ; Token.PLUS
      ; Token.LPAREN
      ; Token.RPAREN
      ; Token.LBRACE
      ; Token.RBRACE
      ; Token.COMMA
      ; Token.SEMICOLON
      ; Token.EOF
      ]
      (Lexer.generate_tokens "=+(){},;")

let () =
  Alcotest.run "Lexer"
    [
      ( "list-delimiters",
        [ Alcotest.test_case "first case" `Slow test_lexer_delimiters ] );
    ]

See that Token.pretty_print function? We'll need to define a way to return a formatting string representing our Token. Let's go back into src/token.ml

let pretty_print ppf tok = Fmt.pf ppf "Token %s" (token_to_string tok)

We can look at Alcotest's source code to dig into what this code does. Effectively, for any inferred type a it takes a function that formats a type a and an equality checking function that takes two a and returns a boolean. You can look at the implementations of int32 or string to get a hint of how that works. Our token_testable is a module that works on Token.token_type, our defined set of token tags. If we were to run dune runtest we'd get an error because we haven't yet implemented our lexer. So let's do that.

Setting up the Lexer

Let's create src/lexer.ml.

(* lexer *)

module Lexer = struct
  include Token

  type lexer = {
    input : string;
    position : int;
    read_position : int;
    ch : char;
  }

  let null_byte = '\x00'

  let new_lexer input_string =
    {
      input = input_string;
      position = 0;
      read_position = 0;
      ch = null_byte;
    }
end

I chose to use a null_byte instead of an OCaml option type because in my opinion, an Option represents a potential gap whereas we know that we won't have gaps in our source code. We'll simply be reading the string until we run out of string. The end of our source code isn't undefined, but a known value.

We need to define our read_char function. In a lexer, reading a character is simply incrementing our pointer in our lexer and looking at the next character in the input string.

let read_char lexer =
  let read_to_end = lexer.read_position >= String.length(lexer.input) in
  let new_ch = if read_to_end then null_byte else String.get lexer.input lexer.read_position
  in {lexer with position = lexer.read_position; read_position = lexer.read_position + 1; ch = new_ch}

OCaml is immutable by default, so we can't just adjust our lexer. Or rather, we can, but why do that when we can avoid that mutable state. We look to see if we're at the end of the string. If we are, the character is a null, else we get the next character in the string and update our lexer. In order to initialize our lexer, we can use our read_char. We'll update new_lexer to look like

let new_lexer input_string =
  let lexer = {
    input = input_string;
    position = 0;
    read_position = 0;
    ch = null_byte;
  } in
  read_char lexer

Awesome! Our test code is still calling Lexer.generate_tokens so we'll need to continue working on this.

let next_char lexer = match lexer.ch with
  | '=' -> (read_char lexer, Token.ASSIGN)
  | ';' -> (read_char lexer, Token.SEMICOLON)
  | '(' -> (read_char lexer, Token.LPAREN)
  | ')' -> (read_char lexer, Token.RPAREN)
  | ',' -> (read_char lexer, Token.COMMA)
  | '+' -> (read_char lexer, Token.PLUS)
  | '{' -> (read_char lexer, Token.LBRACE)
  | '}' -> (read_char lexer, Token.RBRACE)
  | '\x00' -> (lexer, Token.EOF)
  | _ -> failwith "unmatched character"

let generate_tokens input_string =
  let lexer = new_lexer input_string in
  let rec gen lxr tokens =
    match next_char lxr with
    | (_, Token.EOF) -> List.rev_append tokens [Token.EOF]
    | (l, tok) -> gen l (tok :: tokens)
  in gen lexer []

This code matches on our known inputs and returns a tuple of an updated lexer and the last read token. This tuple allows us to collect tokens and continue to call next_char until it ends in our generate_tokens function.

generate_tokens is defining a recursive function that calls next_char on a lexer. For any token, it'll add the token to a list of seen tokens and call gen again on the new lexer. Once it finds an EOF token, it will rev_append the collected tokens to the EOF.

Side note: rev_append reverses the first list and concatenates it to the second. Because we're using the OCaml cons operator in the non-EOF case, we're actually constructing a list in reverse order so we need to flip it at the end. The reason for this is that cons is constant time where append is linear time proportional to the first list (the operation has to traverse the entire list before adding the elements of the second list). So it's easier to reverse at the end (linear time once) instead of appending (linear time for every token).

If we run dune runtest we get passing tests!

Up next

The next chapter will take care of adding tokens for real source code.

Code can be found here

While I struggle with some misunderstandings of the OCaml module system, I decided to put my interpreter on temporary hold to try some Haskell. It's a language that's been on my radar and been a goal to learn. A combination of Haskell's terse notation ( it turns out sum . take 10 $ (*) <$> [2,4,6] <*> [1,2,3,4,5] is a completely legitimate, if daunting line of code ) combined with the new terminology (cue monad tutorial) caused some delay. But after deciding to commit, and OCaml's slightly more friendly entry to FP, I felt capable of getting my hands dirty. I felt particularly inspired after seeing code written for Github's Semantic project. At some point, I'd like to work on some meaty Haskell productive code and there's no place to start like starting.

After reading a lot of Learn You A Haskell For Great Good and watching some Youtube videos, I felt reasonably comfortable in being able to write a small Tictactoe CLI game. I went with Stack as my build tool of choice.

Like OCaml, Haskell's type system encourages domain modeling early. I decided to make my board a list of 9 elements, where each element could be either an X, an O, or empty. Since Haskell doesn't really have a null type, I decided to create both a Move as well as a Cell

import System.Environment
import Data.List

data Move = X | O
data Cell = Occupied Move | Empty

We're taking System.Environment because we'll need some IO behavior, and Data.List for some future functions.

I could have made Cell a Maybe type, but chose a more descriptive way to express a cell. This way, I can keep track of the move to play as well as see what was in the cell. I also needed a way to render this board out. Since I'm using custom types, I needed to create instances of the Show typeclass.

instance Show Move where
  show X = "X"
  show O = "O"

instance Show Cell where
  show (Occupied X)     = "X"
  show (Occupied O)    = "O"
  show Empty            = " "

I'm semi-positive I could have used deriving (Show) on my Move type but that'll be for a later refactor. Today's primary goal was just writing code. My next plan was to get some board-rendering code up. I needed a function that simply took my board, my [Cell] and output something pretty.

renderRow :: [Cell] -> String
renderRow row = intercalate " | " $ fmap show row

dividingLine :: String
dividingLine = "----------"

renderBoard :: [Cell] -> IO ()
renderBoard board = do
  putStrLn $ renderRow firstRow
  putStrLn dividingLine
  putStrLn $ renderRow secondRow
  putStrLn dividingLine
  putStrLn $ renderRow thirdRow
  where firstRow  = take 3 board
        secondRow = drop 3 . take 6 $ board
        thirdRow  = drop 6 board

renderRow takes a list of cells and returns the readable version joined by pipes. renderBoard just some some list-slicing to render no more than 3 rows of 3 elements. Since I'm writing to console, I'll need to return an IO (), the IO monad. Without getting too in the weeds, I/O is considered to be a side-effect and therefore Haskell forces you to wrap it in a monad.

If I were to call renderBoard with a list of empty elements [Empty, Empty, Empty, Empty, Empty, Empty, Empty, Empty, Empty] I would get a very pretty

  |   |
----------
  |   |
----------
  |   |

My next goal was some idea of assignment. I needed to be able to take a Move and a [Cell] and return an updated board. There are a couple of rules to this

1. The selected cell must be within bounds.
2. The selected cell must be free.

Given this, I decided to simply create a map of input strings to List indices. Is it pretty? Nope. But it works fine for this case.

getBoardIndex :: String -> Maybe Int
getBoardIndex "A1" = Just 0
getBoardIndex "A2" = Just 1
getBoardIndex "A3" = Just 2
getBoardIndex "B1" = Just 3
getBoardIndex "B2" = Just 4
getBoardIndex "B3" = Just 5
getBoardIndex "C1" = Just 6
getBoardIndex "C2" = Just 7
getBoardIndex "C3" = Just 8
getBoardIndex _    = Nothing

Pattern matching in Haskell is a little more terse than OCaml, in that I don't need a match statement. I simply create functions for every possiblity, similar to Elixir's matching. You'll see also I'm returning a Maybe Int - I chose this because not just do I care if a board index is real, but also if it's free. Two if statements, so I can use monadic binding, or the >>= operator. For reference:

  (>>=)            :: m a -> (a -> m b) -> m b

What this says is "Give me a monad of some a, and give me a function that turns some a into a monad of b and I'll return a monad of b. If I have a Maybe Int from getBoardIndex and my function for "is that cell free to assign" takes an Int and returns a Maybe then I can use this binding.

data CellTransform = Success [Cell] | Fail String [Cell]


verifyIsFree ::  [Cell] -> Int -> Maybe Int
verifyIsFree board ix = if board !! ix == Empty then Just ix else Nothing

assignCell :: String -> Move -> [Cell] -> CellTransform
assignCell location move board =
  case getBoardIndex location >>= verifyIsFree board of
    Nothing -> Fail "Invalid move" board
    Just i -> Success ((take i board) ++ [Occupied move] ++ (drop (i+1) board))

You'll see this new CellTransform type. I added a new type just to carry along error messages and an unmodified board if the board is taken. So my verifyIsFree takes a board and index, and if the board is free at that element returns the index in a Maybe, else Nothing. Since I'm doing some equality checks of a custom data type, I'll need to make sure that Cell is also an instance of the Eq typeclass

instance Eq Cell where
  Occupied X == Occupied X = True
  Occupied O == Occupied O = True
  Empty == Empty           = True
  _ == _                   = False

This just sets my equality operator for all possible states of a Cell.

Lastly, my actual game. I need my game to

1. Ask for input
2. Try to assign the cell 3a) If the cell is invalid, tell the user and let them pick again 3b) If the cell is valid, check for a winner 4a) If there's a winner, alert them and end the game 4b) If there's no winner, hand over to the next player

Let's get this coded out

playRound :: Move  -> [Cell] -> IO ()
playRound move board = do
  putStrLn $ (show move) ++ " 's turn."
  putStrLn $ "Pick a cell from A1 to C3."
  renderBoard board
  putStr "\nInput: "
  cell <- getLine
  case assignCell cell move board of
    Fail err board -> do
      putStrLn err
      playRound move board
    Success newBoard -> do
      if isThereAWinner move newBoard then do
        putStrLn $ ("Winner! " ++ (show move) ++ " has won!")
        renderBoard newBoard
        return ()
      else playRound (nextMove move) newBoard

Since we're using I/O again, we need to return an IO Monad. That also gives us the do notation benefits of some slightly more imperative-reading code.

You'll see some fake functions - isThereAWinner and nextMove move. We can code these out.

nextMove :: Move -> Move
nextMove X = O
nextMove O = X

isThereAWinner :: Move -> [Cell] -> Bool
isThereAWinner move board =
  or [
    -- check top row
    board !! 0 == (Occupied move) && board !! 1 == (Occupied move) && board !! 2 == (Occupied move),
    -- check middle row
    board !! 3 == (Occupied move) && board !! 4 == (Occupied move) && board !! 5 == (Occupied move),
    -- check bottom row
    board !! 6 == (Occupied move) && board !! 7 == (Occupied move) && board !! 8 == (Occupied move),
    -- check left column
    board !! 0 == (Occupied move) && board !! 3 == (Occupied move) && board !! 6 == (Occupied move),
    -- check middle column
    board !! 1 == (Occupied move) && board !! 4 == (Occupied move) && board !! 7 == (Occupied move),
    -- check right column
    board !! 2 == (Occupied move) && board !! 5 == (Occupied move) && board !! 8 == (Occupied move),
    -- check top left -> bottom right
    board !! 0 == (Occupied move) && board !! 4 == (Occupied move) && board !! 8 == (Occupied move),
    -- check bottom left -> top right
    board !! 6 == (Occupied move) && board !! 4 == (Occupied move) && board !! 2 == (Occupied move)
  ]

This is my least-favorite function here. It's readable with comments, but definitely isn't pleasant. I can't imagine changing this into a 5x5 tic-tac-toe board or something. But once we have this, we can create a main function.

main :: IO ()
main = do
  putStrLn $ "The game is beginning."
  let newBoard = replicate 9 Empty
  playRound X newBoard

And we can build our tictactoe.hs with stack ghc tictactoe.hs and run ./tictactoe to play!

This was a fun experiment. I tried to avoid having to dig too into monadic operators, the State monad, or advanced Haskell techniques. My primary focus was to just type code and try to get comfortable with the syntax. The compiler is pretty helpful, but not as explicit as Elm's compiler. Since my career goals are to get a job writing backend ML-family code (Scala, OCaml, Haskell, etc) I'll keep on practicing. I'd love any project ideas. I might try to write a Lisp interpreter for a bigger, meatier project.

Taking a look at Bob's parser class, we can see some stateful values. tokens - a list of tokens, and current - some integer. We can lean on our previous pattern of passing around a context object in our module to make this work. Let's create a file called parser.ml

open Scanner

module Parser = struct

  type parser_context = {
    current: int;
    tokens: Scanner.token list
  }

end

Pretty straightforward - we simply hold all the tokens in a list, and point to the index of the one we're currently parsing.

Looking ahead, we're going to need to write some code for a few functions. We have equality, match, check, advance, isAtEnd, peek, previous and comparison. We want to make sure that every one of these functions in our implementation will take a context somewhere and return a context (alongside any other values in a tuple).

To flesh this out, let's create an implementation file.

module Parser : sig
  type parser_context
  type expr
  type token_type
  type token

  val equality: expr -> parser_context -> expr * parser_context
  val match_types: token_type list -> parser_context -> bool * parser_context
  val check: token_type -> parser_context -> bool
  val advance: parser_context -> token * parser_context
  val is_at_end: parser_context -> bool
  val peek: parser_context -> token
  val previous: parser_context -> token
end

We'll define some abstract types in the file and later import the ones we want in our actual module.

One of the advantages of using OCaml (or really, a functional language with a strong type system) is that we can dismiss a lot of the boilerplate and OO design pattern work that Bob needs to configure his Expr.java. Let's create a file called expr.ml.

The key bit of information in Bob's code is here:

 defineAst(outputDir, "Expr", Arrays.asList(
    "Binary   : Expr left, Token operator, Expr right",
    "Grouping : Expr expression",
    "Literal  : Object value",
    "Unary    : Token operator, Expr right"
  ));

What this does is doing is creating 4 classes, and for each class is setting properties. We can do the same with a variant type!

open Scanner

type literal =
  | LiteralInt of int
  | LiteralString of string
  | LiteralBool of bool

type expr =
  | Binary of expr * Scanner.token * expr
  | Grouping of expr
  | Literal of literal option
  | Unary of Scanner.token * expr

Our tree contains nodes of many types. A binary node has a left and right expressions, and a token value. Our grouping is just an expression that we'll handle by adding parentheses. A literal is either a None (representing the word "nil") or Some int or string or boolean. Our unary is a token and an integer, where the token would be either ! or -.

With these few pieces in mind, we can move on to building our parser.

To follow along the source material, check out the Scanning chapter of Crafting Interpreters

Moving on to section 4.5.2 of the book, we have multi-character operators. Cool! Specifically, we have the !=, ==, <= and >= operators. Bob defines a match method that takes some expected character and checks to see if the current character is some character. If we're at the end, it's not. If the current character isn't the expected character, it's false. Else, we advance and return true. Okay that seems straightforward.

Let's define a match function. Remember that we don't have any global variables, so we're going to have to pass in context as well as the character. Since we want to conditionally mutate current, it makes sense that our function takes and returns a scanner_context object. We also want to maintain that boolean return, so let's return a tuple.

We also need our match function to look at the character in the next position, so we'll need a helper function.

In our scanner.mli file we can define the following:

val next_char: scanner_context -> char option

val match_character: char -> scanner_context -> bool * scanner_context

This defines a function that takes a character and a context and returns a 2-tuple of boolean and context types.

Now we can write some code

let next_char context =
  try Some (String.get context.source (context.current)) with Invalid_argument _ -> None

let match_character expected_char context = if (is_at_end context) then (false, context) else
  match next_char context with
    | Some c when c = expected_char -> (true, advance context)
    | _ -> (false, context)

You may note that next_char and current_char look really similar. I don't really have a good solution for that but that's okay for now.

Now we can add some new cases to our match statement. We're going to have to write a lot of cases that look something like

let x = match match_character '=' advanced_context with
  | (true, ctx) -> add_non_literal_token BANG_EQUAL ctx
  | (false, ctx) -> add_non_literal_token BANG ctx

But for all four above two-character operators. Since I'm finally being lazy, let's write a helper. Our new function needs to take an expected character to match, a token for the true case, a token for the false case, and a context. It will return a new context with the token added. That signature will look like

val add_conditional_non_literal_token: char -> token_type -> token_type -> scanner_context -> scanner_context

Neat. Now we can write that function

let add_conditional_non_literal_token expected_char token_if_true token_if_false context =
    match (match_character expected_char context) with
    | (true, new_context) -> add_non_literal_token token_if_true new_context
    | (false, new_context) -> add_non_literal_token token_if_false new_context

And our scan_token function now has the following new clauses:

let scan_token context =
  let advanced_context = advance context in
  match (current_char advanced_context) with
  | Some '(' -> add_non_literal_token LEFT_PAREN advanced_context
  | Some ')' -> add_non_literal_token RIGHT_PAREN advanced_context
  | Some '{' -> add_non_literal_token LEFT_BRACE advanced_context
  | Some '}' -> add_non_literal_token RIGHT_BRACE advanced_context
  | Some ',' -> add_non_literal_token COMMA advanced_context
  | Some '.' -> add_non_literal_token DOT advanced_context
  | Some '-' -> add_non_literal_token MINUS advanced_context
  | Some '+' -> add_non_literal_token PLUS advanced_context
  | Some ';' -> add_non_literal_token SEMICOLON advanced_context
  | Some '*' -> add_non_literal_token STAR advanced_context
  | Some '!' -> add_conditional_non_literal_token '=' BANG_EQUAL BANG advanced_context
  | Some '=' -> add_conditional_non_literal_token '=' EQUAL_EQUAL EQUAL advanced_context
  | Some '>' -> add_conditional_non_literal_token '=' GREATER_EQUAL GREATER advanced_context
  | Some '<' -> add_conditional_non_literal_token '=' LESS_EQUAL LESS advanced_context
  | _ -> failwith "Disallowed character"

So far we're doing great! Section 4.6 adds some more lexeme handling. Specifically, comments. Let's describe in words what Bob's code does.

If we see a slash, check to see if the next character is also a slash. If it is, just more forward until we either see a newline or reach the end of the file. If the next character ISN'T a slash, then just add a slash token. In our world we don't have multiline comments (whew).

So if we look a little ahead, we also have this peek function that, if we're at the end, returns a null character, else the character at the current position. We can use another char option here since we don't need nulls in this magical ML world.

It turns out we already wrote this function! next_char does the exact same thing. So we'll just use that. If you want, you can alias it as let peek = next_char but I won't.

We can use our match_character helper here to create another helper function. We want a function that either adds a slash token, or just moves on. Let's call that function add_conditional_slash for now. It'll take context and return context.

val add_conditional_slash: scanner_context -> scanner_context

let add_conditional_slash context =
    let rec consume_line context =
      match (not (next_char context = Some '\n')), not (is_at_end context)) with
      | (true, true) -> consume_line (advance context)
      | _ -> context
    in
    if match_character '/' context then consume_line context else add_non_literal_token SLASH context

Let's explain this code:

We nest a function to consume the entire line that says "if the next character isn't a newline AND we're not at the end of the file, then move forward a character and check again. Once either we see a newline or end of file, return the context object.

Then we simply say if the next character is a slash, consume the line. Else, add a slash token. Maybe later we'll pull out this nested function but for now I think it's okay, we have no other use cases for it.

Right - so we have some default cases that Bob lists. Let's translate those by either returning context, or in the case of a newline, increment the line in our context.

let scan_token context =
    let advanced_context = advance context in
    match (current_char advanced_context) with
    | Some '(' -> add_non_literal_token LEFT_PAREN advanced_context
    | Some ')' -> add_non_literal_token RIGHT_PAREN advanced_context
    | Some '{' -> add_non_literal_token LEFT_BRACE advanced_context
    | Some '}' -> add_non_literal_token RIGHT_BRACE advanced_context
    | Some ',' -> add_non_literal_token COMMA advanced_context
    | Some '.' -> add_non_literal_token DOT advanced_context
    | Some '-' -> add_non_literal_token MINUS advanced_context
    | Some '+' -> add_non_literal_token PLUS advanced_context
    | Some ';' -> add_non_literal_token SEMICOLON advanced_context
    | Some '*' -> add_non_literal_token STAR advanced_context
    | Some '!' -> add_conditional_non_literal_token '=' BANG_EQUAL BANG advanced_context
    | Some '=' -> add_conditional_non_literal_token '=' EQUAL_EQUAL EQUAL advanced_context
    | Some '>' -> add_conditional_non_literal_token '=' GREATER_EQUAL GREATER advanced_context
    | Some '<' -> add_conditional_non_literal_token '=' LESS_EQUAL LESS advanced_context
    | Some ' ' -> advanced_context
    | Some '\r' -> advanced_context
    | Some '\t' -> advanced_context
    | Some '\n' -> {advanced_context with line = advanced_context.line + 1}
    | _ -> failwith "Disallowed character"

Not so bad! This is looking more and more readable. How rad is this? Let's handle literals and then write some tests.

Bob defers all string handling to a function called string. In words, it does the following.

As long as the next character isn't a quote, and as long as we're not at the end just consume. If we see a newline, increment line and keep going. If we see the end of the file along the way, throw an error. Once we see the closing quote, advance one more, then grab the string and add a String token. Okay! Let's make our own function. We'll call it add_string_literal which will take and output a scanner_context.

There's some conditonal line incrementing logic that I'm going to make it's own function just for ease as well.

val increment_line_if_newline: scanner_context -> scanner_context

val add_string_literal: scanner_context -> scanner_context

 let increment_line_if_newline context =
    match next_char context with
    | Some '\n' -> {context with line = context.line+1}
    | _ -> context

  let rec add_string_literal context =
     if( not ( next_char context = Some('"')) && (not (is_at_end context)) ) then
      context |> increment_line_if_newline |> advance |> add_string_literal
    else if (is_at_end context) then failwith "Unterminated string" else
    let str = (String.sub context.source (context.start + 1) (context.current - 1)) in
    add_literal_token STRING (Some (STRING_LITERAL str)) context

It's a bit gnarly and imperative. I'd like to find a way to pattern match out. BUT now we can add the following to our scan_tokens

| Some '"' -> add_string_literal advanced_context

Starting to get exciting now. Let's also handle number literals in section 4.6.2. The rules of numbers are simple - if there is a . character, it must have 1 or more digits on both sides. So let's try and implement this.

We need a function is_digit that takes a char option and returns true or false, and then a add_number_literal function that takes and returns a scanner_context. We'll also create a peek_next along the way that takes a scanner_context and returns char option.

To make things easy, we should write a function that simply captures digits. Let's call it capture_digits. It'll just abstract out the Java code of while (isDigit(peek())) advance();. We can also write a function to defer to above with capture_decimal

In our interface:

val is_digit: char option -> bool

val capture_digits: scanner_context -> scanner_context

val capture_decimal: scanner_context -> scanner_context

val peek_next: scanner_context -> char option

val add_number_literal: scanner_context -> scanner_context

And back in scanner.ml

let peek_next context =
    try Some (String.get context.source (context.current + 1)) with Invalid_argument _ -> None

  let is_digit character = match character with
    | Some c when c >= '0' && c <= '9' -> true
    | _ -> false

  let rec capture_digits context =
    if is_digit (next_char context) then context |> advance |> capture_digits else context

  let capture_decimal context = match (next_char context) with
    | Some '.' when (is_digit peek_next context) -> context |> advance |> capture_digits
    | _ -> context

  let add_number_literal context =
    let new_context = context |> capture_digits |> capture_decimal in
    let stringified_number = (String.sub new_context.source new_context.start (new_context.current - new_context.start)
    let parsed_number = int_of_string stringified_number in
    add_literal_token NUMBER NUMBER_LITERAL(stringified_number) new_context

Then just add in scan_tokens the following clause

| digit when (is_digit digit) -> add_number_literal advanced_context

Spicy. If you go ahead and dune runtest you should get some compiler errors about unused characters but no syntax or interface errors. Woohoo!

Let's finish up identifiers real quick. Once again, skimming ahead in Bob's chapter we see the claim if isAlpha or in English, "if the next lexeme begins with a letter or underscore, capture everything that's alphanumeric after and assume it's an identifier.

Our version of is_alpha will take a char option and return boolean. So will is_alpha_numeric. Our add_identifier_literal will need to take a scanner context and return one. Let's create a new literal_type

type literal_type = STRING_LITERAL of string | NUMBER_LITERAL of float | IDENTIFIER_LITERAL of string

This new type will just wrap our identifiers that users create. If a user defines a top levek function, it's an IDENTIFIER_LITERAL. If it's a usual keyword like AND, it's just an IDENTIFIER.

To our interface to define implementations.

 val is_alpha: char option -> bool

val is_alpha_numeric: char option -> bool

val add_identifier_literal: scanner_context -> scanner_context

And our implementation

let is_alpha character = match character with
  | Some c when c >= 'a' && c <='z' -> true
  | Some c when c >= 'A' && c <='Z' -> true
  | Some c when c = '_' -> true
  | _ -> false

  let is_alpha_numeric character = (is_alpha character) || (is_digit character)

  let rec add_identifier_literal context =
    if (is_alpha_numeric (next_char context)) then context |> advance |> add_identifier_literal
    else
    let substring = (String.sub context.source context.start (context.start - context.current)) in
    match substring with
    | "and" -> add_non_literal_token AND context
    | "class" -> add_non_literal_token CLASS context
    | "else" -> add_non_literal_token ELSE context
    | "false" -> add_non_literal_token FALSE context
    | "for" -> add_non_literal_token FOR context
    | "fun" -> add_non_literal_token FUN context
    | "if" -> add_non_literal_token IF context
    | "nil" -> add_non_literal_token NIL context
    | "or" -> add_non_literal_token OR context
    | "print" -> add_non_literal_token PRINT context
    | "return" -> add_non_literal_token RETURN context
    | "super" -> add_non_literal_token SUPER context
    | "this" -> add_non_literal_token THIS context
    | "true" -> add_non_literal_token TRUE context
    | "var" -> add_non_literal_token VAR context
    | "while" -> add_non_literal_token WHILE context
    | _ -> add_literal_token IDENTIFIER (Some (IDENTIFIER_LITERAL substring)) context

and then we can add in our scan_tokens clause the last piece.

| alpha when (is_alpha alpha) -> add_identifier_literal advanced_context

And now if you dune runtest one more time the only issue you should get is the call to Scanner.scan_tokens in olox.ml. Just go ahead and open Scanner at the top to fix that.

We now have a working scanner. In the next chapter move onto the code representation step.

*To follow along the source material, check out the Scanning chapter of Crafting Interpreters

For ease, I'll recommend you clone down this starter repo. We'll be building a scanner for a Lox implementation in OCaml, which we'll call Olox. Let's create a file called olox.ml. This is going to be our entry point for the application, similar to Lox.java in the source material.

Bob's Java code handles the following three cases:

1. If more than one argument is passed in, output a line telling the user the proper usage and exit.
2. If one argument is passed in, run a file with that name.
3. Else, just open a prompt.

Okay, let's start by writing out the following code in our olox.ml

  let main = fun _ -> match Array.length Sys.argv with
    | 0 -> run_prompt ()
    | 1 -> run_file (Array.get Sys.argv 0)
    | _ -> print_endline "Usage: olox [script]"; exit 64;

We're pattern matching on the length of the arguments vector passed in. Sys.argv takes an array of strings that we can use. If no arguments are passed in, we run a prompt. If one is passed in, we run a file with that first element. Else, we'll print an error and exit. We can't really run this code, since we have two undefined functions in run_prompt and run_file but we're just following along.

So now need to define those two functions. Bob's implementation defers run_file to a function called run by reading the bytes out of a file. run_prompt also just takes a line from input and calls run. So we now need to define those three functions.

Let's add the following in our module above our main function

  let run source =
    let tokens = Scanner.scan_tokens in
    List.iter (fun token -> print_endline token) tokens

  let run_prompt = fun _ ->
    while true do
      print_string "> ";
      let input = read_line ()
      in run input;
    done

  let run_file filename =
    let channel = open_in filename in
    try
      (* read entire file *)
      let line = really_input_string channel (in_channel_length channel) in
      run line;
      flush stdout;
      close_in channel
    with e ->
      close_in_noerr channel;
      raise e

Like Bob's Java code, we have a run function that just gets tokens from a scanner - in his case, an instance of a Scanner class. We'll just use a to-be-defined module. Then we'll just iterate over the returned tokens and print them out.

Our run_prompt function will print a string as a prompt, then read in the next line and pass it to our run function.

Our run_file function takes some filename and creates an input channel, positioned at the beginning of the file. It then gets the entirety of the file by reading in_channel_length channel number of characters, then passes into run. For clarity, see the following type signatures.

val really_input_string : in_channel -> int -> string val in_channel_length : in_channel -> int

really_input_string takes a channel and an int, and returns a string. in_channel_length returns the length of a channel.

So far so good! If you were to copy and paste this into utop, you'd error on the Scanner module. That's fine - feel free to replace run with something like let run = print_endline and test all those functions. In fact, let's try it right now. Create a file called test.md and add the following code.

Hello, world!
This is the second line!
Now on line 3 with spec!@l characters

In your terminal, run utop and copy and paste the following code which replaces run with a simple print.

let run = print_endline

let run_prompt = fun _ ->
  while true do
    print_string "> ";
    let input = read_line ()
    in run input;
  done

let run_file filename =
  let channel = open_in filename in
  try
    (* read entire file *)
    let line = really_input_string channel (in_channel_length channel) in
    run line;
    flush stdout;
    close_in channel
  with e ->
    close_in_noerr channel;
    raise e

Try running run_file "test.md";; and then run_prompt ();; Anything in your prompt should be returned back. Neat! Now ctrl+D to exit.

Next up is error handling! Bob makes the following point and I think it's very valid

The other reason I pulled the error reporting out here instead of stuffing it into the scanner and other phases where the error occurs is to remind you that it’s a good engineering practice to separate the code that generates the errors from the code that reports them.

However, in the interest of not having a global mutable variable, and keeping some of my logic contained in the scanner (for now), I'm going to go ahead and handle all error management in the scanner. This way we can maintain all our scanner logic (line number, character positions) safely isolated and still report on it.

Let's jump right into it. Create a file called scanner.ml. Bob's examples creates classes for Token and TokenType. In our FP world, those can just be types that our scanner knows about so we'll wrap it all up inside one file. If we need to refactor later we can.

Looking at Token.java in the source, we see a giant enum. When I see enum, I think variant types - read up here if you need. Let's take the enum and create a variant type in our scanner.

module Scanner = struct
  type token_type =
    (* single-character tokens *)
    | LEFT_PAREN
    | RIGHT_PAREN
    | LEFT_BRACE
    | RIGHT_BRACE
    | COMMA
    | DOT
    | MINUS
    | PLUS
    | SEMICOLON
    | SLASH
    | STAR

    (* One or two character tokens *)
    | BANG
    | BANG_EQUAL
    | EQUAL
    | EQUAL_EQUAL
    | GREATER
    | GREATER_EQUAL
    | LESS
    | LESS_EQUAL

    (* Literals *)
    | IDENTIFIER
    | STRING
    | NUMBER
    (* Keywords *)
    | AND
    | CLASS
    | ELSE
    | FALSE
    | FUN
    | FOR
    | IF
    | NIL
    | OR
    | PRINT
    | RETURN
    | SUPER
    | THIS
    | TRUE
    | VAR
    | WHILE
    | EOF
end

Just to quote section 4.2.2,

There are lexemes for literal values—numbers and strings and the like. Since the scanner has to walk each character in the literal to correctly identify it, it can also convert it to the real runtime value that will be used by the interpreter later.

Before we move on, we'll need to have some way of wrapping literals. Our token is going to want to have a reference of token type AND optional literals - e.g. a token could be of type TRUE with no literal, or of type NUMBER with a literal 2. So, let's start by just making two types for literals - string and number

  type literal_type = STRING_LITERAL of string | NUMBER_LITERAL of float

This doesn't solve all our problems, but at least we now have some idea of what a literal can be. Moving on to make our token, we should define a type that represents a token. Bob's Token class has a type of TokenType, a lexeme of String, a literal of some Object, and a line of Int. We can do something similar with

type token = {
  lexeme: string;
  literal: literal_type option;
  line: int;
  token_type: token_type;
}

Nikhil why is that literal an option? Well, as we had mentioned before - a token does not need to contain a literal. There is no literal for the token_type VAR, so we may not need a literal in our token. Options represent that. Real World OCaml's chapter on options is a great primer for more info.

Lets take a moment to look at the initial Scanner code in section 4.4. The file defined a class Scanner that has a field source, a string that represents the code we're scanning. scanTokens takes a list of tokens, checks to see if we're at the end of the source and if not, scans tokens. Once at the end, we add a token representing the end of the file and return the list of tokens. We also have the use of a helper function that determines if we're at the end of the source code.

We see that the original Java code has some fields for use. Let's embrace our FP-world and rather than create a class, we're going to create a type that represents some record of fields. We can then ensure all our functions take and emit that type.

type scanner_context = {
  source: string;
  start: int;
  current: int;
  line: int;
}

Now that we have a type that contains all the fields we know about, let's define our methods. To start, let's create an interface file. Create a file called scanner.mli with the following

module Scanner : sig
  type token_type

  type literal_type

  type token

  type scanner_context

  val scan_tokens: string -> token list

  val is_at_end: scanner_context -> bool
end

In our interface, I'll be using abstract types. type token and type scanner_context now hide the implementation of the type from any module that implements this signature. You can read more about them on the OCaml website and this Cornell class's notes.

Our signatures implement what Bob's Java code does - is_at_end returns true or false for the state of the application. scan_tokens will go through the source code and return all the tokens.

Let's prove that our interface actually does something! Open dune and replace with the following code`

(executable
  (name olox)
  (libraries oUnit)
)

(alias
  (name    runtest)
  (deps    (:x olox.exe))
  (action  (run %{x})))

Now just run dune build and dune runtest. You should see the following errors.

File "scanner.ml", line 1:
Error: The implementation scanner.ml
       does not match the interface .olox.eobjs/byte/scanner.cmi:
       ...
       In module Scanner:
       The value `is_at_end' is required but not provided
       File "scanner.mli", line 56, characters 2-40: Expected declaration
       In module Scanner:
       The value `scan_tokens' is required but not provided
       File "scanner.mli", line 54, characters 2-48: Expected declaration

There's a bit more above it but let's come back to that later. Our interface works! We stated that the module must have two specific functions but did not implement them. Now to do that.

We'll first write is_at_end. The function takes our wrapping context and returns a boolean. I wrote

let is_at_end ctx = ctx.current >= (String.length ctx.source)

Then we'll have our scan_tokens function. This function takes just the source code string as an input, and will return a list of tokens. Somewhere in the middle we'll need to wrap up the rest of our context. OCaml has local let bindings we can use for this. We also now realize we need to maintain a running list of tokens that we're adding. Let's add tokens to our scanner_context. In the definition of type scanner_context just add tokens: token list; inside.

Looking at Bob's implementation, we have a few more pieces of information. We have a scanToken function that does...something. We can at least assume it changes the value of current globally due to the reset inside the loop. We also have some addition of an EOF token. Let's sketch out what a scan_token function might look like. Given that it'll change two values, current and our token list, let's have it receive and output a scanner_context. In our .mli file add

  val scan_token: scanner_context -> scanner_context

Bob's token addition basically just instantiates a token object and adds it to the global list. Let's do the same, by writing a function add_token that takes all the fields of a token as well as context and returns the new context.

  val add_token: token_type -> string -> literal_type option -> int -> scanner_context -> scanner_context

Okay! We have some loosely defined code interfaces. Let's implement. In scanner.ml we'll do the eaier of the two first.

let add_token token_type lexeme literal line ctx =
  {ctx with tokens = ctx.tokens @ [{
      token_type;
      lexeme;
      literal;
      line
    }]
  }

There's a performance issue here where OCaml's list appending operator (either @ or List.append) runs in O(n) time by walking through the length of the first list. It's not ideal but that's what we get for using a linked list. For now, we'll ignore this and come back to it when it's a performance issue.

Our next function will be our scanning. Look at Bob's implementation. In words - he gets the next character, checks to see what it is, then adds a token. I'm going to break this into three functions - one function to just increment the current counter, one to get the next character, and our add token above.

We can add the following two interfaces

  val advance: scanner_context -> scanner_context

  val current_char: scanner_context -> char option

And then define our functions

  let advance context = {context with current = context.current + 1}

  let current_char context =
    try Some (String.get context.source (context.current -1)) with Invalid_argument _ -> None

I'm using an option for the current_char because it's possible to get an out of bounds exception on string access, and it's better to just be safe than sorry.

We also have sone unary function addToken that Bob defines, that just defers to a binary function. I'm going to define these two functions as either adding a literal, or a nonliteral. That means we'll need a function add_literal_token and add_non_literal_token - the former takes both a token_type and Some literal and the latter can defer to the former with a None. We're going to again have context passed along as the last argument in order to make pipelining our functions easy.

val add_literal_token: token_type -> literal_type option -> scanner_context -> scanner_context

val add_non_literal_token: token_type -> scanner_context -> scanner_context

Great. Now to just write those little helpers...

let add_literal_token token_type literal_type context =
  let text = (String.sub context.source context.start (context.current - context.start)) in
  add_token token_type text literal_type context.line context

let add_non_literal_token token_type context = add_literal_token token_type None context

We create a substring from the start position and capture the difference between start and current, then add the token. String.sub in OCaml takes a start and length, rather than a start and end so we need to do a little math.

Now that we have these, we can write out our scan_token function and finally write scan_tokens. Again, since we see that scan_token in Bob's example cares about two fields - the current character, and the token list, let's pass in our context and get out context.

val scan_token: scanner_context -> scanner_context

let scan_token context =
  let advanced_context = advance context in
  match (current_char advanced_context) with
  | Some '(' -> add_non_literal_token LEFT_PAREN advanced_context
  | Some ')' -> add_non_literal_token RIGHT_PAREN advanced_context
  | Some '{' -> add_non_literal_token LEFT_BRACE advanced_context
  | Some '}' -> add_non_literal_token RIGHT_BRACE advanced_context
  | Some ',' -> add_non_literal_token COMMA advanced_context
  | Some '.' -> add_non_literal_token DOT advanced_context
  | Some '-' -> add_non_literal_token MINUS advanced_context
  | Some '+' -> add_non_literal_token PLUS advanced_context
  | Some ';' -> add_non_literal_token SEMICOLON advanced_context
  | Some '*' -> add_non_literal_token STAR advanced_context
  | _ -> failwith "Disallowed character"

WHEW. That's a whole lot of work but we did it. We now have a way to scan tokens from a string. Now for the fun part.

In our next post let's try to write more matching cases. And if we're lucky, some tests.

There are multiple steps to an interpreter that Bob Nystrom covers in detail on his website. For the first pass, we'll be building a tree-walking interpreter. This shrinks the problem domain a bit. In a nutshell, we'll have to solve the following problems.

Step one is lexical analysis - taking in some stream of text (the code input) and breaking it into chunks or lexemes. For example, if I were to type

const x = 2;

we have 5 lexemes - the keyword const, the variable x, the equals sign, the integer 2, and the semicolon. However, in order to maintain the context or type of lexeme, we're going to have to tag them with a type. You can think of each chunk broken into a token, which might look like the following

  const token1 = {
    lexeme: "=",
    type: "EQUALS"
  }

  const token2 = {
    lexeme: "2",
    type: "NUMBER"
  }

in OCaml we'd use the built in type system and end up with something more like

type token_type = EQUALS | NUMBER
type token = {
  lexeme: string;
  type: token_type;
}

let token1 = {
  lexeme = "=";
  type: EQUALS
}

let token1 = {
  lexeme = "2";
  type: NUMBER
}

Step two is parsing - taking our list of tokens and building an abstract syntax tree. This tree representation lets us build out a data structure that can throw errors when leaf nodes are invalid, recurse and find subexpressions, etc.

If you were to imagine the following

  (1 + 2) / 3

This might be represented as nodes of

  type fn = Add | Subtract | Multiply | Divide
  let tree =
    | Literal of int
    | Node of fn * tree * tree

  Node(
    Divide,
    Node(Add, 1, 2)
    3
  )

The following post will focus on building out step one.

If you want to just clone a repo and move on, check out this sample

The easiest way to get an environment set up is to just follow the Real World OCaml guide. I'm going to enumerate the steps here in less detail. Since I'm running on macOS, this guide will be tailored to that. Check out the RWO guide for more options.

1. Get OPAM installed. Opam is the OCaml package manager and is available on Homebrew. If you have Homebrew installed, it's as easy as brew install opam

2. This should have installed OCaml for you. To check, run ocaml -version and make sure you have a response.

3. Next up, we need to configure OPAM. Run opam init and walk through the questions. I answered yes to everything.

4. Now we need some packages. We're going to start by installing just Core and utop. Core is a replacement to the OCaml standard library. It has some different APIs for labeled arguments, more functionality in every module, and has a consistent interface for Container structures. To be honest, we could just install Base instead of Core, where Base supplies those features, but RWO uses Core so let's stick with it. utop is a toplevel (like the Ruby REPL irb) that is really easy to work with for testing code on the fly. For the purpose of this tutorial, I'm actually not going to use Base or Core so you can omit it.

5. Let's also install dune for our build tooling, with opam install dune.

6. open up ~/.ocamlinit and add the following to get all of Core set up in the toplevel

#use "topfind";;
#thread;;

7. Configure your editor. I use Visual Studio Code and really like the ReasonML tool Reason Language Server

And that's a minimal install! If you check out the repo I linked above, you can run tests with dune runtest and play with some code.

1. I want to challenge myself and step out of the comfort of usual web development and all the ease of the JS ecosystem
2. I want to better understand how programming languages actually work.

To learn OCaml, I've worked through a Cornell class's online notes ¹ as well as read much of version 1 of Real World OCaml. When version 2 comes out, I'll try to update these notes. I've also been devouring YouTube videos from Jane Street as well as spending a lot of time in the ReasonML / OCaml Discord

To understand more about interpreters, I'll be using two primary resources.

• Crafting Interpreters² by Bob Nystrom
• Writing an Interpreter in Go³ by Thorsten Ball

By the time this series is done, I hope to have written an interpreter for the Lox language used in Crafting Interpreters using OCaml. I'll be establishing a few rules in order to throw myself into the deep end.

1. Where possible, I will use the OCaml data type system to help ensure exhaustive checking of various states. Variant types are a powerful tool available in OCaml (and Haskell, F# and many other ML-inspired langauges) that can serve as a way of flagging different cases. For example, a chunk of code can be a string, a number, or some keyword. Using these types, I could state that a code chunk is

type Chunk =
| Keyword
| String of string
| Number of int

2. Where possible, I'll avoid mutation, classes and global state. In chapter 1, Bob uses a Scanner class with fields of start and current, both ints, to track lexeme parsing (aka just parsing out phrases). Rather than this, I'll be writing a Scanner module and passing around a context object between functions.

3. Where possible, I'll use recursion over looping. In the book, there are times Bob uses while loops to pick out phrases. For example, continue parsing out a chunk of code while the next character is not a space, then stop. That code might look something like (in JS).

  let start = 0;
  let current = 0;
  let phrase = "";
  const testString = "Hello world!";

  while (testString[current] != " ") {
    phrase = phrase + testString[current];
    current++;
  }

  // phrase = "Hello"

Rather than a while loop, I'll be using pattern matching and recursion. In OCaml, that might look something like the following.

  type context = {
    start: int;
    current: int;
    phrase: string;
    source: string;
  }

  let ctx = {
    start = 0;
    current = 0;
    phrase = "";
    source = "Hello world!";
  }

  let rec capture_until_space context =
    let next_char = (String.get context.source context.current) in
    match next_char with
    | ' ' -> context
    | _ -> capture_until_space {
        context with current = context.current + 1;
                     phrase = (context.phrase ^ Char.escaped next_char);
      }

  (*
    # capture_until_space ctx;;
    - : context = {start = 0; current = 5; phrase = "Hello"; source = "Hello world!"}
  *)

If I succeed in doing this, I can maintain purity.

I'll be also attempting to write unit tests along the way using OUnit2. I'll also attempt to implement a setup with Dune to make it easier to build and follow along. I know the Reason community has tools like Esy as well. If I'm ambitious, I'll use that as well to get everything set up.

Wish me luck!