DataStructures | Portfolio

Algorithms

Thu, 10 Oct 2024 00:00:00 +0000

1. Overview

This note compiles the key concepts covered in an undergraduate algorithms course. Use it to refresh important ideas quickly and to understand how the individual topics are connected.

2. Sorting Algorithms

Sorting is the preparatory step for many problems. When choosing an algorithm, consider input size, data distribution, auxiliary space, and whether stability (preserving the order of equal keys) matters.

Algorithm	Average Time	Worst Time	Extra Space	Highlights
Merge Sort	O(n log n)	O(n log n)	O(n)	Stable; great for external sorting
Quick Sort	O(n log n)	O(n²)	O(log n)	Divide and conquer; pivot choice is critical
Selection Sort	O(n²)	O(n²)	O(1)	Few swaps but generally slow; unstable
Bubble Sort	O(n²)	O(n²)	O(1)	Simple; fast on nearly sorted data with early exit
Insertion Sort	O(n²)	O(n²)	O(1)	Excellent for small or nearly sorted arrays; stable
Radix Sort	O(d·n)	O(d·n)	O(n + k)	Digit-wise; bypasses comparisons
Counting/Bucket Sort	O(n + k)	O(n + k)	O(n + k)	Strong when value range is small or uniform

2.1 Merge Sort

Recursively splits the array and merges the sorted halves.
Stable because equal keys retain their relative order.
Drawbacks: needs extra memory and incurs recursion overhead.

2.2 Quick Sort

Partitions around a pivot into smaller and larger elements and recurses.
Typically the fastest in practice, but degenerates to O(n²) on sorted data unless you randomise or choose smart pivots (median-of-three, etc.).

2.3 Selection Sort

Repeatedly selects the minimum (or maximum) remaining element and swaps it into place.
Swap count is limited to O(n), but the algorithm is mainly pedagogical due to its O(n²) comparisons.

2.4 Bubble Sort

Repeatedly swaps adjacent elements that are out of order so the largest “bubbles” to the end.
Each pass places one element; adding a flag for “already sorted” speeds up nearly sorted inputs.

2.5 Insertion Sort

Inserts each element into the correct position within the already sorted prefix.
Stable and efficient for tiny datasets or partially ordered inputs.

2.6 Radix Sort

Sorts integers or fixed-length strings digit by digit: LSD (least significant digit first) or MSD (most significant digit first).
Requires a stable subroutine (e.g., counting sort) to guarantee global stability.

2.7 Counting & Bucket Sort

Counting sort achieves O(n + k) for small value range k; bucket sort achieves expected O(n) when inputs are uniformly distributed.
Both need extra memory and require advance knowledge of the key range.

3. Trees: Red-Black Trees

Red-black trees are balanced binary search trees that maintain O(log n) height by enforcing colour-based invariants.

Invariants:
(1) Every node is red or black.
(2) The root and NIL leaves are black.
(3) Red nodes have black children.
(4) Every path from a node to its descendant NIL nodes contains the same number of black nodes.
Operations:
Insert like a BST, then fix colours/rotations. Delete by replacing with a successor and resolving the “double black” case.
Why use them: Fewer rotations than AVL trees, making them practical when inserts/deletes are frequent.
Examples: std::map, std::set, Java TreeMap, Linux CFS scheduler.

4. Hash Tables

Idea: Hash a key to an array index so average lookup/insertion/deletion becomes O(1).
Collision: When multiple keys hash to the same bucket. Quality of the hash function and load factor (n/m) dictate performance.

4.1 Collision-Resolution Strategies

Separate chaining: Linked lists or dynamic arrays per bucket. Simple and flexible, but needs pointer storage.
Open addressing: Probe for the next empty slot (linear, quadratic probing, double hashing). Cache-friendly, yet suffers from clustering.
Rehashing: Enlarge the table when the load factor exceeds a threshold and reinsert all entries.

4.2 Design Tips

Aim for a hash function that distributes keys evenly, is fast to compute, and resists adversarial input.
For concurrent scenarios, look into lock-free buckets, striped locks, or partitioned tables.

5. Graph Algorithms

Graphs model networks, dependency orders, and relationships. Sparse graphs typically use adjacency lists; dense graphs favour adjacency matrices.

5.1 Topological Sort

Produces a linear order of a DAG’s prerequisites.
Implement via Kahn’s algorithm (in-degree queue) or DFS finishing times; both detect cycles when sorting fails.
Useful for course prerequisites, build systems, and task scheduling.

5.2 Bellman–Ford Algorithm

Handles graphs with negative edge weights to find single-source shortest paths.
Runs |V|−1 relaxations and checks for updates on the |V|th iteration to detect negative cycles.
Critical for routing protocols such as RIP where Dijkstra cannot be applied.

5.3 Pebble Game

Places “pebbles” on DAG nodes to study computation order and resource usage.
Helps derive lower bounds for register allocation, memory minimisation, and proof complexity.
Solving the optimal pebbling strategy is challenging beyond restricted graph classes.

5.4 Additional Highlights

DFS/BFS: Core search routines; basis for connectivity, cycle detection, and topological sort.
Strongly Connected Components: Kosaraju’s or Tarjan’s algorithms condense graphs into DAGs.
Minimum Spanning Tree: Kruskal and Prim minimise connection cost in undirected weighted graphs.

6. String Matching: Rabin–Karp

Uses a rolling hash to compare the pattern hash with substrings of the text.
Offers expected O(n + m) time; collisions trigger explicit substring comparisons.
Particularly handy for multi-pattern search after precomputing pattern hashes.

7. Study Tips

Implement algorithms and analyse both average and worst cases using your own test datasets.
Visualise algorithm steps with tools such as VisuAlgo or CS Academy.
In coding interviews or contests, inspect input bounds and memory constraints before choosing a strategy.
Suggested reading: CLRS, Sedgewick’s Algorithms, and Problem Solving with Algorithms and Data Structures by Miller & Ranum.

Data Structures

Mon, 15 Apr 2024 00:00:00 +0000

1. Overview

Data structures organise information in memory and on disk so that algorithms can run efficiently. When solving problems, consider both the abstract data type (what operations are supported) and the concrete implementation (time/space trade-offs).

2. Arrays and Wrapper Classes

2.1 Arrays

Traits: Contiguous memory, O(1) random access by index.
Limitations: Fixed size; mid-array insert/delete costs O(n) shifts.
Use cases: Cache-friendly for numerical computing, DP tables, game state loops.

2.2 Class Wrappers

Wrapping arrays inside classes adds safety features such as bounds checking, dynamic resizing, and iterators.
std::vector (C++) and ArrayList (Java) double their capacity when full, yielding amortised O(1) append operations.

3. Stack and Queue

3.1 Stack

Last-in-first-out (LIFO). Underpins call stacks, delimiter checking, DFS backtracking.
Core ops: push, pop, top. Implementable with arrays or linked lists.

3.2 Queue

First-in-first-out (FIFO). Used in BFS, job scheduling, buffering.
Core ops: enqueue, dequeue, front. Circular arrays eliminate index wrap-around.

3.3 Deque

Inserts and deletes on both ends. Handy for sliding windows and monotonic queues.

4. Pointers and Linked Lists

4.1 Pointers

Store memory addresses and reference dynamically allocated objects.
Beware of null dereferences, dangling pointers, and double frees.

4.2 Singly Linked List

Nodes store (data, next). Inserts/deletes run in O(1); random access takes O(n).
Techniques: head insertion, tail pointer maintenance, sentinel (dummy) heads.

4.3 Doubly Linked List

Each node keeps prev and next, enabling bidirectional traversal.
Useful for LRU caches and editors. Twice as many pointer updates—test carefully.

5. List Abstract Data Type

List ADTs define ordered collections; performance depends on implementation.
Dynamic array list: O(1) indexing, O(n) mid-insert/delete.
Linked list: O(1) insert/delete (with references), O(n) search; cursor iterators help frequent traversals.

6. Recursion

Functions call themselves to break a problem into smaller cases.
Essential elements: base case, recursive relation, and state propagation.
Watch call-stack depth; tail recursion can often be transformed into iteration.

7. Trees

Hierarchical, acyclic structures branching from a root.
Terminology: depth, height, degree, subtree.
Traversals: preorder, inorder, postorder, level-order—implemented via recursion or queues.

8. Binary Search Tree (BST)

Maintains the invariant left subtree < root < right subtree.
Average O(log n) lookup/insert/delete; skewed trees degrade to O(n).
Balancing options: AVL, red-black tree, treap, splay tree—guarantee logarithmic height.

9. Priority Queue

Stores elements with priorities; retrieves the highest (or lowest) priority element.
Typically implemented with heaps: insert and extract in O(log n), peek in O(1).
Applications: Dijkstra’s algorithm, schedulers, event simulation, running medians.

10. Graphs

Comprised of vertices (V) and edges (E). Consider direction, multiedges, self-loops.
Representations:
- Adjacency list: O(V + E) space—ideal for sparse graphs.
- Adjacency matrix: O(V²) space—constant-time edge existence checks.
Traversals: DFS (stack/recursion) and BFS (queue) underpin connectivity, topological order, and cycle detection.

11. Weighted Graphs

Each edge carries a cost. Models maps, networks, logistics.
Single-source shortest paths: Dijkstra (non-negative weights), Bellman–Ford (with negatives), SPFA (practical improvements).
All-pairs shortest paths: Floyd–Warshall (O(V³)), Johnson’s algorithm (sparse graphs).
Minimum spanning tree: Kruskal (Union-Find) and Prim (priority queue).

12. Sorting (Review)

Reorders data to enable faster search or analytics.
Comparison-based: Quick, merge, heap sort—lower bound O(n log n).
Non-comparison: Counting, radix, bucket sort—use key range/digits to reach O(n + k).
Evaluate stability, in-place behaviour, memory footprint, and whether divide-and-conquer fits the scenario.

13. Searching

Linear search: Scan sequentially; works on unsorted collections; O(n).
Binary search: Halves the range on a sorted array; O(log n).
Hash lookup: Average O(1) with well-designed collisions handling.
Tree search: Balanced BSTs or B-Trees deliver O(log n); maintaining balance is vital.

14. Study Tips

Build your own complexity tables and revisit them regularly.
Step through pointer manipulations with a debugger to internalise recursion and linked structures.
Analyse input size, update frequency, and random access needs before choosing a structure.