This is part 1 of a two part series. Skip over to part 2 you'd like .
For coding interviews, we are interested in gauging the asymptotic efficiency of algorithms both in terms of running time and space. The formal study of analysis of algorithms is called complexity theory, a rich field with fascinating and complicated math. For interviews, we only need a few basic concepts. Asymptotic efficiency is concerned with how the running time or memory requirements of an algorithm grows with the input size, so it is intimately concerned with how well algorithms scale to larger inputs. This is important in Computer Science and in practice because whereas some algorithms work well enough for small inputs of say < 10 inputs, the running time and space grows far faster than the input size and thus large inputs of say 10s to millions of inputs becomes impractical (usually meaning taking hours or even years of execution time). Consider sorting. Say for the sake of argument that sorting algorithm A takes 1, 2, 4, 16, 32, and so on seconds to sort 1, 2, 3, 4, 5, and so on elements and sorting algorithm B takes 1, 2, 3, 4, 5, and so on seconds for the same. The former algorithm would be considered exponential time or O(2^n) and the latter linear time O(n). Note that exponential time is really bad because for just 30 elements, you'd be talking about 12,427 hours of compute time. Instead of actually measuring the execution time of an already implemented algorithm, we can calculate a rough model for it via asymptotic notation.
First, to analyze the complexity of algorithms, there is a need to be precise: Are you interested in complexity in the worst case or average case? There is also best case performance, but you touch that less frequently. Some algorithms such as quicksort have good average cases (O(n log n) for quicksort) but poor worst case (O(n^2)). For interviews, usually the interviewer is interested in worst-case complexity, but if in doubt, ask. Before going further, let's get a good working definition of the most common form complexity analysis: Big O notation. Big O is the asymptotic upper bound of the growth of complexity.
Some common Big-O classes are, in the order of efficiency: O(1) constant < O(log n) logarithmic < O(n) linear < O(n log n) < O(n^2) quadratic < O(n^3) cubic < ... < O(2^n) exponential < ... < O(n!) factorial.
"Big-O" Class | Name | Representative algorithms |
---|---|---|
O(1) | constant-time | insertion into a linked list, lookup of a hash table with no collisions |
O(log n) | logarithmic-time | binary search on sorted array |
O(n) | linear-time | linear search, counting sort, iterate thru array a constant number of times, Boyer-Moore string search |
O(n log n) | quasilinear time | mergesort, heapsort worst case, quicksort avg case, comparison-based sort, Dijkstra's shortest path (nonnegative edge weights) |
O(n^2) | quadratic-time | |
O(n^3) | cubic-time | Floyd-Warshall all-pairs shortest-path algorithm, Bellman-Ford |
O(n^k) | polynomial-time | dynamic programming |
O(2^n) | exponential-time | backtracking algorithm |
O(n!) | factorial-time | brute-force search traveling salesman |
Comments
Post a Comment