Longest increasing subsequence

The Longest Increasing Subsequence problem is to find the longest increasing subsequence of a given sequence. It also reduces to a Graph Theory problem of finding the longest path in a Directed acyclic graph.

Overview
Formally, the problem is as follows:

Given a sequence $$a_1, a_2, \ldots, a_n$$, find the longest subset such that for every $$i < j$$, $$a_i < a_j$$.

Longest Common Subsequence
A simple way of finding the longest increasing subsequence is use the Longest Common Subsequence (Dynamic Programming) algorithm.


 * 1) Make a sorted copy of the sequence $$A$$, denoted as $$B$$. $$O(n \lg(n) )$$ time.
 * 2) Use Longest Common Subsequence on with $$A$$ and $$B$$. $$O(n^2)$$ time.

Dynamic Programming
There is a straight-forward Dynamic Programming solution in $$O(n^2)$$ time. Though this is asymptotically equivalent to the Longest Common Subsequence version of the solution, the constant is lower, as there is less overhead.

Given the sequence $$a_1,a_2,\ldots,a_k$$, the optimal way to add $$a_k+1$$ would simply be the longest of the longest subsequence from $$a_i$$ to $$a_k+1$$, adding it to each list if needed. For each $$a_k+1$$, there are two possibilities:
 * The number is lower than the last number of the subsequence and greater than the second last. In this case the last number of the subsequence is replaced by the new number.
 * The number is greater than the last number of the subsequence: The number is appended to the subsequence and if this subsequence is the best subsequence with this length, it will be stored.

The pseudo-code is show below:

func lis( a ) initialize best to an array of 0's.  for ( i from 1 to n ) best[i] = 1 for ( j from 1 to i - 1 ) if ( a[i] > a[j] ) best[i] = max( best[i], best[j] + 1 ) return max( best )

Faster Algorithm
There's also an $$O(n \log{n})$$ solution based on some observations. Let $$A_{i,j}$$ be the smallest possible tail out of all increasing subsequences of length $$j$$ using elements $$a_1, a_2, a_3, \ldots, a_i$$.

Observe that, for any particular $$i$$, $$A_{i,1} < A_{i,2} < \ldots < A_{i,j}$$. This suggests that if we want the longest subsequence that ends with $$a_{i+1}$$, we only need to look for a j such that $$A_{i,j} < a_{i+1} <= A_{i,j+1}$$ and the length will be $$j+1$$.

Notice that in this case, $$A_{i+1,j+1}$$ will be equal to $$a_{i+1}$$, and all $$A_{i+1, k}$$ will be equal to $$A_{i,k}$$ for $$k \ne j + 1$$.

Furthermore, there is at most one difference between the set $$A_i$$ and the set $$A_{i+1}$$, which is caused by this search.

Since $$A$$ is always ordered in increasing order, and the operation does not change this ordering, we can do a binary search for every single $$a_1, a_2, \ldots, a_n$$.

Implementation

 * C
 * C++ ($$O(n \log n)$$ algorithm - output sensitive - $$O(n \log k)$$)