Design and Analysis - Time Complexity

---

In this chapter, let us discuss the time complexity of algorithms and the factors that influence it.

Time complexity of an algorithm, in general, is simply defined as the time taken by an algorithm to implement each statement in the code. It is not the execution time of an algorithm. This entity can be influenced by various factors like the input size, the methods used and the procedure. An algorithm is said to be the most efficient when the output is produced in the minimal time possible.

The most common way to find the time complexity for an algorithm is to deduce the algorithm into a recurrence relation. Let us look into it further below.

Solving Recurrence Relations

A recurrence relation is an equation (or an inequality) that is defined by the smaller inputs of itself. These relations are solved based on Mathematical Induction. In both of these processes, a condition allows the problem to be broken into smaller pieces that execute the same equation with lower valued inputs.

These recurrence relations can be solved using multiple methods; they are −

• Substitution Method

• Recurrence Tree Method

• Iteration Method

• Master Theorem

Substitution Method

The substitution method is a trial and error method; where the values that we might think could be the solution to the relation are substituted and check whether the equation is valid. If it is valid, the solution is found. Otherwise, another value is checked.

Procedure

The steps to solve recurrences using the substitution method are −

• Guess the form of solution based on the trial and error method

• Use Mathematical Induction to prove the solution is correct for all the cases.

Example

Let us look into an example to solve a recurrence using the substitution method,

T(n) = 2T(n/2) + n

Here, we assume that the time complexity for the equation is O(nlogn). So according the mathematical induction phenomenon, the time complexity for T(n/2) will be O(n/2logn/2); substitute the value into the given equation, and we need to prove that T(n) must be greater than or equal to nlogn.

≤ 2n/2Log(n/2) + n
= nLogn – nLog2 + n
= nLogn – n + n
≤ nLogn

Recurrence Tree Method

In the recurrence tree method, we draw a recurrence tree until the program cannot be divided into smaller parts further. Then we calculate the time taken in each level of the recurrence tree.

Procedure

• Draw the recurrence tree for the program

• Calculate the time complexity in every level and sum them up to find the total time complexity.

Example

Consider the binary search algorithm and construct a recursion tree for it −

Since the algorithm follows divide and conquer technique, the recursion tree is drawn until it reaches the smallest input level $\mathrm{T\left ( \frac{n}{2^{k}} \right )}$.

$$\mathrm{T\left ( \frac{n}{2^{k}} \right )=T\left ( 1 \right )}$$

$$\mathrm{n=2^{k}}$$

Applying logarithm on both sides of the equation,

$$\mathrm{log\: n=log\: 2^{k}}$$

$$\mathrm{k=log_{2}\:n}$$

Therefore, the time complexity of a binary search algorithm is O(log n).

Master’s Method

Master’s method or Master’s theorem is applied on decreasing or dividing recurrence relations to find the time complexity. It uses a set of formulae to deduce the time complexity of an algorithm.

To learn more about Master’s theorem, please click here