English

not defined

no text concepts found

Discrete Structures Lecture 11: Algorithms Miss, Yanyan,Ji United International College Thanks to Professor Michael Hvidsten Objective and Outline • Objective: Construct and analysis algorithm • Outline: Algorithm definition and properties Using Pseudocode describe sorting and search algorithm Algorithm analysis: Big oh, Big Theta, Big Omega Algorithms • Definition: An algorithm is a finite set of precise instructions for performing a computation or for solving a problem. • Example: Find the largest (maximum) number in a list. • Algorithm: (in Pseudocde) procedure max(a1, a2 , a3 , …, an: integers) max = a1 for i = 2 to n if max < ai then max = ai return max Algorithm Properties • All algorithms share these properties: – – – – Input: The set of input value(s) for the algorithm Output: Value(s) that form the solution to the problem Well-Defined: The steps of the algorithm must be precise Correctness: An algorithm should produce the correct input for a given set of input values – Finiteness: An algorithm should produce an output after a finite number of steps – Effectiveness and Generality: The algorithm should be applicable for all possible input values. Algorithms • Example: Find the maximum number in a list. procedure max(a1, a2 , a3 , …, an: integers) max = a1 for i = 2 to n if max < ai then max = ai return max • Does this algorithm have all of the properties listed above? Solution • Input :a sequence of integers • Output: the largest integer in the sequence • Definiteness: only assignments, a finite loop, and condition statements occur. • Correctness: initial value of max is the first term of the sequence, as successive terms of the sequence are examined. max is updated to the value of a term if the term exceeds the maximum of the terms previously examined. • Finiteness: it terminates after all the integers in the sequence have been examined. • Effectiveness: the algorithm can be carried out in a finite amount of time since each step is either a comparison or an assignment. • Generality: it can be used to find the maximum of any finite sequence of integers. Algorithm Properties • Note: An algorithm can be considered as a function that takes an input and returns an output. • Pseudocode: Provides an intermediate step between an English language description and a implementation in a programming language. We want to use instructions resembling those of a programming language. We don’t use any particular language (like C or Java) to describe the steps in an algorithm, since we want to be able to implement the algorithm in every language. Search Algorithms • General Problem: Locate an element x in a list of elements a1, a2 , a3 , …, an. • Example: Locate a word in a dictionary. • Linear Search Algorithm: Compare x with a1. If x = a1, return 1. Otherwise, compare x with a2. If x = a2, return 2. Continue in this process until we either find x, then the solution will be the location of x or we exhaust the list (in which case we return 0). Linear Search • Linear Search Algorithm: (Pseudocode) procedure linear search (x: integer, a1,a2,a3,…, an: distinct integers) i=1 while (i ≤ n and x ≠ ai ) i = i+1 if i ≤ n then return i else return 0 Binary Search • Binary Search Algorithm: If the terms in the list are in order of increasing size, a1 ≤ a2 ≤ a3 ≤ … ≤ an we can use the following procedure: Compare x to the middle term of the list am where m = ⌊(n+1)/2⌋ . If x > am, continue searching in the second half of the list. Otherwise, continue searching in the first half of the list. In either case, we now have a new list to search of half the size as the original. Use the same procedure on this new list, on so on. Binary Search • Binary Search Algorithm: i = left endpoint of search interval j = right endpoint of search interval Start: i = 1, j = n, x = (term to find) While i<j do the following: Compute middle index of list, m = ⌊(i+j)/2⌋ If x > am, search in second half of list (am+1, am+2 , … aj) and set i = m+1 Else search in first half of list (ai, ai+1 , … am) and set j = m Binary Search Example: Find index of 13 in (1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16) Start: i=1, j=12, x = 13 (1) m = ⌊(i+j)/2 =13/2⌋ = 6 and am = 7. Since x > 7, we set i = m+1 = 7 and continue searching in (8 10 12 13 15 16). [i=7, j=12] (2) m = ⌊(i+j)/2⌋ = 9 and am = 12. Since x > 12, we set i = m+1 = 10 and continue searching in (13 15 16). [i=10, j=12] (3) m = ⌊22/2⌋ = 11 and am = 15. Since x < 15, set j= m = 11 and continue searching in (13 15). [i=10, j=11] (4) m = ⌊21/2⌋ =10 and am = 13. Since x = 13, set j=m=10. We are done, since i=j=10. Return 10. Binary Search • Binary Search Algorithm: (Pseudocode) procedure Binary search(x: integer, a1,a2,a3,…, an: increasing integers) i = 1 (left endpoint of search interval) j = 1 (right endpoint of search interval) while (i < j) begin m = ⌊(i+j)/2⌋ if x > am then i = m+1 else j = m end if x = ai return i else return 0 public class BinarySearch { public static int search(int x, int[] list) { int i=0, j = list.length-1, m =0; while (i < j) { m = (int)Math.floor( ((double)(i+j)) / 2.0); System.out.println("i = " + i + ", j = " + j + ", m = " + m); System.out.println(" x = " + x + ", list[m]= " + list[m]); if(x > list[m]) i = m+1; else j = m; } System.out.println("last step: i = " + i + ", j = " + j + ", list[i] = " + list[i]); if(x == list[i]) return i; else return 0; } public static void main(String arg[]) { int list[] = {1,2,3,5,6,7,8,10,12,13,15,16}; int x = 13; int m = BinarySearch.search(x,list); System.out.println("index of x in list = " + m); } } Java compiling • Third, we would type “javac BinarySearch.java” to compile our java program. If successful, a file called “BinarySearch.class” is created. This is the bytecode for our program. • Lastly, to run the program type “java BinarySearch” Java Compiling Question Question Class exercise: List all the steps used to search for 9 in the sequence 1,3,4,5,6,8,9,11 using (a) a linear search (b) a binary search Answer: (a) i=1,i=2,i=3,i=4,i=5,i=6,i=7 So Location=7 (b) i=1,j=8,m=4,i=5,j=8,m=6,i=7,j=8,m=7,j=7,location=7 Sorting • Input sequence <a1,a2,….,an> • Output sequence<a1’,a2’,…,an’> such that: a1’<=a2’<=a3’…<=an’ or a1’>=a2’>=a3’…>=an’ Sorting • Bubble sort is one of the simplest sorting algorithms, but not one of the most efficient. The smaller elements “bubble” to the top ,the larger elements “sink” to the bottom. • The Bubble sort algorithm: Procedure bubblesort(a1,…,an :real numbers with n>=2) for i:=1 to n-1 for j=1 to n-I if aj > aj+1 then interchange aj and aj+1 {a1,…,an is in increasing order} Sorting • Use the bubble sort to put 3,2,4,1,5 into increasing order. Sorting • Insertion sort is a simple sorting algorithm, a comparison sort in which the sorted array is built one entry at a time. • The insertion sort algorithm: Procedure insertion sort(A[1:n]) for j:=2 to n do key=A [j] i=j-1; while i>0 and A [i]>key do A [i+1]:=A [i] i:=i-1 end A [i+1]:=key End {A [n] are sorted} j i key Sorting • Insertion Example: 824936 284936 248936 248936 234896 2 3 4 6 8 9 done Question • Class Exercise: Sorting the sequence numbers: 6,2,3,1,5,4 using (a) Bubble sort (b) Insertion sort Algorithm Analysis • Why need algorithm analysis ? Writing a working program is not good enough The program may be inefficient! If the program is run on a large data set, then the running time becomes an issue The growth of functions • Without worry about the hardware and software used to implement an algorithm. • Assume the different operations used in an algorithm take the same time, which simplifier the analysis. • Determine whether it is practical to use a particular algorithm to solve a problem as the size of the input increase • Compare two algorithms to determine which is more efficient as the size of input grows. Big-O Notation • Big-O notation is used extensively to estimate the number of operations an algorithm as its input grows. • Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers. • c , n0 > 0 such that f(N) c g(N) when N n0 • g(N) is an upper bound on f(N) • f(x) is big –O of g(x) Big-O Notation • F(x)=x2+2x+1 is O(x2) • We observe that we can readily estimate of f(x) when x>1 because x<x2 and 1<x2 when x>1. • x2<=x2+2x+1<=x2+2x2+x2=4x2 • Consequently, we can take c=4 and n0=1 Big-Omega • Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers. • c , n0 > 0 such that f(N) c g(N) when N n0 • g(N) is an lower bound on f(N) • f(x) is big – of g(x) Big-Theta • Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers. • f(N) = (g(N)) iff f(N) = O(g(N)) and f(N) = (g(N)) • g(N) is both an upper and a lower bound on the size of f(N) Big-Theta • Show that 3x2+8x log x is (x2) 0 8x log x 8 x2, it follows that 3x2+8x log x 11 x2 for x>1. Consequently, 3x2+8x log x is O(X2). Clearly. X2 is O(3x2+8x log x ). So, 3x2+8x log x is (x2) Exercise Class Exercise: Using the definition of Big-O notation to proof: (1) Suppose that f1(x) is O(g1(x)) and f2x is O(g2(x)). Then (f1+f2)(x) is O(max(|g1(x)|,|g2(x)|). (2) Suppose that f1(x) is O(g1(x)) and f2x is O(g2(x)). Then (f1f2)(x) is O(g1(x)g2(x)). Complexity of Algorithm • Best Case – constraints on the input, other than size, resulting in the fastest possible running time. • Worst Case – constraints on the input, other than size, resulting in the slowest possible running time. • Average Case – average running time over every possible type of input (usually involve probabilities of different types of input). Three Analyses of Insertion Sorting Best Case A[1] ≤ A[2] ≤ A[3] ≤ ··· ≤ A[n] The number of comparisons needed is equal to Worst Case A[1] ≥ A[2] ≥ A[3] ≥ ··· ≥ A[n] The number of comparisons needed is equal to Average Case Θ(n2) assuming that each of the n! instances are equally like Complexity of Algorithm Commonly Used Terminology for the Complexity of Algorithm Complexity Terminology Θ(1) Constant Complexity Θ(log n) Logarithmic Complexity Θ(n) Linear Complexity Θ(n log n) N log N complexity Θ(n b) Polynomial Complexity Θ(b n) Exponential Complexity Θ(n!) Factorial Complexity