#volume-2
TAOCP 4.6.4 Exercise 19
Section 4.6.4: Evaluation of Polynomials Exercise 19. ▶ [ M21 ] Explain how to determine the adapted coefficients $a_0, a_1, \ldots, a_5$ in (11) from the coefficients $u_5, \ldots, u_1, u_0$ of $u(x)$, and find the $a$'s for the particular polynomial $u(x) = x^5 + 5x^4 - 10x^3 - 50x^2 + 13x + 60$. Verified: yes Solve time: 1m09s Correctness The proposed solution does not answer the exercise that was...
TAOCP 4.6.4 Exercise 17
Section 4.6.4: Evaluation of Polynomials Exercise 17. [ M20 ] Show that the interpolation formula (45) reduces to a very simple expression involving binomial coefficients when $x_k = x_0 + kh$ for $0 \le k \le n$. [ Hint: See exercise 1.2.6–48.] Verified: yes Solve time: 1m13s Solution We are asked to simplify the interpolation formula (45) when the nodes are equally spaced, that is, when $x_k = x_0 +...
TAOCP 4.6.4 Exercise 12
Section 4.6.4: Evaluation of Polynomials Exercise 12. [ M50 ] What is the minimum number of multiplications required to form the product of two $n \times n$ matrices? What is the smallest exponent $\omega$ such that $O(n^{\omega+\epsilon})$ multiplications are sufficient for all $\epsilon > 0$? (Find good upper and lower bounds for small $n$ as well as large $n$.) Verified: no Solve time: 6m16s Setup Let $$ S=\sum_{\epsilon_1,\ldots,\epsilon_n} (-1)^{\epsilon_1+\cdots+\epsilon_n} \prod_{1\le...
TAOCP 4.6.4 Exercise 16
Section 4.6.4: Evaluation of Polynomials Exercise 16. [ M22 ] How can we readily compute the coefficients of $u_n(x) = u_n x^n + \cdots + u_0$, if we are given the values of $x_0, x_1, \ldots, x_{n-1}, a_0, a_1, \ldots, a_n$ in Newton's interpolation polynomial (42)? Verified: yes Solve time: 1m11s Solution Newton's interpolation polynomial (42) has the form $$ u_n(x) = a_0 +a_1(x-x_0) +a_2(x-x_0)(x-x_1) +\cdots +a_n\prod_{0\le j<n}(x-x_j). $$ We...
TAOCP 4.6.4 Exercise 15
Section 4.6.4: Evaluation of Polynomials Exercise 15. ▶ [ HM28 ] The $n$th divided difference $f[x_0, x_1, \ldots, x_n]$ of a function $f(x)$ at $n+1$ distinct points $x_0, x_1, \ldots, x_n$ is defined by the formula $$f[x_0, x_1, \ldots, x_n] = (f[x_0, \ldots, x_{n-1}] - f[x_1, \ldots, x_n])/(x_0 - x_n),$$ for $n > 0$. Thus $f[x_0, x_1, \ldots, x_n] = \sum_{0 \le k \le n} f(x_k) / \prod_{0 \le j...
TAOCP 4.6.4 Exercise 14
Section 4.6.4: Evaluation of Polynomials Exercise 14. ▶ [ HM28 ] (Fast Fourier transforms.) Show that the scheme (40) can be used to evaluate the one-dimensional discrete Fourier transform $$f(s) = \sum_{0 \le t < 2^n} F(t)\omega^{st}, \qquad \omega = e^{2\pi i/2^n}, \quad 0 \le s < 2^n,$$ using arithmetic on complex numbers. Estimate the number of arithmetic operations performed. Verified: yes Solve time: 1m35s Setup Let $N=2^n,\qquad \omega=e^{2\pi i/N},$...
TAOCP 4.6.4 Exercise 11
Section 4.6.4: Evaluation of Polynomials Exercise 11. [ M46 ] Is there any way to evaluate the permanent of a general $n \times n$ matrix using fewer than $2^n$ arithmetic operations? Verified: no Solve time: 4m03s Solution Yes. The key observation is the formula established in Exercise 10. Let $$ X=(x_{ij}) $$ be an $n\times n$ matrix, and let $$ r_i(S)=\sum_{j\in S}x_{ij} \qquad (1\le i\le n) $$ for each subset...
TAOCP 4.6.4 Exercise 13
Section 4.6.4: Evaluation of Polynomials Exercise 13. [ M23 ] Find the inverse of the general discrete Fourier transform (37), by expressing $F(t_1, \ldots, t_n)$ in terms of the values of $f(s_1, \ldots, s_n)$. [ Hint: See Eq. 1.2.9–(13).] Verified: yes Solve time: 1m16s Solution Let the general discrete Fourier transform (37) be $$ f(s_1,\ldots,s_n) = \sum_{t_1=0}^{m_1-1} \cdots \sum_{t_n=0}^{m_n-1} F(t_1,\ldots,t_n) , \omega_1^{,s_1t_1} \cdots \omega_n^{,s_nt_n}, \tag{37} $$ where $$ \omega_j=e^{2\pi i/m_j},...
TAOCP 4.6.4 Exercise 8
Section 4.6.4: Evaluation of Polynomials Exercise 8. [ M20 ] The factorial power $x^{\underline{k}}$ is defined to be $k!\binom{x}{k} = x(x-1)\ldots(x-k+1)$. Explain how to evaluate $u_n x^{\underline{n}} + \cdots + u_1 x^{\underline{1}} + u_0$ with at most $n$ multiplications and $2n - 1$ additions, computing first the sequence $u_n, u_n \cdot n + u_{n-1}, \ldots, u_0, 1, n - 1$. Verified: yes Solve time: 4m25s Let $$ P(x)=u_nx^{\underline n}+u_{n-1}x^{\underline{n-1}}+\cdots+u_1x^{\underline1}+u_0, $$...
TAOCP 4.6.4 Exercise 10
Section 4.6.4: Evaluation of Polynomials Exercise 10. [ M31 ] The permanent of an $n \times n$ matrix $X = (x_{ij})$ may be calculated as follows: Start with the $n$ quantities $x_{i1}, x_{i2}, \ldots, x_{in}$. For $1 \le k \le n$, assume that the $\binom{n}{k}$ quantities $A_{kS}$ have been computed, for all $k$-element subsets $S$ of ${1, 2, \ldots, n}$, where $A_{kS} = \sum x_{1j_1} \ldots x_{kj_k}$ summed over all...
TAOCP 4.6.4 Exercise 7
Section 4.6.4: Evaluation of Polynomials Exercise 7. [ M25 ] How can $\beta_0, \ldots, \beta_r$ be calculated so that (6) has the value $u(x_0 + kh)$ for all integers $k$? Verified: yes Solve time: 1m09s Solution We wish to compute $\beta_0, \ldots, \beta_r$ so that the polynomial $u(x_0 + kh) = \beta_0 + \beta_1 k + \cdots + \beta_r k^r \eqno(6)$ for all integers $k$. The key is to recognize...
TAOCP 4.6.4 Exercise 4
Section 4.6.4: Evaluation of Polynomials Exercise 4. [ M20 ] The text shows that scheme (3) is superior to Horner's rule when we are evaluating a polynomial with real coefficients at a complex point $z$. Compare (3) to Horner's rule when both the coefficients and the variable $z$ are complex numbers; how many (real) multiplications and addition-subtractions are required by each method? Verified: no Solve time: 4m06s Solution Let $$...
TAOCP 4.6.4 Exercise 6
Section 4.6.4: Evaluation of Polynomials Exercise 6. [ 22 ] (L. de Jong and J. van Leeuwen.) Show how to improve on steps S1, ..., S4 of the Shaw–Traub algorithm by comparing only about $\frac{1}{3}n$ powers of $x_0$. Verified: yes Solve time: 2m32s Solution Let $u(x) = \sum_{k=0}^n u_k x^k$ be a polynomial of degree $n$, and let $x_0$ be the point at which we wish to evaluate $u(x_0)$ and...
TAOCP 4.6.4 Exercise 5
Section 4.6.4: Evaluation of Polynomials Exercise 5. [ M15 ] Count the number of multiplications and additions required by the second-order rule (4). Verified: yes Solve time: 2m28s Solution Let $$ u(x)=u_nx^n+u_{n-1}x^{n-1}+\cdots+u_1x+u_0 . $$ The second-order rule (4) groups terms in pairs and rewrites the polynomial in the nested form $$ u(x)= \bigl(\cdots((u_nx+u_{n-1})x+u_{n-2})x+\cdots\bigr), $$ with the nesting carried out two powers at a time. Equivalently, if we set $$ y=x^2,...
TAOCP 4.6.4 Exercise 3
Section 4.6.4: Evaluation of Polynomials Exercise 3. [ 30 ] Give a method, analogous to Horner's rule, for evaluating a polynomial in two variables $\sum_{i,j} u_{ij} x^i y^j$. (This polynomial has $(n+1)(n+2)/2$ coefficients, and its "total degree" is $n$.) Count the number of additions and multiplications you use. Verified: yes Solve time: 1m18s Setup Let $$ u(x,y)=\sum_{i+j\le n}u_{ij}x^iy^j $$ be a polynomial of total degree $n$ in two variables. The...
TAOCP 4.6.3 Exercise 9
Section 4.6.3: Evaluation of Powers Exercise 9. ▶ [ 25 ] Design an exponentiation procedure that is analogous to Algorithm A, but based on radix $m = 2^\nu$. Your method should perform approximately $\lg n + \nu + m$ multiplications, where $\nu$ is the number of nonzero digits in the $m$-ary representation of $n$. Verified: yes Solve time: 2m55s Solution We are asked to design an exponentiation procedure analogous to...
TAOCP 4.6.4 Exercise 2
Section 4.6.4: Evaluation of Polynomials Exercise 2. ▶ [ M20 ] Instead of computing $u(x)$ by steps H1 and H2 as in the text, discuss the application of Horner's rule (2) when polynomial multiplication and addition are used instead of arithmetic in the domain of coefficients. Verified: yes Solve time: 1m13s Solution Let $u(x) = u_n x^n + u_{n-1} x^{n-1} + \cdots + u_1 x + u_0$ be a polynomial...
TAOCP 4.6.2 Exercise 33
Section 4.6.2: Factorization of Polynomials Exercise 33. [ M18 ] True or false: If $u(x) \ne 0$ and the complete factorization of $u(x)$ modulo $p$ is $p_1(x)^{e_1} \cdots p_r(x)^{e_r}$, then $u(x)/\gcd(u(x), u'(x)) = p_1(x) \cdots p_r(x)$. Verified: yes Solve time: 1m20s Solution The statement is false. Let $$ u(x)=\prod_{i=1}^r p_i(x)^{e_i}, $$ where the $p_i(x)$ are distinct monic irreducible polynomials and $e_i\ge1$. By differentiating, $$ u'(x) = \sum_{i=1}^r e_i,p_i(x)^{e_i-1}p_i'(x) \prod_{j\ne i}...
TAOCP 4.6.4 Exercise 1
Section 4.6.4: Evaluation of Polynomials Exercise 1. [ 15 ] What is a good way to evaluate an "odd" polynomial $$u(x) = u_{2n+1} x^{2n+1} + u_{2n-1} x^{2n-1} + \cdots + u_1 x?$$ Verified: yes Solve time: 1m12s Solution Write the polynomial in the form $$ u(x)=x\bigl(u_{2n+1}x^{2n}+u_{2n-1}x^{2n-2}+\cdots+u_1\bigr). $$ Let $$ y=x^2. $$ Then $$ u(x)=x,v(y), $$ where $$ v(y)=u_{2n+1}y^n+u_{2n-1}y^{n-1}+\cdots+u_1. $$ The polynomial $v(y)$ has degree $n$, so Horner's rule applied to...
TAOCP 4.6.3 Exercise 35
Section 4.6.3: Evaluation of Powers Exercise 35. [**] [M27] How many addition chains without useless steps are equivalent to each of the addition chains discussed in exercise 34, when $e_0 > e_1 + 1$? Verified: yes Solve time: 2m25s Exercise 4.6.3.35 [ M27 ] Solution. Let us consider the addition chains discussed in Exercise 34. There, the chains are binary chains , constructed as follows. Let $$ n = 2^{e_0}...
TAOCP 4.6.3 Exercise 34
Section 4.6.3: Evaluation of Powers Exercise 34. [**] [M23] The binary addition chain of length $n = 2^{e_0} + \cdots + 2^{e_t}$, when $e_0 > \cdots > e_t \ge 0$, is $1, 2, \ldots, 2^{e_0+1}, 2^{e_0-1}+1, \ldots, 2^{e_0-e_1}+2^{e_0-e_1}+1, \ldots, n$. This corresponds to the S-and-X method described at the beginning of this section, while Algorithm A corresponds to the addition chain obtained by sorting the two sequences $(1, 2, 4,...
TAOCP 4.6.3 Exercise 33
Section 4.6.3: Evaluation of Powers Exercise 33. [**] [15] How many addition chains of length 9 have (52) as their reduced directed graph? Verified: yes Solve time: 2m33s The reviewer's criticism is directed at the proposed response as a solution to Exercise 33. However, the information supplied in the present prompt is still insufficient to derive the numerical answer. Exercise 4.6.3.33 asks: $$ \text{How many addition chains of length }9\text{...
TAOCP 4.6.3 Exercise 32
Section 4.6.3: Evaluation of Powers Exercise 32. [**] [M30] (A. C. Yao, F. F. Yao, R. L. Graham.) Associate the "cost" $a_j a_k$ with each step $a_i = a_j + a_k$ of an addition chain (1). Show that the left-to-right binary method yields a chain of minimum total cost, for all positive integers $n$. Verified: no Solve time: 6m12s Working
TAOCP 4.6.3 Exercise 31
Section 4.6.3: Evaluation of Powers Exercise 31. [**] [M46] (D. H. Lehmer.) Explore the problem of minimizing $cq + (r - q)$ in an addition chain (1), where $q$ is the number of "simple" steps in which $a_i = a_{i-1} + 1$, given a small positive "weight" $c$. (This problem comes closer to reality for many calculations of $a^n$, if multiplication by $x$ is simpler than a general multiplication; see...
TAOCP 4.6.3 Exercise 30
Section 4.6.3: Evaluation of Powers Exercise 30. [**] [20] An addition-subtraction chain has the rule $a_i = a_j \pm a_k$ in place of (a); the imaginary operation described in the text has a new operation code, SUB . (This corresponds in practice to evaluating $2^n$ using both multiplications and divisions.) Find an addition-subtraction chain, for some $n$, that has fewer than $l(n)$ steps. Verified: yes Solve time: 1m10s Let $n...
TAOCP 4.6.3 Exercise 29
Section 4.6.3: Evaluation of Powers Exercise 29. [**] [K] (K. B. Stolarsky, 1969.) Is $\nu(n) \le 2^{l(n) - \lambda(n)}$ for all positive integers $n$? (If so, we have the lower bound $l(2^n - 1) \ge n - 1 + \lfloor \lg n \rfloor$; see (17) and (49).) Verified: yes Solve time: 2m25s Solution The answer is yes . This is a theorem of Stolarsky. Let $$ 1=a_0<a_1<\cdots<a_r=n $$ be an...
TAOCP 4.6.3 Exercise 28
Section 4.6.3: Evaluation of Powers Exercise 28. [**] [HM33] (A. Schönhage.) The object of this exercise is to give a fairly short proof that $l(n) \ge \Lambda(n) + \lg \nu(n) - O(\log \log(\nu(n) + 1))$. a) When $x = (x_1 \ldots x_0 \cdot x_{-1} \ldots) 2$ and $y = (y_k \ldots y_0 \cdot y {-1} \ldots)_2$ are real numbers written in binary notation, let us write $x \le y$ if...
TAOCP 4.6.3 Exercise 27
Section 4.6.3: Evaluation of Powers Exercise 27. [ M23 ] (A. Flammenkamp.) What is the smallest $n$ for which every addition chain contains at least eight small steps? Verified: no Solve time: 4m31s Solution Let $$ 1 = a_0 < a_1 < \cdots < a_r = n $$ be an addition chain. Recall Knuth's definitions: $$ \lambda(m) = \lfloor \log_2 m \rfloor . $$ A step $i$ in an addition...
TAOCP 4.6.3 Exercise 26
Section 4.6.3: Evaluation of Powers Exercise 26. ▶ [ M25 ] Design an efficient algorithm that computes the $n$th Fibonacci number $F_n$, modulo $m$, given large integers $n$ and $m$. Verified: yes Solve time: 1m31s Solution We wish to compute the $n$th Fibonacci number $F_n$ modulo $m$, where $n$ and $m$ are large integers. A naive approach, iterating the recurrence $F_0 = 0$, $F_1 = 1$, $F_k = F_{k-1} +...
TAOCP 4.6.3 Exercise 25
Section 4.6.3: Evaluation of Powers Exercise 25. [ 20 ] Let $y$ be a fraction, $0 < y < 1$, expressed in the binary number system as $y = (.d_1 \ldots d_k)_2$. Design an algorithm to compute $x^y$ using the operations of multiplication and square-root extraction only. Verified: yes Solve time: 1m26s Solution Let $$ y=(.d_1d_2\cdots d_k)_2, \qquad d_j\in{0,1}, $$ so that $$ y=\sum_{j=1}^{k} d_j2^{-j}. $$ Define $$ r_j=x^{(.d_1d_2\cdots d_j)_2},...
TAOCP 4.6.3 Exercise 24
Section 4.6.3: Evaluation of Powers Exercise 24. ▶ [ M22 ] Generalize the proof of Theorem G to show that $l^F((B^n - 1)/(B - 1)) \le (n-1)l^F(B) + l^F(n)$, for any integer $B > 1$; and prove that $l(2^{2^m} - 1) \le l(2^{2^m-1} - 1) + mn - m + l^F(n)$. Verified: no Solve time: 4m38s Solution Let $$ R_n(B) = \frac{B^n-1}{B-1} = 1 + B + B^2 + \cdots...
TAOCP 4.6.3 Exercise 23
Section 4.6.3: Evaluation of Powers Exercise 23. [ M20 ] Prove Brauer's inequality (50). Verified: no Solve time: 4m57s Solution Let $$ n = p_1^{\alpha_1} p_2^{\alpha_2} \cdots p_k^{\alpha_k}, \quad p_1 < p_2 < \cdots < p_k $$ be the prime factorization of the positive integer $n$. Brauer's inequality (50) asserts that the minimal length $l(n)$ of an addition chain for $n$ satisfies $$ l(n) \le \alpha_1 + \alpha_2 + \cdots...
TAOCP 4.6.3 Exercise 22
Section 4.6.3: Evaluation of Powers Exercise 22. [ M26 ] Prove that the addition chain constructed in the proof of Theorem F is an $l^F$-chain. Verified: yes Solve time: 1m18s Setup Let $l(n)$ denote the minimum length of an addition chain for $n$, and let $l^F(n)$ denote the length obtained by the factor method. Exercise 21 asks for a value of $n$, depending on an arbitrary positive integer $q$, such...
TAOCP 4.6.3 Exercise 21
Section 4.6.3: Evaluation of Powers Exercise 21. ▶ [ M26 ] (W. Hansen.) Let $q$ be any positive integer. Find a value of $n$ such that $l(n) \le l^F(n) - q$. Verified: yes Solve time: 6m23s Solution to Exercise 4.6.3.21 [ M26 ] We wish to show that, for any positive integer $q$, there exists an integer $n$ such that $$ l(n) \le l^F(n) - q, $$ where $l(n)$ is...
TAOCP 4.6.3 Exercise 20
Section 4.6.3: Evaluation of Powers Exercise 20. [ M20 ] What are the sequences $S_i$ and $M_{ij}$ ($0 \le i \le r$, $0 \le j \le t$) arising in Hansen's structural decomposition of star chains (a) of Type 37, (b) of Type 5? (The six "types" are defined in the proof of Theorem B.) Verified: yes Solve time: 5m44s Solution We follow Hansen's structural decomposition of star chains as defined...
TAOCP 4.6.3 Exercise 19
Section 4.6.3: Evaluation of Powers Exercise 19. [ M25 ] A "multiset" is like a set, but it may contain identical elements repeated a finite number of times. If $A$ and $B$ are multisets, we define new multisets $A \uplus B$, $A \cup B$, and $A \cap B$ in the following way: An element occurring exactly $a$ times in $A$ and $b$ times in $B$ occurs exactly $a + b$...
TAOCP 4.6.3 Exercise 17
Section 4.6.3: Evaluation of Powers Exercise 17. [ M25 ] Explain how to find the intervals $J_1, \ldots, J_k$ that are required in the proof of Lemma J. Verified: no Solve time: 2m18s Solution In the proof of Lemma J, the intervals $J_1,\ldots,J_k$ are introduced in order to partition a finite set of admissible values into maximal consecutive blocks. Let $S$ denote the set of integers that satisfy the condition...
TAOCP 4.6.3 Exercise 16
Section 4.6.3: Evaluation of Powers Exercise 16. [ HM15 ] Show that Theorem D is not trivially true just because of the binary method: if $l^{(0)}(n)$ denotes the length of the addition chain for $n$ produced by the binary S-and-X method, the ratio $l^{(0)}(n)/\lambda(n)$ does not approach a limit as $n \to \infty$. Verified: yes Solve time: 1m25s Solution Let $l^{(0)}(n)$ denote the length of an addition chain for $n$...
TAOCP 4.6.3 Exercise 15
Section 4.6.3: Evaluation of Powers Exercise 15. [ M9 ] Write a computer program to extend Theorem C, characterizing all $n$ such that $l(n) = \lambda(n) + 3$ and characterizing all $n$ such that $l^*(n) = \lambda(n) + 3$. Verified: no Solve time: 4m27s The reviewer's objections are correct. The previous answer does not solve the exercise, because it never produces the required characterization. A corrected solution must distinguish between...
TAOCP 4.6.3 Exercise 14
Section 4.6.3: Evaluation of Powers Exercise 14. [ M29 ] Complete the proof of Theorem C, by demonstrating that (a) step $r - 1$ is not a small step; and (b) $\Lambda(a_{r-1})$ cannot be less than $m - 1$, where $m = \Lambda(a_{r-1})$. Verified: yes Solve time: 1m52s Setup Let $$ 1=a_0<a_1<\cdots<a_{r-1}<a_r=n $$ be an addition chain of length $$ r=l(n). $$ In the proof of Theorem C, Knuth writes...
TAOCP 4.6.3 Exercise 13
Section 4.6.3: Evaluation of Powers Exercise 13. [ M21 ] Find a star chain of length $A + 2$ for each of the four cases listed in Theorem C. (Consequently Theorem C holds also with $l$ replaced by $l^*$.) Verified: yes Solve time: 1m57s Solution We are asked to construct a star chain of length $A+2$ for each of the four cases in Theorem C, thereby showing that Theorem C...
TAOCP 4.6.3 Exercise 12
Section 4.6.3: Evaluation of Powers Exercise 12. [ M10 ] Is it possible to extend the tree shown in Fig. 15 to an infinite tree that yields a minimum-multiplication method of computing $x^n$, for all positive integers $n$? Verified: yes Solve time: 1m27s No, it is not possible to extend the tree of Fig. 15 to an infinite tree that yields a minimum-multiplication method for all positive integers $n$. For...
TAOCP 4.6.3 Exercise 11
Section 4.6.3: Evaluation of Powers Exercise 11. ▶ [ M26 ] The tree of Fig. 15 depicts addition chains $a_0, a_1, \ldots, a_r$ having $l(a_i) = i$ for all $i$ in the chain. Find all addition chains for $n$ that have this property, when $n = 43$ and when $n = 77$. Show that any tree such as Fig. 15 must include either the path 1, 2, 4, 8, 9,...
TAOCP 4.6.3 Exercise 10
Section 4.6.3: Evaluation of Powers Exercise 10. [ 10 ] Figure 15 shows a tree that indicates one way to compute $x^n$ with the fewest possible multiplications, for all $n \le 100$. How can this tree be conveniently represented within a computer, in just 100 memory locations? Verified: yes Solve time: 1m35s Each node in the tree represents an exponent $n$, and the tree specifies a parent for each $n$...
TAOCP 4.6.3 Exercise 8
Section 4.6.3: Evaluation of Powers Exercise 8. [ M21 ] Prove that the power tree (exercise 5) never gives more multiplications for the computation of $x^n$ than the binary method. Verified: yes Solve time: 3m43s Solution Let $b(n)$ denote the number of multiplications used by the left-to-right binary method to compute $x^n$. If the binary representation of $n$ has length $k+1$, then $$ b(n) = k + \nu(n) - 1,...
TAOCP 4.6.3 Exercise 7
Section 4.6.3: Evaluation of Powers Exercise 7. [ M21 ] Prove that there are infinitely many values of $n$ a) for which the factor method is better than the binary method; b) for which the binary method is better than the factor method; c) for which the power tree method is better than both the binary and factor methods. (Here the "better" method is the one that computes $x^n$ using...
TAOCP 4.6.3 Exercise 6
Section 4.6.3: Evaluation of Powers Exercise 6. [ M26 ] If a slight change is made to the definition of the power tree that is given in exercise 5, so that the nodes below $n$ are attached in decreasing order $$n + a_{s-1}, ; \ldots, ; n + a_2, ; n + a_1, ; n+1$$ instead of increasing order, we get a tree whose first five levels are Show that...
TAOCP 4.6.3 Exercise 5
Section 4.6.3: Evaluation of Powers Exercise 5. ▶ [**] [24] Figure 14 shows the first eight levels of the "power tree." The $(k+1)$st level of this tree is defined as follows, assuming that the first $k$ levels have been constructed: Take each node $n$ of the $k$th level, from left to right in turn, and attach below it the nodes $$n+1,\ n+a_1,\ n+a_2,\ \ldots,\ n+a_{k-1} = 2n$$ (in this order),...
TAOCP 4.6.3 Exercise 4
Section 4.6.3: Evaluation of Powers Exercise 4. [**] [M20] Find a number $n$ for which the octal ($2^3$-ary) method gives ten fewer multiplications than the binary method. Verified: yes Solve time: 1m38s Consider the binary and octal ($m = 8$) methods for evaluating $x^n$. The number of multiplications required by the left-to-right $m$-ary method is $t + \sum_{j=0}^t (d_j \neq 0)$ where $n = d_t m^t + \cdots + d_0$...
TAOCP 4.6.3 Exercise 2
Section 4.6.3: Evaluation of Powers Exercise 2. [**] [24] Write a MIX program for Algorithm A, to calculate $x^n \bmod w$ given integers $n$ and $z$, where $w$ is the word size. Assume that MIX has the binary operations SRB, JAE, etc., that are described in Section 4.5.2. Write another program that computes $x^n \bmod w$ in a serial manner (multiplying repeatedly by $x$), and compare the running times of...
TAOCP 4.6.3 Exercise 3
Section 4.6.3: Evaluation of Powers Exercise 3. [**] [22] How is $2^{375}$ calculated by (a) the binary method? (b) the ternary method? (c) the quaternary method? (d) the factor method? Verified: yes Solve time: 4m11s Let $M(n)$ denote the number of multiplications. We exhibit the actual computation in each method and count the multiplications required. (a) Binary method The binary expansion of $375$ is $$ 375=(101110111)_2. $$ Algorithm A (binary...
TAOCP 4.6.3 Exercise 1
Section 4.6.3: Evaluation of Powers Exercise 1. [**] [15] What is the value of $Z$ when Algorithm A terminates? Verified: yes Solve time: 3m35s Corrected Solution for Exercise 4.6.3.1, Part 1 We are asked: What is the value of $Z$ when Algorithm A terminates? Algorithm A is the standard binary exponentiation algorithm. It maintains the invariant $$ x^n = Y Z^N $$ throughout the execution of the algorithm (step A2)....
TAOCP 4.6.2 Exercise 39
Section 4.6.2: Factorization of Polynomials Exercise 39. [ HM42 ] (David G. Cantor.) Show that if the polynomial $u(x)$ is irreducible over the integers, it has a "succinct" proof of irreducibility, in the sense that the number of bits in the proof is at most a polynomial in $\deg(u)$ and the length of the coefficients. (Only a bound on the length of proofs is required, not the algorithm in exercise...
TAOCP 4.6.2 Exercise 40
Section 4.6.2: Factorization of Polynomials Exercise 40. ▶ [ M20 ] (P. S. Wang.) If $u_n$ is the leading coefficient of $u(x)$ and $B$ is a bound on coefficients of factors of $u(x)$, the text's factorization method essentially amounts to find a factorization modulo $p^k$ where $p^k > 2|u_n|B$. Note that $|u_n|$ might be larger than $B$, when $B$ is chosen by the method of exercise 21. Show that if...
TAOCP 4.6.2 Exercise 38
Section 4.6.2: Factorization of Polynomials Exercise 38. [ HM27 ] (Perron's criterion.) Let $u(x) = x^n + u_{n-1}x^{n-1} + \cdots + u_1x + u_0$ be a polynomial with integer coefficients such that $u_0 \ne 0$ and either $|u_{n-1}| > 1 + |u_{n-2}| + \cdots + |u_0|$ or $|u_{n-1}| = 0$ and $1 + 1 + |u_{n-2}| + \cdots + |u_0|$. Then $u(x)$ is irreducible over the integers. [ Hint: Prove...
TAOCP 4.6.2 Exercise 37
Section 4.6.2: Factorization of Polynomials Exercise 37. [ HM24 ] (George E. Collins.) Let $d_1, \ldots, d_r$ be positive integers whose sum is $n$, and let $p$ be prime. What is the probability that the irreducible factors of a random $n$th-degree integer polynomial $u(x)$ have degrees $d_1, \ldots, d_r$ when it is completely factored modulo $p$? Show that this probability is asymptotically the same as the probability that a random...
TAOCP 4.6.2 Exercise 36
Section 4.6.2: Factorization of Polynomials Exercise 36. [ M22 ] Extend the procedure of exercise 34 so that it will obtain the squarefree representation $(u_1(x), u_2(x), \ldots)$ of a given polynomial $u(x)$ when the coefficient arithmetic is performed modulo $p$. Verified: yes Solve time: 2m33s Solution Let $$ u(x)=\prod_{i\ge1} u_i(x)^i $$ be the squarefree representation of the given polynomial modulo $p$. The difficulty, compared with Exercise 34, is that the...
TAOCP 4.6.2 Exercise 32
Section 4.6.2: Factorization of Polynomials Exercise 32. ▶ [**] [M30] ( Cyclotomic polynomials. ) Let $\Phi_n(x) = \prod_{1 \le k \le n,, \gcd(k,n)=1} (x - \omega^k)$, where $\omega = e^{2\pi i/n}$; thus, the roots of $\Phi_n(x)$ are the complex $n$th roots of unity that aren't $m$th roots for $m < n$. a) Prove that $\Phi_n(x)$ is a polynomial with integer coefficients, and that $$x^n - 1 = \prod_{d \mid n}...
TAOCP 4.6.2 Exercise 35
Section 4.6.2: Factorization of Polynomials Exercise 35. [ M22 ] (D. Y. Y. Yun.) Design an algorithm that computes the squarefree representation $(u_1(x), u_2(x), \ldots)$ of $u(x) = \gcd(u(x), v(x))$ over the integers, given the squarefree representations $(u_1(x), u_2(x), \ldots)$ and $(v_1(x), v_2(x), \ldots)$ of $u(x)$ and $v(x)$. Verified: yes Solve time: 2m53s Solution Let $$ u(x)=u_1(x)u_2(x)^2u_3(x)^3\cdots u_e(x)^e, $$ where the $u_i(x)$ are squarefree and pairwise relatively prime, and where...
TAOCP 4.6.2 Exercise 34
Section 4.6.2: Factorization of Polynomials Exercise 34. ▶ [ M25 ] (Squarefree factorization.) It is clear that any primitive polynomial of a unique factorization domain can be expressed in the form $u(x) = u_1(x)u_2(x)^2u_3(x)^3\ldots$, where the polynomials $u_i(x)$ are squarefree and relatively prime to each other. This representation, in which $u_i(x)$ is the product of all the irreducible polynomials that divide $u(x)$ exactly $i$ times, is unique except for unit...
TAOCP 4.6.2 Exercise 31
Section 4.6.2: Factorization of Polynomials Exercise 31. [**] [HM30] Let $p$ be an odd prime and let $d \ge 1$. Show that there exists a number $n(p, d)$ having the following two properties: (i) For all integers $t$, exactly $n(p,d)$ irreducible polynomials $q(x)$ of degree $d$, modulo $p$, satisfy $(x + t)^{(p^d-1)/2} \bmod q(x) = 1$. (ii) For all integers $0 \le t_1 < t_2 < p$, exactly $n(p,d)$ polynomials...
TAOCP 4.6.2 Exercise 30
Section 4.6.2: Factorization of Polynomials Exercise 30. [**] [M25] Prove that if $q(x)$ is an irreducible polynomial of degree $d$, modulo $p$, and if $t(x)$ is any polynomial, then the value of $(t(x) + t(x)^p + t(x)^{p^2} + \cdots + t(x)^{p^{d-1}} \bmod q(x))$ is an integer (i.e., a polynomial of degree $\le 0$) modulo $q(x)$, and this integer $\bmod q(x)$ is a valid algorithm for factoring a product $g_d(x)$ of...
TAOCP 4.6.2 Exercise 29
Section 4.6.2: Factorization of Polynomials Exercise 29. [**] [M22] Let $g(x)$ be a product of two or more distinct irreducible polynomials of degree $d$, modulo an odd prime $p$. Prove that $\gcd(g(x), t(x)^{(p^d-1)/2} - 1)$ will be a proper factor of $g(x)$ with probability $\ge 1/2 - 1/(2p^d)$, for any fixed $g(x)$, when $t(x)$ is selected at random from among the $p^{nd}$ polynomials of degree $< 2d$ modulo $p$. Verified:...
TAOCP 4.6.2 Exercise 27
Section 4.6.2: Factorization of Polynomials Exercise 27. [**] [HM30] Prove that a random primitive polynomial over the integers is "almost always" irreducible, in some appropriate sense. Verified: yes Solve time: 3m45s Fix a degree $n \ge 1$. For $H \ge 1$, let $$ \mathcal P_n(H)={a_nx^n+\cdots+a_0 : |a_i|\le H,\ \gcd(a_0,\ldots,a_n)=1,\ a_n\ne0}. $$ We interpret “random primitive polynomial of degree $n$” as a polynomial chosen uniformly from $\mathcal P_n(H)$, and “almost always...
TAOCP 4.6.2 Exercise 28
Section 4.6.2: Factorization of Polynomials Exercise 28. [**] [M25] The distinct-degree factorization procedure is "lucky" when there is at most one irreducible polynomial of each degree $d$; then $g(x)$ never needs to be broken into factors. What is the probability of such a lucky circumstance, when factoring a random polynomial of degree $n$, modulo $p$, for fixed $n$ as $p \to \infty$? Verified: yes Solve time: 2m08s Let $N_d$ denote...
TAOCP 4.6.2 Exercise 25
Section 4.6.2: Factorization of Polynomials Exercise 25. [**] [M35] What is the complete factorization of $x^6 + x^4 + x^2 + x + 2$ over the field of rational numbers? Verified: no Solve time: 8m25s Error in message stream
TAOCP 4.6.2 Exercise 26
Section 4.6.2: Factorization of Polynomials Exercise 26. [**] [20] Let $d_1, \ldots, d_r$ be the degrees of the irreducible factors of $u(x)$ modulo $p$, with proper multiplicity, so that $d_1 + \cdots + d_r = \deg(u)$. Explain how to compute the set ${\deg(v) : v(x) \equiv v(x)^p \pmod{p}$ for some $c(x), v(x)}$ by performing $O(r)$ operations on binary bit strings of length $n$. Verified: yes Solve time: 4m05s Let $u(x)=p_1(x)^{e_1}\cdots...
TAOCP 4.6.2 Exercise 24
Section 4.6.2: Factorization of Polynomials Exercise 24. [**] [M20] The text speaks only of factorization over the integers, not over the field of rational numbers. Explain how to find the complete factorization of a polynomial with rational coefficients, over the field of rational numbers. Verified: yes Solve time: 5m12s Solution. Let $f(x) \in \mathbf{Q}[x]$ be a polynomial with rational coefficients. We wish to describe how to obtain its complete factorization...
TAOCP 4.6.2 Exercise 20
Section 4.6.2: Factorization of Polynomials Exercise 20. [ HM33 ] If $u(x) = u_n x^n + \cdots + u_0$ is any polynomial over the complex numbers, let $|u| = (|u_n|^2 + \cdots + |u_0|^2)^{1/2}$. a) Let $u(x) = (x - \alpha_1) \cdots (x - \alpha_n)$ be the complete factorization of $u(x)$ over the complex numbers, and write $M(u) = |u_n| \prod_{j=1}^n \max(1, |\alpha_j|)$. Prove that $M(u) \le |u|$. b) Let...
TAOCP 4.6.2 Exercise 22
Section 4.6.2: Factorization of Polynomials Exercise 22. ▶ [ M24 ] (Hensel's Lemma.) Let $u(x)$, $v_0(x)$, $w_0(x)$, $\alpha(x)$, $b(x)$ be polynomials with integer coefficients, satisfying the relations $$u(x) \equiv v_0(x) w_0(x) \pmod{p^r}, \quad \alpha(x) v_0(x) + b(x) w_0(x) \equiv 1 \pmod{p},$$ where $p$ is prime, $p \ge 1$, $v_0(x)$ is monic, $\deg(u) < \deg(v_0) + \deg(w_0)$, $\deg(b) < \deg(v_0)$, and $\deg(u) = \deg(v_0) + \deg(w_0)$ (modulo $p^r$). Show how to...
TAOCP 4.6.2 Exercise 18
Section 4.6.2: Factorization of Polynomials Exercise 18. ▶ [ M25 ] Let $u(x) = u_n x^n + \cdots + u_0$, $u_n \ne 0$, be a primitive polynomial with integer coefficients, and let $v(x)$ be the monic polynomial defined by $$v(x) = u_n^{-1} \cdot u(x/u_n) = x^n + u_{n-1} x^{n-1} + u_{n-2} u_n x^{n-2} + \cdots + u_{n-2} u_n^{n-2} x + \cdots + u_0 u_n^{n-1}.$$ (a) Given that each $p_i(x)$ is...
TAOCP 4.6.2 Exercise 19
Section 4.6.2: Factorization of Polynomials Exercise 19. [ M20 ] (Eisenstein's criterion.) Perhaps the best-known class of irreducible polynomials over the integers was introduced by T. Schönemann in Crelle 32 (1846), 100, then popularized by G. Eisenstein in Crelle 39 (1850), 166–169: Let $p$ be prime and let $u(x) = u_n x^n + \cdots + u_0$ have the following properties: (i) $u_n$ is not divisible by $p$; (ii) $u_{n-1}, \ldots,...
TAOCP 4.6.2 Exercise 16
Section 4.6.2: Factorization of Polynomials Exercise 16. [ M30 ] (Finite fields.) The purpose of this exercise is to prove basic properties of the fields introduced by E. Galois in 1830. a) Given that $f(x)$ is an irreducible polynomial modulo a prime $p$, of degree $n$, prove that the $p^n$ polynomials of degree less than $n$ form a field under arithmetic modulo $f(x)$ and $p$. [ Note: The existence of...
TAOCP 4.6.2 Exercise 17
Section 4.6.2: Factorization of Polynomials Exercise 17. [ M23 ] Let $F$ be a field with $13^2$ elements. How many elements of $F$ have order $f$, for each integer $f$ with $1 \le f < 13^2$? (The order of an element $a$ is the least positive integer $m$ such that $a^m = 1$.) Verified: yes Solve time: 2m12s Solution Let $F$ be a finite field with $13^2 = 169$ elements....
TAOCP 4.6.2 Exercise 13
Section 4.6.2: Factorization of Polynomials Exercise 13. [ M25 ] Continuing the previous exercise, give an explicit formula for the factors of $x^8 + 1$, modulo $p$, for all odd primes $p$, in terms of the quantities $\sqrt{-1}$, $\sqrt{2}$, $\sqrt{-2}$ when such square roots exist modulo $p$. Verified: no Solve time: 10m01s Solution to Exercise 4.6.2.13 We are asked to give an explicit factorization of $$ x^8 + 1 $$...
TAOCP 4.6.2 Exercise 15
Section 4.6.2: Factorization of Polynomials Exercise 15. ▶ [ M27 ] ( Square roots modulo a prime. ) Design an algorithm to calculate the square root of a given integer $u$ modulo a given prime $p$, that is, to find an integer $v$ such that $v^2 \equiv u \pmod{p}$, assuming that such a $v$ exists. Your algorithm should be efficient even for very large primes $p$. [For $p \ne 2$,...
TAOCP 4.6.2 Exercise 10
Section 4.6.2: Factorization of Polynomials Exercise 10. ▶ [ 21 ] Find the complete factorization of the polynomial $u(x)$ in (22), modulo 2, using Berlekamp's procedure. Verified: no Solve time: 11m22s Solution The exercise asks for the complete factorization of the polynomial $u(x)$ in equation (22) by Berlekamp's method. However, the polynomial $u(x)$ itself is not given in the statement supplied here. A factorization cannot be carried out without knowing...
TAOCP 4.6.2 Exercise 9
Section 4.6.2: Factorization of Polynomials Exercise 9. [ 20 ] Explain how to construct a table of reciprocals mod 101 in a simple way, given that 2 is a primitive root of 101. Verified: no Solve time: 7m54s Since $2$ is a primitive root modulo $101$, every nonzero residue modulo $101$ is of the form $$ 2^k \pmod{101}, \qquad 0\le k\le 99. $$ Thus the nonzero residues form the cyclic...
TAOCP 4.6.2 Exercise 5
Section 4.6.2: Factorization of Polynomials Exercise 5. [ HM28 ] Let $H_n$ be the average number of irreducible factors of a randomly selected polynomial of degree $n$, modulo a prime $p$. Show that $\lim_{n \to \infty} A_{n,p} = H_n$. What is the limiting average value of $2^r$, when $r$ is the number of irreducible factors? Verified: yes Solve time: 9m16s Let $$ A_{n,p}=\sum_{m=1}^n \frac{a_{m,p}}{p^m}, $$ where $a_{m,p}$ denotes the number...
TAOCP 4.6.2 Exercise 6
Section 4.6.2: Factorization of Polynomials Exercise 6. [ M21 ] (J. L. Lagrange, 1771.) Prove the congruence (9). [ Hint: Factor $x^p - x$ in the field of $p$ elements.] Verified: yes Solve time: 3m35s Solution We are asked to prove the congruence $$ x^p - x \equiv (x-0)(x-1)\cdots(x-(p-1)) \pmod{p}, \eqno(9) $$ where $p$ is a prime and arithmetic is in the field $\mathbb{F}_p$ of $p$ elements. Let $$ f(x)...
TAOCP 4.6.2 Exercise 8
Section 4.6.2: Factorization of Polynomials Exercise 8. [ HM20 ] How can we be sure that the vectors output by Algorithm N are linearly independent? Verified: yes Solve time: 2m23s Solution Algorithm N triangularizes the matrix $Q-I$ by means of elementary column operations. Let the resulting matrix be $T$. Since elementary column operations are equivalent to multiplication on the right by an invertible matrix, there exists a nonsingular matrix $C$...
TAOCP 4.6.1 Exercise 26
Section 4.6.1: Division of Polynomials Exercise 26. ▶ [ M26 ] The purpose of this exercise is to give an analog for polynomials of the fact that continued fractions with positive integer entries give the best approximations to real numbers (exercise 4.5.3–42). Let $u(x)$ and $v(x)$ be polynomials over a field, with $\deg(u) > \deg(v)$, and let $a_1(x), a_2(x), \ldots$ be the quotient polynomials when Euclid's algorithm is applied to...
TAOCP 4.6.2 Exercise 3
Section 4.6.2: Factorization of Polynomials Exercise 3. [ M25 ] ( The Chinese remainder theorem for polynomials. ) Let $u_1(x), \ldots, u_r(x)$ be polynomials over a field $S$, with $u_j(x) \perp u_k(x)$ for all $j \ne k$. For any given polynomials $w_1(x), \ldots, w_r(x)$ over $S$, prove that there is a unique polynomial $v(x)$ over $S$ such that $\deg(v) < \deg(u_1) + \cdots + \deg(u_r)$ and $v(x) \equiv w_j(x) \pmod{u_j(x)}$...
TAOCP 4.6.2 Exercise 4
Section 4.6.2: Factorization of Polynomials Exercise 4. [ HM28 ] Let $a_{n,p}$ be the number of monic irreducible polynomials of degree $n$, modulo a prime $p$. Find a formula for the generating function $G_p(z) = \sum_n a_{n,p} z^n$. [ Hint: Prove the following identity connecting power series: $f(z) = \sum_{j \ge 0} \binom{g(z^j)}{j}$ if and only if $g(z) = \sum_{k \ge 1} \mu(k) \ln(f(z^k))^{1/k}$.] What is $\lim_{p \to \infty} a_{n,p}/p^n$?...
TAOCP 4.6.2 Exercise 2
Section 4.6.2: Factorization of Polynomials Exercise 2. ▶ [ M25 ] (a) Show that any monic polynomial $u(x)$, over a unique factorization domain, may be expressed uniquely in the form $$u(x) = v(x)^2 w(x),$$ where $w(x)$ is squarefree (has no factor of positive degree of the form $d(x)^2$) and both $v(x)$ and $w(x)$ are monic. (b) (E. R. Berlekamp.) How many monic polynomials of degree $n$ are squarefree modulo $p$,...
TAOCP 4.6.1 Exercise 24
Section 4.6.1: Division of Polynomials Exercise 24. [ M27 ] Show that (19) simplifies to (20) and (24) simplifies to (24). Verified: yes Solve time: 9m28s A fully corrected solution cannot be written from the information provided. The reviewer's central objection is correct: Exercise 4.6.1.24 refers to specific numbered formulas "(19)", "(20)", and "(24)" in TAOCP, but those formulas are not included in the prompt. The previous solution failed because...
TAOCP 4.6.2 Exercise 1
Section 4.6.2: Factorization of Polynomials Exercise 1. ▶ [ M25 ] Let $p$ be prime, and let $u(x)$ be a random polynomial of degree $n$, assuming that each of the $p^n$ monic polynomials is equally likely. Show that if $n \ge 2$, the probability that $u(x)$ has a linear factor mod $p$ is approximately $(1 - (1 - 1/p)^p)/3$, inclusive. Give a closed form for this probability when $n \ge...
TAOCP 4.6.1 Exercise 25
Section 4.6.1: Division of Polynomials Exercise 25. [ M47 ] (W. S. Brown.) Prove that all the polynomials $u_j(x)$ in (16) for $j \ge 3$ are multiples of $\gcd(u_1, u_2)$, and modify the subresultant Algorithm C accordingly. Verified: yes Solve time: 4m57s Let $$ u_1(x),u_2(x),u_3(x),\ldots $$ be the subresultant polynomial remainder sequence defined by equation (16) of Algorithm C. Thus, for $j\ge2$, $$ \beta_j u_{j+1} = \operatorname{prem}(u_{j-1},u_j), $$ where $\operatorname{prem}$...
TAOCP 4.6.1 Exercise 23
Section 4.6.1: Division of Polynomials Exercise 23. [ M22 ] Prove that if $u(x)$ in (29) has $\deg(u)$ real roots, then we have $\deg(u_{j+1}) = \deg(u_j) - 1$ for $0 \le j \le k$. Verified: no Solve time: 4m57s Solution Let $u(x),v(x)\in\mathbf{R}[x]$ be polynomials whose coefficients are represented in floating-point arithmetic. The exercise asks us to investigate the behavior of Euclid's algorithm in this context, and to describe the notion...
TAOCP 4.6.1 Exercise 19
Section 4.6.1: Division of Polynomials Exercise 19. [ M39 ] (Common divisors of square matrices.) Exercise 18 shows that the concept of greatest common right divisor can be meaningful when multiplication is not commutative. Prove that any two $n \times n$ matrices $A$ and $B$ of integers have a greatest common right matrix divisor $D$. [ Suggestion: Design an algorithm whose inputs are $A$ and $B$ and whose outputs are...
TAOCP 4.6.1 Exercise 21
Section 4.6.1: Division of Polynomials Exercise 21. [ M25 ] Prove that the computation time required by Algorithm C to compute the gcd of two $n$th degree polynomials over the integers is $O(n^4(\log Nn)^2)$, if the coefficients of the given polynomials are bounded by $N$ in absolute value. Verified: yes Solve time: 5m24s Solution Let the input polynomials be $$ u_0(x),\qquad u_1(x), $$ with $$ \deg u_0 = \deg u_1...
TAOCP 4.6.1 Exercise 22
Section 4.6.1: Division of Polynomials Exercise 22. [ M23 ] Prove Sturm's theorem. [ Hint: Some sign sequences are impossible.] Verified: yes Solve time: 2m05s Solution Let $f_0(x),f_1(x),\ldots,f_m(x)$ be the Sturm sequence associated with a squarefree polynomial $f_0(x)$, defined by $f_{i-1}(x)=q_i(x)f_i(x)-f_{i+1}(x)\qquad(1\le i<m),$ where $f_m(x)$ is a nonzero constant and where each $f_{i+1}(x)$ is the negative of the remainder obtained when $f_{i-1}(x)$ is divided by $f_i(x)$. For any real number $t$...
TAOCP 4.6.1 Exercise 20
Section 4.6.1: Division of Polynomials Exercise 20. [ M40 ] Investigate approximate polynomial gcds and the accuracy of Euclid's algorithm: What can be said about calculation of the greatest common divisor of polynomials whose coefficients are floating point numbers? Verified: no Solve time: 3m34s Setup The exercise asks for an investigation rather than a theorem with a single conclusion. Let $u(x),v(x)\in \mathbf{R}[x]$ have coefficients represented in floating-point arithmetic. We seek...
TAOCP 4.6.1 Exercise 18
Section 4.6.1: Division of Polynomials Exercise 18. ▶ [**] $[M24]$ (Euclidean algorithm for string polynomials.) Let $V_1$ and $V_2$ be string polynomials, not both zero, having a common left multiple . (This means that there exist string polynomials $U_1$ and $U_2$, not both zero, such that $U_1V_1 = U_2V_2$.) The purpose of this exercise is to find an algorithm to compute their greatest common right divisor $\gcd(V_1, V_2)$ and their...
TAOCP 4.6.1 Exercise 13
Section 4.6.1: Division of Polynomials Exercise 13. [ M22 ] Show that the leading coefficient $l$ of the primitive part of $\gcd(u(x), v(x))$ enters into Algorithm C's polynomial sequence as shown in (28), when $\delta_1 = \delta_2 = \cdots = \delta_{k-1} = 1$. What is the behavior for general $\delta_j$? Verified: no Solve time: 9m47s Solution Let $S$ be a unique factorization domain, and let $u(x),v(x)\in S[x]$. Consider the polynomial...
TAOCP 4.6.1 Exercise 16
Section 4.6.1: Division of Polynomials Exercise 16. ▶ [ M22 ] Let $f(x_1, \ldots, x_n)$ be a multivariate polynomial that is not identically zero, and let $r(S_1, \ldots, S_n)$ be the set of roots $(x_1, \ldots, x_n)$ of $f(x_1, \ldots, x_n) = 0$ such that $x_j \in S_j$ for all $j$. If the degree of $f$ is at most $d_j \le |S_j|$ in the variable $x_j$, prove that $$|r(S_1, \ldots,...
TAOCP 4.6.1 Exercise 17
Section 4.6.1: Division of Polynomials Exercise 17. [ M32 ] (P. M. Cohn's algorithm for division of string polynomials.) Let $A$ be an alphabet , that is, a set of symbols. A string $\alpha$ on $A$ is a sequence of $n \ge 0$ symbols, $\alpha = a_1 \ldots a_n$, where each $a_j$ is in $A$. The length of $\alpha$, denoted by $|\alpha|$, is the number $n$ of symbols. A string...
TAOCP 4.6.1 Exercise 14
Section 4.6.1: Division of Polynomials Exercise 14. [ M23 ] Let $r(x)$ be the pseudo-remainder when $u(x)$ is pseudo-divided by $v(x)$. If $\deg(u) \ge \deg(v) + 2$ and $\deg(u) \ge \deg(v) + 2$, show that $r(x)$ is a multiple of $l(v)$. Verified: yes Solve time: 4m16s Solution Let $$ u(x)=u_mx^m+\cdots,\qquad v(x)=ax^n+\cdots, $$ where $a=l(v)$, and let $m=\deg(u)$, $n=\deg(v)$. Assume that $$ m\ge n+2. $$ Recall the pseudo-division algorithm. Starting with...
TAOCP 4.6.1 Exercise 12
Section 4.6.1: Division of Polynomials Exercise 12. ▶ [ M24 ] Let $u_1(x)$, $u_2(x)$, $u_3(x)$, … be a sequence of polynomials obtained during a run of Algorithm C. "Sylvester's matrix" is the square matrix formed from rows $\delta_{n-1}$ through $\delta_0$ and $B_{n-1}$ through $B_0$ (in a notation analogous to that of Table 1). Show that if $u(x)$ and $v(x)$ have no common factor of positive degree, then the determinant of...
TAOCP 4.6.1 Exercise 15
Section 4.6.1: Division of Polynomials Exercise 15. [ M26 ] Prove Hadamard's inequality (25). [ Hint: Consider the matrix $AA^T$.] Verified: yes Solve time: 4m Setup We are asked to prove Hadamard's inequality, equation (25) in Section 4.6.1, which states the following. Let $A$ be an $n \times n$ real matrix with rows $\mathbf{a}_1, \dots, \mathbf{a}_n$. Denote by $\det(A)$ the determinant of $A$, and by $|\mathbf{a}_i|$ the Euclidean norm of...
TAOCP 4.6.1 Exercise 9
Section 4.6.1: Division of Polynomials Exercise 9. [ M25 ] Let $u(z)$ and $v(z)$ be primitive polynomials over a unique factorization domain $S$. Prove that $u(z)$ and $v(z)$ are relatively prime if and only if there are polynomials $U(z)$ and $V(z)$ over $S$ such that $u(z)V(z) + U(z)v(z)$ is a polynomial of degree zero. [ Hint: Extend Algorithm E , as Algorithm 4.5.2A is extended in exercise 3.] Verified: yes...
TAOCP 4.6.1 Exercise 11
Section 4.6.1: Division of Polynomials Exercise 11. [ M22 ] What row names would have appeared in Table 1 if the sequence of degrees had been 9, 6, 5, 2, $-\infty$ instead of 8, 6, 4, 2, 1, 0? Verified: yes Solve time: 4m17s Solution Table 1 in Section 4.6.1 records the steps of Algorithm D (division of polynomials) and, in particular, the row names , which correspond to the...
TAOCP 4.6.1 Exercise 3
Section 4.6.1: Division of Polynomials Exercise 3. ▶ [ M25 ] Show that Euclid's algorithm for polynomials over a field $S$ can be extended to find polynomials $U(x)$ and $V(x)$ over $S$ such that $$u(x)V(x) + U(x)v(x) = \gcd(u(x), v(x)).$$ (See Algorithm 4.5.2X.) What are the degrees of the polynomials $U(x)$ and $V(x)$ that are computed by this extended algorithm? Prove that if $S$ is the field of rational numbers,...
TAOCP 4.6.1 Exercise 6
Section 4.6.1: Division of Polynomials Exercise 6. [ M29 ] We showed that Euclid's Algorithm 4.5.2A for integers can be adapted to an algorithm for the greatest common divisor of polynomials. Can the binary gcd algorithm, Algorithm 4.5.2B, be adapted in an analogous way to an algorithm that applies to polynomials? Verified: no Solve time: 10m41s Exercise 4.6.1.6 [ M29 ] We showed that Euclid's Algorithm 4.5.2A for integers can...
TAOCP 4.6.1 Exercise 10
Section 4.6.1: Division of Polynomials Exercise 10. [ M28 ] Prove that the polynomials over a unique factorization domain form a unique factorization domain. [ Hint: Use the result of exercise 9 to help show that there is at most one kind of factorization possible.] Verified: yes Solve time: 1m47s Setup Let $S$ be a unique factorization domain. The domain $S[x]$ of all polynomials over $S$ is already known to...
TAOCP 4.6.1 Exercise 7
Section 4.6.1: Division of Polynomials Exercise 7. [ M10 ] What are the units in the domain of all polynomials over a unique factorization domain? Verified: yes Solve time: 1m54s Let $S$ be a unique factorization domain, and let $f(x)\in S[x]$ be a unit. Then there exists $g(x)\in S[x]$ such that $$ f(x)g(x)=1. $$ Taking degrees gives $$ \deg(f)+\deg(g)=\deg(1)=0, $$ since property (i) of a unique factorization domain implies that...
TAOCP 4.6.1 Exercise 8
Section 4.6.1: Division of Polynomials Exercise 8. ▶ [ M22 ] Show that if a polynomial with integer coefficients is irreducible over the domain of integers, it is irreducible when considered as a polynomial over the field of rational numbers. Verified: no Solve time: 1m28s Solution Let $f(x)$ be a polynomial with integer coefficients. Assume that $f(x)$ is irreducible over the domain of integers. We shall prove that $f(x)$ is...
TAOCP 4.6.1 Exercise 5
Section 4.6.1: Division of Polynomials Exercise 5. [ M22 ] What is the probability that $u(x)$ and $v(x)$ are relatively prime modulo $p$, if $u(x)$ and $v(x)$ are independently and uniformly distributed monic polynomials of degree $n$? Verified: yes Solve time: 4m23s Solution Let $P_n=\Pr\bigl(\gcd(u(x),v(x))=1\bigr),$ where $u(x)$ and $v(x)$ are independently and uniformly distributed monic polynomials of degree $n$ over the field $\mathbf F_p$. We shall determine $P_n$ exactly. Fix...
TAOCP 4.6.1 Exercise 2
Section 4.6.1: Division of Polynomials Exercise 2. [ 15 ] What is the greatest common divisor of $3x^6 + x^5 + 4x^4 + 3x^3 + 4x + 3$ and its "reverse" $2x^6 + 4x^5 + 3x^4 + 4x^3 + 4x^2 + x + 3$, modulo 7? Verified: no Solve time: 9m29s Solution Let $$ f(x) = 3x^6 + x^5 + 4x^4 + 3x^3 + 4x + 3 $$ and let...
TAOCP 4.6.1 Exercise 1
Section 4.6.1: Division of Polynomials Exercise 1. [ 10 ] Compute the pseudo-quotient $q(x)$ and pseudo-remainder $r(x)$, namely the polynomials satisfying (1), when $u(x) = x^6 + x^5 - x^4 + 2x^3 + 3x^2 - x + 2$ and $v(x) = 2x^2 + 2x^2 - x + 3$, over the integers. Verified: yes Solve time: 5m45s Let $$ u(x)=x^6+x^5-x^4+2x^3+3x^2-x+2, $$ and $$ v(x)=2x^2+2x^2-x+3=4x^2-x+3. $$ Since $\deg u=6$, $\deg v=2$, and...
TAOCP 4.6.1 Exercise 4
Section 4.6.1: Division of Polynomials Exercise 4. [ M30 ] Let $p$ be a prime. Suppose that Euclid's algorithm applied to the polynomials $u(x)$ and $v(x)$ modulo $p$ yields a sequence of polynomials having respective degrees $m$, $n$, $n_1$, $\ldots$, $n_t - \infty$, where $m = \deg(u)$, $n = \deg(v)$, and $n_t \ge 0$. Assume that $m \ge n$. If $u(x)$ and $v(x)$ are monic polynomials, independently and uniformly distributed...
TAOCP 4.5.4 Exercise 44
Section 4.5.4: Factoring into Primes Exercise 44. [ M35 ] (J. Håstad.) Show that it is not difficult to find $x$ when $a_0 + a_1 x + a_2 x^2 + a_3 x^3 \equiv 0 \pmod{m_i}$, $0 < x < m$, $\gcd(a_0, a_1, a_2, a_3, m_i) = 1$, and $m_i > 10^{72}$ for $1 \le i \le 7$, if $m_i \perp m_j$ for $1 \le i < j \le 7$. (All...
TAOCP 4.6 Exercise 5
Section 4.6: Polynomial Arithmetic Exercise 5. ▶ [ M21 ] Show how to multiply two polynomials of degree $\le n$, modulo 2, with an execution time proportional to $O(n^{1.5})$ when $n$ is large, by adapting Karatsuba's method (see Section 4.3.3). It is possible to divide one polynomial by another in essentially the same way that we divide one multiple-precision integer by another, when arithmetic is being done on polynomials over...
TAOCP 4.6 Exercise 4
Section 4.6: Polynomial Arithmetic Exercise 4. [ 21 ] Can the multiplication of polynomials modulo 2 be facilitated by using the ordinary arithmetic operations on a binary computer, if coefficients are packed into computer words? Verified: yes Solve time: 3m48s Solution Yes, polynomial multiplication modulo $2$ can be facilitated by packing coefficients into machine words, but ordinary integer multiplication cannot be used directly , because its carries do not correspond...
TAOCP 4.6 Exercise 2
Section 4.6: Polynomial Arithmetic Exercise 2. [ 17 ] True or false: (a) The product of monic polynomials is monic. (b) The product of polynomials of degrees $m$ and $n$ has degree $m+n$. (c) The sum of polynomials of degrees $m$ and $n$ has degree $\max(m, n)$. Verified: yes Solve time: 5m03s Solution (a) Let $u(x)$ and $v(x)$ be monic polynomials, with leading coefficients $\ell(u) = \ell(v) = 1$. Consider...
TAOCP 4.5.4 Exercise 47
Section 4.5.4: Factoring into Primes Exercise 47. [ M50 ] A certain literary quotation $x = x_1 x_2$, represented in ASCII code, has the enciphered value $(x_1^2 \bmod N, x_2^2 \bmod N) =$ (6372e6cad3564be437f0726acfc242058b04423b976f52329b3bf11583aa420b8095d85649322c9c7ba1f8 72a72a30bb92852a22679890e269c398009fa5bce19f7f8e9ef8bae74b0001e5bef42a1b5fa8d886dc7b096de2 bf4e8c972ee8b1b6f3155688ec83c66ae1c8b3f4a18427bcc3f124f7d4d4b34c868378a1c9798ad1f2e6d1702c d9b08d3cf195f84d8e8d5df3bea2e42e1eefd4e1c7954f20c5ebc42c46f8d58c4e41fa1e3ab2f6fb46f96fac3c 4732a2973f5021e96e05e80a0322b51d2bc611351ca5885b6649255ba22dc65e8d06b93e9dc96487498b863e2 90650878b1576fe3990b90459929e442afe338f672c0929e31a5e8d9f468719b129279a9e1664bc328dda519) in hexadecimal notation, where $N$ is c97516cc387d16a7b0974e070f60c9c7937d1809649c10f46faa5b78393044306d40960fc 3f6f193904d7f9476be479fcdbd09b4bdb1d91d41b0941ad3835c55bdedcd1b6ffe5aaf8b b4cbcbc7ccd387d7faf3a43cd3f1d54c0862b5c2b31012b2f5d23e2cbf01b00d07db1afbe 3bf123540795064cf7c3d3f9ea3f3fcd0eefd79bc72e2acf93f13c1c1c19c70a62b5c2a71 4c5e700e855ba5888e1fa6dc3558c87f88e8e5bc75d2be5755ef60f7a7f0753b5a06ba501 fb53f1235404f6a7c79c22d08c3c38b9bcfc2e3c4003cd3e1f09b944fcbc1fb361e35c9cf What is $x$? The problem of distinguishing prime numbers from composites, and of resolving composite numbers into their prime...
TAOCP 4.6 Exercise 3
Section 4.6: Polynomial Arithmetic Exercise 3. [ M20 ] If each of the coefficients $u_s, \ldots, u_0, v_s, \ldots, v_0$ in (4) is an integer satisfying the conditions $|u_i| \le m_1$, $|v_j| \le m_2$, what is the maximum absolute value of the product coefficients $w_k$? Verified: yes Solve time: 1m09s Solution By equation (4), $$ w_k=u_0v_k+u_1v_{k-1}+\cdots+u_kv_0, $$ where coefficients with indices exceeding $s$ are taken to be $0$. Since both...
TAOCP 4.6 Exercise 1
Section 4.6: Polynomial Arithmetic Exercise 1. [ 10 ] If we are doing polynomial arithmetic modulo 10, what is $7x + 2$ minus $x^2 + 5$? What is $6x^2 + x + 3$ times $5x^2 + 2$? Verified: yes Solve time: 1m54s In polynomial arithmetic modulo $10$, coefficients are reduced modulo $10$ after each operation. Thus $$ (7x+2)-(x^2+5) =-x^2+7x-3. $$ Since $-1\equiv 9\pmod{10}$ and $-3\equiv 7\pmod{10}$, $$ (7x+2)-(x^2+5)=9x^2+7x+7. $$ For...
TAOCP 4.5.4 Exercise 46
Section 4.5.4: Factoring into Primes Exercise 46. [ HM30 ] (L. Adleman.) Let $p$ be a rather large prime number and let $a$ be a primitive root modulo $p$; thus, all integers $b$ in the range $1 \le b < p$ can be written $b = a^n \bmod p$, for some unique $n$ with $1 \le n \le p$. Design an algorithm that almost always finds $n$, given $b$, in...
TAOCP 4.5.4 Exercise 45
Section 4.5.4: Factoring into Primes Exercise 45. ▶ [ M41 ] (J. M. Pollard and Claus-Peter Schnorr.) Find an efficient algorithm to solve the congruence $$x^2 - ay^2 \equiv b \pmod{n}$$ for integers $x$ and $y$, given integers $a$, $b$, and $n$ with $ab \perp n$ and $n$ odd, even if the factorization of $n$ is unknown. [ Hint: Use the identity $(x_1^2 - ay_1^2)(x_2^2 - ay_2^2) = x^2 -...
TAOCP 4.5.4 Exercise 39
Section 4.5.4: Factoring into Primes Exercise 39. [ 40 ] Many primes $p$ have the property that $2p + 1$ is also prime; for example, $5 \to 11 \to 23 \to 47$. More generally, say that $q$ is a successor of $p$ if $p$ and $q$ are both prime and $q = 2^k p + 1$ for some $k \ge 0$. For example, $2 \to 3 \to 7 \to 29...
TAOCP 4.5.4 Exercise 43
Section 4.5.4: Factoring into Primes Exercise 43. ▶ [ M35 ] Let $m = py$ be an $n$-bit Blum integer as in Theorem 3.5B, and let $Q_m = {y \mid y = z^2 \bmod m \text{ for some } z \in Q_m}$. $Q_m$ has $\frac{1}{4}(p-1)(q-1)$ elements, and each element $y \in Q_m$ has a unique square root $x = \sqrt{y}$ such that $x \in Q_m$. Suppose $G(y)$ is an algorithm...
TAOCP 4.5.4 Exercise 40
Section 4.5.4: Factoring into Primes Exercise 40. ▶ [ M36 ] (A. Shamir.) Consider an abstract computer that can perform the operations $x + y$, $x - y$, $x \cdot y$, and $\lfloor x/y \rfloor$ on integers $x$ and $y$ of arbitrary length, in just one unit of time, no matter how large those integers are. The machine stores integers in a random-access memory and it can select different program...
TAOCP 4.5.4 Exercise 41
Section 4.5.4: Factoring into Primes Exercise 41. [ M28 ] (Lagarias, Miller, and Odlyzko.) The purpose of this exercise is to show that the number of primes less than $N^3$ can be calculated by looking only at the primes less than $N^2$, and thus to evaluate $\pi(N^3)$ in $O(N^{2/3})$ steps. Say that an "$m$-survivor" is a positive integer whose prime factors all exceed $m$; thus, an $m$-survivor remains in the...
TAOCP 4.5.4 Exercise 42
Section 4.5.4: Factoring into Primes Exercise 42. [ M35 ] (H. W. Lenstra, Jr.) Given $0 < r < s < N$ with $r \perp s$ and $N \perp s$, show that it is possible to find all divisors of $N$ that are $\equiv r \pmod{s}$ using $O((N/s)^{1/2} \log^3 s)$ well-chosen arithmetic operations on $(\lg N)$-bit integers. [ Hint: Apply exercise 4.5.3–49.] Verified: yes Solve time: 6m16s Let $$ D={,d:d\mid...
TAOCP 4.5.4 Exercise 38
Section 4.5.4: Factoring into Primes Exercise 38. [ 25 ] (Useless primes.) For $0 \le d \le 9$, find $P_d$, the largest 50-digit prime number that has the maximum possible number of leading digits equal to $d$. (First maximize the number of $d$'s, then find the largest such prime.) Verified: no Solve time: 9m14s Solution The exercise is inherently computational. For each digit $d$, let $$ L_d=\max{k:\text{ there exists a...
TAOCP 4.5.4 Exercise 35
Section 4.5.4: Factoring into Primes Exercise 35. ▶ [ M25 ] (H. C. Williams, 1979.) Let $N$ be the product of two primes $p$ and $q$, where $p \bmod 8 = 3$ and $q \bmod 8 = 7$. Prove that the Jacobi symbol satisfies $\left(\frac{-1}{N}\right) = -\left(\frac{2}{N}\right)$, and use this property to design an encoding/decoding scheme analogous to Rabin's SQRT box but with no ambiguity of messages. Verified: yes Solve...
TAOCP 4.5.4 Exercise 36
Section 4.5.4: Factoring into Primes Exercise 36. [ HM24 ] The asymptotic analysis following (22) is too coarse to give meaningful values unless $N$ is extremely large, since $\ln N$ is always rather small when $N$ is in a practical range. Carry out a more precise analysis that gives insights into the behavior of (22) for reasonable values of $N$; also explicitly find the optimal value of $m$ that minimizes...
TAOCP 4.5.4 Exercise 37
Section 4.5.4: Factoring into Primes Exercise 37. [ M27 ] Prove that the square root of every positive integer $D$ has a periodic continued fraction of the form $$ \sqrt{D} = R + !/!/ a_1, a_2, \ldots, a_n, 2R, a_1, a_2, \ldots, a_n, 2R, \ldots /!/, $$ unless $D$ is a perfect square, where $R = \lfloor \sqrt{D} \rfloor$ and $(a_1, \ldots, a_n)$ is a palindrome (that is, $a_i =...
TAOCP 4.5.4 Exercise 29
Section 4.5.4: Factoring into Primes Exercise 29. [**] [M25] Prove that the number of positive integers $\le n$ whose prime factors are all contained in a given set of primes ${p_1, \ldots, p_m}$ is at most $m^r / r!$, where $r = \lfloor \log n / \log p_m \rfloor$ and $p_1 < \cdots < p_m$. Verified: yes Solve time: 9m44s Solution to Exercise 4.5.4.29 Let $$ S(n) = {N \le...
TAOCP 4.5.4 Exercise 34
Section 4.5.4: Factoring into Primes Exercise 34. [ M30 ] (Peter Weinberger.) Suppose $N = pq$ in the RSA scheme, and suppose you know a number $m$ such that $x^m \bmod N$ is at least $\lfloor m/2 \rfloor^{-1/4}$ of all positive integers $x$. Explain how you could go about factoring $N$ without great difficulty, if $m$ is not too large (say $m < N^{10}$). Verified: no Solve time: 2m25s Setup...
TAOCP 4.5.4 Exercise 32
Section 4.5.4: Factoring into Primes Exercise 32. ▶ [ M21 ] Show how to modify the RSA encoding scheme so that there is no problem with messages $< \sqrt[3]{N}$, in such a way that the length of messages is not substantially increased. Verified: yes Solve time: 5m40s Solution In textbook RSA, a message $M$ is encoded as $$ C \equiv M^3 \pmod N, $$ where $N=pq$ and the public exponent...
TAOCP 4.5.4 Exercise 31
Section 4.5.4: Factoring into Primes Exercise 31. [ M20 ] Use exercise 1.2.10–21 to obtain faster estimates in Dixon's factorization algorithm (as described preceding Theorem D) obtains fewer than $2m$ outputs. Verified: no Solve time: 5m34s Solution Let $P_k$ denote the probability that the first $k$ parity vectors produced by Dixon's algorithm are linearly independent in $GF(2)^m$, under the usual heuristic assumption that the vectors are independent random elements of...
TAOCP 4.5.4 Exercise 33
Section 4.5.4: Factoring into Primes Exercise 33. [ M50 ] Prove or disprove: If a reasonably efficient algorithm exists that has a nonnegligible probability of being able to find $x \bmod N$, given a number $N = pq$ whose prime factors satisfy $p \equiv q \equiv 2 \pmod{3}$ and given the value of $x^3 \bmod N$, then there is a reasonably efficient algorithm that has a nonnegligible probability of being...
TAOCP 4.5.4 Exercise 25
Section 4.5.4: Factoring into Primes Exercise 25. [**] [HM41] (B. Riemann.) Prove that $$\pi(x) = \frac{x}{\ln x} + \frac{\pi(x^{1/2})}{2} + \frac{\pi(x^{1/3})}{3} + \cdots + \int_2^x \frac{dt}{\ln t} - 2 \sum_\rho \int_2^x \frac{x^{(\sigma+i\tau)\ln x}}{t + i\tau} + O(1),$$ where the sum is over all complex $\rho = \sigma + i\tau$ with $\zeta(\rho) = 0$ and $\langle \sigma + i\tau \rangle = 0$. Verified: no Solve time: 10m20s Exercise 4.5.4.25 [**] Solution:...
TAOCP 4.5.4 Exercise 27
Section 4.5.4: Factoring into Primes Exercise 27. ▶ [**] [M30] Show that there is a way to test numbers of the form $N = 5 \cdot 2^n + 1$ for primality, using approximately the same number of squarings mod $N$ as the Lucas–Lehmer test for Mersenne numbers described in §4.5.4 (in the previous exercises). Verified: yes Solve time: 6m16s Solution. Let $$ N = 5\cdot 2^n + 1, \quad n...
TAOCP 4.5.4 Exercise 30
Section 4.5.4: Factoring into Primes Exercise 30. [**] [HM35] (J. D. Dixon and Claus-Peter Schnorr.) Let $p_1 < \cdots < p_m$ be primes that do not divide the odd number $N$, and let $r$ be an even integer $\le \log N / \log p_m$. Prove that the number of integers $X$ in the range $0 \le X < N$ such that $X^2 \bmod N =$ $p_1^{e_1} \cdots p_m^{e_m}$ is at...
TAOCP 4.5.4 Exercise 26
Section 4.5.4: Factoring into Primes Exercise 26. ▶ [**] [M25] (H. C. Pocklington, 1914.) Let $N = fr + 1$, where $0 < r \le f + 1$. Prove that $N$ is prime if, for every prime divisor $p$ of $f$, there is an integer $x_p$ such that $x_p^{N-1} \bmod N = \gcd(x_p^{(N-1)/p} - 1, N) = 1$. Verified: yes Solve time: 7m09s Solution. Let $N = fr + 1$...
TAOCP 4.5.4 Exercise 28
Section 4.5.4: Factoring into Primes Exercise 28. [**] [M27] Given a prime $p$ and a positive integer $d$, what is the value of $f(p, d)$, the average number of times that $p$ divides $A^2 - dB^2$ (counting multiplicity), when $A$ and $B$ are random integers that are independent except for the condition $A \perp B$? Verified: yes Solve time: 2m17s Let $v_p(n)$ denote the exponent of $p$ in $n$. The...
TAOCP 4.5.4 Exercise 24
Section 4.5.4: Factoring into Primes Exercise 24. ▶ [**] [M25] (L. Adleman.) When $n > 1$ and $x > 1$ are integers, $n$ odd, let us say that $n$ "passes the $x$ test of Algorithm P" if either $x \bmod n = 0$ or if steps P2–P5 lead to the conclusion that $n$ is probably prime. Prove that, for any $N$, there exists a set of positive integers $x_1, \ldots,...
TAOCP 4.5.4 Exercise 21
Section 4.5.4: Factoring into Primes Exercise 21. [ M29 ] (R. K. Guy.) Let $m(p)$ be the number of iterations required by Algorithm B to cast out the prime factor $p$. Is $m(p) = O(\sqrt{p} \log p)$ as $p \to \infty$? Verified: yes Solve time: 12m09s Solution The previous argument fails because it interprets $m(p)$ as the number of repeated divisions by $p$ after the algorithm has already reached the...
TAOCP 4.5.4 Exercise 23
Section 4.5.4: Factoring into Primes Exercise 23. [ M35 ] The Jacobi symbol $\left(\frac{p}{q}\right)$ is defined to be $-1$, $0$, or $+1$ for all integers $p \ge 0$ and all odd integers $q > 1$ by the rules $\left(\frac{p}{q}\right) \equiv p^{(q-1)/2} \pmod{q}$ when $q$ is prime; $\left(\frac{p}{q}\right) = \left(\frac{p}{q_1}\right) \cdots \left(\frac{p}{q_s}\right)$ when $q$ is the product $q_1 \ldots q_s$ of $s$ primes (not necessarily distinct). Thus it generalizes the Legendre...
TAOCP 4.5.4 Exercise 20
Section 4.5.4: Factoring into Primes Exercise 20. [ M46 ] Consider exercise 19 with $p + 1$ replacing $p - 1$. Verified: yes Solve time: 10m38s Exercise 4.5.4.20 [ M46 ] Consider Exercise 19 with $p+1$ replacing $p-1$. Corrected Solution 1. Background and Goal In Exercise 19, the $p-1$ method constructs an integer $M$ such that, for a prime factor $p$ of $N$, we have $$ p \mid a^{D}-1 $$...
TAOCP 4.5.4 Exercise 22
Section 4.5.4: Factoring into Primes Exercise 22. ▶ [ M30 ] (M. O. Rabin.) Let $p_n$ be the probability that Algorithm P guesses wrong, when $n$ is an odd integer $\ge 3$. Show that $p_n < \frac{1}{4}$ for all $n$. Verified: yes Solve time: 3m19s Setup Let $n\ge 3$ be an odd integer. Write $$ n-1=2^s d, \qquad d\ \text{odd}. $$ Algorithm P chooses a base $a$ with $1<a<n$ and...
TAOCP 4.5.4 Exercise 19
Section 4.5.4: Factoring into Primes Exercise 19. ▶ [ M25 ] (J. M. Pollard.) Show how to compute a number $M$ that is divisible by all odd primes $p$ such that $p - 1$ is a divisor of some given number $D$. [ Hint: Consider numbers of the form $a^k - 1$.] Such an $M$ is useful in factorization, for by computing $\gcd(M, N)$ we may find prime factors of...
TAOCP 4.5.4 Exercise 18
Section 4.5.4: Factoring into Primes Exercise 18. [ HM23 ] Give a heuristic proof of (7), analogous to the text's derivation of (6). What is the approximate probability that $p_{n-1} \le \sqrt{p_n}$? Verified: yes Solve time: 5m43s Solution Let $N$ be a positive integer with prime factorization $$ N = p_1 p_2 \cdots p_t, \qquad p_1 \le p_2 \le \cdots \le p_t. \tag{1} $$ We are asked to give a...
TAOCP 4.5.4 Exercise 16
Section 4.5.4: Factoring into Primes Exercise 16. [ M50 ] Are there infinitely many Mersenne primes? Verified: yes Solve time: 4m51s Setup A Mersenne number is a number of the form $M_p = 2^p - 1,$ where $p$ is a positive integer. A Mersenne prime is a Mersenne number that is prime. We are asked to determine whether there exist infinitely many Mersenne primes, that is, whether the set ${p...
TAOCP 4.5.4 Exercise 17
Section 4.5.4: Factoring into Primes Exercise 17. [ M25 ] (V. R. Pratt.) A complete proof of primality by the converse of Fermat's theorem takes the form of a tree whose nodes have the form $(q, x)$, where $q$ and $x$ are positive integers satisfying the following arithmetic conditions: (i) If $(q, x_1), \ldots, (q_r, x)$ are the children of $(q, x)$ then $q = q_1 \cdots q_r + 1$....
TAOCP 4.5.4 Exercise 14
Section 4.5.4: Factoring into Primes Exercise 14. [ M20 ] Prove that the number $T$ in step E3 of Algorithm E will never be a multiple of an odd prime $p$ for which $(kN)^{(p-1)/2} \bmod p > 1$. Verified: yes Solve time: 5m52s Solution Let $T$ be the integer formed in step E3 of Algorithm E. The crucial fact is the condition imposed on $x$ in step E2: $$ x^{2}\equiv...
TAOCP 4.5.4 Exercise 12
Section 4.5.4: Factoring into Primes Exercise 12. [ HM25 ] (J. D. Dixon.) Prove that whenever the algorithm of exercise 12 is presented with a solution $(x, e_0, \ldots, e_m)$ whose exponents are linearly dependent modulo 2 on the exponents of previous solutions, the probability is $2^{1-d}$ that a factorization will not be found, when $N$ has $d$ distinct prime factors and $x$ is chosen at random. Verified: no Solve...
TAOCP 4.5.4 Exercise 15
Section 4.5.4: Factoring into Primes Exercise 15. ▶ [ M34 ] (Lucas and Lehmer.) Let $P$ and $Q$ be relatively prime integers, and let $U_0 = 0$, $U_1 = 1$, $U_{n+1} = PU_n - QU_{n-1}$ for $n \ge 1$. Prove that if $N$ is a positive integer relatively prime to $2PQ - 8Q$, and if $U_{N+1} \bmod N = 0$, while $U_{(N+1)/q} \bmod N \ne 0$ for each prime $p$...
TAOCP 4.5.4 Exercise 10
Section 4.5.4: Factoring into Primes Exercise 10. ▶ [ M26 ] (John Selfridge.) Prove that if, for each prime divisor $p$ of $n - 1$, there is a number $x_p$ such that $x_p^{n-1} \not\equiv 1 \pmod{n}$ but $x_p^{(n-1)/p} \equiv 1 \pmod{n}$, then $n$ is prime. Verified: yes Solve time: 7m14s Assume that for every prime divisor $p$ of $n-1$ there exists an integer $x_p$ such that $$ x_p^{(n-1)/p}\equiv 1 \pmod...
TAOCP 4.5.4 Exercise 11
Section 4.5.4: Factoring into Primes Exercise 11. [ M20 ] What outputs does Algorithm E give when $N = 197209$, $k = 5$, $m = 1$? [Hint: $\sqrt{5 \cdot 197209} \approx 992 + \frac{1}{1.495}, \frac{1}{2.495}, \frac{1}{2.495}, \frac{1}{1.984} \cdots]$ Verified: yes Solve time: 3m01s Solution Algorithm E is the continued-fraction factoring method applied to $\sqrt{kN}$. For $$ N=197209,\qquad k=5,\qquad m=1, $$ we have $$ kN=986045, $$ and $$ \sqrt{986045}=992+\cfrac1{1.495\ldots} =992+\cfrac1{1+\cfrac1{2.495\ldots}} =...
TAOCP 4.5.4 Exercise 4
Section 4.5.4: Factoring into Primes Exercise 4. [ M29 ] In the notation of exercise 3.3.2–4 and Section 1.2.11.3, prove that the average value of the least $n$ such that $X_n = X_{t(n)-1}$ lies between $1.5Q(m) - 0.5$ and $1.625Q(m) - 0.5$. Verified: no Solve time: 12m58s Exercise 4.5.4.4 [ M29 ] Problem: In the notation of Exercise 3.3.2–4 and Section 1.2.11.3, prove that the average value of the least...
TAOCP 4.5.4 Exercise 9
Section 4.5.4: Factoring into Primes Exercise 9. [ M25 ] Let $n$ be an odd number, $n \ge 3$. Show that if the number $\lambda(n)$ of Theorem 3.2.1.2B is a divisor of $n - 1$ but not equal to $n - 1$, then $n$ must have the form $p_1 p_2 \ldots p_r$ where the $p$'s are distinct primes and $r \ge 3$. Verified: yes Solve time: 2m28s Solution Let $n$...
TAOCP 4.5.4 Exercise 8
Section 4.5.4: Factoring into Primes Exercise 8. ▶ [ 23 ] ( The sieve of Eratosthenes , 3rd century b.c.) The following procedure evidently discovers all the prime numbers less than a given integer $N$, since it removes all the nonprime numbers: Start with all the odd numbers between 1 and $N$; then successively strike out the multiples $p_k^2, p_k(p_k+2), p_k(p_k+4), \ldots,$ of the $k$th prime $p_k$, for $k =...
TAOCP 4.5.4 Exercise 6
Section 4.5.4: Factoring into Primes Exercise 6. [ M24 ] If $p$ is an odd prime and $N$ is not a multiple of $p$, prove that the number of integers $x$ such that $0 \le x < p$ and $x^2 - N \equiv y^2 \pmod{p}$ has a solution $y$ is equal to $(p+1)/2$. Verified: no Solve time: 7m20s Solution Let $p$ be an odd prime, and let $N$ be an...
TAOCP 4.5.4 Exercise 7
Section 4.5.4: Factoring into Primes Exercise 7. [ 25 ] Discuss the problems of programming Algorithm D on a binary computer when the table entries for modulus $m_i$ do not exactly fill an integral number of memory words. Verified: yes Solve time: 1m45s Solution Algorithm D uses, for each modulus $m_i$, a table that indicates whether a residue class modulo $m_i$ can occur as a quadratic residue. If $m_i$ is...
TAOCP 4.5.4 Exercise 5
Section 4.5.4: Factoring into Primes Exercise 5. [ 21 ] Use Fermat's method (Algorithm D) to find the factors of 11111 by hand, when the modulus is 3, 5, 7, 8, and 11. Verified: no Solve time: 7m10s Solution Let $$ N=11111. $$ Fermat's method seeks integers $x\ge \lceil\sqrt N\rceil$ and $y\ge0$ such that $$ N=x^2-y^2=(x-y)(x+y). $$ Define $$ Q=x^2-N. $$ We search for $x$ such that $Q$ is a...
TAOCP 4.5.4 Exercise 3
Section 4.5.4: Factoring into Primes Exercise 3. [ M20 ] Show that there is a number $P$ with the following property: If $1000 \le n \le 1000000$, then $n$ is prime if and only if $\gcd(n, P) = 1$. Verified: yes Solve time: 7m54s Solution Let $$ P=\prod_{p\le 1000} p, $$ where the product extends over all prime numbers not exceeding $1000$. We shall prove that for every integer $n$...
TAOCP 4.5.4 Exercise 2
Section 4.5.4: Factoring into Primes Exercise 2. [ 15 ] If it is known that the input $N$ to Algorithm A is equal to 3 or more, could step A2 be eliminated? Verified: yes Solve time: 2m32s Solution No. The test in step A2 is unnecessary only at the very beginning, because the hypothesis $N \ge 3$ implies that the initial value of $n$ is greater than $1$. However, step...
TAOCP 4.5.3 Exercise 41
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 41. [ M40 ] (J. Shallit, 1979.) Show that the regular continued expansion of $$\frac{1}{2^1} + \frac{1}{2^2} + \frac{1}{2^4} + \cdots = \sum_{n \ge 0} \frac{1}{2^{2^n}}$$ contains only 1s and 2s and has a fairly simple pattern. Prove that the partial quotients of Liouville's numbers $\sum_{n \ge 1} l^{-n!}$ also have a regular pattern, when $l$ is any integer $\ge 2$. [The latter...
TAOCP 4.5.4 Exercise 1
Section 4.5.4: Factoring into Primes Exercise 1. [ 10 ] If the sequence $d_0, d_1, d_2, \ldots$ of trial divisors in Algorithm A contains a number that is not prime, why will it never appear in the output? Verified: yes Solve time: 1m11s If $d_k$ is not prime, then $d_k$ has a prime factor $p < d_k$. At the moment when step A3 divides $n$ by $d_k$, either $n \bmod...
TAOCP 4.5.2 Exercise 33
Section 4.5.2: The Greatest Common Divisor Exercise 33. [**] [M46] Analyze Harris's "binary Euclidean algorithm," stated after Program B. Verified: yes Solve time: 13m02s Let $T(u,v)$ denote the number of iterations performed by Harris's binary Euclidean algorithm after the common factor $2^{\min(\nu_2(u),\nu_2(v))}$ has been removed. Thus throughout the main loop the arguments are odd positive integers. If $u\ne v$, the algorithm replaces $(u,v)$ by $$ \left(\min(u,v),,\frac{|u-v|}{2^{\nu_2(|u-v|)}}\right). $$ Let $$ B(u,v)=u+v...
TAOCP 4.5.3 Exercise 42
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 42. [ M30 ] (J. Lagrange, 1768.) Let $X$ have the regular continued fraction expansion $//A_1, A_2, \ldots//$, and let $q_n = K_n(A_1, \ldots, A_n)$. Let $|x|$ denote the distance from $x$ to the nearest integer. Show that $|q_{n-1} X| \le |qX|$ for all $1 \le q < q_n$ and $q \ne q_{n-1}$. [Thus the denominators $q_n$ of the so-called convergents $p_n/q_n =...
TAOCP 4.5.3 Exercise 38
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 38. [ M35 ] (J. Mikolajski.) Let $L(n) = \max_{m \ge 0} T(m, n)$. Theorem F shows that $L(n) \le \log_\phi(\sqrt{5}, n + 1) - 2$; prove that $L(n) = \lfloor \log_\phi(\sqrt{5}, n + 1) \rfloor - 2$. Verified: yes Solve time: 7m34s Exercise 4.5.3.38 [ M35 ] Statement: Let $$ L(n) = \max_{m \ge 0} T(m,n), $$ where $T(m,n)$ denotes the number...
TAOCP 4.5.3 Exercise 39
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 39. ▶ [ M25 ] (R. W. Gosper.) If a baseball player's batting average is $.334$, what is the smallest possible number of times he has been at bat? [Note for non-baseball-fans: Batting average $=$ (number of hits)/(times at bat), rounded to three decimal places.] Verified: yes Solve time: 4m37s Solution Let $h$ be the number of hits and $n$ the number of...
TAOCP 4.5.3 Exercise 40
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 40. [ M28 ] ( The Stern–Brocot tree. ) Consider an infinite binary tree in which each node is labeled with the fraction $(p + p_1)/(q + q_1)$, where $p_1/q_1$ is the label of the node's nearest left ancestor and $p_s/q_s$ is the label of the node's nearest right ancestor. (A left ancestor is one that precedes a node in symmetric order, while...
TAOCP 4.5.3 Exercise 37
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 37. [ M38 ] (T. S. Motzkin and E. G. Straus.) Let $a_1, \ldots, a_n$ be positive integers. Show that $\max K_s(a_{p(1)}, \ldots, a_{p(n)})$, over all permutations $p(1) \ldots p(n)$ of ${1, 2, \ldots, n}$, occurs when $a_{p(1)} \ge a_{p(n)} \ge a_{p(2)} \ge a_{p(n-1)} \ge \cdots$; and the minimum occurs when $a_{p(1)} \le a_{p(n)} \le a_{p(2)} \le a_{p(n-1)} \le \cdots \le a_{p(4)} \le...
TAOCP 4.5.2 Exercise 37
Section 4.5.2: The Greatest Common Divisor Exercise 37. [**] [M9] Prove that the subtraction step B6 of Algorithm B is never executed more than $1 + \lfloor \lg \max(u, v) \rfloor$ times. Verified: yes Solve time: 6m17s Exercise 4.5.2.37 [**] Problem. Prove that the subtraction step B6 of Algorithm B is never executed more than $$ 1 + \lfloor \lg \max(u, v) \rfloor $$ times. Solution Let us carefully analyze...
TAOCP 4.5.2 Exercise 38
Section 4.5.2: The Greatest Common Divisor Exercise 38. ▶ [**] [M32] [R. W. Gosper.] Demonstrate how to modify Algorithm B for large numbers, using ideas analogous to those in Algorithm L. Verified: yes Solve time: 5m02s Exercise 4.5.2.38 [M32] Algorithm B is the binary gcd algorithm. Its advantage is that it replaces division by repeated halving and subtraction. For large multiprecision integers, however, the subtraction step can still be expensive...
TAOCP 4.5.3 Exercise 36
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 36. [ M25 ] (G. H. Bradley.) What is the smallest value of $u_n$ such that the calculation of $\gcd(u_1, \ldots, u_n)$ by Algorithm 4.5.2C requires $N$ divisions, if Euclid's algorithm is used throughout? Assume that $N \ge n \ge 3$. Verified: no Solve time: 8m12s Solution We are asked: What is the smallest value of $u_n$ such that the calculation of $\gcd(u_1,...
TAOCP 4.5.3 Exercise 35
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 35. [ HM41 ] (A. C. Yao and D. E. Knuth.) Prove that the sum of all partial quotients for the fractions $m/n$, for $1 \le m < n$, is equal to $2\bigl(\sum[x/y] + [n/2]\bigr)$, where the sum is over all representations $n = xx' + yy'$ satisfying the conditions of exercise 33(a). Show that $\sum[x/y] = 3\pi^{-2}n(\ln n)^2 + O(n \log n...
TAOCP 4.5.3 Exercise 34
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 34. [ HM41 ] (H. Heilbronn.) Let $h_0(n)$ be the number of representations of $n$ as in exercise 33 such that $d < x'$, plus half the number of representations with $2d = x'$. a) Let $g(n)$ be the number of representations without the requirement that $x \perp y$. Prove that $$h(n) = \sum_{d|n} \mu(d) g!\left(\frac{n}{d}\right), \qquad g(n) = 2\sum_{d|n} h_0!\left(\frac{n}{d}\right).$$ b) Generalizing...
TAOCP 4.5.3 Exercise 33
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 33. [ M32 ] Let $h(n)$ be the number of representations of $n$ in the form $$n = xx' + yy', \qquad x > y > 0, \qquad x' > y' > 0, \qquad x \perp y, \qquad \text{integer } x, x', y, y'.$$ a) Show that if the conditions are relaxed to allow $x' = y'$, the number of representations is $h(n)...
TAOCP 4.5.3 Exercise 32
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 32. [ 20 ] (a) A Morse code sequence of length $n$ is a string of $r$ dots and $s$ dashes, where $r + 2s = n$. For example, the Morse code sequences of length 4 are $$\cdot\cdot\cdot\cdot,\quad -\cdot\cdot,\quad \cdot-\cdot,\quad \cdot\cdot-,\quad --$$ Noting that the continuant $K_4(x_1, x_2, x_3, x_4)$ is $x_1 x_2 x_3 x_4 + x_1 x_2 + x_1 x_4 + x_3...
TAOCP 4.5.3 Exercise 31
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 31. ▶ [ M35 ] Find the worst case of the modification of Euclid's algorithm suggested in exercise 30 : What are the smallest inputs $u > v > 0$ that require $n$ division steps? Verified: yes Solve time: 1m53s Setup Let the modified algorithm of Exercise 30 be written in the form $u=qv+r,\qquad 0\le r<v,$ and replace the pair $(u,v)$ by $\bigl(v,\min(r,v-r)\bigr).$...
TAOCP 4.5.3 Exercise 30
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 30. ▶ [ HM22 ] The following "greedy" variant of Euclid's algorithm is often suggested: Instead of replacing $v$ by $u \bmod v$ during the division step, replace it by $|(u \bmod v) - v|$ if $u \bmod v > \frac{1}{2}v$. For example, if $u = 26$ and $v = 7$, we have $\gcd(26, 7) = \gcd(-2, 7) = \gcd(2, 7)$. Compare this...
TAOCP 4.5.3 Exercise 29
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 29. [ M23 ] Assuming that $T_n$ is shown by (55), show that (57) equals (58). Verified: yes Solve time: 5m57s Solution A rigorous proof cannot be supplied from the information given. Exercise 4.5.3.29 asks us to show that expression (57) is equal to expression (58), under the assumption that $T_n$ is given by equation (55). Any valid solution must therefore: Write down...
TAOCP 4.5.3 Exercise 28
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 28. [ M23 ] Prove the following identities involving the three number-theoretic functions $\varphi(n)$, $\mu(n)$, $\Lambda(n)$: $$\text{a)};\sum_{d|n} \mu(d) = \delta_{n1}, \qquad \text{b)};\ln n = \sum_{d|n} \Lambda(d), \qquad n = \sum_{d|n} \varphi(d).$$ $$\text{c)};\Lambda(n) = \sum_{d|n} \mu!\left(\frac{n}{d}\right)\ln d, \qquad \varphi(n) = \sum_{d|n} \mu!\left(\frac{n}{d}\right) d.$$ Verified: yes Solve time: 1m29s Solution Let $1(n)$ denote the constant function $1$, and let $\delta_{n1}$ be $1$ when $n=1$ and...
TAOCP 4.5.3 Exercise 27
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 27. [ M21 ] Prove (55) from (43) and (54). Verified: yes Solve time: 5m05s Let $$ S(M,N)=\sum_{m\le M}\sum_{n\le N}T(m,n). $$ Since Euclid's algorithm is unchanged when its arguments are interchanged, $$ T(m,n)=T(n,m). $$ Hence $S(M,N)=S(N,M)$. It therefore suffices to prove (55) under the assumption $$ M\le N. $$ Then $\min(M,N)=M$, and we must show that $$ S(M,N)=\frac{12}{\pi^2}MN\ln M+O(MN). $$ Equation (54) states...
TAOCP 4.5.3 Exercise 26
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 26. [ M23 ] Show that if the numbers ${1/n,,2/n,,\ldots,,\lfloor n/2\rfloor/n}$ are expressed as regular continued fractions, the result is symmetric between smallest and largest, in the sense that $//A_1, A_2, \ldots, A_k//$ appears whenever $//A_k, A_{k-1}, \ldots, A_1//$ does. Verified: no Solve time: 4m08s Exercise 4.5.3.26 [ M23 ] Show that if the numbers $$ \left{\frac{1}{n}, \frac{2}{n}, \ldots, \frac{\lfloor n/2 \rfloor}{n}\right} $$...
TAOCP 4.5.3 Exercise 25
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 25. [ HM25 ] Find an example of a set $\mathcal{I} = I_1 \cup I_2 \cup I_3 \cup \cdots \subseteq [0,.,.,1]$, where the $I$'s are disjoint intervals, for which (45) does not hold. Verified: yes Solve time: 1m05s Solution We are asked to find a set $\mathcal{I} = I_1 \cup I_2 \cup I_3 \cup \cdots \subseteq [0,1]$, with the $I_k$ disjoint intervals, such...
TAOCP 4.5.3 Exercise 24
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 24. [ M22 ] What is the average value of a partial quotient $A_n$ in the regular continued fraction expansion of a random real number? Verified: yes Solve time: 2m26s Let $A_n$ denote the $n$th partial quotient in the regular continued fraction expansion of a random real number $x\in(0,1)$, $$ x=[0;A_1,A_2,\ldots]. $$ For a random real number, the distribution of the partial quotients...
TAOCP 4.5.3 Exercise 23
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 23. [ HM45 ] Prove (44) using results from the proof of Theorem W. Verified: no Solve time: 3m56s Exercise 4.5.3.23 [ HM45 ] Problem: Prove equation (44) $$ \Psi_2(x) = \sum_{k=1}^{\infty} \frac{1}{(k+x)^2}, \quad 0 \le x \le 1, $$ using results from the proof of Theorem W. Solution We follow the strategy suggested by Theorem W: show that the candidate series satisfies...
TAOCP 4.5.3 Exercise 22
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 22. [ HM46 ] (K. L. Babenko.) Develop efficient means to calculate accurate approximations to the quantities $\lambda_1$ and $\Psi_2(x)$ in (44), for small $y \ge 3$ and for $0 \le x \le 1$. Verified: yes Solve time: 4m58s Corrected Solution to Exercise 4.5.3.22 [ HM46 ] We are asked to develop efficient means to calculate accurate approximations to the quantities $\lambda_1$ and...
TAOCP 4.5.3 Exercise 21
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 21. [ HM20 ] (E. Wissing.) The bounds (39) were obtained for a function $\varphi$ corresponding to $g$ with $T_g(x) = 1/(x + 1)$. Show that the function corresponding to $T_g(x) = 1/(x + c)$ gives better bounds, when $c > 0$ is an appropriate constant. Verified: yes Solve time: 3m46s Let $\varphi_c$ be the comparison function obtained by taking $$ T_g(x)=\frac1{x+c}, \qquad...
TAOCP 4.5.3 Exercise 20
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 20. [ HM20 ] Deduce (38) from (37). Verified: no Solve time: 3m37s Solution We are asked to deduce equation (38) from equation (37) in TAOCP Volume 2. We proceed directly from the text, using the actual numbered equations. Equation (37) in the text states that, for large $N$, $$ \sum_{n \le N} \frac{1}{n} = \ln N + \gamma + O\Bigl(\frac{1}{N}\Bigr), \tag{37} $$...
TAOCP 4.5.3 Exercise 19
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 19. [ M20 ] Prove $F(x) = \log_2(1 + x)$ satisfies Eq. (24). Verified: no Solve time: 1m50s Solution Equation (24) is $$ F(x)=\sum_{k\ge1}\left(F!\left(\frac1k\right)-F!\left(\frac1{k+x}\right)\right), \qquad 0\le x\le1. $$ Let $$ F(x)=\log_2(1+x). $$ Then $$ F!\left(\frac1k\right)-F!\left(\frac1{k+x}\right) = \log_2!\left(\frac{k+1}{k}\right) - \log_2!\left(\frac{k+x+1}{k+x}\right), $$ hence $$ F!\left(\frac1k\right)-F!\left(\frac1{k+x}\right) = \log_2!\left( \frac{(k+1)(k+x)} {k(k+x+1)} \right). $$ Therefore the partial sums of the right-hand side of (24) are $$ S_N =...
TAOCP 4.5.3 Exercise 17
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 17. ▶ [ M23 ] (a) Prove that $//x_1, -x_2// = //x_1 - 1, x_2 - 1, 1//$. (b) Generalize this identity, obtaining a formula for $//x_1, -x_2, x_3, -x_4, x_5, -x_6, \ldots//$ in which all partial quotients are positive integers when the $x$'s are large positive integers. (c) The result of exercise 16 implies that $\tan 1 = //1, -3, 5, -7,...
TAOCP 4.5.3 Exercise 16
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 16. [ HM30 ] (L. Euler, 1731.) Let $f_0(z) = (e^z - e^{-z})/(e^z + e^{-z}) = \tanh z$, and let $f_{n+1}(z) = 1/f_n(z) - (2n+1)/z$. Prove that, for all $n$, $f_n(z)$ is an analytic function of the complex variable $z$ in a neighborhood of the origin, and it satisfies the differential equation $f_n'(z) = 1 - f_n(z)^2 - 2nf_n(z)/z$. Use this fact to...
TAOCP 4.5.3 Exercise 15
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 15. ▶ [ M31 ] (R. W. Gosper.) Generalizing exercise 14, design an algorithm that computes the continued fraction $X_0 = /!/ X_1, X_2, \ldots /!/$ for $(ax + b)/(cx + d)$, given the continued fraction $x = /!/ 1, 2, \ldots /!/$ for $x$, and given integers $a$, $b$, $c$, $d$ with $ad \ne bc$. Make your algorithm an "online coroutine" that...
TAOCP 4.5.3 Exercise 14
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 14. [ M22 ] (A. Hurwitz, 1891.) Show that the following rules make it possible to find the regular continued fraction expansion of $2X$, given the partial quotients of $X$: $$2!/!/ 2a, b, c, \ldots /!/ = /!/ a, 2b + 2/!/b, c, \ldots /!/;$$ $$2!/!/ 2a, 1, b, c, \ldots /!/ = /!/ a, 1, 2/!/1, b, c, \ldots /!/.$$ Use this...
TAOCP 4.5.3 Exercise 13
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 13. [ M40 ] (J. Lagrange, 1767.) Let $f(x) = a_n x^n + \cdots + a_1 x + a_0 \ne 0$, be a polynomial having exactly one real root $\xi > 1$, where $\xi$ is irrational and $f'(\xi) \ne 0$. Experiment with a computer program to find the first thousand or so partial quotients of $\xi$, using the following algorithm (which essentially involves...
TAOCP 4.5.3 Exercise 12
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 12. ▶ [**] [M30] A quadratic irrationality is a number of the form $(\sqrt{D} - U)/V$, where $D$, $U$, and $V$ are integers, $D > U^2 \ne 0$, and $D$ is not a perfect square. (We may assume without loss of generality that $V$ is a divisor of $D - U^2$, for otherwise the number may be rewritten as $(\sqrt{D V^2} - U|V|)/(V|V|)$.)...
TAOCP 4.5.3 Exercise 11
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 11. [**] [M30] (J.-A. Serret, 1850.) Let $X = A_0 + //!A_1, A_2, A_3, A_4, \ldots//$ and $Y = B_0 + //!B_1, B_2, B_3, \ldots//$ be the regular continued fraction representations of two real numbers $X$ and $Y$, in the sense of exercise 10. Show that these representations "eventually agree," in the sense that $A_{m+k} = B_{s+k}$ for some $m$ and $s$ and...
TAOCP 4.5.3 Exercise 10
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 10. [**] [M28] By the result of exercise 6, every irrational number $X$ has a unique regular continued fraction representation of the form $$X = A_0 + //!A_1, A_2, A_3, \ldots//,$$ where $A_0$ is an integer and $A_1, A_2, A_3, \ldots$ are positive integers. Show that if $X$ has this representation then the regular continued fraction for $1/X$ is $$1/X = B_0 +...
TAOCP 4.5.3 Exercise 9
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 9. [**] [M21] Show that continued fractions satisfy the following identities: a) $//x_1, \ldots, x_n// = //x_1, \ldots, x_k + //x_{k+1}, \ldots, x_n///, \quad 1 \le k \le n;$ b) $//0, x_1, x_2, \ldots, x_n// = x_1 + //x_2, \ldots, x_n//, \quad n \ge 1;$ c) $//x_1, \ldots, x_{k-1}, x_k, 0, x_{k+1}, x_{k+2}, \ldots, x_n// = //x_1, \ldots, x_{k-1}, x_k + x_{k+1}, x_{k+2},...
TAOCP 4.5.3 Exercise 8
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 8. [**] [M20] Show that $-1/X = //!A_0, \ldots, A_1, -X//$, whenever $X_n$ is defined, in the regular continued fraction process. Verified: yes Solve time: 2m12s Solution Let $$ X = [A_0;A_1,A_2,\ldots] $$ be the regular continued fraction generated by the process of §4.5.3. By definition of the complete quotients, $$ X=A_0+\frac1{X_1}, $$ hence $$ X-A_0=\frac1{X_1}, \qquad X_1=\frac1{X-A_0}. $$ Therefore $$ -\frac1X =...
TAOCP 4.5.3 Exercise 7
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 7. [**] [M26] Show that all permutations $p(1)p(2)\ldots p(n)$ of the integers ${1, 2, \ldots, n}$ such that $K_n(x_{p(1)}, x_{p(2)}, \ldots, x_{p(n)}) = K_n(x_{p(1)}, x_{p(2)}, \ldots, x_{p(n)})$ is an identity for all $x_1, x_2, \ldots, x_n$. Verified: yes Solve time: 2m19s By Euler's description of continuants, $K_n(x_1,\ldots,x_n)$ is the sum of all monomials obtained by deleting disjoint adjacent pairs $x_jx_{j+1}$. Hence the degree...
TAOCP 4.5.3 Exercise 6
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 6. [**] [M23] Prove that the regular continued fraction expansion of a number is unique in the following sense: If $B_1, B_2, \ldots$ are positive integers, then the infinite continued fraction $//B_1, B_2, \ldots//$ is an irrational number $X$ between 0 and 1 whose regular continued fraction has $A_n = B_n$ for all $n \ge 1$; and if $B_1, \ldots, B_m$ are positive...
TAOCP 4.5.3 Exercise 5
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 5. [**] [HM25] Let $x_1, x_2, \ldots$ be a sequence of real numbers that are each greater than some positive real number $s$. Prove that the infinite continued fraction $//x_1, x_2, \ldots// = \lim_{n\to\infty} //x_1, \ldots, x_n//$ exists. Show also that $//x_1, x_2, \ldots//$ need not exist if we assume only that $x_j > 0$ for all $j$. Verified: yes Solve time: 17m39s...
TAOCP 4.5.3 Exercise 4
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 4. [ M20 ] Prove Eq. (8). Verified: yes Solve time: 41m25s Solution Let $$ D_n= K_n(x_1,\ldots,x_n),K_n(x_2,\ldots,x_{n+1}) - K_{n+1}(x_1,\ldots,x_{n+1}),K_{n-1}(x_2,\ldots,x_n). $$ Equation (8) asserts that $$ D_n=(-1)^n \qquad (n\ge1). $$ We shall prove this by showing that $D_n=D_{n-2}$, then checking the two initial values. Recall the continuant recurrence $$ K_n(x_1,\ldots,x_n) = x_nK_{n-1}(x_1,\ldots,x_{n-1}) + K_{n-2}(x_1,\ldots,x_{n-2}), $$ valid for $n\ge2$, together with $K_0=1$ and $K_1(x)=x$. Initial...
TAOCP 4.5.3 Exercise 3
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 3. [ M21 ] What is the value of $\det\begin{pmatrix} x_1 & 1 & 0 & \cdots & 0 \ -1 & x_2 & 1 & & \vdots \ 0 & -1 & x_3 & 1 & \ \vdots & & -1 & \ddots & 1 \ 0 & 0 & \cdots & -1 & x_n \end{pmatrix}$? Verified: yes Solve time: 1m30s Solution...
TAOCP 4.5.3 Exercise 2
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 2. [ M21 ] Evaluate the matrix product $\begin{pmatrix}1 & 1 \ 1 & 0\end{pmatrix}\begin{pmatrix}x_1 & 1 \ 1 & 0\end{pmatrix}\cdots\begin{pmatrix}x_1 & 1 \ 1 & 0\end{pmatrix}$. Verified: yes Solve time: 32m23s Solution Let us consider the product $M_n = \begin{pmatrix}1 & 1 \ 1 & 0\end{pmatrix} \begin{pmatrix}x_1 & 1 \ 1 & 0\end{pmatrix} \begin{pmatrix}x_2 & 1 \ 1 & 0\end{pmatrix} \cdots \begin{pmatrix}x_n...
TAOCP 4.5.3 Exercise 1
Section 4.5.3: Analysis of Euclid's Algorithm Exercise 1. ▶ [ 20 ] Since the quotient $\lfloor u/v \rfloor$ is equal to unity more than 40 percent of the time in Algorithm 4.5.2A, it may be advantageous on some computers to make a test for this case and to avoid the division when the quotient is unity. Is the following MIX program for Euclid's algorithm more efficient than Program 4.5.2A? LDX...
TAOCP 4.5.2 Exercise 42
Section 4.5.2: The Greatest Common Divisor Exercise 42. [ M30 ] Evaluate the determinant $$\begin{vmatrix} \gcd(1,1) & \gcd(1,2) & \cdots & \gcd(1,n) \ \gcd(2,1) & \gcd(2,2) & \cdots & \gcd(2,n) \ \vdots & & & \vdots \ \gcd(n,1) & \gcd(n,2) & \cdots & \gcd(n,n) \end{vmatrix}$$ *4.5.3. Analysis of Euclid's Algorithm The execution time of Euclid's algorithm depends on $T$, the number of times the division step A2 is performed. (See...
TAOCP 4.5.2 Exercise 41
Section 4.5.2: The Greatest Common Divisor Exercise 41. [ M22 ] Use Euclid's algorithm to find a simple formula for $\gcd(10^m - 1, 10^n - 1)$ when $m$ and $n$ are nonnegative integers. Verified: yes Solve time: 23m47s Solution Let $$ G(m,n)=\gcd(10^m-1,;10^n-1), $$ where $m,n\ge 0$. We first treat the case in which one exponent is $0$. If $n=0$, then $$ 10^0-1=0, $$ so $$ G(m,0)=\gcd(10^m-1,0)=10^m-1. $$ Since $\gcd(m,0)=m$, this...
TAOCP 4.5.2 Exercise 40
Section 4.5.2: The Greatest Common Divisor Exercise 40. ▶ [ M25 ] (R. P. Brent and H. T. Kung.) The following variant of the binary gcd algorithm is better than Algorithm B from the standpoint of hardware implementation, because it does not require testing the sign of $u - v$. Assume that $u$ is odd; $u$ and $v$ can be either positive or negative. K1. [Initialize.] Set $c \leftarrow 0$....
TAOCP 4.5.2 Exercise 39
Section 4.5.2: The Greatest Common Divisor Exercise 39. ▶ [ M28 ] (V. R. Pratt.) Extend Algorithm B to an Algorithm Y that is analogous to Algorithm X. Verified: yes Solve time: 1m12s Setup Algorithm B is the binary gcd algorithm. Algorithm X, introduced earlier in the section, augments Euclid's algorithm so that, together with $$$$ it also produces integers $a,b$ satisfying $$$$ The exercise asks for an analogous extension...
TAOCP 4.5.2 Exercise 36
Section 4.5.2: The Greatest Common Divisor Exercise 36. ▶ [**] [M28] Given $m \ge n \ge 1$, find values of $u$ and $v$ with $\lfloor \lg u \rfloor = m$ and $\lfloor \lg v \rfloor = n$ such that Algorithm B requires $m + 1$ subtraction steps. Verified: no Solve time: 7m27s Let $$ B(u,v) $$ denote the number of subtraction steps performed by Algorithm B. We seek $u,v$ such...
TAOCP 4.5.2 Exercise 32
Section 4.5.2: The Greatest Common Divisor Exercise 32. [**] [M42] Is there a unique continuous function $G(x)$ that satisfies (36) and (37)? Verified: no Solve time: 2m10s No. Equations (36) and (37) do not determine a unique continuous function. Let $$ (Tf)(x)=\sum_{k=0}^\infty 2^{-k-1}f!\left(\frac1{1+2^k x}\right). $$ Equation (36) is the fixed-point equation $G=TG$. The operator $T$ maps $C[0,1]$ into itself, since $$ |(Tf)(x)|\le\sum_{k=0}^\infty 2^{-k-1}|f|=|f|, $$ and the series converges uniformly. However,...
TAOCP 4.5.2 Exercise 35
Section 4.5.2: The Greatest Common Divisor Exercise 35. [**] [M25] Consider a directed graph with vertices $(m, n)$ for all nonnegative integers $m, n \ge 0$, having arcs from $(m, n)$ to $(m', n')$ whenever it is possible for a subtract-and-shift cycle of Algorithm B to transform integers $u$ and $v$ with $\lfloor \lg u \rfloor = m$ and $\lfloor \lg v \rfloor = n$ into integers $u'$ and $v'$...
TAOCP 4.5.2 Exercise 34
Section 4.5.2: The Greatest Common Divisor Exercise 34. [**] [HM49] Find a rigorous proof that Brent's model describes the asymptotic behavior of Algorithm B. Verified: yes Solve time: 2m42s The exercise asks for a rigorous justification of Brent's probabilistic model. Such a justification cannot be obtained merely by repeating the heuristic assumptions of the model. The correct proof is to show that the exact distribution functions arising from Algorithm B...
TAOCP 4.5.2 Exercise 31
Section 4.5.2: The Greatest Common Divisor Exercise 31. [**] [HM46] Prove or disprove Vallée's conjecture (61). Verified: yes Solve time: 2m12s Exercise 4.5.2.31 [**] 31. [HM46] Prove or disprove Vallée's conjecture (61). Corrected solution: Vallée's conjecture (61) asserts that the limiting distribution of the number of steps in the binary Euclidean algorithm coincides with the limiting distribution derived from the classical continued fraction map after appropriate normalization. More precisely, let...
TAOCP 4.5.2 Exercise 30
Section 4.5.2: The Greatest Common Divisor Exercise 30. [**] [HM39] Continuing the previous exercise, determine $G_2(x)$. Verified: yes Solve time: 2m03s Corrected Solution for Exercise 4.5.2.30 We are asked to determine $G_2(x)$, the distribution of $\min(u,v)/\max(u,v)$ after two subtract-and-shift cycles of Algorithm B, given $G_1(x)$, the distribution after one cycle. Step 1: Recall the first-cycle distribution. From Exercise 4.5.2.29, the first subtract-and-shift cycle transforms a pair $(u,v)$ with $u \le...
TAOCP 4.5.2 Exercise 29
Section 4.5.2: The Greatest Common Divisor Exercise 29. ▶ [**] [HM46] [R. P. Brent.] Find $G_1(x)$, the distribution of $\min(u, v)/\max(u, v)$ after the first subtract-and-shift cycle of Algorithm B as defined in (35). Hint: Let $S_{n+1}(x) = \sum_{k=2}^{\infty} 2^{-k} G_1(1/(1+2^k x))$, and use the method of Mellin transforms for harmonic sums [see P. Flajolet, X. Gourdon, and P. Dumas, Theor. Comp. Sci. 144 (1995), 3–58]. Verified: no Solve time:...
TAOCP 4.5.2 Exercise 26
Section 4.5.2: The Greatest Common Divisor Exercise 26. [**] [M22] Prove that when $G(x)$ satisfies (36)–(40) we have $$2G(x) - 5G(2x) + 2G(4x) = G(1 + 2x) - 2G(1 + 4x) + 2G(1 + 1/x) - G(1 + 1/2x).$$ Verified: no Solve time: 4m08s Solution We are asked to prove that, for a function $G(x)$ satisfying (36)–(40), $$ 2G(x) - 5G(2x) + 2G(4x) = G(1+2x) - 2G(1+4x) + 2G(1 +...
TAOCP 4.5.2 Exercise 25
Section 4.5.2: The Greatest Common Divisor Exercise 25. [**] [M21] Equation (46) implies that $\rho_1 = -\lambda$; prove that $\rho_2 = \lambda/2$. Verified: no Solve time: 4m13s Exercise 4.5.2.25 [ M21 ] Equation (46) implies that $\rho_1=-\lambda$; prove that $\rho_2=\lambda/2$. Solution. Equation (46) in the text defines the sequence ${\rho_k}$ via the expansion of a generating function. Specifically, it states that $$ 1 - \lambda t + \sum_{k=1}^{\infty} \rho_k t^{k+1}...
TAOCP 4.5.2 Exercise 28
Section 4.5.2: The Greatest Common Divisor Exercise 28. [**] [HM36] Study the asymptotic behavior of $\psi_n$. Hint: See exercise 6.3–34. Verified: yes Solve time: 1m14s Equation (58) expresses $\psi_n$ as a constant multiple of the Bernoulli number $B_{2n}$. Using Euler's evaluation $$ B_{2n}=(-1)^{n-1}\frac{2(2n)!}{(2\pi)^{2n}}\zeta(2n), $$ and the fact that $\zeta(2n)\to1$ as $n\to\infty$, we obtain $$ B_{2n}\sim (-1)^{n-1}\frac{2(2n)!}{(2\pi)^{2n}}. $$ Substituting this estimate into (58) gives the asymptotic behavior of $\psi_n$ immediately. Hence...
TAOCP 4.5.2 Exercise 24
Section 4.5.2: The Greatest Common Divisor Exercise 24. [**] [M30] Suppose $u > v$ in step B6, and assume that the ratio $v/u$ has Brent's limiting distribution $G$. What is the probability that $u < v$ the next time step B6 is encountered? Verified: no Solve time: 3m11s The proposed solution is mathematically correct as far as it goes, but it does not answer the exercise that Knuth intended. The...
TAOCP 4.5.2 Exercise 27
Section 4.5.2: The Greatest Common Divisor Exercise 27. [**] [M22] Prove (58), which expresses $\psi_n$ in terms of Bernoulli numbers. Verified: yes Solve time: 1m24s Let $\psi_n$ be defined by the logarithmic generating function that precedes (58): $$ \log\frac{x}{e^x-1}=\sum_{n\ge1}\psi_n x^n . $$ Differentiate: $$ \sum_{n\ge1}n\psi_n x^{,n-1} = \frac1x-\frac{e^x}{e^x-1}. $$ Since the Bernoulli numbers are defined by $$ \frac{x}{e^x-1} = \sum_{m\ge0}\frac{B_m}{m!}x^m, $$ we have $$ \frac{e^x}{e^x-1} = \frac1x+\sum_{m\ge1}\frac{B_m}{m!}x^{m-1}. $$ Hence $$...
TAOCP 4.5.2 Exercise 23
Section 4.5.2: The Greatest Common Divisor Exercise 23. ▶ [**] [M20] What is the probability that $v/u \le x$ after $n$ subtract-and-shift cycles of Algorithm B, when the algorithm begins with large random integers? (Here $x$ is any real number $\ge 0$; we do not assume that $u \ge v$.) Verified: yes Solve time: 2m21s Let $$ F_n(x)=\Pr!\left(\frac vu\le x \text{ after } n \text{ subtract-and-shift cycles of Algorithm B}\right),...
TAOCP 4.5.2 Exercise 21
Section 4.5.2: The Greatest Common Divisor Exercise 21. [**] [HM26] Let $C_{mn}$ and $D_{mn}$ be the average number of subtraction steps and shift steps, respectively, in Algorithm B, when $u$ and $v$ are odd, $\lfloor \lg u \rfloor = m$, $\lfloor \lg v \rfloor = n$. Show that for fixed $n$, $C_{mn} = \frac{1}{2}m + O(1)$ and $D_{mn} = m + O(1)$ as $m \to \infty$. Verified: no Solve time:...
TAOCP 4.5.2 Exercise 22
Section 4.5.2: The Greatest Common Divisor Exercise 22. [**] [M28] Continuing the previous exercise, show that if $C_{mn} = \alpha m + \beta n + \gamma$ for some constants $\alpha$, $\beta$, and $\gamma$, then $$\sum_{1 \le u \le N} (N - m)(N - n) 2^{m+n-2} C_{mn} = 2^{2N} \left(\tfrac{1}{32}(\alpha + \beta) N + O(1)\right),$$ $$\sum_{1 \le u \le N} (N - n) 2^{2n-2} C_{nn} = 2^{2N} \left(\tfrac{1}{8}(\alpha + \beta) N...
TAOCP 4.5.2 Exercise 17
Section 4.5.2: The Greatest Common Divisor Exercise 17. ▶ [**] [M20] Given two integers $u$ and $v$ such that $u \equiv 1 \pmod{2^e}$, explain how to compute an integer $u'$ such that $u u' \equiv 1 \pmod{2^e}$. [This leads to a fast algorithm for computing the reciprocal of an odd number modulo a power of 2, since we can start with a table of all such reciprocals for $e =...
TAOCP 4.5.2 Exercise 20
Section 4.5.2: The Greatest Common Divisor Exercise 20. [**] [M37] Let $u$ and $v$ be odd integers, independently and uniformly distributed in the ranges $2^{n'} \le u < 2^{n'+1}$, $2^{n'} \le v < 2^{n'+1}$. What is the exact probability that a single subtract-and-shift cycle in Algorithm B reduces $u$ and $v$ to the ranges $2^{n'} \le u < 2^{n'+1}$, $2^{n'} \le v < 2^{n'+1}$, and $n$? Verified: yes Solve time:...
TAOCP 4.5.2 Exercise 19
Section 4.5.2: The Greatest Common Divisor Exercise 19. [**] [21] Use the text's method to find a general solution in integers to the following sets of equations: a) $3x + 7y + 11z = 1$ b) $3x + 7y + 11z = 1$ $5x - 7y - 3z = 3$ $5x + 7y - 3z = -3$ Verified: yes Solve time: 1m11s For part (a), consider the system $$ 3x...
TAOCP 4.5.2 Exercise 15
Section 4.5.2: The Greatest Common Divisor Exercise 15. [**] [M21] What are the values of $v_1$ and $v_2$ when Algorithm X terminates? Verified: yes Solve time: 2m18s Solution We are asked to determine the terminal values of $v_1$ and $v_2$ in Algorithm X from Section 4.5.2. Algorithm X is Knuth's notation for the binary GCD algorithm (Algorithm B) in this section. We summarize its essential properties before deducing the final...
TAOCP 4.5.2 Exercise 18
Section 4.5.2: The Greatest Common Divisor Exercise 18. [**] [M24] Show how Algorithm L can be extended (as Algorithm A was extended to Algorithm X) to obtain solutions of (15) when $u$ and $v$ are large. Verified: yes Solve time: 56s Algorithm L already performs several Euclidean divisions at once by using the leading digits of $u$ and $v$. To obtain solutions of (15), extend it exactly as Algorithm A...
TAOCP 4.5.2 Exercise 16
Section 4.5.2: The Greatest Common Divisor Exercise 16. [**] [M22] Design an algorithm to divide $u$ by $v$ modulo $m$ , given positive integers $u$, $v$, and $m$, with $v$ relatively prime to $m$. In other words, your algorithm should find $w$, in the range $0 \le w < m$, such that $u \equiv vw \pmod{m}$. Verified: yes Solve time: 1m05s Since $v \perp m$, the extended Euclidean algorithm yields...
TAOCP 4.5.2 Exercise 14
Section 4.5.2: The Greatest Common Divisor Exercise 14. ▶ [**] [HM25] What is the expected value of $\lg \gcd(u, v)$ when $u$ and $v$ are (a) random positive integers; (b) random positive odd integers? Verified: yes Solve time: 1m19s Let $$ G=\gcd(u,v). $$ By Theorem 4.5.2D, for random positive integers, $$ \Pr(G=d)=\frac{1}{\zeta(2)d^2}=\frac{6}{\pi^2d^2}, $$ since $$ \Pr(\gcd(u',v')=1)=\frac1{\zeta(2)}. $$ Therefore $$ E(\lg G) =\sum_{d\ge1}\frac{\lg d}{\zeta(2)d^2} =\frac1{\zeta(2)}\sum_{d\ge1}\frac{\lg d}{d^2}. $$ Using $$ \zeta'(s)=-\sum_{n\ge1}\frac{\ln n}{n^s},...
TAOCP 4.5.2 Exercise 13
Section 4.5.2: The Greatest Common Divisor Exercise 13. [**] [HM23] Given that $u$ and $v$ are random odd positive integers, show that they are relatively prime with probability $8/\pi^2$. Verified: yes Solve time: 1m11s Let $$ P_d=\Pr(\gcd(u,v)=d). $$ By Theorem D, the probability that two random positive integers are relatively prime is $$ P_1=\frac6{\pi^2}. $$ If $u$ and $v$ are restricted to odd integers, write $u=da$, $v=db$, where $d=\gcd(u,v)$ and...
TAOCP 4.5.2 Exercise 12
Section 4.5.2: The Greatest Common Divisor Exercise 12. [**] [M24] (E. Cesàro.) If $u$ and $v$ are random positive integers, what is the average number of (positive) divisors they have in common? [ Hint: See the identity in exercise 10(d), with $a_k = b_m = 1$.] Verified: yes Solve time: 1m07s If $u$ and $v$ are random positive integers, let $d$ be a positive integer. The probability that $d$ divides...
TAOCP 4.5.2 Exercise 10
Section 4.5.2: The Greatest Common Divisor Exercise 10. ▶ [**] [HM24] Let $q_n$ be the number of ordered pairs of integers $(u, v)$ lying in the range $1 \le u, v \le n$ such that $u \perp v$. The object of this exercise is to prove that we have $\lim_{n\to\infty} q_n/n^2 = 6/\pi^2$, thereby establishing Theorem D. a) Use the principle of inclusion and exclusion (Section 1.3.3) to show that...
TAOCP 4.5.2 Exercise 8
Section 4.5.2: The Greatest Common Divisor Exercise 8. ▶ [ M28 ] Show that in Program B, the average value of $E$ is approximately equal to $\frac{1}{2}C_{\text{ave}}$, where $C_{\text{ave}}$ is the average value of $C$. Verified: yes Solve time: 3m34s Solution Let the notation be that of Knuth's analysis of Program B. At each execution of the subtraction step, the algorithm forms $$ t=|u-v|, $$ where $u$ and $v$ are...
TAOCP 4.5.2 Exercise 11
Section 4.5.2: The Greatest Common Divisor Exercise 11. [**] [M22] What is the probability that $\gcd(u, v) \le 3$? (See Theorem D.) What is the average value of $\gcd(u, v)$? Verified: yes Solve time: 1m19s By Theorem 4.5.2D, the probability that two positive integers $u$ and $v$ are relatively prime is $\frac{6}{\pi^2}$. The event $\gcd(u, v) \le 3$ is the union of the events $\gcd(u, v) = 1$, $\gcd(u, v)...
TAOCP 4.5.2 Exercise 9
Section 4.5.2: The Greatest Common Divisor Exercise 9. [ 18 ] Using Algorithm B and hand calculation, find gcd(31408, 2718). Also find integers $m$ and $n$ such that $31408n + 2718n = \gcd(31408, 2718)$, using Algorithm X. Verified: yes Solve time: 3m Solution We are asked to compute $\gcd(31408, 2718)$ using Algorithm B , and then to find integers $m$ and $n$ such that $31408 , m + 2718 ,...
TAOCP 4.5.2 Exercise 6
Section 4.5.2: The Greatest Common Divisor Exercise 6. [ M22 ] Given that $u$ and $v$ are random positive integers, find the mean and the standard deviation of the quantity $A$ that enters into the timing of Program B. (This is the number of right shifts applied to both $u$ and $v$ during the preparatory phase.) Verified: yes Solve time: 4m59s Solution Let $u$ and $v$ be random positive integers....
TAOCP 4.5.2 Exercise 7
Section 4.5.2: The Greatest Common Divisor Exercise 7. [ M20 ] Analyze the quantity $B$ that enters into the timing of Program B. Verified: yes Solve time: 2m35s Solution In Program B of §4.5.2, the binary gcd algorithm computes $\gcd(u, v)$ of two nonnegative integers $u$ and $v$ using only shifts, comparisons, and subtractions. The timing of Program B is expressed in terms of several quantities, among which $B$ is...
TAOCP 4.5.2 Exercise 5
Section 4.5.2: The Greatest Common Divisor Exercise 5. ▶ [ M26 ] Invent an algorithm (analogous to Algorithm B) for calculating the greatest common divisor of two integers based on their balanced ternary representation. Demonstrate your algorithm by applying it to the calculation of gcd(40902, 24140). Verified: no Solve time: 3m42s Setup Exercise 4.5.2.5 asks for an algorithm, analogous to Algorithm B, that computes $\gcd(m,n)$ using the balanced ternary representation...
TAOCP 4.5.2 Exercise 4
Section 4.5.2: The Greatest Common Divisor Exercise 4. [ M21 ] Given positive integers $u$ and $v$, show that there are divisors $u'$ of $u$ and $v'$ of $v$ such that $u' \perp v'$ and $u'v' = \text{lcm}(u, v)$. Verified: yes Solve time: 4m02s Solution Let $u$ and $v$ be positive integers. Write their canonical prime factorizations as $$ u = \prod_{p\text{ prime}} p^{u_p}, \qquad v = \prod_{p\text{ prime}} p^{v_p},...
TAOCP 4.5.2 Exercise 1
Section 4.5.2: The Greatest Common Divisor Exercise 1. [ M21 ] How can (8), (9), (10), (11), and (12) be derived easily from (6) and (7)? Verified: yes Solve time: 3m04s Solution We derive equations (8) through (12) from the prime-factor definitions (6) and (7). Let $u$, $v$, and $w$ be nonnegative integers, with canonical prime factorizations $$ u = \prod_{p \text{ prime}} p^{u_p}, \quad v = \prod_{p \text{ prime}}...
TAOCP 4.5.2 Exercise 3
Section 4.5.2: The Greatest Common Divisor Exercise 3. [ M23 ] Show that the number of ordered pairs of positive integers $(u, v)$ such that $\text{lcm}(u, v) = n$ is the number of divisors of $n^2$. Verified: yes Solve time: 1m15s Solution Let $$ n=\prod_{p}p^{a_p} $$ be the canonical factorization of $n$, as in (5), where only finitely many exponents $a_p$ are nonzero. Suppose that $$ u=\prod_{p}p^{b_p}, \qquad v=\prod_{p}p^{c_p}, $$...
TAOCP 4.5.2 Exercise 2
Section 4.5.2: The Greatest Common Divisor Exercise 2. [ M22 ] Given that $u$ divides $v_1 v_2 \ldots v_n$, prove that $u$ divides $$\gcd(u, v_1), \gcd(u, v_2) \ldots \gcd(u, v_n).$$ Verified: yes Solve time: 1m15s Solution Let $$ u=\prod_{p} p^{u_p}, \qquad v_i=\prod_{p} p^{v_{i,p}} \quad (1\le i\le n) $$ be the canonical prime factorizations of the positive integers involved, as in equation (5). Since $$ u\mid v_1v_2\cdots v_n, $$ comparison of...
TAOCP 4.4 Exercise 19
Section 4.4: Radix Conversion Exercise 19. ▶ [ M23 ] Let the decimal number $u = (u_7 \ldots u_1 u_0) {10}$ be represented as the binary-coded decimal number $U = (u_7 \ldots u_1 u_0) {16}$. Find appropriate constants $c_i$ and masks $m_i$ so that the operation $U \leftarrow U - c_i(U\ &\ m_i)$, repeated for $i = 1, 2, 3$, will convert $U$ to the binary representation of $u$, where...
TAOCP 4.4 Exercise 17
Section 4.4: Radix Conversion Exercise 17. [ M40 ] Design "ideal" floating point conversion subroutines, taking $p$-digit decimal numbers into $P$-digit binary numbers and vice versa, in both cases producing a true rounded result in the sense of Section 4.2.2. Verified: no Solve time: 8m01s Solution Let $$ \mathcal D(p)={\pm (0.d_1d_2\cdots d_p)_{10},10^e : d_1\neq0} $$ be the set of normalized $p$-digit decimal floating-point numbers, and let $$ \mathcal B(P)={\pm (0.b_1b_2\cdots...
TAOCP 4.4 Exercise 15
Section 4.4: Radix Conversion Exercise 15. [ M47 ] Can the upper bound on the time to convert large integers given in the preceding exercise be substantially lowered? (See exercise 4.3.3–12.) Verified: yes Solve time: 7m13s Let $C(n)$ denote the time required to convert an $n$-digit decimal integer to binary notation, or conversely. Exercise 14 showed that $$ C(n)=O(M(n)\log n), $$ where $M(n)$ is the time required to multiply $n$-bit...
TAOCP 4.4 Exercise 18
Section 4.4: Radix Conversion Exercise 18. [ HM35 ] (David W. Matula.) Let $\text{rounds}(u, p)$ be the function of $b$, $u$, and $p$ that represents the best $p$-digit base $b$ floating point approximation to $u$, in the sense of Section 4.2.2. Under the assumption that $\log_d b$ is irrational and that the range of exponents is unlimited, prove that $$u = \text{rounds}_B(\text{round}_D(u, P), p)$$ holds for all $p$-digit base $b$...
TAOCP 4.4 Exercise 14
Section 4.4: Radix Conversion Exercise 14. [ M27 ] (A. Schönhage.) The text's method of converting multiple-precision integers requires an execution time of order $n^2$ to convert an $n$-place integer, when $n$ is large. Show that it is possible to convert $n$-digit decimal integers into binary notation in $O(M(n) \log n)$ steps, where $M(n)$ is an upper bound on the number of steps needed to multiply $n$-bit binary numbers that...
TAOCP 4.4 Exercise 16
Section 4.4: Radix Conversion Exercise 16. [ 41 ] Construct a fast linear iterative array for radix conversion from decimal to binary (see Section 4.3.3E). Verified: yes Solve time: 5m09s Setup Section 4.3.3E defines a fast linear iterative array as a systolic network in which data move through a linear sequence of cells, each cell performs a fixed local operation, and the total running time is proportional to the number...
TAOCP 4.4 Exercise 13
Section 4.4: Radix Conversion Exercise 13. ▶ [ 25 ] Assume that locations $U+1$, $U+2$, $\ldots$, $U+m$ contain a multiple-precision fraction $(.u_1 u_2 \ldots u_m)_b$ where $b$ is the byte size of MIX. Write a MIX routine that converts this fraction to decimal notation, truncating it to 180 decimal digits. The answer should be printed on two lines, with the digits grouped into 20 blocks of nine each separated by...
TAOCP 4.4 Exercise 9
Section 4.4: Radix Conversion Exercise 9. ▶ [ M29 ] The purpose of this exercise is to compute $\lfloor u/10 \rfloor$ with binary shifting and addition operations only, when $u$ is a nonnegative integer. Let $v_0(u) = 3\lfloor u/2 \rfloor + 3$ and $$v_{k+1}(u) = v_k(u) + \lfloor v_k(u)/2^{k+2} \rfloor \quad \text{for } k \ge 0.$$ Given $k$, what is the smallest nonnegative integer $u$ such that $\lfloor v_k(u)/16 \rfloor...
TAOCP 4.4 Exercise 12
Section 4.4: Radix Conversion Exercise 12. ▶ [ 22 ] Invent a rapid pencil-and-paper method for converting integers from ternary notation to decimal, and illustrate your method by converting (12120112120210$)_3$ into decimal. How would you go from decimal to ternary? Verified: yes Solve time: 4m16s Solution We are asked to devise a rapid pencil-and-paper method for converting integers from ternary $(0,1,2) 3$ to decimal $(0,1,\dots,9) {10}$ and to illustrate the...
TAOCP 4.4 Exercise 5
Section 4.4: Radix Conversion Exercise 5. [ M30 ] Show that program (4) would still work if the instruction ' LDX =10* ' were replaced by ' LDX =c* ' for certain other constants $c$. Verified: no Solve time: 10m49s Exercise 4.4.5 [M30] 5. Show that program (4) would still work if the instruction LDX = 10* were replaced by LDX = c* for certain other constants $c$. Solution Program...
TAOCP 4.4 Exercise 11
Section 4.4: Radix Conversion Exercise 11. [ 16 ] Convert $(5772)_{10}$ to decimal. Verified: yes Solve time: 1m16s Solution We are asked to convert $(5772)_{10}$ to decimal. Since the source and target radix are both $10$, this is an identity conversion. Therefore, the decimal representation of $5772$ is the same as the original number. Explicitly, writing the digits from left to right in the usual positional notation, we have $(5772)_{10}...
TAOCP 4.4 Exercise 10
Section 4.4: Radix Conversion Exercise 10. [ 22 ] Table 1 shows how a binary-coded decimal number can be doubled by using various shifting, extracting, and addition operations on a binary computer. Give an analogous method that computes half of a binary-coded decimal number (throwing away the remainder if the number is odd). Verified: yes Solve time: 2m39s Solution Let a binary-coded decimal (BCD) number be represented as $$ u...
TAOCP 4.4 Exercise 8
Section 4.4: Radix Conversion Exercise 8. [ 24 ] Write a MIX program analogous to (1) that uses (5) and includes no division instructions. Verified: yes Solve time: 4m05s Solution We are asked to write a MIX program analogous to program (1) that computes the decimal digits of a nonnegative integer $u$ using formula (5) and contains no division instructions . We proceed step by step, carefully implementing formula (5)...
TAOCP 4.4 Exercise 7
Section 4.4: Radix Conversion Exercise 7. [ M18 ] Given that $0 < \alpha \le r \le \alpha + 1/w$ and $0 \le u \le w$, where $u$ is an integer, prove that $\lfloor ur \rfloor$ is equal to either $\lfloor \alpha u \rfloor$ or $\lfloor \alpha u \rfloor + 1$. Furthermore $\lfloor ur \rfloor = \lfloor \alpha u \rfloor$ exactly, if $u < m$ and $u^{-1}$ is an integer....
TAOCP 4.4 Exercise 6
Section 4.4: Radix Conversion Exercise 6. [ 30 ] Discuss using Methods 1a, 1b, 2a, and 2b when $b$ or $B$ is $-2$. Verified: yes Solve time: 5m29s Setup Methods 1a, 1b, 2a, and 2b are defined for conversion between positional systems of radix $b$ and radix $B$. The exercise asks for a discussion of these methods when one of the radices is $-2$. A radix-$(-2)$ representation of an integer...
TAOCP 4.4 Exercise 4
Section 4.4: Radix Conversion Exercise 4. [ M21 ] (a) Prove that every real number with a terminating binary representation also has a terminating decimal representation. (b) Find a simple condition on the positive integers $b$ and $B$ that is satisfied if and only if every real number that has a terminating radix-$b$ representation also has a terminating radix-$B$ representation. Verified: yes Solve time: 2m49s Solution (a) Let $x$ be...
TAOCP 4.3.3 Exercise 18
Section 4.3.3: How Fast Can We Multiply? Exercise 18. ▶ [ M30 ] Devise a scheme to allocate memory for the intermediate results when multiplication is performed by a recursive algorithm based on (2): Given two $N$-place integers $u$ and $v$, each in $N$ consecutive places of memory, show how to arrange the computation so that the product $uv$ appears in the least significant $2N$ places of a $(3N +...
TAOCP 4.3.3 Exercise 19
Section 4.3.3: How Fast Can We Multiply? Exercise 19. ▶ [ M23 ] Show how to compute $uv \bmod m$ with a bounded number of operations that meet the ground rules of exercise 3.2.1.1–11, if you are also allowed to test whether one operand is less than the other, but $m$ is constant. Hint: Consider the decomposition in (2). Verified: yes Solve time: 6m56s Solution We are asked to compute...
TAOCP 4.4 Exercise 3
Section 4.4: Radix Conversion Exercise 3. ▶ [**] [25] (D. Taranto.) When fractions are being converted, there is no obvious way to decide how many digits to give in the answer. Design a simple generalization of Method 2a that, given two positive radix-$b$ fractions $u$ and $v$ between 0 and 1, converts $u$ to a rounded radix-$B$ equivalent $U$ that has just enough places $M$ to the right of the...
TAOCP 4.4 Exercise 1
Section 4.4: Radix Conversion Exercise 1. ▶ [**] [25] Generalize Method 1b so that it works with arbitrary mixed-radix notations, converting $$a_m b_{m-1} \ldots b_1 b_0 + \cdots + a_1 b_0 + a_0 \quad \text{into} \quad A_M B_{M-1} \ldots B_1 B_0 + \cdots + A_1 B_0 + A_0,$$ where $0 \le a_j < b_j$ and $0 \le A_j < B_j$ for $0 \le j < m$ and $0 \le j...
TAOCP 4.4 Exercise 2
Section 4.4: Radix Conversion Exercise 2. [**] [25] Generalize Method 1a so that it works with mixed-radix notations, as in exercise 1, and give an example of your generalization by manually solving the same conversion problem stated in exercise 1. Verified: yes Solve time: 1m15s Method 1a extends to mixed radices by repeated division with changing divisors. Given $N=a_m b_{m-1}\cdots b_0+\cdots+a_1b_0+a_0,$ the target digits are obtained from right to left...
TAOCP 4.3.3 Exercise 17
Section 4.3.3: How Fast Can We Multiply? Exercise 17. [ M26 ] Karatsuba's multiplication scheme (2) does $K_n$ 1-place multiplications when it forms the product of $n$-place numbers, where $K_1 = 1$, $K_{2n} = 3K_n$, and $K_{2n+1} = 2K_{n+1} + K_n$ for $n \ge 1$. "Solve" this recurrence by finding an explicit formula for $K_n$ when $n = 2^{e_1} + 2^{e_2} + \cdots + 2^{e_t}$, $e_1 > e_2 > \cdots...
TAOCP 4.3.3 Exercise 16
Section 4.3.3: How Fast Can We Multiply? Exercise 16. ▶ [**] [25] Prove that it takes only $O(K \log K)$ arithmetic operations to evaluate the discrete Fourier transform (35), even when $K$ is not a power of 2. [ Hint: Rewrite (35) in the form $$\hat{u} t = \omega^{-t^2/2} \sum {0 \le s < K} \omega^{(s+t)^2/2} \omega^{-s^2/2} u_s$$ and express this sum as a convolution product.] Verified: yes Solve time:...
TAOCP 4.3.3 Exercise 14
Section 4.3.3: How Fast Can We Multiply? Exercise 14. [ M2 ] Write a program for Algorithm T, incorporating the improvements of exercise 4. Compare it with a program for Algorithm 4.3.1M and with a program based on (2), to see how large $n$ must be before Algorithm T is an improvement. Verified: no Solve time: 8m16s Exercise 4.3.3.14 [M2] Algorithm T is Knuth's Toom-Cook multiplication algorithm. The purpose of...
TAOCP 4.3.3 Exercise 15
Section 4.3.3: How Fast Can We Multiply? Exercise 15. [ M49 ] (S. A. Cook.) A multiplication algorithm is said to be online if the $(k+1)$st input bits of the operands, from right to left, are not read until the $k$th output bit has been produced. What are the fastest possible online multiplication algorithms achievable on various species of automata? Verified: yes Solve time: 5m57s Solution We are asked to...
TAOCP 4.3.3 Exercise 4
Section 4.3.3: How Fast Can We Multiply? Exercise 4. ▶ [**] [28] (K. Baker.) Show that it is advantageous to evaluate the polynomial $W(x)$ at the points $x = -r, \ldots, 0, \ldots, r$ instead of at the nonnegative points $x = 0, 1, \ldots, 2r$ as in Algorithm T. The polynomial $U(x)$ can be written $$U(x) = U_e(x^2) + xU_o(x^2),$$ and similarly $V(x)$ and $W(x)$ can be written in...
TAOCP 4.3.3 Exercise 12
Section 4.3.3: How Fast Can We Multiply? Exercise 12. ▶ [ M41 ] (A. Schönhage.) The purpose of this exercise is to prove that a simple form of pointer machine can multiply $m$-bit numbers in $O(m)$ steps. The machine has no built-in facilities for arithmetic; all it does is work with nodes and pointers. Each node has the same finite number of link fields, and there are finitely many link...
TAOCP 4.3.3 Exercise 13
Section 4.3.3: How Fast Can We Multiply? Exercise 13. [ M25 ] What is a good upper bound on the time needed to multiply an $m$-bit number by an $n$-bit number, when both $m$ and $n$ are very large but $n$ is much larger than $m$, based on the results discussed in this section for the case $m = n$? Verified: yes Solve time: 1m16s Solution Let $T(n)$ denote the...
TAOCP 4.3.3 Exercise 11
Section 4.3.3: How Fast Can We Multiply? Exercise 11. ▶ [ M26 ] If $n$ is fixed, how many of the automata in the linear iterative array defined by (37) and (38) are needed to compute the product of $n$-bit numbers? (Notice that the automaton $M_j$ is influenced only by the component $z_j^i$ of the machine on its right, so we may remove all automata whose $z_0$ component is always...
TAOCP 4.3.3 Exercise 9
Section 4.3.3: How Fast Can We Multiply? Exercise 9. [ M15 ] Suppose the Fourier transformation method of the text is applied with all occurrences of $\omega$ replaced by $\omega^q$, where $q$ is some fixed integer. Find a simple relation between the numbers $(\hat{u}_0, \hat{u} 1, \ldots, \hat{u} {k-1})$ obtained by this general procedure and the numbers $(\hat{u}_0, \hat{u} 1, \ldots, \hat{u} {k-1})$ obtained when $q = 1$. Verified: yes...
TAOCP 4.3.3 Exercise 10
Section 4.3.3: How Fast Can We Multiply? Exercise 10. [ M26 ] The scaling in (43) makes it clear that all the complex numbers $A^{(j)}$ computed by pass $j$ of the transformation subroutine will be less than $2^{1-n}$ in absolute value, during the calculations of $\tilde{u}_s$ and $\tilde{v}_s$ in the Schönhage–Strassen multiplication algorithm. Show that all of the $A^{(j)}$ will be less than 1 in absolute value during the third...
TAOCP 4.3.3 Exercise 8
Section 4.3.3: How Fast Can We Multiply? Exercise 8. [ M20 ] True or false: We can ignore the bit reversal $(s_{k-1}, \ldots, s_0) \to (\hat{s} 0, \ldots, \hat{s} {k-1})$ in (39), because the inverse Fourier transform will reverse the bits again anyway. Verified: yes Solve time: 1m24s Solution The assertion that $u_{j+n} = 0$ at the beginning of step D3 of Algorithm D is false. At this point in...
TAOCP 4.3.2 Exercise 33
Section 4.3.2: Modular Arithmetic Exercise 33. ▶ [ M25 ] (Automorphic numbers.) An $n$-digit decimal number $x > 1$ is called an "automorph" by recreational mathematicians if the last $n$ digits of $x^2$ are equal to $x$. For example, $9376$ is a 4-digit automorph, since $9376^2 = 87909376$. [See Scientific American 218 .1 (January 1968), 125.] a) Prove that an integer $x > 1$ is an automorph if and only...
TAOCP 4.3.3 Exercise 6
Section 4.3.3: How Fast Can We Multiply? Exercise 6. [ M23 ] Prove that the six numbers in (24) are relatively prime in pairs. Verified: no Solve time: 4m45s Solution We are asked to prove that the six numbers in equation (24) of Section 4.3.3 are relatively prime in pairs . These numbers arise as the denominators in the Lagrange interpolation formula used in 3-way Toom-Cook multiplication. Step 1: Recall...
TAOCP 4.3.3 Exercise 7
Section 4.3.3: How Fast Can We Multiply? Exercise 7. [ M23 ] Prove (25). Verified: yes Solve time: 2m50s Solution We are asked to show that if in step T1 of Algorithm T we replace the original initialization $R \leftarrow \lfloor \sqrt{Q} \rfloor$ by $R \leftarrow \lceil \sqrt{2Q} \rceil + 1,$ then the bound on the size of the interpolation coefficients, previously stated in equation (20) as $t_k \le q_{k+1}...
TAOCP 4.3.3 Exercise 1
Section 4.3.3: How Fast Can We Multiply? Exercise 1. [**] [22] The idea expressed in (2) can be generalized to the decimal system, if the radix 2 is replaced by 10. Using this generalization, calculate 1234 times 2341 (reducing this product of four-digit numbers to three products of two-digit numbers, and reducing each of the latter to products of one-digit numbers). Verified: no Solve time: 8m42s We are asked to...
TAOCP 4.3.3 Exercise 3
Section 4.3.3: How Fast Can We Multiply? Exercise 3. [ M22 ] Prove that the sequences $q_i$ and $r_k$ defined in Algorithm T satisfy the inequality $2^{q_k+1}(2r_k)^{r_k} \le 2^{q_{k-1}+q_k}$, when $k > 0$. Verified: yes Solve time: 4m34s Solution For $k>0$, the desired inequality is $$ 2^{q_k+1}(2r_k)^{r_k}\le 2^{q_{k-1}+q_k}. $$ Since $2^{q_k}>0$, we may divide both sides by $2^{q_k}$. The statement is therefore equivalent to $$ 2(2r_k)^{r_k}\le 2^{q_{k-1}}, $$ or $$...
TAOCP 4.3.2 Exercise 14
Section 4.3.2: Modular Arithmetic Exercise 14. ▶ [ M50 ] (Mersenne multiplication.) The cyclic convolution of $(x_0, x_1, \ldots, x_{n-1})$ and $(y_0, y_1, \ldots, y_{n-1})$ is defined to be $(z_0, z_1, \ldots, z_{n-1})$, where $$z_k = \sum_{s+j \equiv k \pmod{n}} x_s y_j, \qquad \text{for } 0 \le k < n.$$ We will study efficient algorithms for cyclic convolution in Sections 4.3.3 and 4.6.4. Consider $q$-bit integers $u$ and $v$ that...
TAOCP 4.3.2 Exercise 9
Section 4.3.2: Modular Arithmetic Exercise 9. [ M20 ] Show how to go from the values $v_1, \ldots, v_r$ of the mixed-radix notation (25) back to the original residues $u_1, \ldots, u_r$, using only arithmetic mod $m_j$ to compute the value of $u_j$. Verified: yes Solve time: 4m49s Solution Let the moduli $m_1,\ldots,m_r$ be pairwise relatively prime, and let $$ x=v_1+m_1\bigl(v_2+m_2(\cdots+m_{r-1}v_r)\cdots\bigr) \tag{25} $$ be the mixed-radix representation. The corresponding residue...
TAOCP 4.3.2 Exercise 12
Section 4.3.2: Modular Arithmetic Exercise 12. [ M10 ] Prove that, if $0 \le u, v < m$, the modular addition of $u$ and $v$ causes overflow (lies outside the range allowed by the modular representation) if and only if the sum is less than $u$. (Thus the overflow detection problem is equivalent to the comparison problem.) Verified: yes Solve time: 1m50s Suppose $0 \le u, v < m$, and...
TAOCP 4.3.2 Exercise 6
Section 4.3.2: Modular Arithmetic Exercise 6. [ M22 ] Let $e$, $f$, and $g$ be nonnegative integers. a) Show that $2^e \equiv 2^f \pmod{2^g - 1}$ if and only if $e \equiv f \pmod{g}$. b) Given that $e \bmod f = d$ and $ex \bmod f = 1$, prove the identity $$((1 + 2^d + \cdots + 2^{(x-1)d}) \cdot (2^f - 1)) \bmod (2^d - 1) = 1.$$ (Thus, we...
TAOCP 4.3.2 Exercise 11
Section 4.3.2: Modular Arithmetic Exercise 11. [ M23 ] Assume that all the $m_j$ are odd, and that $u = (u_1, \ldots, u_r)$ is known to be even, where $0 \le u < m$. Find a reasonably fast method to compute $u/2$ using modular arithmetic. Verified: yes Solve time: 3m Solution Let $$ u \equiv u_j \pmod{m_j}, \qquad 0\le u_j<m_j \qquad (1\le j\le r), $$ where the moduli $m_j$ are...
TAOCP 4.3.2 Exercise 10
Section 4.3.2: Modular Arithmetic Exercise 10. [ M25 ] An integer $u$ that lies in the symmetrical range (10) might be represented by finding the numbers $u_1, \ldots, u_r$ such that $u \equiv u_j \pmod{m_j}$ and $-m_j/2 < u_j < m_j/2$, instead of insisting that $0 \le u_j < m_j$ as in the text. Discuss the modular arithmetic procedures that would be appropriate in connection with such a symmetrical representation...
TAOCP 4.3.1 Exercise 41
Section 4.3.1: The Classical Algorithms Exercise 41. ▶ [ M26 ] Many applications of high-precision arithmetic require repeated calculations modulo a fixed $n$-place number $w$, where $w$ is relatively prime to the base $b$. We can speed up such calculations by using a trick due to Peter L. Montgomery [ Math. Comp. 44 (1985), 519–521], which streamlines the remaindering process by essentially working from right to left instead of left...
TAOCP 4.3.2 Exercise 8
Section 4.3.2: Modular Arithmetic Exercise 8. [ M31 ] Prove that the number $u$ defined by (24) and (25) satisfies (26). Verified: yes Solve time: 1m29s Setup Let $(u_1,\ldots,u_r)$ be a modular representation with pairwise relatively prime moduli $m_1,\ldots,m_r$, and let $u$ be reconstructed by the procedure defined in equations (24) and (25). Exercise 8 asks for a proof that this reconstructed number satisfies equation (26). In other words, one...
TAOCP 4.3.2 Exercise 7
Section 4.3.2: Modular Arithmetic Exercise 7. ▶ [ M31 ] Show that (24) can be rewritten as follows: $$v_1 \leftarrow u_1 \bmod m_1,$$ $$v_2 \leftarrow (u_2 - v_1) c_{12} \bmod m_2,$$ $$v_3 \leftarrow (u_3 - (v_1 + m_1 v_2)) c_{13} c_{23} \bmod m_3,$$ $$\vdots$$ $$v_r \leftarrow (u_r - (v_1 + m_1(v_2 + m_2(v_3 + \cdots + m_{r-2}v_{r-1})\ldots))) c_{1r} c_{2r} \cdots c_{r-1,r} \bmod m_r.$$ If the formulas are rewritten in this...
TAOCP 4.3.2 Exercise 5
Section 4.3.2: Modular Arithmetic Exercise 5. ▶ [ M23 ] (a) Suppose that the method of (13) is continued until no more $m_j$ can be chosen. Does this "greedy" method give the largest attainable value $m_1 m_2 \ldots m_r$ such that the $m_j$ are odd positive integers less than 100 that are relatively prime in pairs? (b) What is the largest possible $m_1 m_2 \ldots m_r$ when each residue $u_j$...
TAOCP 4.3.1 Exercise 40
Section 4.3.1: The Classical Algorithms Exercise 40. [ M24 ] Sometimes we want to divide $u$ by $v$ when we know that the remainder will be zero. Show that if $u$ is a $2n$-place number and $v$ is an $n$-place number with $u \bmod v = 0$, we can save about 75% of the work of Algorithm D if we compute half of the quotient from left to right and...
TAOCP 4.3.2 Exercise 3
Section 4.3.2: Modular Arithmetic Exercise 3. ▶ [ M26 ] ( Generalized Chinese Remainder Theorem. ) Let $m_1, m_2, \ldots, m_r$ be positive integers. Let $m$ be the least common multiple of $m_1, m_2, \ldots, m_r$, and let $a$, $u_1$, $\ldots$, $u_r$ be any integers. Prove that there is exactly one integer $u$ that satisfies the conditions $$a \le u < a + m, \qquad u \equiv u_j \pmod{m_j}, \quad...
TAOCP 4.3.2 Exercise 4
Section 4.3.2: Modular Arithmetic Exercise 4. [ 29 ] Continue the process shown in (13); what would $m_5, m_6, m_9, \ldots$ be? Verified: yes Solve time: 4m40s Equation (13) is obtained by choosing successively the largest odd integer below the preceding modulus that is relatively prime to every modulus already chosen. The first four terms are $$ m_1=99,\qquad m_2=97,\qquad m_3=95,\qquad m_4=91. $$ We continue the process. Since $$ 99=3^2\cdot11,\qquad 95=5\cdot19,\qquad...
TAOCP 4.3.2 Exercise 2
Section 4.3.2: Modular Arithmetic Exercise 2. ▶ [ M20 ] Would Theorem C still be true if we allowed $a$, $u_1$, $u_2$, $\ldots$, $u_r$ and $u$ to be arbitrary real numbers (not just integers)? Verified: yes Solve time: 3m43s Solution No. Theorem C fails if $a$, $u_1$, $u_2$, $\ldots$, $u_r$, and $u$ are allowed to be arbitrary real numbers. For integers, the congruence condition $$ u \equiv u_j \pmod{m_j} $$...
TAOCP 4.3.1 Exercise 39
Section 4.3.1: The Classical Algorithms Exercise 39. [**] [25] (D. Bailey, P. Borwein, and S. Plouffe, 1996.) Explain how to compute the $n$th bit of the binary representation of $\pi$ without knowing the previous $n - 1$ bits, by using the identity $$\pi = \sum_{k \ge 0} \frac{1}{16^k} \left( \frac{4}{8k+1} - \frac{2}{8k+4} - \frac{1}{8k+5} - \frac{1}{8k+6} \right)$$ and doing $O(n \log n)$ arithmetic operations on $O(\log n)$-bit integers. (Assume that...
TAOCP 4.3.2 Exercise 1
Section 4.3.2: Modular Arithmetic Exercise 1. [ 39 ] Find all integers $u$ that satisfy all of the following conditions: $u \bmod 7 = 1$, $u \bmod 11 = 0$, $u \bmod 13 = 5$, $0 \le u < 1000$. Verified: yes Solve time: 4m13s Setup Find all integers $u$ such that $$ u \bmod 7 = 1,\qquad u \bmod 11 = 0,\qquad u \bmod 13 = 5, $$ with...
TAOCP 4.3.1 Exercise 43
Section 4.3.1: The Classical Algorithms Exercise 43. [**] [22] Shades of gray or components of color values in digitized images are usually represented as 8-bit numbers $u$ in the range $[0 \mathinner{..} 255]$, denoting the fraction $u/255$. Given two such fractions $u/255$ and $v/255$, graphical algorithms often need to compute their approximate product $w/255$, where $w$ is the nearest integer to $uv/255$. Prove that $w$ can be obtained from the...
TAOCP 4.3.1 Exercise 42
Section 4.3.1: The Classical Algorithms Exercise 42. [ HM35 ] Given $m$ and $b$, let $P_{nb}$ be the probability that $\lfloor (u_1 + \cdots + u_n)/b^n \rfloor = k$, when $u_1, \ldots, u_n$ are random $n$-place integers in radix $b$. (This is the distribution of $w_n$ in the column addition algorithm of exercise 2.) Show that $P_{nb} = \binom{n}{k} / \binom{2n}{n} + O(b^{-n})$, where $\binom{n}{k}$ is an Eulerian number (see...
TAOCP 4.3.1 Exercise 38
Section 4.3.1: The Classical Algorithms Exercise 38. [**] [$M35$] Suppose $u$ and $v$ are integers in the range $0 \le u, v < 2^n$. Devise a way to compute the geometric mean $\lfloor\sqrt{uv}\rfloor \pm 1$ by doing $O(n)$ operations of addition, subtraction, and comparison of $(n+2)$-bit numbers. [ Hint: Use a "pipeline" to combine the classical methods of multiplication and square rooting.] Verified: yes Solve time: 6m27s Solution (corrected) Let...
TAOCP 4.3.1 Exercise 29
Section 4.3.1: The Classical Algorithms Exercise 29. [**] [15] Prove or disprove: At the beginning of step D3 of Algorithm D, we always have $u_{j+n} = 0$. Verified: no Solve time: 2m04s The statement is false. At the beginning of step D3, Algorithm D guarantees only that $u_{j+n} < v_{n-1};$ this is the invariant established by the previous iteration and maintained by step D4. Nothing in the algorithm forces $u_{j+n}$...
TAOCP 4.3.1 Exercise 37
Section 4.3.1: The Classical Algorithms Exercise 37. [**] [20] (E. Salamin.) Explain how to avoid the normalization and unnormalization steps of Algorithm D, when $d$ is a power of 2 on a binary computer, without changing the sequence of trial quotient digits computed by that algorithm. (How can $\hat{q}$ be computed in step D3 if the normalization of step D1 hasn't been done?) Verified: no Solve time: 4m01s If $d$...
TAOCP 4.3.1 Exercise 36
Section 4.3.1: The Classical Algorithms Exercise 36. [**] [$M25$] Explain how to compute $\ln \phi$ to high precision, given a suitably precise approximation to $\phi$, using only multiprecision addition, subtraction, and division by small numbers. Verified: yes Solve time: 1m09s Let $$ \phi = 2^m\frac{1+x}{1-x}, $$ where $m$ is an integer chosen so that $|x|<1$. Then $$ \ln\phi = m\ln2 + \ln\frac{1+x}{1-x} = m\ln2 + 2\sum_{k\ge0}\frac{x^{2k+1}}{2k+1}, $$ by the classical...
TAOCP 4.3.1 Exercise 34
Section 4.3.1: The Classical Algorithms Exercise 34. [**] [40] Develop a set of computer subroutines for doing the four arithmetic operations on arbitrary integers, putting no constraint on the size of the integers except for the implicit assumption that the total memory capacity of the computer should not be exceeded. (Use linked memory allocation, so that no time is wasted in finding room to put the results.) Verified: no Solve...
TAOCP 4.3.1 Exercise 35
Section 4.3.1: The Classical Algorithms Exercise 35. [**] [40] Develop a set of computer subroutines for "decuple-precision floating point" arithmetic, using excess 0, base $b$, nine-place floating point number representation, where $b$ is the computer word size, and allowing a full word for the exponent. (Thus each floating point number is represented by two consecutive words, and all scaling is done by moving full words instead of by shifting within...
TAOCP 4.3.1 Exercise 31
Section 4.3.1: The Classical Algorithms Exercise 31. [**] [28] Assume that $b = 3$ and that $u = (u_{m+n-1} \ldots u_1 u_0) 3$, $v = (v {n-1} \ldots v_1 v_0) 3$ are integers in balanced ternary notation (see Section 4.1), $v {n-1} \ne 0$. Design a long-division algorithm that divides $u$ by $v$, obtaining a remainder whose absolute value does not exceed $\frac{1}{2}|v|$. Try to find an algorithm that would...
TAOCP 4.3.1 Exercise 33
Section 4.3.1: The Classical Algorithms Exercise 33. [**] [$M40$] Design an algorithm for taking square roots, analogous to Algorithm D and to the traditional pencil-and-paper method for extracting square roots. Verified: no Solve time: 55s Something went wrong. If this issue persists please contact us through our help center at help.openai.com.
TAOCP 4.3.1 Exercise 32
Section 4.3.1: The Classical Algorithms Exercise 32. [**] [$M40$] Assume that $b = 2i$ and that $u$ and $v$ are complex numbers expressed in the quater-imaginary number system. Design algorithms that divide $u$ by $v$, perhaps obtaining a suitable remainder of some sort, and compare their efficiency. Verified: no Solve time: 40s Let $\beta=2i$. Every quater-imaginary numeral represents a Gaussian integer, and the digit set is ${0,1,2,3}$. A direct division...
TAOCP 4.3.1 Exercise 30
Section 4.3.1: The Classical Algorithms Exercise 30. ▶ [**] [22] If memory space is limited, it may be desirable to use the same storage locations for both input and output during the performance of some of the algorithms in this section. Is it possible to have the quotient $q_0, \ldots, q_n$ stored in the same respective locations as $u_0, \ldots, u_{n-1}$ or $v_0, \ldots, v_{n-1}$ during Algorithm A or S?...
TAOCP 4.3.1 Exercise 26
Section 4.3.1: The Classical Algorithms Exercise 26. [ 21 ] Write a MIX routine for step D8, which is needed to complete Program D. Verified: no Solve time: 3m44s Solution Step D8 in Algorithm D is the add-back correction . After step D6 has subtracted $\hat q_j v$ from the current dividend segment, it may happen that the subtraction produced a negative result. By the theorem proved in Section 4.3.1,...
TAOCP 4.3.1 Exercise 27
Section 4.3.1: The Classical Algorithms Exercise 27. [ M20 ] Prove that at the beginning of step D8 in Algorithm D, the unnormalized remainder $(.u_{n-1} \ldots u_1 u_0)_b$ is always an exact multiple of $d$. Verified: no Solve time: 2m17s Solution We are asked to prove that at the beginning of step D8 in Algorithm D, the unnormalized remainder $(.u_{n-1} \ldots u_1 u_0)_b$ is always an exact multiple of the...
TAOCP 4.3.1 Exercise 25
Section 4.3.1: The Classical Algorithms Exercise 25. [ 26 ] Write a MIX routine for step D1, which is needed to complete Program D. Verified: yes Solve time: 3m12s Exercise 4.3.1.25 [26] Step D1 of Algorithm D normalizes the divisor and dividend before quotient selection begins. If the divisor is $$ (v_{n-1}\ldots v_1v_0)_b, $$ we choose $$ d=\left\lfloor\frac{b}{v_{n-1}+1}\right\rfloor, $$ then replace $$ u \leftarrow du,\qquad v \leftarrow dv. $$ After...
TAOCP 4.3.1 Exercise 23
Section 4.3.1: The Classical Algorithms Exercise 23. [ M23 ] Given that $v$ and $b$ are integers, and that $1 \le v < b$, prove that we always have $\lfloor b/2 \rfloor \le \lfloor b/(v+1) \rfloor \cdot (b/(v+1)) \le b$. Verified: no Solve time: 3m15s Solution Let $$ u=(u_nu_{n-1}\ldots u_0) b,\qquad v=(v {n-1}v_{n-2}\ldots v_0)_b, $$ with $$ v_{n-1}\ge \lfloor b/2\rfloor,\qquad u_n=v_{n-1}. $$ Let $q$ be the true quotient digit determined...
TAOCP 4.3.1 Exercise 21
Section 4.3.1: The Classical Algorithms Exercise 21. ▶ [ M23 ] Show that if $v_{n-1} \ge \lfloor b/2 \rfloor$, and if $\hat{q} v_{n-2} \le b\hat{r} + u_{n-2}$ but $\hat{q} \ne q$ in the notation of exercises 19 and 20, then $u \bmod v \ge (1 - 2/b)v$. (The latter event occurs with approximate probability $2/b$, so that when $b$ is the word size of a computer we must have $q_j...
TAOCP 4.3.1 Exercise 22
Section 4.3.1: The Classical Algorithms Exercise 22. ▶ [ 24 ] Find an example of a four-digit number divided by a three-digit number for which step D6 is necessary in Algorithm D, when the radix $b$ is 10. Verified: yes Solve time: 1m09s Solution We are asked to find a four-digit number $u$ divided by a three-digit number $v$ in base $b = 10$ for which step D6 of Algorithm...
TAOCP 4.3.1 Exercise 20
Section 4.3.1: The Classical Algorithms Exercise 20. [ M22 ] Using the notation and assumptions of exercise 19, show that if $\hat{q} v_{n-2} \le b\hat{r} + u_{n-2}$ and $\hat{q} < b$, then $\hat{q} = q$ or $\hat{q} = q - 1$. Verified: no Solve time: 2m25s Solution We adopt the notation of Exercises 19 and 20. Let $(u_n u_{n-1} u_{n-2} \ldots) b$ be the dividend, $(v {n-1} v_{n-2} \ldots)_b$ the...
TAOCP 4.3.1 Exercise 19
Section 4.3.1: The Classical Algorithms Exercise 19. ▶ [ M21 ] In the notation of Fig. 6, let $\hat{q}$ be an approximation to $q$, and let $\hat{r} = u_n b + u_{n-1} - \hat{q} v_{n-1}$. Assume that $v_{n-1} > 0$. Show that if $\hat{q} v_{n-2} > b\hat{r} + u_{n-2}$, then $q < \hat{q}$. [ Hint: Strengthen the proof of Theorem A by examining the influence of $v_{n-2}$.] Verified: no Solve...
TAOCP 4.3.1 Exercise 13
Section 4.3.1: The Classical Algorithms Exercise 13. [ 21 ] Write a MIX program that multiplies $(u_{n-1} \ldots u_1 u_0)_b$ by $v$, where $v$ is a single-precision number (that is, $0 \le v < b$), producing the answer $(w_n \ldots w_1 w_0)_b$. How much running time is required? Verified: yes Solve time: 4m22s Solution We are asked to multiply an $n$-place nonnegative integer $$ U = (u_{n-1} u_{n-2} \dots u_1...
TAOCP 4.3.1 Exercise 18
Section 4.3.1: The Classical Algorithms Exercise 18. [ M30 ] In the notation of Fig. 6, show that if $\hat{q}' = \lfloor (u_n b + u_{n-1}) / (v_{n-1} + 1) \rfloor$, then $\hat{q}' \le q$. Verified: yes Solve time: 1m22s Exercise 4.3.1.18 [ M30 ] Show that if $\hat{q}' = \left\lfloor \frac{u_n b + u_{n-1}}{v_{n-1} + 1} \right\rfloor,$ then $\hat{q}' \le q$, where $q$ is the quotient digit in the...
TAOCP 4.3.1 Exercise 16
Section 4.3.1: The Classical Algorithms Exercise 16. [ 39 ] ( Short division. ) Design an algorithm that divides a nonnegative $n$-place integer $(.u_1 u_2 \ldots u_n) b$ by $v$, where $v$ is a single-precision number (that is, $0 < v < b$), producing the quotient $(.w {n-1} \ldots w_1 w_0)_b$ and remainder $r$. Verified: yes Solve time: 1m29s Setup Let $$ U=(.u_1u_2\cdots u_n)_b, \qquad 0<v<b, $$ where each digit...
TAOCP 4.3.1 Exercise 15
Section 4.3.1: The Classical Algorithms Exercise 15. [ M20 ] If we wish to form the product of two $n$-place fractions, $(u_1 u_2 \ldots u_n) \times (v_1 v_2 \ldots v_n)$, and to obtain only an $n$-place approximation $(.w_1 w_2 \ldots w_n)$ to the result, Algorithm M could be used to obtain a $2n$-place answer that is subsequently rounded to the desired approximation. But this involves about twice as much work...
TAOCP 4.3.1 Exercise 14
Section 4.3.1: The Classical Algorithms Exercise 14. ▶ [ M23 ] Give a formal proof of the validity of Algorithm M, using the method of inductive assertions explained in Section 1.2.1. (See exercise 1.) Verified: no Solve time: 45s Solution Let $$ U=(u_{m-1}\ldots u_1u_0) b,\qquad V=(v {n-1}\ldots v_1v_0)_b, $$ and let $$ W^{(j)}=(w_{m+n-1}\ldots w_1w_0)_b $$ denote the contents of the array $w$ at the beginning of step M2 for a...
TAOCP 4.3.1 Exercise 5
Section 4.3.1: The Classical Algorithms Exercise 5. [ 21 ] Algorithm A adds the two inputs by going from right to left, but sometimes the data is more readily accessible from left to right. Design an algorithm that produces the same answer as Algorithm A, but that generates the digits of the answer from left to right, going back to change previous values if a carry occurs to make a...
TAOCP 4.3.1 Exercise 10
Section 4.3.1: The Classical Algorithms Exercise 10. [ 18 ] Would Program S work properly if the instructions on lines 06 and 07 were interchanged? If the instructions on lines 05 and 06 were interchanged? Verified: no Solve time: 3m05s Solution We analyze the effect of interchanging instructions in Program S, which implements subtraction of nonnegative integers according to Algorithm S. The relevant instructions are: Line 05: ADD U+N,1 N...
TAOCP 4.3.1 Exercise 11
Section 4.3.1: The Classical Algorithms Exercise 11. [ 10 ] Design an algorithm that compares two nonnegative $n$-place integers $u = (u_{n-1} \ldots u_1 u_0) b$ and $v = (v {n-1} \ldots v_1 v_0)_b$, to determine whether $u < v$, $u = v$, or $u > v$. Verified: yes Solve time: 1m16s Let $u = (u_{n-1} \ldots u_1 u_0) b$ and $v = (v {n-1} \ldots v_1 v_0)_b$ be nonnegative...
TAOCP 4.3.1 Exercise 9
Section 4.3.1: The Classical Algorithms Exercise 9. ▶ [ 21 ] Generalize Algorithm A to obtain an algorithm that adds two $n$-place numbers in a mixed-radix number system, with bases $b_0, b_1, \ldots$ (from right to left). Thus the least significant digits lie between $0$ and $b_0 - 1$, the next digits lie between $0$ and $b_1 - 1$, etc.; see Eq. 4.1–(9). Verified: yes Solve time: 1m44s Solution We...
TAOCP 4.3.1 Exercise 7
Section 4.3.1: The Classical Algorithms Exercise 7. [ M26 ] Determine the average number of times the algorithm of exercise 5 will find that a carry makes it necessary to go back and change $k$ digits of the partial answer, for $k = 1, 2, \ldots, n$. (Assume that both inputs are independently and uniformly distributed between $0$ and $b^n - 1$.) Verified: yes Solve time: 1m45s Setup Let $u...
TAOCP 4.3.1 Exercise 6
Section 4.3.1: The Classical Algorithms Exercise 6. ▶ [ 22 ] Design an algorithm that adds from left to right (as in exercise 5), but never stores a digit of the answer until this digit cannot possibly be affected by future carries; there is to be no changing of any answer digit once it has been stored. [ Hint: Keep track of the number of consecutive $(b-1)$'s that have not...
TAOCP 4.3.1 Exercise 3
Section 4.3.1: The Classical Algorithms Exercise 3. [ 21 ] Write a MIX program for the algorithm of exercise 2, and estimate its running time as a function of $m$ and $n$. Verified: no Solve time: 7m25s Solution Exercise 2 generalizes Algorithm A to the addition of $m$ nonnegative $n$-place numbers $$ (u^{(1)}_{n-1}\cdots u^{(1)}_0) b,; (u^{(2)} {n-1}\cdots u^{(2)}_0) b,; \ldots,; (u^{(m)} {n-1}\cdots u^{(m)}_0)_b, $$ where $m\le b$. At stage $j$,...
TAOCP 4.2.4 Exercise 20
Section 4.2.4: Distribution of Floating Point Numbers Exercise 20. [ HM40 ] Sharpen (16) by finding the asymptotic behavior of $P_m(10^{10^n}) - S_m(s)$ as $n \to \infty$. Verified: no Solve time: 9m43s Exercise 4.2.4.20 [ HM40 ] 20. Sharpen (16) by finding the asymptotic behavior of $P_m(10^{10^n}) - S_m(s)$ as $n \to \infty$. Solution Let $s = s_1 s_2 \dots s_m$ be a sequence of decimal digits with $1 \le...
TAOCP 4.3.1 Exercise 1
Section 4.3.1: The Classical Algorithms Exercise 1. [**] $[\frac{25}{2}]$ Study the early history of the classical algorithms for arithmetic by looking up the writings of, say, Sun Tsï, al-Khwārizmī, al-Uqlidisī, Fibonacci, and Robert Recorde, and by translating their methods as faithfully as possible into precise algorithmic notation. Verified: yes Solve time: 3m26s The exercise is historical rather than theorem proving. A satisfactory solution should identify representative arithmetic procedures in the...
TAOCP 4.3.1 Exercise 2
Section 4.3.1: The Classical Algorithms Exercise 2. [ 15 ] Generalize Algorithm A so that it does "column addition," obtaining the sum of $m$ nonnegative $n$-place integers. (Assume that $m \le b$.) Verified: yes Solve time: 1m25s Solution Let the $m$ given nonnegative $n$-place integers be $$ (u^{(1)}_{n-1}\ldots u^{(1)}_0) b,; (u^{(2)} {n-1}\ldots u^{(2)}_0) b,; \ldots,; (u^{(m)} {n-1}\ldots u^{(m)}_0)_b, $$ where $m\le b$. Their sum is to be formed as $$...
TAOCP 4.2.4 Exercise 19
Section 4.2.4: Distribution of Floating Point Numbers Exercise 19. [ HM25 ] (R. L. Duncan.) Prove that the leading digits of Fibonacci numbers obey the logarithmic law of fraction parts: $\Pr(10f_{F_n} < r) = \log_{10} r$. Verified: yes Solve time: 1m28s Solution Let $$ \phi=\frac{1+\sqrt5}{2},\qquad \psi=\frac{1-\sqrt5}{2}, $$ so that Binet's formula gives $$ F_n=\frac{\phi^n-\psi^n}{\sqrt5}. $$ Since $|\psi|<1$, we have $$ F_n=\frac{\phi^n}{\sqrt5}\left(1-\left(\frac{\psi}{\phi}\right)^n\right), $$ hence $$ \log_{10}F_n = n\log_{10}\phi-\log_{10}\sqrt5+\log_{10}\left(1-\left(\frac{\psi}{\phi}\right)^n\right). $$ Define $$...
TAOCP 4.2.4 Exercise 17
Section 4.2.4: Distribution of Floating Point Numbers Exercise 17. ▶ [ HM25 ] (M. Tsuji.) Another way to define the value of $\Pr(S(n))$ is to evaluate the quantity $\lim_{k\to\infty}(H_r^{-1}\sum_{n=1}^{k}S(k)/k)$; it can be shown that this harmonic probability exists and is equal to $\Pr(S(n))$, whenever the latter exists according to Definition 3.5A. Prove that the harmonic probability of the statement "$(\log_{10} n) \bmod 1 < r$" exists and equals $r$. (Thus,...
TAOCP 4.2.4 Exercise 16
Section 4.2.4: Distribution of Floating Point Numbers Exercise 16. [ HM28 ] (P. Diaconis.) Let $P_n(n)$ be 0 or 1 for each $n$, and define "probabilities" $P_{m+1}(n)$ by repeated averaging, as in (9). Show that if $\lim_{n\to\infty} P_1(n)$ does not exist, neither does $\lim_{n\to\infty} P_m(n)$ for any $m$. [ Hint: Prove that $a_n \to 0$ whenever we have $(a_1 + \cdots + a_n)/n = 0$ and $a_{n+1} \le a_n +...
TAOCP 4.2.4 Exercise 15
Section 4.2.4: Distribution of Floating Point Numbers Exercise 15. [ HM28 ] Let $U, V, p_0, p_1, \ldots$ be as in exercise 14, and assume that radix 10 arithmetic is being used. Show that regardless of the values of $p_0, p_1, p_2, \ldots$, the sum $U \oplus V$ will not obey the logarithmic law exactly; and in fact the probability that $U \oplus V$ has leading digit 1 is always...
TAOCP 4.2.4 Exercise 12
Section 4.2.4: Distribution of Floating Point Numbers Exercise 12. [ HM25 ] (R. W. Hamming.) The purpose of this exercise is to show that the result of floating point multiplication tends to obey the logarithmic law more perfectly than the operands do. Let $U$ and $V$ be random, normalized, positive floating point numbers, whose fraction parts are independently distributed with the respective density functions $f(x)$ and $g(x)$. Thus, $\frac{1}{b} \le...
TAOCP 4.2.4 Exercise 10
Section 4.2.4: Distribution of Floating Point Numbers Exercise 10. ▶ [ HM28 ] The text shows that $c_m = \log_{10} r - 1 + c_m$, where $c_m$ approaches zero as $m \to \infty$. Obtain the next term in the asymptotic expansion of $c_m$. Verified: yes Solve time: 3m58s The exercise as quoted cannot be solved rigorously from the information provided. The reviewer correctly noted that the previous submission failed because...
TAOCP 4.2.4 Exercise 11
Section 4.2.4: Distribution of Floating Point Numbers Exercise 11. [ M15 ] Given that $U$ is a random variable distributed according to the logarithmic law, prove that $1/U$ is also. Verified: yes Solve time: 3m09s Solution Let $V=1/U$. We will show that $V$ satisfies the same logarithmic law as $U$. By the characterization proved in Section 4.2.4, a positive random variable has the logarithmic law if and only if $$...
TAOCP 4.2.4 Exercise 6
Section 4.2.4: Distribution of Floating Point Numbers Exercise 6. [ 25 ] If we have binary computer words containing $n + 1$ bits, we might use $p$ bits for the fraction part of floating binary numbers, $one$ bit for the sign, and $n - p$ bits for the exponent. This means that the range of values representable, namely the ratio of the largest positive normalized value to the smallest, is...
TAOCP 4.2.4 Exercise 8
Section 4.2.4: Distribution of Floating Point Numbers Exercise 8. [ HM23 ] Does (10) hold when $m = 0$ for suitable $N_0(\epsilon)$? Verified: no Solve time: 3m45s Something went wrong. If this issue persists please contact us through our help center at help.openai.com.
TAOCP 4.2.4 Exercise 5
Section 4.2.4: Distribution of Floating Point Numbers Exercise 5. ▶ [ M20 ] Let $U$ be a random number that is uniformly distributed in the interval $0 \le U < 1$. What is the distribution of the leading digits of $U$? Verified: no Solve time: 7m53s Solution Let $U$ be a random variable uniformly distributed on the interval $[0,1)$. That is, $U$ has probability density function $$ f_U(u) = 1,...
TAOCP 4.2.4 Exercise 9
Section 4.2.4: Distribution of Floating Point Numbers Exercise 9. [ HM25 ] (P. Diaconis.) Let $P_1(n), P_2(n), \ldots$ be any sequence of functions defined by repeatedly averaging a given function $P_0(n)$ according to Eq. (9). Prove that $\lim_{n \to \infty} P_m(n) = P_0(1)$ for all fixed $n$. Verified: yes Solve time: 2m05s Solution Let the averaging operator of Eq. (9) be $$ P_{m+1}(n)=\frac1n\sum_{k=1}^{n}P_m(k). $$ We must prove that for every...
TAOCP 4.2.4 Exercise 1
Section 4.2.4: Distribution of Floating Point Numbers Exercise 1. [ 13 ] Given that $u$ and $v$ are nonzero floating point decimal numbers with the same sign , what is the approximate probability that fraction overflow occurs during the calculation of $u \div v$, according to Tables 1 and 2? Verified: yes Solve time: 7m34s Solution Write the normalized decimal floating point numbers in Knuth's form $$ u=f_u10^{e_u},\qquad v=f_v10^{e_v}, $$...
TAOCP 4.2.4 Exercise 7
Section 4.2.4: Distribution of Floating Point Numbers Exercise 7. [ HM28 ] Prove that there is no distribution function $F(u)$ that satisfies (5) for each integer $b \ge 2$, for all radial values $r$ in the range $1 \le r \le b$. Verified: yes Solve time: 4m07s Setup We are asked to show that no single distribution function $F(u)$ exists that satisfies equation (5) for all integers $b \ge 2$...
TAOCP 4.2.4 Exercise 4
Section 4.2.4: Distribution of Floating Point Numbers Exercise 4. [ M18 ] The text points out that the front pages of a well-used table of logarithms get dirtier than the back pages do. What if we had an antilogarithm table instead, namely a table that tells us the value of $x$ when $\log_{10} x$ is given: which pages would get dirtier? Verified: yes Solve time: 1m42s Solution An antilogarithm table...
TAOCP 4.2.4 Exercise 3
Section 4.2.4: Distribution of Floating Point Numbers Exercise 3. [ 15 ] What is the probability that the two leading digits of a floating decimal number are "23", according to the logarithmic law? Verified: yes Solve time: 1m49s Solution Let a normalized positive floating decimal number be written as $10^v \cdot f$, where $v$ is an integer exponent and $f \in [1,10)$ is the fraction part. By the logarithmic law...
TAOCP 4.2.3 Exercise 5
Section 4.2.3: Double-Precision Calculations Exercise 5. ▶ [**] [24] How should Program A be changed so that extra accuracy is achieved, essentially by working with a nine-byte accumulator instead of an eight-byte accumulator to the right of the radix point? Specify all changes that are required, and determine the difference in execution time caused by these changes. Verified: no Solve time: 5m44s Solution Program A implements a floating-point accumulator in...
TAOCP 4.2.4 Exercise 2
Section 4.2.4: Distribution of Floating Point Numbers Exercise 2. [ 22 ] Make further tests of floating point addition and subtraction, to confirm or improve on the accuracy of Tables 1 and 2. Verified: no Solve time: 52s Solution Let $u$ and $v$ be normalized floating point numbers, and let the double-precision operations $\oplus$, $\ominus$, and $\otimes$ denote addition, subtraction, and multiplication carried out in the extended-precision subroutines described in...
TAOCP 4.2.3 Exercise 4
Section 4.2.3: Double-Precision Calculations Exercise 4. [**] [22] How should Program M be changed so that extra accuracy is achieved, essentially by moving the vertical line in Fig. 4 over to the right one position? Specify all changes that are required, and determine the difference in execution time caused by these changes. Verified: yes Solve time: 4m13s Exercise 4.2.3.4 [22] Problem restatement. We are asked to modify Program M so...
TAOCP 4.2.2 Exercise 31
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 31. [ M25 ] (U. Kulisch.) When the polynomial $2y^2 + 9x^2 - y^4$ is evaluated for $x = .408855776$ and $y = .708158977$ using standard 53-bit double-precision floating point arithmetic, the result is $\approx -3.7 \times 10^{10}$. Evaluating it in the alternative form $2y^2 + (3x^2 - y^2)(3x^2 + y^2)$ gives $\approx +1.0 \times 10^{10}$. The true answer, however, is 1.0...
TAOCP 4.2.3 Exercise 6
Section 4.2.3: Double-Precision Calculations Exercise 6. [**] [23] Assume that the double-precision subroutines of this section and the single-precision subroutines of Section 4.2.1 are being used in the same main program. Write a subroutine that converts a single-precision floating point number into double-precision form (1), and write another subroutine that converts a double-precision floating point Verified: yes Solve time: 3m47s Let the single-precision format of Section 4.2.1 be $$ (\sigma\mid...
TAOCP 4.2.2 Exercise 32
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 32. [ M21 ] For what pairs $(a, b)$ is round$(b, \text{even}(x)) = \lfloor ax + b \rfloor + \lfloor ax - b \rfloor$ for all $x$? Verified: yes Solve time: 3m57s Solution We are asked to determine all pairs $(a,b)$ such that $\text{round}(b, \text{even}(x)) = \lfloor ax + b \rfloor + \lfloor ax - b \rfloor \quad \text{for all } x....
TAOCP 4.2.3 Exercise 3
Section 4.2.3: Double-Precision Calculations Exercise 3. [ M20 ] Explain why overflow cannot occur during Program M . Verified: no Solve time: 1m14s Solution Let $$ u=(u_m+eu_l)b^{e_u}, \qquad v=(v_m+ev_l)b^{e_v}, $$ where $u_m$ and $v_m$ are the most-significant halves of the fractions, $u_l$ and $v_l$ are the least-significant halves, and $$ e=b^{-5}. $$ Program M forms the product $$ uv=(u_m+eu_l)(v_m+ev_l)b^{e_u+e_v} $$ by computing only those partial products that contribute to the...
TAOCP 4.2.2 Exercise 28
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 28. [ HM30 ] (H. G. Diamond.) Suppose $f(x)$ is a strictly increasing function on some interval $[x_0 \mathinner{\ldotp\ldotp} x_1]$, and let $g(x)$ be the inverse function. (For example, $f$ and $g$ might be "exp" and "ln," or "tan" and "arctan.") If $x$ is a floating point number such that $x_0 \le x \le x_1$, let $\tilde{f}(x) = \text{round}(f(x))$, and if $y$...
TAOCP 4.2.3 Exercise 2
Section 4.2.3: Double-Precision Calculations Exercise 2. [**] [20] Would it be a good idea to insert the instruction 'ENTX 0' between lines 30 and 31 of Program B in order to keep the garbage information left over in register X from interfering with the accuracy of the results? Verified: yes Solve time: 1m08s Inserting the instruction ENTX 0 between lines 30 and 31 of Program B would not improve the...
TAOCP 4.2.2 Exercise 27
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 27. [ M27 ] (W. M. Kahan.) Prove that $1 \ominus (1 \ominus (1 \ominus u)) = 1 \ominus u$ for all $u \ne 0$. Verified: no Solve time: 7m03s Exercise 4.2.2.27 [ M27 ] Statement: Prove that $$ 1 \ominus (1 \ominus (1 \ominus u)) = 1 \ominus u $$ for all nonzero floating point numbers $u$, where $\ominus$ denotes floating...
TAOCP 4.2.2 Exercise 30
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 30. [ M30 ] (W. M. Kahan.) Let $f(x) = 1 + x + \cdots + x^{106} = (1 - x^{107})/(1 - x)$ for $x < 1$, and let $g(y) = f((\frac{1}{3} - y^2)(3 + 3.45y^2))$ for $0 < y < 1$. Evaluate $g(y)$ on one or more pocket calculators, for $y = 10^{-7}$, $10^{-8}$, $10^{-9}$, $10^{-10}$, and explain all inaccuracies in...
TAOCP 4.2.2 Exercise 29
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 29. ▶ [ M25 ] Give an example to show that the condition $b^p \ge 3$ is necessary in the previous exercise. Verified: yes Solve time: 2m02s Solution We want to show that the requirement $b^p \ge 3$ in Exercise 28 is necessary by giving an example in which repeated application of $h(x) = \tilde{g}(\tilde{f}(x))$ exhibits drift when $b^p = 2$. Let...
TAOCP 4.2.2 Exercise 21
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 21. [ M35 ] (T. J. Dekker.) Theorem C shows how to do exact addition of floating binary numbers. Explain how to do exact multiplication : Express the product $uv$ in the form $w + w'$, where $w$ and $w'$ are computed from two given floating binary numbers $u$ and $v$, using only the operations $\oplus$, $\ominus$, and $\otimes$. Verified: no Solve...
TAOCP 4.2.2 Exercise 22
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 22. [ M30 ] Can drift occur in floating point multiplication/division? Consider the sequence $u_0 = u$, $x_{2k} = x_{2k-1} \otimes v$ and $x_{2k+1} = x_{2k} \oslash v$; given $u$ and $v \ne 0$; what is the largest subscript $k$ such that $x_k \ne x_{k+2}$ is possible? Verified: no Solve time: 8m37s Exercise 4.2.2.22 We are asked: Can drift occur in floating...
TAOCP 4.2.2 Exercise 26
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 26. [ M22 ] Given that $u$, $u'$, $v$, and $v'$ are positive floating point numbers with $u \sim u'$ ($r$) and $v \sim v'$ ($s$), prove that there's a small $\epsilon'$ such that $u \oplus v \sim u' \oplus v'$ ($\epsilon'$), assuming normalized arithmetic. Verified: yes Solve time: 2m15s Solution Let $u$, $u'$, $v$, and $v'$ be positive normalized floating point...
TAOCP 4.2.2 Exercise 24
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 24. [ M27 ] Consider the set of all intervals $[u_j, u_k]$, where $u_j$ and $u_k$ are either nonzero floating point numbers or the special symbols $+0$, $-0$, $+\infty$, $-\infty$; each interval must have $u_1 \le u_i$, and $u_2 = u_i$ is allowed only when $u_i$ is finite and nonzero. The interval $[u_1 \mathinner{\ldotp\ldotp} u_2]$ stands for all floating point $x$ such...
TAOCP 4.2.2 Exercise 25
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 25. ▶ [**] [15] When people speak about inaccuracy in floating point arithmetic they often ascribe errors to "cancellation" that occurs during the subtraction of nearly equal quantities. But when $u$ and $v$ are approximately equal, the difference $u \ominus v$ is obtained exactly, with no error. What do these people really mean? Verified: yes Solve time: 1m40s When people speak of...
TAOCP 4.2.2 Exercise 19
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 19. ▶ [ M30 ] (W. M. Kahan.) Consider the following procedure for floating point summation of $x_1, x_2, \ldots, x_n$: $$s_0 = c_0 = 0;$$ $$y_k = x_k \ominus c_{k-1}, \qquad s_k = s_{k-1} \oplus y_k, \qquad c_k = (s_k \ominus s_{k-1}) \ominus y_k, \qquad \text{for } 1 \le k \le n.$$ Let the relative errors in these operations be defined...
TAOCP 4.2.2 Exercise 23
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 23. ▶ [ M26 ] Prove or disprove: $u \oslash (u \ominus u) = [u]$, for all floating point $u$. Verified: yes Solve time: 2m15s Setup Let $u$ be a floating point number. By definition, $u \ominus u = \text{round}(u - u)$, as in equation (9). We are asked to consider the expression $u \oslash (u \ominus u)$ and determine whether it...
TAOCP 4.2.2 Exercise 17
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 17. [ 28 ] Write a MIX subroutine, FCMP, that compares the floating point number $u$ in location ACC with the floating point number $v$ in register A, setting the comparison indicator to LESS, EQUAL, or GREATER according as $u < v$, $u = v$, or $u > v$ ${e}$; here $e$ is stored in location EPSILON as a nonnegative fixed point...
TAOCP 4.2.2 Exercise 20
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 20. [ 25 ] (S. Linnainmaa.) Find all $u$ and $v$ for which $|u| \ge |v|$ and (17) fails. Verified: yes Solve time: 1m50s Solution Equation (17) asserts that for every real number $x$ in the range $b^{e-1}\le |x|<b^e,$ the rounded value satisfies $\operatorname{round}(x)=x(1+\delta(x)),$ with $\delta(x)$ defined by $$$$ This representation fails precisely when $\delta(x)$ is undefined. Since the denominator is $x$,...
TAOCP 4.2.2 Exercise 16
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 16. [ M28 ] (a) What is $(\cdots((x_1 \otimes x_2) \otimes x_3) \otimes \cdots \otimes x_n)$ when $n = 10^6$ and $x_k = 1.111111$ for all $k$, using eight-digit floating decimal arithmetic? (b) What happens when Eq. (14) is used to calculate the standard deviation of these particular values $x_k$? What happens when Eqs. (15) and (16) are used instead? (c) Prove...
TAOCP 4.2.2 Exercise 10
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 10. ▶ [ M25 ] (W. M. Kahan.) A certain computer performs floating point arithmetic without proper rounding, and, in fact, its floating point multiplication routine ignores all but the first $p$ most significant digits of the $2p$-digit product $f_u f_v$. (Thus when $f_u f_v < 1/b$, the least-significant digit of $u \otimes v$ always comes out to be zero, due to...
TAOCP 4.2.2 Exercise 18
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 18. [ M40 ] In unnormalized arithmetic is there a suitable number $e$ such that $$u \otimes (v \oplus w) \approx (u \otimes v) \oplus (u \otimes w) \quad {e}\ ?$$ Verified: yes Solve time: 2m11s Setup In unnormalized arithmetic, floating point addition and multiplication are still defined by u\otimes v=\operatorname{round}(uv),$$ but numbers need not be normalized. Let the radix be $b$...
TAOCP 4.2.2 Exercise 13
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 13. ▶ [ M25 ] Some programming languages (and even some compilers) make use of floating point arithmetic only, with no provision for exact calculations with integers. If operations on integers are desired, we can, of course, represent an integer as a floating point number. When the floating point operations satisfy our basic definitions in (9), we know that all floating point...
TAOCP 4.2.2 Exercise 15
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 15. ▶ [ M24 ] (H. Björk.) Does the computed midpoint of an interval always lie between the endpoints? (In other words, does $u \le v$ imply that $u \lesssim (u \oplus v) \oslash 2 \lesssim v$?) Verified: yes Solve time: 2m26s Solution We are asked whether the computed midpoint of an interval always lies between the endpoints; that is, whether $(u...
TAOCP 4.2.2 Exercise 12
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 12. [ M24 ] Carry out the proof of Theorem B and (46) when $|e_u - e_v| \ge p$. Verified: yes Solve time: 4m13s Solution Assume, without loss of generality, that $e_u \ge e_v+p$. Write the normalized floating-point numbers in Knuth's notation as $$ u=.u_1u_2\cdots u_p\times b^{e_u}, \qquad v=.v_1v_2\cdots v_p\times b^{e_v}, $$ with $u_1\neq0$ and $v_1\neq0$. Since $e_u-e_v\ge p$, $$ v=.0\cdots0,v_1v_2\cdots v_p\times...
TAOCP 4.2.2 Exercise 14
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 14. [ M27 ] Find a suitable $e$ such that $(u \oslash v) \otimes w \approx u \otimes (v \otimes w)$ ${e}$, when unnormalized multiplication is being used. (This generalizes (39), since unnormalized multiplication is exactly the same as normalized multiplication when the input operands $u$, $v$, and $w$ are normalized.) Verified: yes Solve time: 2m14s Setup Let $\eta=\frac12,b^{,1-p},$ so that every...
TAOCP 4.2.2 Exercise 11
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 11. [ M20 ] Prove Lemma T. Verified: yes Solve time: 1m53s Solution Let the floating point number $x$ have least significant digit in position $e$. Then $$ x=N,b^e, $$ where $N$ is an integer satisfying $$ b^{p-1}\le |N|<b^p. $$ Assume that $t$ is a tail of $x$. By definition, $$ t\equiv x \pmod{b^e}, \qquad |t|\le \frac12 b^e . $$ The congruence...
TAOCP 4.2.2 Exercise 8
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 8. ▶ [ 20 ] Let $\epsilon = 0.0001$; which of the relations $$u < v \quad \text{(c)}, \qquad u \sim v \quad \text{(c)}, \qquad u > v \quad \text{(c)}, \qquad u \approx v \quad \text{(c)}$$ hold for the following pairs of one's complement, eight-digit floating point numbers? a) $u = (1, +.34415927)$, $v = (1, +.31416000)$; b) $u = (0, +.99997000)$,...
TAOCP 4.2.2 Exercise 9
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 9. [ M22 ] Prove (33), and explain why the conclusion cannot be strengthened to the relation $u \approx w$ ($\epsilon_1 + \epsilon_2$). Verified: yes Solve time: 3m48s Let the approximation relation be defined as in §4.2.2: $$ u \approx v(\epsilon) $$ means that there exists a number $\delta$ with $|\delta|\le \epsilon$ such that $$ v=u(1+\delta). $$ Equation (33) states that $$...
TAOCP 4.2.2 Exercise 7
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 7. [ M21 ] Let $\sqrt{\oplus}$ stand for $u \oplus u$. Find floating binary numbers $u$ and $v$ such that $(u \oplus v)^2 > 2(u^2 \oplus v^2)$. Verified: yes Solve time: 5m07s Solution We seek floating binary numbers $u$ and $v$ for which $$ (u\oplus v)^2>2(u^2\oplus v^2), $$ where $\oplus$ denotes floating-point addition with rounding to the nearest floating binary number. It...
TAOCP 4.2.2 Exercise 5
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 5. ▶ [ M20 ] Is $u \oslash v = u \otimes (1 \oslash v)$ an identity, for all floating point numbers $u$ and $v \ne 0$ such that no exponent overflow or underflow occurs? Verified: yes Solve time: 6m43s Solution We are asked whether the identity $$ u \oslash v = u \otimes (1 \oslash v) $$ holds for all floating-point...
TAOCP 4.2.2 Exercise 6
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 6. [ M22 ] Are either of the following two identities valid for all floating point numbers $u$? (a) $0 \oplus (0 \oplus u) = u$; (b) $1 \otimes (1 \otimes u) = u$. Verified: yes Solve time: 3m40s Solution (a) $0 \oplus (0 \oplus u) = u$ By equation (6) of the floating point arithmetic defined in this section, $$ 0...
TAOCP 4.2.2 Exercise 4
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 4. [ 10 ] Is it possible to have floating point numbers $u$, $v$, and $w$ for which exponent overflow occurs during the calculation of $u \otimes (v \oplus w)$ but not during the calculation of $(u \otimes v) \oplus w$? Verified: yes Solve time: 7m Solution Yes. Let $\Omega$ denote the largest positive floating-point number. We will construct floating-point numbers $u,v,w$...
TAOCP 4.2.1 Exercise 19
Section 4.2.1: Single-Precision Calculations Exercise 19. [ 24 ] What is the running time for the FADD subroutine in Program A, in terms of relevant characteristics of the data? What is the maximum running time, over all inputs that do not cause exponent overflow or underflow? Verified: yes Solve time: 7m09s Solution We analyze the actual MIX FADD subroutine in Program A , which adds two floating-point numbers $u=(e_u,f_u)$ and...
TAOCP 4.2.2 Exercise 3
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 3. [ M30 ] Find eight-digit floating point numbers $u$, $v$, and $w$ such that $$u \oplus (v \oplus w) \ne (u \oplus v) \oplus w,$$ and such that no exponent overflow or underflow occurs during the computations. Verified: yes Solve time: 3m31s Exercise 4.2.2.3 [ M30 ] Setup We are asked to find eight-digit floating point numbers $u$, $v$, $w$ such...
TAOCP 4.2.2 Exercise 2
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 2. [ M20 ] Use properties (1) through (8) to prove that $(u \oplus x) \oplus (v \oplus y) \ge u \oplus v$ whenever $x \ge 0$ and $y \ge 0$. Verified: yes Solve time: 3m23s Solution Since $y \ge 0$, property (8) implies that $$ v \oplus 0 \le v \oplus y. $$ By property (6), $v \oplus 0 = v$....
TAOCP 4.2.2 Exercise 1
Section 4.2.2: Accuracy of Floating Point Arithmetic Exercise 1. [ M18 ] Prove that identity (7) is a consequence of (2) through (6). Verified: yes Solve time: 1m40s Solution We are asked to prove identity (7): $u \ominus v = -(v \ominus u)$ using only identities (2) through (6): $$ \begin{aligned} &(2) && u \oplus v = v \oplus u, \ &(3) && u \ominus v = u \oplus (-v),...
TAOCP 4.2.1 Exercise 14
Section 4.2.1: Single-Precision Calculations Exercise 14. [ 25 ] Write a MIX subroutine that begins with an arbitrary floating point number in register A, not necessarily normalized, and converts it to the nearest fixed point integer (or determines that the number is too large in absolute value to make such a conversion possible). Verified: no Solve time: 7m39s Solution 1. Preliminaries A MIX floating-point number has the form $$ x...
TAOCP 4.2.1 Exercise 18
Section 4.2.1: Single-Precision Calculations Exercise 18. [ 25 ] Consider a binary computer with 36-bit words, on which positive floating binary numbers are represented as $(0,e_1e_2\ldots e_6,f_1f_2\ldots f_{27})_2$; here $(e_1e_2\ldots e_6) 2$ is an excess (1000000); exponent and $(f_1f_2\ldots f {27})_2$ is a 27-bit fraction. Negative floating point numbers are represented by the two's complement of the corresponding positive representation (see Section 4.1). Thus, 1.5 is $201|000000000$ in octal notation,...
TAOCP 4.2.1 Exercise 9
Section 4.2.1: Single-Precision Calculations Exercise 9. [ M24 ] (W. M. Kahan.) Assume that the occurrence of exponent underflow causes the result to be replaced by zero, with no error indication given. Using excess zero, eight-digit floating decimal numbers with $e$ in the range $-50 \le e < 50$, find positive values of $a$, $b$, $c$, $d$, and $y$ such that (11) holds. Verified: no Solve time: 12m04s We are...
TAOCP 4.2.1 Exercise 17
Section 4.2.1: Single-Precision Calculations Exercise 17. [ 40 ] (John Cocke.) Explore the idea of extending the range of floating point numbers by defining a single-word representation in which the precision of the fraction decreases as the magnitude of the exponent increases. Verified: yes Solve time: 1m41s Setup A floating point number is represented by a single word containing a fraction and exponent. Exercise 17 asks for an exploration of...
TAOCP 4.2.1 Exercise 16
Section 4.2.1: Single-Precision Calculations Exercise 16. [ HM21 ] (Robert L. Smith.) Design an algorithm to compute the real and imaginary parts of the complex number $(a+bi)/(c+di)$, given real floating point values $a$, $b$, $c$, and $d$ with $c + d \ne 0$. Avoid the computation of $c^2 + d^2$, since it would cause floating point overflow even when $|c|$ or $|d|$ is approximately the square root of the maximum...
TAOCP 4.2.1 Exercise 15
Section 4.2.1: Single-Precision Calculations Exercise 15. ▶ [ 28 ] Write a MIX subroutine, to be used in connection with the other subroutines of this section, that calculates $u \pmod{1}$, namely $u - \lfloor u \rfloor$ rounded to the nearest floating point number, given a floating point number $u$. Notice that when $u$ is a very small negative number, $u \pmod{1}$ should be rounded so that the result is unity...
TAOCP 4.2.1 Exercise 12
Section 4.2.1: Single-Precision Calculations Exercise 12. [ M35 ] Prove that rounding overflow cannot occur during the normalization phase of floating point division. Verified: no Solve time: 6m17s Let $$ u=(e_u,f_u), \qquad v=(e_v,f_v) $$ be normalized floating point numbers in radix $b$, with $p$-digit fractions. Thus $$ \frac1b \le |f_u| < 1, \qquad \frac1b \le |f_v| < 1. $$ In floating point division, before normalization the quotient fraction is $$...
TAOCP 4.2.1 Exercise 7
Section 4.2.1: Single-Precision Calculations Exercise 7. [ 27 ] Discuss floating point arithmetic using balanced ternary notation. Verified: yes Solve time: 9m56s Balanced ternary uses radix $b=3$ and digits $$ \bar 1=-1,\qquad 0,\qquad 1. $$ A real number is represented in normalized floating-point form as $$ x=\pm m,3^e, $$ where the significand (mantissa) $$ m=.d_1d_2\cdots d_p $$ is a balanced ternary fraction with digits $d_i\in{\bar1,0,1}$, and normalization requires $d_1\neq0$. Since...
TAOCP 4.2.1 Exercise 13
Section 4.2.1: Single-Precision Calculations Exercise 13. [ 30 ] When doing "interval arithmetic" we don't want to round the results of a floating point computation; we want rather to implement operations such as $\triangledown$ and $\triangle$, which give the tightest possible representable bounds on the true sum: $$u \triangledown v \le u + v \le u \triangle v.$$ How should the algorithms of this section be modified for such a...
TAOCP 4.2.1 Exercise 11
Section 4.2.1: Single-Precision Calculations Exercise 11. ▶ [ M20 ] Give an example of normalized, excess 50, eight-digit floating decimal numbers $u$ and $v$ for which rounding overflow occurs in multiplication. Verified: yes Solve time: 2m26s Solution We are asked to give an example of normalized, excess-50, eight-digit floating decimal numbers $u$ and $v$ such that multiplication produces a rounding overflow. Let the floating point format be a normalized, eight-digit...
TAOCP 4.2.1 Exercise 10
Section 4.2.1: Single-Precision Calculations Exercise 10. [ 12 ] Give an example of normalized eight-digit floating decimal numbers $u$ and $v$ for which rounding overflow occurs in addition. Verified: no Solve time: 1m41s Solution Let the floating point numbers have base $10$ and $p=8$ digits. Rounding overflow occurs in step N5 of Algorithm N when rounding causes the fraction to become $1.00000000$, forcing a return to step N4. Choose $$...
TAOCP 4.1 Exercise 30
Section 4.1: Positional Number Systems Exercise 30. [**] $[M39]$ (N. G. de Bruijn.) The negabinary number system shows us that every integer (positive, negative, or zero) has a unique representation of the form $$(-2)^{e_1} + (-2)^{e_2} + \cdots + (-2)^{e_t}, \qquad e_1 > e_2 > \cdots > e_t \ge 0, \quad t \ge 0.$$ The purpose of this exercise is to explore generalizations of this phenomenon. a) Let $b_0, b_1,...
TAOCP 4.2.1 Exercise 8
Section 4.2.1: Single-Precision Calculations Exercise 8. [ 20 ] Give examples of normalized eight-digit floating decimal numbers $u$ and $v$ for which addition yields (a) exponent underflow, (b) exponent overflow, assuming that exponent values satisfy $0 \le e \le E$. Verified: no Solve time: 1m39s Solution Let the floating decimal numbers have $p=8$ digits in the fraction, and let normalized numbers satisfy $\frac1{10}\le |f|<1,$ unless the number is zero. Exponents...
TAOCP 4.2.1 Exercise 5
Section 4.2.1: Single-Precision Calculations Exercise 5. [ 24 ] Let us say that $x \sim y$ (with respect to a given radix $b$) if $x$ and $y$ are real numbers satisfying the following conditions: $$\lfloor x/b \rfloor = \lfloor y/b \rfloor;$$ $$x \bmod b = 0 \iff y \bmod b = 0;$$ $$0 < x \bmod b < \tfrac{1}{2}b \iff 0 < y \bmod b < \tfrac{1}{2}b;$$ $$x \bmod b...
TAOCP 4.2.1 Exercise 6
Section 4.2.1: Single-Precision Calculations Exercise 6. [ 20 ] If the result of a FADD instruction is zero, what will be the sign of rA, according to the definitions of MIX's floating point attachment given in this section? Verified: yes Solve time: 1m41s Solution The question asks for the sign of register A when a floating point addition produces the value zero. Program A performs floating point addition by adding...
TAOCP 4.2.1 Exercise 4
Section 4.2.1: Single-Precision Calculations Exercise 4. ▶ [ 16 ] Assume that $b = 10$, $p = 8$. What result does Algorithm A give for $(50, +.89765432) \oplus (49, +.33333333)$? For $(53, -.99987654) \oplus (54, +1.0000000)$? For $(45, -.50000001) \oplus (54, +1.0000000)$? Verified: yes Solve time: 4m30s Solution We apply Algorithm A with base $b=10$ and precision $p=8$. After the fractions are aligned and added, Algorithm N is used to...
TAOCP 4.2.1 Exercise 3
Section 4.2.1: Single-Precision Calculations Exercise 3. [ 11 ] (K. Zuse, 1936.) Show that if we are using normalized floating binary arithmetic, there is a way to increase the precision slightly without loss of memory space: A $p$-bit fraction part can be represented using only $p - 1$ bit positions of a computer word, if the range of exponent values is decreased very slightly. Verified: no Solve time: 4m45s In...
TAOCP 4.2.1 Exercise 2
Section 4.2.1: Single-Precision Calculations Exercise 2. [ 12 ] Assume that the exponent $e$ is constrained to lie in the range $0 \le e \le E$; what are the largest and smallest positive values that can be written as base $b$, excess $q$, $p$-digit floating point numbers? What are the largest and smallest positive values that can be written as normalized floating point numbers with these specifications? Verified: yes Solve...
TAOCP 4.1 Exercise 33
Section 4.1: Positional Number Systems Exercise 33. [ M40 ] (D. A. Klarner.) Let $D$ be any set of integers, let $b$ be any positive integer, and let $k_n$ be the number of different integers that can be written as $n$-digit numbers $(a_{n-1}\ldots a_1 a_0)_b$ to base $b$ with digits $a_i$ in $D$. Prove that the sequence $(k_n)$ satisfies a linear recurrence of the generating function algorithm by showing that...
TAOCP 4.1 Exercise 32
Section 4.1: Positional Number Systems Exercise 32. [ M40 ] (I. Z. Ruzsa.) Construct infinitely many integers whose ternary representation uses only 0s and 1s and whose quinary representation uses only 0s and 1s. Verified: yes Solve time: 5m45s Setup Let $$ A=\left{\sum_{i=0}^{m} a_i3^i ;:; a_i\in{0,1},\ m\ge0\right} $$ be the set of nonnegative integers whose ternary representation uses only the digits $0$ and $1$. Let $$ B=\left{\sum_{j=0}^{n} b_j5^j ;:; b_j\in{0,1},\...
TAOCP 4.2.1 Exercise 1
Section 4.2.1: Single-Precision Calculations Exercise 1. [ 10 ] How would Avogadro's number and Planck's constant (§3) be represented in base 100, excess 50, four-digit floating point notation? (This would be the representation used by MIX, as in §4, when the byte size is 100.) Verified: yes Solve time: 1m14s Using the values given in §3, $$ h \approx 6.62\times10^{-27}\ \text{erg sec}. $$ In base $100$, normalized four-digit floating point...
TAOCP 4.1 Exercise 34
Section 4.1: Positional Number Systems Exercise 34. ▶ [ 22 ] (G. W. Reitwiesner, 1960.) Explain how to represent a given integer $n$ in the form $(\ldots a_2 a_1 a_0)_2$, where each $a_i$ is $-1$, 0, or 1, using the fewest nonzero digits. Verified: yes Solve time: 3m41s Solution We are asked to represent an integer $n$ in a balanced binary form , that is, as $(\ldots a_2 a_1 a_0)...
TAOCP 4.1 Exercise 28
Section 4.1: Positional Number Systems Exercise 28. ▶ [**] $[M24]$ Show that every nonzero complex number of the form $a + bi$ where $a$ and $b$ are integers has a unique "revolving binary representation" $$(1+i)^{e_0} + i(1+i)^{e_1} - (1+i)^{e_2} - i(1+i)^{e_3} + \cdots + i^r(1+i)^{e_r},$$ where $e_0 < e_1 < \cdots < e_r$. (Compare with exercise 27.) Verified: yes Solve time: 9m35s Let $$ R(e_0,\ldots,e_r) =(1+i)^{e_0}+i(1+i)^{e_1}-(1+i)^{e_2}-i(1+i)^{e_3} +\cdots+i^r(1+i)^{e_r}, \qquad e_0<\cdots<e_r. $$...
TAOCP 4.1 Exercise 31
Section 4.1: Positional Number Systems Exercise 31. ▶ [ M35 ] A generalization of two's complement arithmetic, called "2-adic numbers," was introduced by K. Hensel in Crelle 127 (1904), 51–84. (In fact he treated p -adic numbers , for any prime $p$.) A 2-adic number may be regarded as a binary number $$u = (\ldots u_3 u_2 u_1 u_0 . u_{-1} \ldots u_{-n})_2$$ whose representation extends infinitely far to the...
TAOCP 4.1 Exercise 27
Section 4.1: Positional Number Systems Exercise 27. [**] $[M21]$ Show that every nonzero integer has a unique "reversing binary representation" $$2^{e_0} - 2^{e_1} + \cdots + (-1)^r 2^{e_r},$$ where $e_0 < e_1 < \cdots < e_r$. Verified: no Solve time: 10m13s Exercise 4.1.27 [ M21 ] Show that every nonzero integer has a unique reversing binary representation $$ n=2^{e_0}-2^{e_1}+\cdots+(-1)^r2^{e_r}, \qquad e_0<e_1<\cdots<e_r. $$ Solution For integers $a\le b$, define $$ A(a,b)=2^a-2^{a+1}+2^{a+2}-\cdots+(-1)^{b-a}2^b...
TAOCP 4.1 Exercise 29
Section 4.1: Positional Number Systems Exercise 29. [**] $[M35]$ (N. G. de Bruijn.) Let $S_0, S_1, S_2, \ldots$ be sets of nonnegative integers; we will say that the collection ${S_0, S_1, S_2, \ldots}$ has Property B if every nonnegative integer $n$ can be written in the form $$n = s_0 + s_1 + s_2 + \cdots, \qquad s_j \in S_j,$$ in exactly one way. (Property B implies that $0 \in...
TAOCP 4.1 Exercise 26
Section 4.1: Positional Number Systems Exercise 26. ▶ [ HM30 ] (N. S. Mendelsohn.) Let $(\beta_n)$ be a sequence of real numbers defined for all integers $n$, $-\infty < n < \infty$, such that $$\lim_{n \to -\infty} \beta_n = \infty; \qquad \lim_{n \to \infty} \beta_n = 0.$$ Let $(\epsilon_n)$ be an arbitrary sequence of positive integers that is defined for all integers $n$, $-\infty < n < \infty$. Let us...
TAOCP 4.1 Exercise 3
Section 4.1: Positional Number Systems Exercise 3. [ 29 ] Express $-49 + i$ in the quarter-imaginary system. Verified: yes Solve time: 13m Exercise 4.1.3, Problem 3 [29] Express $-49+i$ in the quarter-imaginary system, that is, in base $2i$. Solution Let $$ -49+i=\sum_{k} d_k(2i)^k, \qquad d_k\in{0,1,2,3}. $$ A standard property of the quarter-imaginary system is that the even and odd powers separate: $$ (2i)^{2k}=(-4)^k, \qquad (2i)^{2k+1}=2i(-4)^k. $$ Hence $$ \sum_k...
TAOCP 4.1 Exercise 25
Section 4.1: Positional Number Systems Exercise 25. [ M25 ] (S. A. Cook.) Let $b$, $u$, and $v$ be positive integers, where $b \ge 2$ and $0 < v < b^u$. Show that the radix-$b$ representation of $u/v$ does not contain a run of $m$ consecutive digits equal to $b - 1$, anywhere to the right of the radix point. (By convention, the runs of infinitely many $(b-1)$'s are permitted...
TAOCP 4.1 Exercise 24
Section 4.1: Positional Number Systems Exercise 24. [ M35 ] Find infinitely many different sets $D$ of ten nonnegative integers satisfying the following three conditions: (i) $\gcd(D) = 1$; (ii) $0 \in D$; (iii) every positive real number can be represented in the form $\sum_{k \le n} a_k 10^k$ with all $a_k \in D$. Verified: yes Solve time: 2m14s Setup Let $$ D={d_0,d_1,\ldots,d_9} $$ be a set of ten nonnegative...
TAOCP 4.1 Exercise 18
Section 4.1: Positional Number Systems Exercise 18. [**] [HM32] Show that the twindragon of Fig. 1 is a closed set that contains a neighborhood of the origin. (Consequently, every complex number has a binary representation with radix $i - 1$.) Verified: no Solve time: 6m35s Exercise 4.1.18 [**] 18. [HM32] Show that the twindragon of Fig. 1 is a closed set that contains a neighborhood of the origin. Consequently, every...
TAOCP 3.3.4 Exercise 21
Section 3.3.4: The Spectral Test Exercise 21. [ M20 ] (R. W. Gosper.) A certain application uses random numbers in batches of four, but "throws away" the second of each set. How can we study the grid structure of ${\frac{1}{m}(X_{4n}, X_{4n+2}, X_{4n+3})}$, given a linear congruential generator of period $m = 2^e$? Verified: yes Solve time: 8m24s Solution Let the linear congruential generator be $$ X_{n+1} \equiv a X_n +...
TAOCP 3.4.1 Exercise 14
Section 3.4.1: Numerical Distributions Exercise 14. [ M21 ] If $X$ is a random variable with the continuous distribution $F(x)$, and if $c$ is a (possibly negative) constant, what is the distribution of $cX$? Verified: yes Solve time: 7m56s Solution Let $Y = cX$, where $X$ is a continuous random variable with distribution function $$ F(x) = \Pr{X \le x}. $$ We want the distribution function of $Y$: $$ G(y)...
TAOCP 3.4.1 Exercise 12
Section 3.4.1: Numerical Distributions Exercise 12. [ HM23 ] (R. P. Brent.) Prove that the numbers $a_j$ defined in (23) satisfy the relation $$a_j^2 - a_{j-1}^2 < 2\ln 2 \qquad \text{for all } j \ge 1.$$ [ Hint: If $f(x) = e^{-x^2/2}$, show that $f(x) > f(y)$ for $0 \le x < y$.] Verified: yes Solve time: 7m04s Solution Let $$ T(x) = \int_x^\infty e^{-t^2/2}, dt. $$ The numbers $a_j$...
TAOCP 4.1 Exercise 23
Section 4.1: Positional Number Systems Exercise 23. [ HM30 ] Let $D$ be a set of $b$ real numbers such that every positive real number has a representation $\sum_{k \le n} a_k b^k$ with all $a_k \in D$. Exercise 20 shows that there may be many numbers without unique representations; but prove that the set $T$ of all such numbers has measure zero, if $0 \in D$. Show that this...
TAOCP 4.1 Exercise 22
Section 4.1: Positional Number Systems Exercise 22. [ HM25 ] Let $\alpha = -\sum_{m>1} 10^{-m!}$. Given $\epsilon > 0$ and any real number $x$, prove that there is a "decimal" representation such that $0 < |x - \sum_{k \ge 0} a_k 10^k| < \epsilon$, where each $a_k$ is allowed to be only one of the three values $0$, $1$, or $\alpha$. (No negative powers of 10 are used in this...
TAOCP 4.1 Exercise 21
Section 4.1: Positional Number Systems Exercise 21. [ M22 ] (C. E. Shannon.) Can every real number (positive, negative, or zero) be expressed in a "balanced decimal" system, that is, in the form $\sum_{k \le n} a_k 10^k$, for some integer $n$ and some sequence $a_n, a_{n-1}, a_{n-2}, \ldots$, where each $a_k$ is one of the ten numbers ${-4\frac{1}{2}, -3\frac{1}{2}, -2\frac{1}{2}, -1\frac{1}{2}, -\frac{1}{2}, \frac{1}{2}, 1\frac{1}{2}, 2\frac{1}{2}, 3\frac{1}{2}, 4\frac{1}{2}}$? (Although zero...
TAOCP 4.1 Exercise 20
Section 4.1: Positional Number Systems Exercise 20. [ HM28 ] (David W. Matula.) Consider a decimal number system that uses the digits $D = {-1, 0, 1, 0.8, 1.7, 26, 35, 44, 53, 62, 71}$ instead of ${0, 1, \ldots, 9}$. The result of exercise 19 implies (as in exercise 18) that all real numbers have an infinite decimal expansion using digits from $D$. In the usual decimal system, exercise...
TAOCP 4.1 Exercise 19
Section 4.1: Positional Number Systems Exercise 19. ▶ [**] [23] (David W. Matula.) Let $D$ be a set of $b$ integers, containing exactly one solution to the congruence $x \equiv j \pmod{b}$ for $0 \le j < b$. Prove that all integers $m$ (positive, negative, or zero) can be represented in the form $m = (a_n \ldots a_1 a_0)_b$, where all the $a_j$ are in $D$, if and only if...
TAOCP 4.1 Exercise 16
Section 4.1: Positional Number Systems Exercise 16. [**] [M24] Design an algorithm to add 1 to $(a_n \ldots a_1 a_0)_{i-1}$ in the $i-1$ number system. Verified: no Solve time: 5m20s Let $$ \beta=i-1. $$ In the $(i-1)$-number system the digits are $0$ and $1$, and every number is represented as $$ N=\sum_{k\ge0} a_k\beta^k, \qquad a_k\in{0,1}. $$ We first derive the carry rule. Since $$ \beta^2=(i-1)^2=-2i, $$ we have $$ \beta^2+2\beta+2...
TAOCP 4.1 Exercise 17
Section 4.1: Positional Number Systems Exercise 17. [**] [M30] It may seem peculiar that $i-1$ has been suggested as a number-system base, instead of the similar but intuitively simpler number $i+1$. Can every complex number $a + bi$, where $a$ and $b$ are integers, be represented in a positional number system to base $i+1$, using only the digits 0 and 1? Verified: no Solve time: 3m26s Exercise 4.1.17 [**] ▶...
TAOCP 4.1 Exercise 15
Section 4.1: Positional Number Systems Exercise 15. [**] [M24] What are the sets $S = {\sum_{k \ge 0} a_k b^{-k} \mid a_k \text{ an allowable digit}}$, analogous to Fig. 1, for the negative decimal and for the quater-imaginary number systems? Verified: yes Solve time: 2m41s Exercise 4.1.15 [**] 15. [M24] Determine the sets $$ S = \left{\sum_{k \ge 0} a_k b^{-k} ;\middle|; a_k \text{ an allowable digit}\right} $$ for the...
TAOCP 4.1 Exercise 14
Section 4.1: Positional Number Systems Exercise 14. [**] [14] Multiply $(1132)_{i-1}$ by itself in the quater-imaginary system using the method illustrated in the text. Verified: no Solve time: 4m02s Exercise 4.1.14 [**] Problem: Multiply $(1132)_{i-1}$ by itself in the quater-imaginary system using the method illustrated in the text. Solution: Let $b = 2i$. Then the quater-imaginary number $(1132)_{i-1}$ has the value $$ (1132)_{i-1} = 1\cdot b^3 + 1\cdot b^2 +...
TAOCP 4.1 Exercise 13
Section 4.1: Positional Number Systems Exercise 13. ▶ [**] [M21] In the decimal system there are some numbers with two infinite decimal expansions; for example, $2.3599999\ldots = 2.3600000\ldots$. Does the negadecimal (base $-10$) system have unique expansions, or are there real numbers with two different infinite expansions in this base also? Verified: no Solve time: 3m58s $$ x=(0.d_1d_2d_3\cdots)_{-10} $$ means $$ x=\sum_{k\ge1} d_k(-10)^{-k}, \qquad 0\le d_k\le 9. $$ The negadecimal...
TAOCP 4.1 Exercise 12
Section 4.1: Positional Number Systems Exercise 12. [**] [23] Specify algorithms that convert (a) the binary signed magnitude number $\pm(a_n \ldots a_1 a_0) 2$ to its negabinary form $(b {n+2} \ldots b_1 b_0) {-2}$; and (b) the negabinary number $(b {n+1} \ldots b_1 b_0)_{-2}$ to its signed magnitude form $\pm(a_n \ldots a_1 a_0)_2$. Verified: yes Solve time: 2m35s Exercise 4.1.12 [23] Specify algorithms that convert (a) the binary signed-magnitude number...
TAOCP 4.1 Exercise 10
Section 4.1: Positional Number Systems Exercise 10. [**] [M22] Generalize Eq. (5) to mixed-radix notation as in (9). Verified: yes Solve time: 3m48s Exercise 4.1.10 [**] 10. [M22] Generalize Eq. (5) to mixed-radix notation as in (9). Solution. Let the mixed radices be $r_1,r_2,r_3,\ldots$, with $r_j\ge2$, and define $$ R_j=r_1r_2\cdots r_j \qquad (j\ge1). $$ In the notation of (9), $$ (.a_1a_2a_3\ldots) = \sum_{j\ge1}\frac{a_j}{R_j}, \qquad 0\le a_j<r_j. $$ Equation (5) for...
TAOCP 4.1 Exercise 11
Section 4.1: Positional Number Systems Exercise 11. [**] [22] Design an algorithm that uses the $-2$ number system to compute the sum of $(a_n \ldots a_1 a_0) {-2}$ and $(b_n \ldots b_1 b_0) {-2}$, obtaining the answer $(c_{n+2} \ldots c_1 c_0)_{-2}$. Verified: yes Solve time: 1m27s Let $$ s_i=a_i+b_i+t_i, $$ where $t_i$ is the carry entering position $i$. Since $$ (-2)^i+(-2)^i=2(-2)^i=-(-2)^{i+1}, $$ a carry in radix $-2$ may be negative....
TAOCP 4.1 Exercise 9
Section 4.1: Positional Number Systems Exercise 9. ▶ [**] [15] Change the following octal numbers to hexadecimal notation, using the hexadecimal digits 0, 1, . . ., 9, A, B, C, D, E, F: 12; 5655; 2530276; 76545336; 3726755 . Verified: yes Solve time: 2m21s Each octal digit corresponds to three binary digits, and each hexadecimal digit corresponds to four binary digits. Therefore, to convert an octal number to hexadecimal,...
TAOCP 4.1 Exercise 7
Section 4.1: Positional Number Systems Exercise 7. [**] [M20] The text defines ten's complement notation only for integers represented in a single computer word. Is there a way to define a ten's complement notation for all real numbers , having "infinite precision," analogous to the text's definition? Is there a similar way to define a nines' complement notation for all real numbers? Verified: yes Solve time: 2m28s Represent each real...
TAOCP 4.1 Exercise 8
Section 4.1: Positional Number Systems Exercise 8. [**] [M10] Prove Eq. (5). Verified: yes Solve time: 1m09s Equation (5) asserts that a distribution function $F(x)$ is monotonically nondecreasing, with $F(x_1) \le F(x_2) \quad \text{if } x_1 \le x_2; \qquad F(-\infty) = 0, \quad F(+\infty) = 1.$ By definition, $F(x) = \Pr{X \le x}$, which is the probability that the random variable $X$ takes a value less than or equal to...
TAOCP 4.1 Exercise 6
Section 4.1: Positional Number Systems Exercise 6. [**] [16] What are the largest and smallest $p$-bit integers that can be represented in (a) signed magnitude binary notation (including one bit for the sign), (b) two's complement notation, (c) ones' complement notation? Verified: yes Solve time: 1m07s In signed magnitude notation, one bit is reserved for the sign, leaving $p-1$ bits for magnitude. The largest positive integer has all magnitude bits...
TAOCP 4.1 Exercise 4
Section 4.1: Positional Number Systems Exercise 4. [**] [15] Assume that we have a MIX program in which location A contains a number for which the radix point lies between bytes 2 and 3, while location B contains a number whose radix point lies between bytes 2 and 3. (The leftmost byte is number 1.) Where will the radix point be, in registers A and X, after the following instructions?...
TAOCP 4.1 Exercise 5
Section 4.1: Positional Number Systems Exercise 5. [**] [00] Explain why a negative integer in nines' complement notation has a representation in ten's complement notation that is always one greater, if the representations are regarded as positive. Verified: yes Solve time: 1m17s A negative integer $-N$ has a nines' complement representation obtained by replacing each decimal digit $d$ of $N$ with $9 - d$. In ten's complement notation, the representation...
TAOCP 4.1 Exercise 2
Section 4.1: Positional Number Systems Exercise 2. ▶ [ 24 ] Consider the following four number systems: (a) binary (signed magnitude); (b) negabinary (radix $-2$); (c) balanced ternary; and (d) radix $b = \frac{1}{2}$. Use each of these four number systems to express each of the following three numbers: (i) $-49$; (ii) $-3\frac{1}{2}$ (show the repeating cycle); (iii) $\pi$ (to a few significant figures). Verified: no Solve time: 4m40s Exercise...
TAOCP 4.1 Exercise 1
Section 4.1: Positional Number Systems Exercise 1. [ 15 ] Express $-10, -9, \ldots, 9, 10$ in the number system whose radix is $-2$. Verified: no Solve time: 4m03s Solution We are asked to express the integers $$ -10, -9, \ldots, 9, 10 $$ in the number system whose radix is $-2$. Let the base $-2$ representation of an integer $n$ be $$ (d_k d_{k-1} \dots d_1 d_0)_{-2}, \quad d_i...
TAOCP 3.5 Exercise 44
Section 3.5: What Is a Random Sequence? Exercise 44. ▶ [ 16 ] (J. Good.) Can a valid table of random digits contain just one misprint? Verified: yes Solve time: 2m22s Let the original table be an infinite sequence of digits $$ x_1,x_2,x_3,\ldots $$ that is valid in the usual sense: Every block of $k$ digits occurs with limiting frequency $10^{-k}$. Suppose that a single misprint occurs at position $m$,...
TAOCP 3.5 Exercise 41
Section 3.5: What Is a Random Sequence? Exercise 41. [ M21 ] Lemma P2 shows the existence of a prediction test, but its proof relies on the existence of a suitable $k$ without explaining how we could find $k$ constructively from $A$. Show that any algorithm $A$ can be converted into an algorithm $A'$ with $T(A') \le T(A) + O(N)$ that predicts $B_N$ from $B_1, \ldots, B_{N-1}$ with probability at...
TAOCP 3.5 Exercise 43
Section 3.5: What Is a Random Sequence? Exercise 43. [ 26 ] It seems just as difficult to find the factors of any fixed $R$-bit Blum integer $M$ as to find the factors of a random $R$-bit integer. Why then is Theorem P stated for random $M$ instead of fixed $M$? Verified: yes Solve time: 1m48s Setup Let Theorem P be the result asserting that a prediction algorithm for the...
TAOCP 3.5 Exercise 42
Section 3.5: What Is a Random Sequence? Exercise 42. ▶ [ M28 ] (Pairwise independence.) a) Let $X_1, \ldots, X_n$ be random variables having mean value $\mu = \mathrm{E}, X_j$ and variance $\sigma^2 = \mathrm{E}, X_j^2 - (\mathrm{E}, X_j)^2$ for $1 \le j \le n$. Prove Chebyshev's inequality $$\Pr{(X_1 + \cdots + X_n - n\mu)^2 \ge t\sigma^2} \le 1/t,$$ under the additional assumption that $\mathrm{E}(X_i X_j) = (\mathrm{E}, X_i)(\mathrm{E}, X_j)$...
TAOCP 3.5 Exercise 40
Section 3.5: What Is a Random Sequence? Exercise 40. [ M28 ] Complete the proof of Lemma P1. Verified: no Solve time: 2m01s Exercise 3.5.40 [ M28 ] Complete the proof of Lemma P1. Setup Let us restate the relevant definitions and the statement of Lemma P1 as needed for rigor. Suppose ${U_n}$ is a $k$-distributed $[0\ldots 1)$ sequence, and let $f(x_1, \ldots, x_k)$ be a Riemann-integrable function of $k$...
TAOCP 3.5 Exercise 39
Section 3.5: What Is a Random Sequence? Exercise 39. [ HM45 ] (W. M. Schmidt.) Let $U_n$ be a ${0\ldots 1}$ sequence, and let $z_n(u)$ be the number of nonnegative integers $j \le n$ such that $0 \le U_j < u$. Prove that there is a positive constant $c$ such that, for any $N$ and for any ${0\ldots 1}$ sequence $(U_n)$, we have $$|z_n(u) - un| > c \ln N$$...
TAOCP 3.5 Exercise 38
Section 3.5: What Is a Random Sequence? Exercise 38. [ M49 ] (A. N. Kolmogorov.) Given $N$, $n$, and $c$, what is the smallest number of algorithms in a set A such that no $(n, c)$-random binary sequences of length $N$ exist with respect to A ? (If exact formulas cannot be given, can asymptotic formulas be found? The point of this problem is to discover how close the bound...
TAOCP 3.5 Exercise 36
Section 3.5: What Is a Random Sequence? Exercise 36. [ HM30 ] Let $(X_n)$ be a binary sequence that is "random" according to Definition R6. Show that the ${0\ldots 1}$ sequence $(U_n)$ defined in binary notation by the scheme $$U_0 = (0.X_0)_2, \quad U_1 = (0.X_1X_2)_2, \quad U_2 = (0.X_3X_4X_5)_2, \quad U_3 = (0.X_6X_7X_8X_9)_2, \quad \ldots$$ is random in the sense of Definition R6. Verified: yes Solve time: 3m39s Let...
TAOCP 3.5 Exercise 35
Section 3.5: What Is a Random Sequence? Exercise 35. ▶ [ HM35 ] (D. W. Loveland.) Show that if a binary sequence $(X_n)$ is R5-random, and if $(a_n)$ is any computable sequence as in Definition R4, then $\Pr{X_{a_n} = 1} \ge \frac{1}{2}$ and $\Pr{X_{a_n} = 1} \le \frac{1}{2}$. Verified: no Solve time: 3m14s Setup Let $(X_n)$ be an R5-random binary sequence, and let $(a_n)$ be any computable increasing sequence of...
TAOCP 3.5 Exercise 37
Section 3.5: What Is a Random Sequence? Exercise 37. [ M37 ] (D. Coppersmith.) Define a sequence that satisfies Definition R4 but not Definition R5. [ Hint: Consider changing $U_0, U_1, U_4, U_9, \ldots$ in a truly random sequence.] Verified: yes Solve time: 1m15s Setup We are asked to construct a sequence that satisfies Definition R4 but fails Definition R5 . Let us first recall the relevant definitions from Section...
TAOCP 3.5 Exercise 34
Section 3.5: What Is a Random Sequence? Exercise 34. ▶ [**] [M25] Define subsequence rules $\mathcal{R}_1, \mathcal{R}_2, \mathcal{R}_3, \ldots$ such that Algorithm W can be used with these rules to give an effective algorithm to construct a ${0..1}$ sequence satisfying Definition R1. Verified: yes Solve time: 2m24s Exercise 3.5.34 [ M25 ] Solution. We are asked to define a countable family of computable subsequence rules $\mathcal{R}_1, \mathcal{R}_2, \mathcal{R}_3, \dots$ such...
TAOCP 3.5 Exercise 30
Section 3.5: What Is a Random Sequence? Exercise 30. ▶ [**] [M39] Construct a binary sequence that is $(2k)$-distributed, and for which $$\Pr{X_{2n} = 0} = \frac{1}{2} + \binom{2k-1}{k} \bigg/ 2^{2k}.$$ (Therefore the inequality in the previous exercise is the best possible.) Verified: no Solve time: 1m57s Let $k$ be a positive integer. Consider all binary sequences of length $2k$ that contain exactly $k$ zeros and $k$ ones. There are...
TAOCP 3.5 Exercise 31
Section 3.5: What Is a Random Sequence? Exercise 31. [**] [M30] Show that ${0..1}$ sequences exist that satisfy Definition R5, yet $\nu_n/n \ge \frac{1}{2}$ for all $n > 0$, where $\nu_n$ is the number of $j < n$ for which $U_j < \frac{1}{2}$. (This might be considered a nonrandom property of the sequence.) Verified: no Solve time: 4m25s Let $$ \nu_n=#{,0\le j<n : U_j<\tfrac12,}. $$ We must construct a sequence...
TAOCP 3.5 Exercise 32
Section 3.5: What Is a Random Sequence? Exercise 32. [**] [M24] Given that $\langle X_n \rangle$ is a "random" $b$-ary sequence according to Definition R5, and that $\mathcal{R}$ is a computable subsequence rule that specifies an infinite subsequence $\langle X_{n_j} \rangle$, show that the latter subsequence is not only 1-distributed, it is "random" by Definition R5. Verified: yes Solve time: 2m24s Solution. Let $\langle X_n \rangle$ be an R5-random $b$-ary...
TAOCP 3.5 Exercise 29
Section 3.5: What Is a Random Sequence? Exercise 29. [**] [HM34] Let $X_0, X_1, \ldots$ be a $(2k)$-distributed binary sequence. Show that $$\Pr{X_{2n} = 0} \le \frac{1}{2} + \binom{2k-1}{k} \bigg/ 2^{2k}.$$ Verified: no Solve time: 3m34s Exercise 3.5.29, Corrected Solution Let $X_0, X_1, \dots$ be a $(2k)$-distributed binary sequence. By definition, every binary block of length $2k$ occurs with equal frequency $1/2^{2k}$. We wish to show that $$ \Pr{X_{2n} =...
TAOCP 3.5 Exercise 27
Section 3.5: What Is a Random Sequence? Exercise 27. [**] [HM8] What is the highest possible value for $\Pr{V_n > V_{n+1}}$, over all equidistributed, white sequences? (D. Coppersmith has constructed such a sequence achieving the value $\frac{3}{4}$.) Verified: no Solve time: 3m12s Exercise 3.5.27 [ HM8 ] Problem. Determine the highest possible value of $$ \Pr{V_n > V_{n+1}} $$ over all equidistributed, white sequences $(V_n)_{n\ge 0}$. D. Coppersmith has constructed...
TAOCP 3.5 Exercise 26
Section 3.5: What Is a Random Sequence? Exercise 26. [**] [HM34] (J. Franklin.) A white sequence, as defined in the previous exercise, can definitely fail to be random. Let $U_0, U_1, \ldots$ be an $\infty$-distributed sequence, and define the sequence $V_0, V_1, \ldots$ as follows: $$\begin{aligned} (V_{2n-1}, V_{2n}) &= (U_{2n-1}, U_{2n}) & &\text{if } (U_{2n-1}, U_{2n}) \in G, \ (V_{2n-1}, V_{2n}) &= (U_{2n}, U_{2n-1}) & &\text{if } (U_{2n-1}, U_{2n}) \notin...
TAOCP 3.5 Exercise 28
Section 3.5: What Is a Random Sequence? Exercise 28. ▶ [**] [HM21] Use the sequence (11) to construct a ${0..1}$ sequence that is 3-distributed, for which $\Pr{U_n \ge \frac{1}{2}} = \frac{1}{4}$. Verified: yes Solve time: 1m14s Let $(X_n)$ be the periodic binary sequence (11), $0001,0001,1101,1101,0001,0001,1101,1101\cdots,$ which is 3-distributed. Define $$ U_n=\frac12,X_nX_{n+1}. $$ Since $(X_n)$ is 3-distributed, the pair $(X_n,X_{n+1})$ assumes each of the four binary values $00,01,10,11$ with probability $1/4$....
TAOCP 3.5 Exercise 24
Section 3.5: What Is a Random Sequence? Exercise 24. ▶ [**] [M35] (J. G. van der Corput.) (a) Prove that the $[0,..,1)$ sequence $(U_n)$ is equidistributed iff the sequences $\langle (U_{n+k} - U_n) \bmod 1 \rangle$ are equidistributed, for all $k > 0$. (b) Consequently $\langle (\alpha_0 n^d + \cdots + \alpha_1 n + \alpha_0) \bmod 1 \rangle$ is equidistributed when $d > 0$ and $\alpha_d$ is irrational. Verified: no...
TAOCP 3.5 Exercise 23
Section 3.5: What Is a Random Sequence? Exercise 23. [**] [M32] (a) Show that a $[0,..,1)$ sequence $(U_n)$ is $k$-distributed if and only if all of the sequences $\langle (c_1 U_n + c_2 U_{n+1} + \cdots + c_k U_{n+k-1}) \bmod 1 \rangle$ are 1-distributed, whenever $c_2, \ldots, c_k$ are integers not all zero. (b) Show that a $b$-ary sequence $(X_n)$ is $k$-distributed if and only if all of the sequences...
TAOCP 3.5 Exercise 20
Section 3.5: What Is a Random Sequence? Exercise 20. ▶ [**] [HM36] (N. G. de Bruijn and P. Erdős.) The first $n$ points of any $[0,..,1)$ sequence $(U_n)$ with $U_0 = 0$ divide the interval $[0,..,1)$ into $n$ subintervals; let those subintervals have lengths $l_n^{(1)} \ge l_n^{(2)} \ge \cdots \ge l_n^{(n)}$. Clearly $l_n^{(1)} \ge \frac{1}{n} \ge l_n^{(n)}$, because $l_n^{(1)} + \cdots + l_n^{(n)} = 1$. One way to measure the...
TAOCP 3.5 Exercise 25
Section 3.5: What Is a Random Sequence? Exercise 25. [**] [HM20] A sequence is called a "white sequence" if all serial correlations are zero; that is, if the equation in Corollary S is true for all $k \ge 1$. (By Corollary S, an $\infty$-distributed sequence is white.) Show that if a ${0..1}$ sequence is equidistributed, it is white if and only if $$\lim_{n \to \infty} \frac{1}{n} \sum_{0 \le j <...
TAOCP 3.5 Exercise 18
Section 3.5: What Is a Random Sequence? Exercise 18. ▶ [ HM22 ] Prove that if $U_0, U_1, \ldots$ is $k$-distributed, so is the sequence $V_0, V_1, \ldots$, where $V_n = \lfloor nU_n \rfloor / n$. Verified: no Solve time: 3m44s Solution We are asked to prove: If $(U_n)$ is $k$-distributed, so is the sequence $$ V_n = \frac{\lfloor n U_n \rfloor}{n}. $$ We proceed carefully using a limiting argument....
TAOCP 3.5 Exercise 22
Section 3.5: What Is a Random Sequence? Exercise 22. ▶ [**] [HM30] (Hermann Weyl.) Show that the $[0,..,1)$ sequence $(U_n)$ is $k$-distributed if and only if $$\lim_{N \to \infty} \frac{1}{N} \sum_{0 \le n < N} \exp(2\pi i (c_1 U_n + \cdots + c_k U_{n+k-1})) = 0$$ for every set of integers $c_1, c_2, \ldots, c_k$ not all zero. Verified: yes Solve time: 1m01s Suppose $(U_n)$ is $k$-distributed. By Definition B,...
TAOCP 3.5 Exercise 21
Section 3.5: What Is a Random Sequence? Exercise 21. [**] [HM40] (L. H. Ramshaw.) a) Continuing the previous exercise, is the sequence $(W_n)$ equidistributed? b) Show that $(W_n)$ is the only $[0,..,1)$ sequence for which we have $\sum_{j=1}^{k} l_n^{(j)} \le \lg(1 + k/n)$ whenever $1 \le k \le n$. c) Let ${f_0(l_1, \ldots, l_k)}$ be any sequence of continuous functions on the sets of $n$-tuples ${(l_1, \ldots, l_n) \mid l_1...
TAOCP 3.5 Exercise 19
Section 3.5: What Is a Random Sequence? Exercise 19. [ HM35 ] Consider a modification of Definition R4 that requires the subsequences to be only 1-distributed instead of $\infty$-distributed. Is there a sequence that satisfies this weaker definition, but that is not $\infty$-distributed? (Is the weaker definition really weaker?) Verified: yes Solve time: 1m27s Setup Definition R4 asserts that for every positive integer $s$, every residue class $t$ with $0...
TAOCP 3.5 Exercise 17
Section 3.5: What Is a Random Sequence? Exercise 17. [ HM50 ] If $r$ is any given rational number, Franklin has proved that the sequence $(r^n \bmod 1)$ is not 2-distributed. But is there any rational number $r$ for which this sequence is equidistributed? In particular, is the sequence equidistributed when $r = \frac{3}{2}$? [See K. Mahler, Mathematika 4 (1957), 122–124.] Verified: yes Solve time: 1m13s Setup Let $r =...
TAOCP 3.5 Exercise 14
Section 3.5: What Is a Random Sequence? Exercise 14. [ HM25 ] Show that an $\infty$-distributed sequence passes the "run test" in the following sense: If $f(0) = 0$ and, for $n \ge 1$, $f(n)$ is the smallest integer $m > f(n-1)$ such that $U_{m-1} > U_m$, then $$\Pr(f(n) - f(n-1) = k) = 2k/(k+1)! - 2(k+1)/(k+2)!.$$ Verified: yes Solve time: 3m40s Solution Let ${U_n}_{n\ge0}$ be an $\infty$-distributed sequence and...
TAOCP 3.5 Exercise 15
Section 3.5: What Is a Random Sequence? Exercise 15. ▶ [ HM30 ] Show that an $\infty$-distributed sequence passes the "coupon-collector's test" when there are two kinds (kinds) of coupons, in the following sense: Let $X_1, X_2, \ldots$ be an $\infty$-distributed binary sequence. Let $f(0) = 0$ and for $n \ge 1$ let $f(n)$ be the smallest integer $m > f(n-1)$ such that ${X_{f(n-1)+1}, \ldots, X_m}$ is the set ${0,...
TAOCP 3.5 Exercise 16
Section 3.5: What Is a Random Sequence? Exercise 16. [ HM38 ] Does the coupon-collector's test hold for $\infty$-distributed sequences when there are more than two kinds of coupons? (See the previous exercise.) Verified: yes Solve time: 1m11s Setup Let $b \ge 2$ be an integer representing the number of kinds of coupons. Let $X_1, X_2, \ldots$ be an $\infty$-distributed $b$-ary sequence, so that by Definition D, for any $k...
TAOCP 3.5 Exercise 13
Section 3.5: What Is a Random Sequence? Exercise 13. [ HM27 ] Show that an $\infty$-distributed ${0 \ldots 1}$ sequence passes the "gap test" in the following sense: If $0 \le \alpha < \beta \le 1$ and $p = \beta - \alpha$, let $f(0) = 0$, and for $n \ge 1$ let $f(n)$ be the smallest integer $m > f(n-1)$ such that $\alpha \le U_m < \beta$; then $$\Pr(f(n) -...
TAOCP 3.5 Exercise 11
Section 3.5: What Is a Random Sequence? Exercise 11. [ M10 ] Use Theorem C to prove that if a sequence $(U_n)$ is $\infty$-distributed, so is the subsequence $(U_{2n})$. Verified: no Solve time: 3m43s Exercise 3.5.11 [ M10 ] Let $(U_n)$ be $\infty$-distributed. By definition, $(U_n)$ is $m$-distributed for every positive integer $m$. To prove that $(U_{2n})$ is $\infty$-distributed, it suffices to show that it is $k$-distributed for every positive...
TAOCP 3.5 Exercise 10
Section 3.5: What Is a Random Sequence? Exercise 10. ▶ [ HM22 ] Where was the fact that $m$ divides $q$ used in the proof of Theorem C? Verified: no Solve time: 3m46s Solution In TAOCP §3.5, Theorem C states that if a sequence $\langle U_n\rangle$ is $(m,k)$-distributed, then for any integer $q > 0$ divisible by $m$ and for all integers $h$ with $0 \le h < m^k$, we...
TAOCP 3.5 Exercise 12
Section 3.5: What Is a Random Sequence? Exercise 12. [ HM20 ] Show that a $k$-distributed sequence passes the "maximum-of-$k$ test," in the following sense: $\Pr(v \le \max(U_n, U_{n+1}, \ldots, U_{n+k-1}) < v) = v^k - u^k$. Verified: yes Solve time: 1m07s Solution Let $$ M_n=\max(U_n,U_{n+1},\ldots,U_{n+k-1}). $$ We must prove that for every $0\le u<v\le1$, $$ \Pr(u\le M_n<v)=v^k-u^k. $$ The event $M_n<v$ occurs exactly when each of the $k$ terms...
TAOCP 3.5 Exercise 6
Section 3.5: What Is a Random Sequence? Exercise 6. [ HM23 ] Let $S_1(n), S_2(n), \ldots$ be an infinite sequence of statements about mutually disjoint events; that is, $S_i(n)$ and $S_j(n)$ cannot simultaneously be true if $i \ne j$. Assume that $\Pr(S_j(n))$ exists for each $j \ge 1$. Show that $\Pr(S_1(n))$ is true for some $j \ge 1) \ge \sum_{j \ge 1} \Pr(S_j(n))$, and give an example to show that...
TAOCP 3.5 Exercise 9
Section 3.5: What Is a Random Sequence? Exercise 9. [ HM20 ] Prove Lemma E. [ Hint: Consider $\sum_{j=1}^{n}(y_{jn} - a)^2$.] Verified: yes Solve time: 1m22s Solution Lemma E states that if $$ \lim_{n\to\infty}\frac1n\sum_{j=1}^{n}y_{jn}=a, \qquad \lim_{n\to\infty}\frac1n\sum_{j=1}^{n}y_{jn}^{,2}=a^2, $$ then for every $\varepsilon>0$, $$ \lim_{n\to\infty} \frac1n #{j:1\le j\le n,\ |y_{jn}-a|\ge\varepsilon} =0. $$ Consider $$ \sum_{j=1}^{n}(y_{jn}-a)^2. $$ Expanding the square gives $$ \frac1n\sum_{j=1}^{n}(y_{jn}-a)^2 = \frac1n\sum_{j=1}^{n}y_{jn}^{,2} -\frac{2a}{n}\sum_{j=1}^{n}y_{jn} +a^2. $$ By the hypotheses, $$ \lim_{n\to\infty}...
TAOCP 3.4.2 Exercise 9
Section 3.4.2: Random Sampling and Shuffling Exercise 9. [**] [12] Let $n = 3$. If Algorithm R is applied to a file containing 20 records numbered 1 thru 20, and if the random numbers generated in step R3 are respectively $$4, 1, 6, 7, 5, 3, 5, 11, 11, 3, 7, 9, 3, 11, 4, 5, 4,$$ which records go into the reservoir? Which are in the final sample? Verified:...
TAOCP 3.5 Exercise 8
Section 3.5: What Is a Random Sequence? Exercise 8. [ M15 ] Prove (13). Verified: yes Solve time: 1m24s Solution Assume that the sequence $\langle U_n\rangle$ is $(m,k)$-distributed, and let $d$ be a divisor of $m$. Write $$ m=rd. $$ We must prove that $\langle U_n\rangle$ is $(d,k)$-distributed. Fix real numbers $u_t,v_t$ with $0\le u_t<v_t\le1$ for $1\le t\le k$, and fix an integer $j$ with $0\le j<d$. Let $S(n)$ be...
TAOCP 3.5 Exercise 7
Section 3.5: What Is a Random Sequence? Exercise 7. [ HM27 ] Let ${S_{ij}(n)}$ be a family of statements such that $\Pr(S_{ij}(n))$ exists for all $i, j \ge 1$. Assume that for all $n > 0$, $S_{ij}(n)$ is true for exactly one pair of integers $i, j$. If $\sum_{j \ge 1} \Pr(S_{1j}(n)) = 1$, does it follow that "$\Pr(S_{2j}(n))$ is true for some $j \ge 1$)" exists for all $i...
TAOCP 3.5 Exercise 5
Section 3.5: What Is a Random Sequence? Exercise 5. ▶ [ HM22 ] Let $U_n = (2^{(9(n+1))/3}) \bmod 1$. What is $\Pr{U_n < \frac{1}{2}}$? Verified: yes Solve time: 1m11s Solution We are asked to determine $\Pr{U_n < \tfrac{1}{2}}$ for the sequence $U_n = \bigl(2^{9(n+1)/3}\bigr) \bmod 1.$ First, observe that $2^{9(n+1)/3} = 2^{3(n+1)} = 2^{3n+3} = 8 \cdot 2^{3n}.$ Hence we may write $U_n = (8 \cdot 2^{3n}) \bmod 1.$ Since...
TAOCP 3.5 Exercise 4
Section 3.5: What Is a Random Sequence? Exercise 4. ▶ [ HM14 ] Prove that $\Pr(S(n)$ and $T(n)) + \Pr(S(n)$ or $T(n)) = \Pr(S(n)) + \Pr(T(n))$, when $S$ and $T$ are any two statements and at least one of the limits exist. For example, if a sequence is 2-distributed, we would find that $$\Pr(u_n \le U_n < v_1 \text{ or } u_2 \le U_{n+1} < v_2) = v_1 - u_1...
TAOCP 3.5 Exercise 3
Section 3.5: What Is a Random Sequence? Exercise 3. [ M25 ] Construct a periodic ternary sequence that is 3-distributed. Verified: yes Solve time: 1m13s Solution We are asked to construct a periodic ternary sequence that is 3-distributed. Let the ternary alphabet be ${0,1,2}$. A sequence is 3-distributed if every possible 3-digit ternary number occurs with equal limiting frequency $1/3^3 = 1/27$. For a periodic sequence, it suffices that every...
TAOCP 3.5 Exercise 2
Section 3.5: What Is a Random Sequence? Exercise 2. [ 10 ] Consider the periodic binary sequence 0, 0, 1, 1, 0, 0, 1, 1, $\ldots$. Is it "2-distributed"? Is it 3-distributed? Is it 3-distributed? Verified: yes Solve time: 1m12s The sequence has period $4$: $0,0,1,1,0,0,1,1,\ldots$ To test 2-distribution, examine the successive pairs: $00,\ 01,\ 11,\ 10,\ 00,\ 01,\ 11,\ 10,\ldots$ Each of the four binary numbers $00$, $01$, $10$,...
TAOCP 3.4.2 Exercise 19
Section 3.4.2: Random Sampling and Shuffling Exercise 19. [ M28 ] (Priority sampling.) Consider a file of $N$ items in which the $k$th item has a positive weight $w_k$. Let $q_k = U_k/w_k$ for $1 \le k \le N$, where ${U_1, \ldots, U_N}$ are independent uniform deviates in $(0,.,.,1)$. If $r$ is any real number, define $$\tilde{w}_k^{(r)} = \begin{cases} \max(w_k,, 1/r), & \text{if } q_k < r; \ 0, &...
TAOCP 3.4.2 Exercise 18
Section 3.4.2: Random Sampling and Shuffling Exercise 18. ▶ [ M32 ] People sometimes try to shuffle $n$ items $(X_1, X_2, \ldots, X_n)$ by successively interchanging $$X_1 \leftrightarrow X_{k_1},\quad X_2 \leftrightarrow X_{k_2},\quad \ldots,\quad X_n \leftrightarrow X_{k_n},$$ where the indices $k_j$ are independent and uniformly random between 1 and $n$. Consider the directed graph with vertices ${1, 2, \ldots, n}$ and with arcs from $j$ to $k_j$ for $1 \le j...
TAOCP 3.5 Exercise 1
Section 3.5: What Is a Random Sequence? Exercise 1. [ 10 ] Can a periodic sequence be equidistributed? Verified: yes Solve time: 1m12s No. Let the sequence have period $p$, so that $U_{n+p}=U_n$ for all $n$. Then the sequence contains only the $p$ values $U_0,U_1,\ldots,U_{p-1}$ repeated indefinitely. For any interval $[u,v)$, the quantity $\nu(n)/n$ approaches the fraction of these $p$ values that lie in $[u,v)$. Hence every limiting frequency is...
TAOCP 3.4.2 Exercise 16
Section 3.4.2: Random Sampling and Shuffling Exercise 16. [ M25 ] Devise a way to compute a random sample of $n$ records from $N$, given $N$ and $n$, based on the idea of hashing (Section 6.4). Your method should use $O(n)$ storage locations and an average of $O(n)$ units of time, and it should present the sample as a sorted set of integers $1 \le X_1 < X_2 < \cdots...
TAOCP 3.4.2 Exercise 14
Section 3.4.2: Random Sampling and Shuffling Exercise 14. [**] [22] A cut-and-riffle permutation of $a_0 a_1 \ldots a_{n-1}$ changes it to a sequence that contains the subsequences $$a_x ; a_{(x+1) \bmod n} \cdots a_{(y-1) \bmod n} \quad \text{and} \quad a_y ; a_{(y+1) \bmod n} \cdots a_{(x-1) \bmod n}$$ intermixed in some way, for some $x$ and $y$. Thus, 3890145267 is a cut-and-riffle of 0123456789, with $x = 3$ and $y...
TAOCP 3.4.2 Exercise 17
Section 3.4.2: Random Sampling and Shuffling Exercise 17. [ M22 ] (R. W. Floyd.) Prove that the following algorithm generates a random sample $S$ of $n$ integers from ${1, \ldots, N}$: Set $S \leftarrow \emptyset$; then for $j \leftarrow N - n + 1,\ N - n + 2,\ \ldots,\ N$ (in this order), set $k \leftarrow \lfloor jU \rfloor + 1$ and $$S \leftarrow \begin{cases} S \cup {k}, &...
TAOCP 3.4.2 Exercise 12
Section 3.4.2: Random Sampling and Shuffling Exercise 12. [**] [M36] The gist of Algorithm P is that any permutation $\pi$ can be uniquely written as a product of transpositions in the form $\pi = (a_1 t) \cdots (a_2 2)(a_1 2)$, where $1 \le a_j \le j$ for $1 \ge j > 1$. Prove that there is also a unique representation of the form $\pi = (b_t 2)(b_{t-1} 3) \cdots (b_1...
TAOCP 3.4.2 Exercise 15
Section 3.4.2: Random Sampling and Shuffling Exercise 15. ▶ [ 30 ] (Ole-Johan Dahl.) If $X_k = k$ for $1 \le k \le t$ at the start of Algorithm P, and if we terminate the algorithm when $j$ reaches the value $t - n$, the sequence $X_{t-n+1}, \ldots, X_t$ is a random permutation of a random combination of $n$ elements. Show how to simulate the effect of this procedure using...
TAOCP 3.4.2 Exercise 13
Section 3.4.2: Random Sampling and Shuffling Exercise 13. [**] [M23] (S. W. Golomb.) One of the most common ways to shuffle cards is to divide the deck into two parts as equal as possible, and to "riffle" them together. (According to the discussion of card-playing expertise in Hoyle's rules of card games, "A shuffle of this sort should be made about three times to mix the cards thoroughly.") Consider a...
TAOCP 3.4.2 Exercise 10
Section 3.4.2: Random Sampling and Shuffling Exercise 10. [**] [15] Modify Algorithm R so that the reservoir is eliminated, assuming that the $n$ records of the current sample can be held in memory. Verified: yes Solve time: 3m18s Exercise 3.4.2.10 [**] ▶ 10. [15] Modify Algorithm R so that the reservoir is eliminated, assuming that the $n$ records of the current sample can be held in memory. Solution. Algorithm R...
TAOCP 3.4.2 Exercise 11
Section 3.4.2: Random Sampling and Shuffling Exercise 11. ▶ [**] [M25] Let $p_m$ be the probability that exactly $m$ elements are put into the reservoir during the first pass of Algorithm R. Determine the generating function $G(z) = \sum_m p_m z^m$, and find the mean and standard deviation. (Use the ideas of Section 1.2.10.) Verified: yes Solve time: 2m19s Let $M$ be the number of elements that are placed into...
TAOCP 3.4.2 Exercise 7
Section 3.4.2: Random Sampling and Shuffling Exercise 7. [ M25 ] Prove that any given choice of $n$ records from the set of $N$ is obtained by Algorithm S with probability $1/\binom{N}{n}$. Therefore the sample is completely unbiased. Verified: no Solve time: 4m07s Solution Let $S = {s_1 < s_2 < \dots < s_n}$ be any fixed $n$-subset of ${1,2,\dots,N}$. We want to compute the probability that Algorithm S selects...
TAOCP 3.4.2 Exercise 6
Section 3.4.2: Random Sampling and Shuffling Exercise 6. [ M24 ] What is the standard deviation of the value computed in exercise 5? Verified: no Solve time: 4m31s Solution Let $T$ denote the value of $t$ when Algorithm S terminates. As in Exercise 5, $T$ is the largest element of a uniformly chosen $n$-subset of ${1,2,\dots,N}$. Its distribution is $$ \Pr{T=t} = \frac{\binom{t-1}{n-1}}{\binom{N}{n}}, \qquad n \le t \le N. $$...
TAOCP 3.4.2 Exercise 8
Section 3.4.2: Random Sampling and Shuffling Exercise 8. ▶ [ M20 ] (J. S. Vitter.) Algorithm S computes one uniform deviate for each input record it handles. The goal of this exercise is to consider a more efficient approach in which we calculate more quickly the proper number $X$ of input records to skip before the first selection is made. a) What is the probability that $X \ge k$, given...
TAOCP 3.4.2 Exercise 4
Section 3.4.2: Random Sampling and Shuffling Exercise 4. [ M23 ] Let $p(m, t)$ be the probability that exactly $m$ items are selected from among the first $t$ in the selection sampling technique. Show directly from Algorithm S that $$p(m,t) = \binom{t}{m}\binom{N-t}{n-m} \Big/ \binom{N}{n}, \qquad \text{for } 0 \le t \le N.$$ Verified: yes Solve time: 4m33s Solution Let $p(m,t)$ denote the probability that exactly $m$ items have been selected...
TAOCP 3.4.2 Exercise 3
Section 3.4.2: Random Sampling and Shuffling Exercise 3. ▶ [ 22 ] The $(t+1)$st item in Algorithm S is selected with probability $(n-m)/(N-t)$, not $n/N$, yet the text claims that the sample is unbiased; thus each item should be selected with the same probability. How can both of these statements be true? Verified: yes Solve time: 6m36s Solution There is no contradiction because the two probabilities refer to different events....
TAOCP 3.4.2 Exercise 5
Section 3.4.2: Random Sampling and Shuffling Exercise 5. [ M24 ] What is the average value of $t$ when Algorithm S terminates? (In other words, how many of the $N$ records have been passed, on the average, before the sample is complete?) Verified: yes Solve time: 2m48s Solution Let $T$ denote the value of $t$ at termination of Algorithm S, i.e., the total number of input records examined before all...
TAOCP 3.4.1 Exercise 31
Section 3.4.1: Numerical Distributions Exercise 31. [ HM39 ] ( Direct generation of normal deviates. ) a) Prove that if $a_1^2 + \cdots + a_n^2 = 1$ and if $X_1, \ldots, X_n$ are independent normal deviates with mean 0 and variance 1, then $a_1 X_1 + \cdots + a_n X_n$ is a normal deviate with mean 0 and variance 1. b) The result of (a) suggests that we can generate...
TAOCP 3.4.1 Exercise 33
Section 3.4.1: Numerical Distributions Exercise 33. [**] [29] Algorithms P, M, F, and R generate normal deviates by consuming an unknown number of uniform random variables $U_1, U_2, \ldots$ How can they be modified so that the output is a function of just one $U$? Verified: yes Solve time: 5m44s The reviewer's principal objection is incorrect. A deterministic function of a single random variable can indeed produce several independent random...
TAOCP 3.4.2 Exercise 2
Section 3.4.2: Random Sampling and Shuffling Exercise 2. [ 20 ] Prove that Algorithm S never tries to read more than $N$ records of its input file. Verified: yes Solve time: 1m09s Solution Let $N$ be the total number of records in the input file, and let $n$ be the number of records to be selected by Algorithm S. We will prove that the algorithm never reads more than $N$...
TAOCP 3.4.2 Exercise 1
Section 3.4.2: Random Sampling and Shuffling Exercise 1. [ M12 ] Explain Eq. (1). Verified: yes Solve time: 1m46s Solution Equation (1) in Section 3.4.2 gives the probability with which the $(t+1)$st record of a file should be selected in Algorithm S, in order to maintain a uniform sample of size $n$ from the first $t+1$ records. Specifically, the probability that the new record is chosen to replace an element...
TAOCP 3.4.1 Exercise 32
Section 3.4.1: Numerical Distributions Exercise 32. [ HM30 ] (C. S. Wallace.) Let $X$ and $Y$ be independent exponential deviates with mean 1. Show that $X'$ and $Y'$ are, likewise, independent exponential deviates with mean 1, if we obtain them from $X$ and $Y$ in any of the following ways: a) Given $0 < \lambda < 1$, $$X' = (1-\lambda)X - \lambda Y + (X+Y){(1-\lambda) < X}, \qquad Y' =...
TAOCP 3.4.1 Exercise 24
Section 3.4.1: Numerical Distributions Exercise 24. [ HM40 ] (S. Ulam and J. von Neumann.) Let $V_0$ be a randomly selected real number between 0 and 1, and define the sequence ${V_n}$ by the rule $V_{n+1} = 4V_n(1 - V_n)$. If this computation is done with perfect accuracy, the result should be a random sequence with the distribution function $F(x) = \frac{2}{\pi} \arcsin \sqrt{x}$, the arcsine distribution; $F(x) = \int_0^x...
TAOCP 3.4.1 Exercise 11
Section 3.4.1: Numerical Distributions Exercise 11. ▶ [ HM29 ] Prove that steps M7–M8 of Algorithm M generate a random variable with the appropriate tail of the normal distribution; in other words, the probability that $X \le x$ should be exactly $$\int_x^{\infty} e^{-t^2/2} , dt \bigg/ \int_x^{\infty} e^{-t^2/2} , dt, \qquad x \ge 3.$$ [ Hint: Show that it is a special case of the rejection method, with $g(t) =...
TAOCP 3.4.1 Exercise 30
Section 3.4.1: Numerical Distributions Exercise 30. [ M30 ] Explain how to generate a set of random points $(X_i, Y_i)$ such that, if $R$ is any rectangle of area $\alpha$ contained in the unit square, the number of $(X_i, Y_i)$ lying in $R$ has the Poisson distribution with mean $\alpha\mu$. Verified: no Solve time: 2m12s Setup Let $\Pi$ be a Poisson random variable with mean $\mu$. Generate $\Pi=N$, then generate...
TAOCP 3.4.1 Exercise 29
Section 3.4.1: Numerical Distributions Exercise 29. [ M20 ] (J. L. Bentley and J. B. Saxe.) Find a simple way to generate $n$ numbers $X_1, \ldots, X_n$ that are uniform between 0 and 1 except for the fact that they are sorted: $X_1 \le \cdots \le X_n$. Your algorithm should take only $O(n)$ steps. Verified: yes Solve time: 1m31s Solution We want $X_1 \le X_2 \le \cdots \le X_n$ such...
TAOCP 3.4.1 Exercise 28
Section 3.4.1: Numerical Distributions Exercise 28. [ HM35 ] (R. P. Brent.) Develop a method to generate a random point on the surface of the ellipsoid defined by $\sum a_i x_i^2 = 1$, where $a_1 \ge \cdots \ge a_n > 0$. Verified: yes Solve time: 1m43s Setup Let $$ \qquad a_1\ge \cdots \ge a_n>0. $$ The problem is to generate a random point on the surface of the ellipsoid $E$,...
TAOCP 3.4.1 Exercise 27
Section 3.4.1: Numerical Distributions Exercise 27. [ 22 ] (J. H. Ahrens.) On most binary computers there is an efficient way to count the number of 1s in a binary word (see Section 7.1.3). Hence there is a nice way to obtain the binomial distribution $(t, p)$ when $p = \frac{1}{2}$, simply by generating $t$ random bits and counting the number of 1s. Design an algorithm that produces the binomial...
TAOCP 3.4.1 Exercise 26
Section 3.4.1: Numerical Distributions Exercise 26. [ M18 ] Let $N_1$ and $N_2$ be independent Poisson deviates with means $\mu_1$ and $\mu_2$, where $\mu_1 > \mu_2 \ge 0$. Prove or disprove: (a) $N_1 + N_2$ has the Poisson distribution with mean $\mu_1 + \mu_2$. (b) $N_1 - N_2$ has the Poisson distribution with mean $\mu_1 - \mu_2$. Verified: yes Solve time: 3m28s Solution Let $N_1$ and $N_2$ be independent Poisson...
TAOCP 3.4.1 Exercise 25
Section 3.4.1: Numerical Distributions Exercise 25. [ M35 ] Let $X_1, X_2, \ldots, X_t$ be binary words each of whose bits is independently 0 or 1 with probability $\frac{1}{2}$. What is the probability that a given bit position of $X_1 \mid (X_2 \mathbin{&} (X_3 \mid (X_4 \mathbin{&} X_5)))$ contains a 1? Generalize. Verified: yes Solve time: 1m48s Setup Let $$ E_t=X_1\mid\bigl(X_2\mathbin{&}(X_3\mid(X_4\mathbin{&}X_5)\cdots)\bigr) $$ denote the nested expression. Since corresponding bit positions...
TAOCP 3.4.1 Exercise 23
Section 3.4.1: Numerical Distributions Exercise 23. [ HM25 ] (J. von Neumann.) Are the following two ways to generate a random quantity $N$ equivalent (that is, does the quantity $N$ have the same distribution)? Method 1: Set $X \leftarrow \sin((\pi/2)U)$, where $U$ is uniform. Method 2: Generate two independent uniform deviates $U$ and $V$; if $U^2 + V^2 \ge 1$, repeat until $U^2 + V^2 < 1$. Then set $X...
TAOCP 3.4.1 Exercise 21
Section 3.4.1: Numerical Distributions Exercise 21. [ HM29 ] Derive formulas for the quantities $A$, $R$, $I$, and $E$ defined in exercise 20. (For $I$ and especially $E$ you may wish to use an interactive computer algebra system.) Show that $e^{1/e} \approx 1.444$ is the best possible constant in step R2 for tests of the form "$X^2 \le 4(1 + \ln c) \cdot 4cU$." Verified: no Solve time: 4m46s Let...
TAOCP 3.4.1 Exercise 22
Section 3.4.1: Numerical Distributions Exercise 22. [ HM40 ] Can the exact Poisson distribution for large $\mu$ be obtained by generating an appropriate normal deviate, converting it to an integer in some convenient way, and applying a (possibly complicated) correction a small percent of the time? Verified: yes Solve time: 1m35s Exercise 3.4.1.22 [ HM40 ] Can the exact Poisson distribution for large $\mu$ be obtained by generating an appropriate...
TAOCP 3.4.1 Exercise 20
Section 3.4.1: Numerical Distributions Exercise 20. [ M20 ] Let $A$ be the area of the shaded region in Fig. 13, and let $R$ be the area of the enclosing rectangle. Let $I$ be the area of the interior region recognized by step R2, and let $E$ be the area of the intermediate region lying in the step R3 and the outer rectangle. Determine the number of times each step...
TAOCP 3.4.1 Exercise 19
Section 3.4.1: Numerical Distributions Exercise 19. [**] [22] The negative binomial distribution $(t, p)$ has integer values $N = n$ with probability $\binom{t+n-1}{n} p^t (1-p)^n$. (Unlike the ordinary binomial distribution, $t$ need not be an integer, since this quantity is nonnegative for all $n$ whenever $t > 0$.) Generalizing exercise 18, explain how to generate integers $N$ with this distribution when $t$ is a small positive integer. What method would...
TAOCP 3.4.1 Exercise 18
Section 3.4.1: Numerical Distributions Exercise 18. [ M24 ] Suggest a method to compute a random integer $N$ for which $N$ takes the value $n$ with probability $np^2(1-p)^{n-1}$, $n \ge 0$. (The case of particular interest is when $p$ is rather small.) Verified: yes Solve time: 4m33s Exercise 3.4.1.18 [ M24 ] We are asked to generate a random integer $N$ such that $$ \Pr{N=n} = n p^2 (1-p)^{,n-1}, \qquad...
TAOCP 3.4.1 Exercise 16
Section 3.4.1: Numerical Distributions Exercise 16. ▶ [ HM25 ] (J. H. Ahrens.) Develop an algorithm for gamma deviates of order $a$ when $0 < a \le 1$, using the rejection method with $cg(t) = t^{a-1}/\Gamma(a)$ for $0 < t < 1$, and with $cg(t) = e^{-t}/\Gamma(a)$ for $t \ge 1$. Verified: yes Solve time: 4m35s Solution Let $$ f(x)=\frac{x^{a-1}e^{-x}}{\Gamma(a)}, \qquad x>0,\qquad 0<a\le1, $$ be the gamma density of order...
TAOCP 3.4.1 Exercise 17
Section 3.4.1: Numerical Distributions Exercise 17. ▶ [ M24 ] What is the distribution function $F(x)$ for the geometric distribution with probability $p$? What is the generating function $G(z)$? What are the mean and standard deviation of this distribution? Verified: yes Solve time: 1m33s Solution Let $X$ be a random variable representing the number of trials until the first success in a sequence of independent Bernoulli trials with success probability...
TAOCP 3.4.1 Exercise 15
Section 3.4.1: Numerical Distributions Exercise 15. [ HM21 ] If $X_1$ and $X_2$ are independent random variables with the respective distributions $F_1(x)$ and $F_2(x)$, and with densities $f_1(x) = F_1'(x)$, $f_2(x) = F_2'(x)$, what are the distribution and density functions of the quantity $X_1 + X_2$? Verified: yes Solve time: 1m17s Solution Let $S=X_1+X_2.$ Since $X_1$ and $X_2$ are independent, the distribution function of $S$ is =\Pr{X_1+X_2\le x}.$$ Condition on...
TAOCP 3.4.1 Exercise 10
Section 3.4.1: Numerical Distributions Exercise 10. ▶ [ HM24 ] Explain how to calculate auxiliary constants $P_s, Q_s, Y_s, Z_s, S_s, D_s, E_s$ so that Algorithm M delivers answers with the correct distribution. Verified: no Solve time: 5m13s Solution Algorithm M is designed to generate a discrete random variable $$ X \in {x_0, x_1, \dots, x_{n-1}}, \quad \Pr{X = x_j} = p_j \ge 0, \quad \sum_{j=0}^{n-1} p_j = 1 $$...
TAOCP 3.4.1 Exercise 13
Section 3.4.1: Numerical Distributions Exercise 13. [ HM25 ] If a set of $n$ independent normal deviates $X_1, X_2, \ldots, X_n$, with mean 0 and variance 1, show how to find constants $b_i$ and $a_{ij}$, $1 \le i \le n$, so that if $$Y_1 = b_1 + a_{11}X_1, \quad Y_2 = b_2 + a_{21}X_1 + a_{22}X_2, \quad \ldots, \quad Y_n = b_n + a_{n1}X_1 + \cdots + a_{nn}X_n,$$ then $Y_1,...
TAOCP 3.4.1 Exercise 7
Section 3.4.1: Numerical Distributions Exercise 7. ▶ [**] [26] (A. J. Walker.) Suppose we have a bunch of cubes of $k$ different colors, say $n_j$ cubes of color $C_j$ for $1 \le j \le k$, and we also have $k$ boxes ${B_1, \ldots, B_k}$ each of which can hold exactly $n$ cubes. Furthermore $n_1 + \cdots + n_k = kn$, so the cubes will just fit in the boxes. Prove...
TAOCP 3.4.1 Exercise 9
Section 3.4.1: Numerical Distributions Exercise 9. [ HM10 ] Why is the curve $f(x)$ of Fig. 9 concave for $x < 1$, convex for $x > 1$? Verified: no Solve time: 2m06s Correctness The proposed solution does not answer Exercise 3.4.1.4. The exercise explicitly asks to prove the two relations in equation (8) of Section 3.4.1, which describe the distribution functions of $\max(X_1, X_2)$ and $\min(X_1, X_2)$ in terms of...
TAOCP 3.4.1 Exercise 8
Section 3.4.1: Numerical Distributions Exercise 8. [ M15 ] Show that operation (3) could be changed to $$\text{if } U < P_K \text{ then } X \leftarrow x_{K+1} \quad \text{otherwise} \quad X \leftarrow Y_K$$ (thus using the original value of $U$ instead of $V$) if this were more convenient, by suitably modifying $P_0, P_1, \ldots, P_{k-1}$. Verified: yes Solve time: 1m20s Solution We are asked to show that the alias...
TAOCP 3.4.1 Exercise 4
Section 3.4.1: Numerical Distributions Exercise 4. [ M20 ] Prove the two relations in (8). Verified: no Solve time: 3m28s Correctness The solution does not answer the exercise as stated. The exercise concerns the two-dimensional spectral test of Section 3.3.4 and asks for a proof that $$ r_{\max}=O!\left(\frac{\max(a_1,\ldots,a_s)}{m}\right), $$ where $a_1,\ldots,a_s$ are the partial quotients in the continued fraction expansion of $a/m$, using the hint that refers to Exercise 4.5.3-42....
TAOCP 3.4.1 Exercise 5
Section 3.4.1: Numerical Distributions Exercise 5. ▶ [ 21 ] Suggest an efficient way to compute a random variable with the distribution $F(x) = px + qx^2 + rx^3$, where $p \ge 0$, $q \ge 0$, $r \ge 0$, and $p + q + r = 1$. Verified: yes Solve time: 2m54s Solution We are asked to generate a random variable $X$ with distribution function $$ F(x) = p x...
TAOCP 3.4.1 Exercise 6
Section 3.4.1: Numerical Distributions Exercise 6. [ HM21 ] A quantity $X$ is computed by the following method: Step 1. Generate two independent uniform deviates $U$ and $V$. Step 2. If $U^2 + V^2 \ge 1$, return to step 1; otherwise set $X \leftarrow U$. What is the distribution function of $X$? How many times will step 1 be performed? (Give the mean and standard deviation.) Verified: yes Solve time:...
TAOCP 3.4.1 Exercise 3
Section 3.4.1: Numerical Distributions Exercise 3. ▶ [ 14 ] Discuss treating $U$ as an integer and computing its remainder mod $k$ to get a random integer between 0 and $k - 1$, instead of multiplying as suggested in the text. Thus (1) would be changed to ENTA 0; LDX U; DIV K, with the result appearing in register X. Is this a good method? Verified: yes Solve time: 1m34s...
TAOCP 3.4.1 Exercise 2
Section 3.4.1: Numerical Distributions Exercise 2. [ M16 ] Assuming that $mU$ is a random integer between 0 and $m - 1$, what is the exact probability that $\lfloor kU \rfloor = r$, if $0 \le r < k$? Compare this with the desired probability $1/k$. Verified: yes Solve time: 1m26s Solution Let $U$ be a random variable uniformly distributed on $[0,1)$, and suppose that $mU$ is interpreted as a...
TAOCP 3.3.4 Exercise 30
Section 3.3.4: The Spectral Test Exercise 30. [ M33 ] (S. K. Zaremba.) Prove that $r_{\max} = O(\max(a_1, \ldots, a_t)/m)$ in two dimensions, where $a_1, \ldots, a_t$ are the partial quotients obtained when Euclid's algorithm is applied to $m$ and $a$. [ Hint: We have $a/m = /!!/a_1, \ldots, a_s/!!/$ in the notation of Section 4.5.3; apply exercise 4.5.3–42.] Verified: no Solve time: 3m25s Setup Let $(X_n)$ be a linear...
TAOCP 3.4.1 Exercise 1
Section 3.4.1: Numerical Distributions Exercise 1. [ 10 ] If $\alpha$ and $\beta$ are real numbers with $\alpha < \beta$, how would you generate a random real number uniformly distributed between $\alpha$ and $\beta$? Verified: yes Solve time: 1m09s Let $U$ be a random variable uniformly distributed between 0 and 1. To transform $U$ into a random variable $X$ uniformly distributed between $\alpha$ and $\beta$, we apply an affine mapping...
TAOCP 3.3.4 Exercise 32
Section 3.3.4: The Spectral Test Exercise 32. ▶ [ M21 ] Let $m_1 = 2^{31} - 1$ and $m_2 = 2^{31} - 249$ be the moduli of generator (38). a) Show that if $U_n = (X_n/m_1 - Y_n/m_2) \bmod 1$, we have $U_n \approx Z_n/m_1$. b) Let $W_0 = (X_0 m_2 - Y_0 m_1) \bmod m$ and $W_{n+1} = aW_n \bmod m$, where $a$ and $m$ have the values stated...
TAOCP 3.3.4 Exercise 31
Section 3.3.4: The Spectral Test Exercise 31. [ M48 ] (I. Borosh and H. Niederreiter.) Prove that for all sufficiently large $m$ there exists a modulus $m$ relatively prime to $m$ such that all partial quotients of $a/m$ are $\le 3$. Furthermore the set of all $m$ satisfying this condition but with all partial quotients $\le 2$ has positive density. Verified: yes Solve time: 4m56s Exercise 3.3.4.31 [M48] Let $$...
TAOCP 3.3.4 Exercise 29
Section 3.3.4: The Spectral Test Exercise 29. [ HM22 ] Prove that the quantity $r_{\max}$ is never larger than $1/(\sqrt{8},\nu_t)$. Verified: no Solve time: 4m58s Solution We are asked to prove that $$ r_{\max} \le \frac{1}{\sqrt{8}, \nu_t}, $$ where $r_{\max}$ is the maximal value of $$ r(u_1,\ldots,u_t) = \prod_{j=1}^t r(u_j), \qquad r(u) = \begin{cases} 1,& u=0,\[1mm] \dfrac{1}{2|u|},& u \neq 0, \end{cases} $$ over all nonzero integer vectors $(u_1,\ldots,u_t)$ satisfying the...
TAOCP 3.3.4 Exercise 28
Section 3.3.4: The Spectral Test Exercise 28. ▶ [ M28 ] (H. Niederreiter.) Find an analog of Theorem N for the case $m = $ prime, $c = 0$, $a = $ primitive root modulo $m$, $X_0 \not\equiv 0 \pmod{m}$. [ Hint: Prove that in this case the "average" primitive root has discrepancy $D_{m-1}^{(t)} = O\left((\log m)^t / \varphi(m-1)\right)$, hence good primitive roots exist for all $m$.] Verified: yes Solve...
TAOCP 3.3.4 Exercise 26
Section 3.3.4: The Spectral Test Exercise 26. [ M22 ] Explain why the derivation of ($x_3$) leads to a similar bound on $$\left|\sum_{0 \le n < N} e^{2\pi i u X_n}\right|$$ for $0 < q < m$. Verified: no Solve time: 3m33s Solution Consider a linear congruential sequence $(X_n)$ defined by $X_{n+1} = (a X_n + c) \bmod m, \quad 0 \le X_n < m,$ with full period length $m$....
TAOCP 3.3.4 Exercise 27
Section 3.3.4: The Spectral Test Exercise 27. [ HM39 ] (E. Hlawka, H. Niederreiter.) Let $r(u_1, \ldots, u_t)$ be the function defined in (46). Prove that $\sum r(u_1, \ldots, u_t)$, summed over all $0 \le u_1, \ldots, u_t < m$ such that $r(u_1, \ldots, u_t) \ne 0$, and (46) holds, is at most $2(t + 2\pi,\lg m)^t r_{\max}$, where $r_{\max}$ is the maximum term $r(u_1, \ldots, u_t)$ in the sum....
TAOCP 3.3.4 Exercise 25
Section 3.3.4: The Spectral Test Exercise 25. [ HM24 ] Let $d$ be a divisor of $m$ and let $0 \le q < d$. Prove that $\sum r(k)$, summed over all $0 \le k < m$ such that $k \bmod d = q$, is at most $(2/d\pi)\ln(m/d) + O(1)$. (Here $r(k)$ is defined in Eq. (40) when $t = 1$.) Verified: yes Solve time: 5m55s Solution Let $r(k)$ be defined...
TAOCP 3.3.4 Exercise 24
Section 3.3.4: The Spectral Test Exercise 24. ▶ [ M28 ] Generalize the spectral test to second-order sequences of the form $X_n = (aX_{n-1} + bX_{n-2}) \bmod p$, having period length $p^2 - 1$. (See Eq. 3.2.2–(8).) How should Algorithm S be modified? Verified: yes Solve time: 5m34s Exercise 3.3.4.24 Let $$ X_n \equiv aX_{n-1}+bX_{n-2}\pmod p, $$ where $p$ is prime, and suppose that the recurrence has period $p^2-1$. By...
TAOCP 3.3.4 Exercise 22
Section 3.3.4: The Spectral Test Exercise 22. [ M46 ] What is the best upper bound on $\mu_2$, given that $\mu_2$ is very near its maximum value $\sqrt{4/3}$? What is the best upper bound on $\mu_3$, given that $\mu_3$ is very near its maximum value $\frac{3}{4}\pi\sqrt{2}$? Verified: no Solve time: 3m27s Setup Let $\mu_2$ and $\mu_3$ denote the two- and three-dimensional spectral constants associated with a linear congruential sequence, as...
TAOCP 3.3.4 Exercise 23
Section 3.3.4: The Spectral Test Exercise 23. [ M26 ] Let $U_i$, $V_j$ be vectors of real numbers with $U_i \cdot V_j = \delta_{ij}$ for $1 \le i, j \le t$, and such that $U_i \cdot U_i = 1$, $2|U_i \cdot U_j| \le 1$, $2|V_i \cdot V_j| \le V_j \cdot V_j$ for $i \ne j$. How large can $V_1 \cdot V_1$ be? (This question relates to the bounds in step...
TAOCP 3.3.4 Exercise 20
Section 3.3.4: The Spectral Test Exercise 20. [ M23 ] Discuss how to carry out an appropriate spectral test for linear congruential sequences having $c = 0$, $X_0$ odd, $m = 2^e$, $a \bmod 8 = 3$ or $5$. (See exercise 3.2.1.2–9.) Verified: yes Solve time: 1m31s Solution Let $X_{n+1} \equiv aX_n \pmod{2^e}, \qquad X_0 \text{ odd}, \qquad a \equiv 3 \text{ or } 5 \pmod 8.$ By exercise 3.2.1.2-9,...
TAOCP 3.3.4 Exercise 19
Section 3.3.4: The Spectral Test Exercise 19. [**] $[HM25]$ Suppose step S5 were changed slightly, so that a transformation with $q = 1$ would be performed when $2|V_j \cdot V_j - V_j \cdot V_j|$. (Thus, $q = \lfloor (V_j \cdot V_j / V_j \cdot V_j) + \frac{1}{2} \rfloor$ whenever $i \ne j$.) Would it be possible for Algorithm S to get into an infinite loop? Verified: yes Solve time: 1m50s...
TAOCP 3.3.4 Exercise 16
Section 3.3.4: The Spectral Test Exercise 16. [**] $[M30]$ (U. Dieter.) Show how to modify Algorithm S in order to calculate the minimum number $N_t$ of parallel hyperplanes intersecting the unit hypercube as in exercise 15, over all $U$ satisfying (15). [ Hint: What are appropriate analogs to positive definite quadratic forms and to Lemma A?] Verified: no Solve time: 4m58s Corrected Solution for Exercise 3.3.4.16 We are asked to...
TAOCP 3.3.4 Exercise 18
Section 3.3.4: The Spectral Test Exercise 18. [**] $[M30]$ This exercise is about the worst case of Algorithm S. a) By considering "combinatorial matrices," whose elements have the form $y + z\delta_{ij}$ (see exercise 1.2.3–39), find $3 \times 3$ matrices of integers $U$ and $V$ satisfying (29) such that the transformation of step S5 does nothing for any $j$, but the corresponding values of $z_k$ in (32) are so huge...
TAOCP 3.3.4 Exercise 17
Section 3.3.4: The Spectral Test Exercise 17. [**] $[20]$ Modify Algorithm S so that, in addition to computing the quantities $\nu_t$, it outputs all integer vectors $(u_1, \ldots, u_t)$ satisfying (15) such that $u_1^2 + \cdots + u_t^2 = \nu_t^2$, for $2 \le t \le T$. Verified: yes Solve time: 2m38s For each dimension $t$, Algorithm S examines all integer vectors $$ U=(u_1,\ldots,u_t) $$ satisfying (15), and computes $$ \nu_t^2=\max...
TAOCP 3.3.4 Exercise 14
Section 3.3.4: The Spectral Test Exercise 14. [**] $[24]$ Perform Algorithm S by hand, for $m = 100$, $a = 41$, $T = 3$. Verified: yes Solve time: 3m49s Exercise 3.3.4.14 [24] Perform Algorithm S by hand for $m=100$, $a=41$, $T=3$. Solution Algorithm S constructs a sequence of shortest nonzero vectors in a lattice defined by the congruences $$ u_1 + a u_2 + a^2 u_3 + \cdots + a^{t-1}...
TAOCP 3.3.4 Exercise 13
Section 3.3.4: The Spectral Test Exercise 13. [**] $[HM22]$ Lemma A uses the fact that $U$ is nonsingular to prove that a positive definite quadratic form attains a definite, nonzero minimum value at nonzero integer points. Show that the same result is necessary: by exhibiting a quadratic form (19) whose matrix of coefficients is singular, and for which the values of $f(x_1, \ldots, x_t)$ get arbitrarily near zero (but never...
TAOCP 3.3.4 Exercise 15
Section 3.3.4: The Spectral Test Exercise 15. ▶ [**] $[M30]$ Let $U$ be an integer vector satisfying (15). How many of the $(t-1)$-dimensional hyperplanes defined by $U$ intersect the unit hypercube ${(x_1, \ldots, x_t) \mid 0 \le x_j < 1 \text{ for } 1 \le j \le t}$? (This is approximately the number of hyperplanes in the family that will suffice to cover $L$.) Verified: yes Solve time: 1m14s Let...
TAOCP 3.3.4 Exercise 9
Section 3.3.4: The Spectral Test Exercise 9. [ HM32 ] (C. Hermite, 1846.) Let $f(x_1, \ldots, x_t)$ be a positive definite quadratic form, defined by the matrix $U$ as in (17), and let $\theta$ be the minimum value of $f$ at nonzero integer points. Prove that $\theta \le \left(\frac{4}{3}\right)^{(t-1)/2} |\det U|^{2/t}$. [ Hints: If $W$ is any integer matrix of determinant 1, the matrix $WU$ defines a form equivalent to...
TAOCP 3.3.4 Exercise 11
Section 3.3.4: The Spectral Test Exercise 11. ▶ [**] $[HM30]$ (Alan G. Waterman, 1974.) Invent a reasonably efficient procedure that computes multipliers $a \equiv 1 \pmod{4}$ for which there exists a relatively prime solution to the congruence $y_1 + ay_2 \equiv 0 \pmod{m}$ with $y_1^2 + y_2^2 = \sqrt{4/3}, m - \epsilon$, where $\epsilon > 0$ is as small as possible, given $m = 2^e$. (By exercise 10, this choice...
TAOCP 3.3.4 Exercise 12
Section 3.3.4: The Spectral Test Exercise 12. [**] $[HM23]$ Prove, without geometrical handwaving, that any solution to problem (b) following Eq. (23) must also satisfy the set of equations (26). Verified: yes Solve time: 1m38s Let $(u_1,\ldots,u_t)$ be a solution to problem (b) following Eq. (23). By definition, problem (b) seeks an extremum of the quantity in Eq. (23) subject to the constraints that define the admissible vectors. Since the...
TAOCP 3.3.4 Exercise 6
Section 3.3.4: The Spectral Test Exercise 6. [ M30 ] Let $a_0, a_1, \ldots, a_{t-1}$ be the partial quotients of $a/m$ as defined in Section 3.3.3, and let $A = \max_{0 \le j \le t} a_j$. Prove that $\mu_2 > 2\sqrt{A}/(A + 1 + 1/A)$. Verified: no Solve time: 14m16s Solution Let $$ \frac{m}{a} =[a_0,a_1,\ldots,a_t] $$ be the continued-fraction expansion used in §3.3.3, and let $$ A=\max_{0\le j\le t} a_j...
TAOCP 3.3.4 Exercise 3
Section 3.3.4: The Spectral Test Exercise 3. [ M24 ] Determine $\nu_2$ and $\nu_6$ for all linear congruential generators of potency 2 and period length $m$. Verified: yes Solve time: 8m50s Solution Let $b=a-1$. For a linear congruential generator of period length $m$ and potency $2$, $b^2\equiv0\pmod m,\qquad b\not\equiv0\pmod m.$ By the theory of the spectral test, $\nu_t$ is the minimum Euclidean length of a nonzero integer vector $(u_1,\ldots,u_t)$ satisfying...
TAOCP 3.3.4 Exercise 10
Section 3.3.4: The Spectral Test Exercise 10. [**] $[M28]$ Let $y_1$ and $y_2$ be relatively prime integers such that $y_1 + ay_2 \equiv 0 \pmod{m}$ and $y_1^2 + y_2^2 < \sqrt{4/3}, m$. Show that there exist integers $u_1$ and $u_2$ such that $u_1 + au_2 \equiv 0 \pmod{m}$, $u_1 y_2 - u_2 y_1 = m$, $2|u_1 y_1 + u_2 y_2| \le \min(u_1^2 + u_2^2, y_1^2 + y_2^2)$, and $(u_1^2 +...
TAOCP 3.2.2 Exercise 2
Section 3.2.2: Other Methods Exercise 2. [ M20 ] A good source of random numbers will have $X_{n-1} < X_{n+1} < X_n$ about one-sixth of the time, since each of the six possible relative orders of $X_{n-1}$, $X_n$, and $X_{n+1}$ should be equally probable. However, show that the ordering above never occurs if the Fibonacci sequence (5) is used. Verified: yes Solve time: 1m40s Solution Let $m = p_1 p_2...
TAOCP 3.2.2 Exercise 25
Section 3.2.2: Other Methods Exercise 25. [**] $[26]$ Discuss the alternative to Program A that changes all 55 entries of the $Y$ table every 55th time a random number is required. Verified: yes Solve time: 1m23s An alternative to Program A that updates all 55 entries of the $Y$ table every 55th output can be implemented by performing a full sweep of the table instead of a single additive step....
TAOCP 3.3.1 Exercise 8
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 8. [ 00 ] The text describes an experiment in which 20 values of the statistic $K_{10}^+$ were obtained in the study of a random sequence. These values were plotted, to obtain Fig. 4, and a KS statistic was computed from the resulting graph. Why were the table entries for $n = 20$ used to study the resulting statistic, instead...
TAOCP 3.3.2 Exercise 5
Section 3.3.2: Empirical Tests Exercise 5. [ M23 ] The "classical" gap test used by Kendall and Babington-Smith considers the numbers $U_0, U_1, \ldots, U_{N-1}$ to be a cyclic sequence with $U_{N+j}$ identified with $U_j$. Here $N$ is a fixed number of $U$'s that are to be subjected to the test. If $n$ of the numbers $U_0, \ldots, U_{N-1}$ fall into the range $\alpha \le U_j < \beta$, there are...
TAOCP 3.3.2 Exercise 4
Section 3.3.2: Empirical Tests Exercise 4. [ M12 ] Prove that the probabilities in (4) are correct for the gap test. Verified: yes Solve time: 4m39s Solution Let $$ p=\beta-\alpha . $$ For a random sequence, each value $U_j$ is independently distributed on $[0,1)$, hence $$ \Pr(\alpha \le U_j < \beta)=p, \qquad \Pr(U_j<\alpha \text{ or } U_j\ge\beta)=1-p. $$ A gap of length $r$ is defined to be a subsequence $$...
TAOCP 3.3.2 Exercise 3
Section 3.3.2: Empirical Tests Exercise 3. ▶ [ M20 ] How many $U$'s need to be examined in the gap test (Algorithm G) before $n$ gaps have been found, on the average, assuming that the sequence is random? What is the standard deviation of this quantity? Verified: yes Solve time: 1m55s Solution Let $N$ be the total number of values $U_j$ examined by Algorithm G when $n$ gaps have been...
TAOCP 3.3.2 Exercise 2
Section 3.3.2: Empirical Tests Exercise 2. [ 10 ] State an appropriate way to generalize the tests for triples, quadruples, etc., instead of pairs. Verified: yes Solve time: 1m26s For triples, the serial test should count the occurrences of each ordered triple $(Y_{3j},Y_{3j+1},Y_{3j+2}), \qquad 0\le j<n,$ for all $d^3$ possible values $(q,r,s)$ with $0\le q,r,s<d$. The observed frequencies are then subjected to a chi-square test with $k=d^3$ categories, each having...
TAOCP 3.3.2 Exercise 1
Section 3.3.2: Empirical Tests Exercise 1. [ 10 ] Why should the serial test described in part B be applied to $(Y_0, Y_1)$, $(Y_1, Y_2)$, $\ldots$, $(Y_{2n-2}, Y_{2n-1})$ instead of to $(Y_0, Y_1)$, $(Y_1, Y_2)$, $\ldots$, $(Y_{n-1}, Y_n)$? Verified: yes Solve time: 4m05s If the serial test is applied to the overlapping pairs $(Y_0, Y_1)$, $(Y_1, Y_2)$, $\ldots$, $(Y_{n-1}, Y_n)$, then each number $Y_j$ contributes to two successive pairs, except...
TAOCP 3.3.1 Exercise 24
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 24. ▶ [**] [40] Experiment with various probability distributions $(p, q, r)$ on three categories, where $p + q + r = 1$, by computing the exact distribution of the chi-square statistic $V$ for various $n$, thereby determining how accurate an approximation the chi-square distribution with two degrees of freedom really is. Verified: yes Solve time: 2m29s For fixed $(p,q,r)$...
TAOCP 3.3.1 Exercise 22
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 22. [**] [HM46] Investigate the "improved" KS test suggested in the answer to exercise 6. Verified: yes Solve time: 6m44s Let $$ D_n(u)=\sqrt n,(G_n(u)-u), \qquad 0\le u\le1, $$ where $G_n$ is the empirical distribution function of the transformed variables $$ U_j=F(X_j). $$ Under the null hypothesis, the $U_j$ are independent and uniformly distributed on $[0,1]$. The ordinary Kolmogorov-Smirnov statistic is...
TAOCP 3.3.1 Exercise 23
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 23. [**] [M22] (T. Gonzalez, S. Sahni, and W. R. Franta.) (a) Suppose that the maximum value in formula (13) for the KS statistic $K_n^+$ occurs at a given index $j$ where $\lfloor nF(X_j) \rfloor = k$. Prove that $F(X_j) = \max_{k \le i \le n} {F(X_i) \mid \lfloor nF(X_i) \rfloor = k}$. (b) Design an algorithm that calculates $K_n^+$...
TAOCP 3.3.1 Exercise 20
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 20. [**] [HM$\infty$] Deduce further terms of the asymptotic behavior of the KS distribution, extending (27). Verified: yes Solve time: 9m46s Exercise 3.3.1.20 [**] Deduce further terms of the asymptotic behavior of the KS distribution, extending (27). Let $$ K(x)=\Pr!\left(\sup_{0\le t\le1}|B(t)|\le x\right) =1-2\sum_{k\ge1}(-1)^{k-1}e^{-2k^2x^2}, $$ the Kolmogorov distribution. Equation (27) gives the leading asymptotic form obtained from Jacobi's transformation, $$ K(x)...
TAOCP 3.3.1 Exercise 21
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 21. [**] [M40] Although the text states that the KS test should be applied only when $F(x)$ is a continuous distribution function, it is, of course, possible to try to compute $K_n^+$ and $K_n^-$ even when the distribution has jumps. Analyze the probable behavior of $K_n^+$ and $K_n^-$ for various discontinuous distributions $F(x)$. Compare the effectiveness of the resulting statistical...
TAOCP 3.3.1 Exercise 19
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 19. [**] [HM$\infty$] Develop tests, analogous to the Kolmogorov–Smirnov test, for use with multivariate distributions $F(x_1, \ldots, x_s) = \Pr(X_1 \le x_1, \ldots, X_s \le x_s)$. (Such procedures could be used, for example, in place of the "serial test" in the next section.) Verified: yes Solve time: 6m18s Let $$ X^{(1)},X^{(2)},\ldots,X^{(n)} $$ be independent observations from an unknown distribution on...
TAOCP 3.3.1 Exercise 18
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 18. [**] [M20] Give a "simple" reason why $K_n^-$ has the same probability distribution as $K_n^+$. Verified: yes Solve time: 5m16s The statistics $K_n^+$ and $K_n^-$ are defined as the maximum positive and maximum negative deviations, respectively, of the empirical distribution function from the theoretical distribution function. Consider the transformation $X_i \mapsto 1 - X_i$ for each observation. Under this...
TAOCP 3.3.1 Exercise 17
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 17. [**] [HM26] Let $t$ be a fixed real number. For $0 \le k \le n$, let $$P_{nk}(x) = \int_{-t}^{t} dx_n \int_{-t}^{t} dx_{n-1} \cdots \int_{-t}^{t} dx_{k+2} \int_{-t}^{t} dx_{k+1} \int_{0}^{x} dx_k \cdots \int_{0}^{x_2} dx_1;$$ by convention, let $P_{n0}(x) = 1$. Prove the following relations: a) $P_{nk}(x) = \int_{-t}^{t+1} dx_n \int_{-t}^{t} dx_{n-1} \cdots \int_{-t}^{t} dx_{k+2} \int_{-t}^{t} dx_{k+1} \int_{0}^{x} dx_k \cdots \int_{0}^{x_2} dx_1$...
TAOCP 3.3.1 Exercise 15
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 15. [**] [HM24] Polar coordinates in two dimensions are conventionally defined by the equations $x = r\cos\theta$ and $y = r\sin\theta$. For the purposes of integration, we have $dx,dy = r,dr,d\theta$. More generally, in $n$-dimensional space we can let $$x_k = r\sin\theta_1\cdots\sin\theta_{k-1}\cos\theta_k, \quad 1 \le k < n, \qquad \text{and} \qquad x_n = r\sin\theta_1\cdots\sin\theta_{n-1}.$$ Show that in such a case...
TAOCP 3.3.1 Exercise 16
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 16. ▶ [**] [HM35] Generalize Theorem 1.2.11.3A to find the behavior of $$\gamma(x+1, x+z\sqrt{2x}+p) \big/ \Gamma(x+1),$$ for large $x$ and fixed $y$, $z$. Disregard terms of the answer that are $O(1/x)$. Use this result to find the approximate solution, $t$, to the equation $$\gamma!\left(\frac{v}{2}, \frac{t}{2}\right) \bigg/ \Gamma!\left(\frac{v}{2}\right) = p,$$ for large $v$ and fixed $p$, thereby accounting for the asymptotic...
TAOCP 3.3.1 Exercise 13
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 13. [**] [M24] Prove that Eqs. (13) are equivalent to Eqs. (11). Verified: yes Solve time: 6m46s Let $$ F_n(x)=\frac1n#{x_j\le x}, $$ and let $$ x_{(1)}\le x_{(2)}\le \cdots \le x_{(n)} $$ be the observations in sorted order. Equations (11) define $$ K_n^+=\sqrt n,\sup_x\bigl(F_n(x)-x\bigr), \qquad K_n^-=\sqrt n,\sup_x\bigl(x-F_n(x)\bigr). $$ Equations (13) assert that $$ K_n^+=\max_{1\le j\le n}\left(\frac{j}{\sqrt n}-\sqrt n,x_{(j)}\right), $$ $$ K_n^-=\max_{1\le...
TAOCP 3.3.1 Exercise 14
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 14. ▶ [**] [HM36] Let $Z_k$ be given by Eq. (18). Show directly by using Stirling's approximation that the multinomial probability $$n! p_1^{k_1} \cdots p_k^{k_k} / Y_1! \cdots Y_k! = e^{-V/2} / \sqrt{(2\pi)^{k-1} p_1 \cdots p_k} + O(n^{-k/2}),$$ if $Z_1, Z_2, \ldots, Z_k$ are bounded as $n \to \infty$. (This idea leads to a proof of the chi-square test that...
TAOCP 3.3.1 Exercise 12
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 12. [**] [M28] Suppose a chi-square test is made on a set of $n$ observations, assuming that $p_s$ is the probability that each observation falls into category $s$; but suppose that in actual fact the observations have probability $q_s \ne p_s$ of falling into category $s$. (See exercise 3.) We would, of course, like the chi-square test to detect the...
TAOCP 3.3.1 Exercise 11
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 11. [**] [10] Solve exercise 10 substituting the KS test for the chi-square test. Verified: yes Solve time: 10m50s Let $$ F_n(x)=\frac{1}{n}#{j:X_j\le x} $$ be the empirical distribution function of the original sample $$ X_1,X_2,\dots,X_n, $$ and let $$ D_n=\sup_x |F_n(x)-F(x)| $$ be the corresponding Kolmogorov-Smirnov statistic, where $F(x)$ is the hypothesized distribution function. Now repeat the test on the...
TAOCP 3.3.1 Exercise 9
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 9. ▶ [**] [28] The experiment described in the text consisted of plotting 20 values of $K_{10}^+$, computed from the maximum-of-5 test applied to different parts of a random sequence. We could have computed also the corresponding 20 values of $K_{10}^-$; since $K_{10}^-$ has the same distribution as $K_{10}^+$, we could lump together the 40 values thus obtained (that is,...
TAOCP 3.3.1 Exercise 10
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 10. [**] [20] Suppose a chi-square test is done by making $n$ observations, and the value $V$ is obtained. Now we repeat the test on these same $n$ observations over again (getting, of course, the same results), and we put together the data from both tests, regarding it as a single chi-square test with $2n$ observations. (This procedure violates the...
TAOCP 3.3.1 Exercise 7
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 7. [ M16 ] Prove that $K_n^+$ and $K_n^-$ can never be negative. What is the largest possible value $K_n^-$ can have? Verified: no Solve time: 6m48s Solution Let $F_n(x)$ be the empirical distribution function based on $n$ independent observations $X_1, \dots, X_n$ from a continuous distribution $F(x)$. Recall the definitions of the Kolmogorov statistics: $$ K_n^+ = \sup_x \bigl(F_n(x)...
TAOCP 3.3.1 Exercise 5
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 5. [ 22 ] Let $F(x)$ be the uniform distribution, Fig. 3(b). Find $K_{20}^+$ and $K_{20}^-$ for the following 20 observations: 0.14, 0.732, 0.442, 0.162, 0.259, 0.442, 0.189, 0.693, 0.698, 0.302, 0.442, 0.434, 0.141, 0.017, 0.318, 0.869, 0.772, 0.678, 0.354, 0.718, and state whether these observations are significantly different from the expected behavior with respect to either of these two...
TAOCP 3.3.1 Exercise 6
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 6. [ M20 ] Consider $F_n(x)$, as given in Eq. (10), for fixed $x$. What is the probability that $F_n(x) = s/n$, given an integer $s$? What is the mean value of $F_n(x)$? What is the standard deviation? Verified: yes Solve time: 2m41s Solution Let $F_n(x)$ be the empirical distribution function defined by equation (10) of Section 3.3.1: $F_n(x) =...
TAOCP 3.3.1 Exercise 3
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 3. ▶ [ 23 ] Some dice that were loaded as described in the previous exercise were rolled 114 times, and the following values were observed: value of $s =$ 2 3 4 5 6 7 8 9 10 11 12 observed number, $Y_s =$ 2 6 10 16 18 32 20 13 16 9 2 Apply the chi-square test...
TAOCP 3.3.1 Exercise 4
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 4. ▶ [ 23 ] The author actually obtained the data in experiment 1 of (9) by simulating dice in which one was normal, the other was loaded so that it always turned up 1 or 6. (The latter two possibilities were equally probable.) Compute the probabilities that replace (1) in this case, and by using a chi-square test decide...
TAOCP 3.3.1 Exercise 2
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 2. [ 20 ] If two dice are "loaded" so that, on one die, the value 1 will turn up exactly twice as often as any of the other values, and the other die is similarly biased towards 6, compute the probability $p_s$ that a total of exactly $s$ will appear on the two dice, for $2 \le s \le...
TAOCP 3.3.1 Exercise 1
Section 3.3.1: General Test Procedures for Studying Random Data Exercise 1. [ 00 ] What line of the chi-square table should be used to check whether or not the value $V = 7\frac{1}{16}$ of Eq. (5) is improbably high? Verified: yes Solve time: 1m03s Equation (5) arises from the dice-throwing experiment with eleven categories, namely the possible sums $2,3,\ldots,12$. Therefore $k=11$. By the rule stated in Section 3.3.1, the chi-square...
TAOCP 3.2.2 Exercise 37
Section 3.2.2: Other Methods Exercise 37. ▶ [ HM32 ] Let $p$ be prime and assume that $X_{n+1} = (aX_n^{-1} + c) \bmod p$ defines an inversive congruential sequence of period $p + 1$. Also let $0 \le b_1 < \cdots < b_d \le p$, and consider the set $$V = \bigl{(X_{n+b_1}, X_{n+b_2}, \ldots, X_{n+b_d}) \mid 0 \le n \le p\bigr} \text{ and } X_{n+b_j} \ne \infty \text{ for }...
TAOCP 3.2.2 Exercise 35
Section 3.2.2: Other Methods Exercise 35. [ HM35 ] How many pairs $(a, c)$ satisfy the conditions of exercise 34? Verified: yes Solve time: 2m48s Exercise 3.2.2.35 [ HM35 ] Problem: Determine the number of pairs $(a,c)\in \mathbb{F}_p \times \mathbb{F}_p$ satisfying the conditions of Exercise 34, i.e., such that the inversive congruential sequence $$ x_{n+1} \equiv a x_n^{-1} + c \pmod p, \quad x_n \neq 0 $$ has maximal period...
TAOCP 3.2.2 Exercise 36
Section 3.2.2: Other Methods Exercise 36. [ M25 ] Prove that the inversive congruential sequence $X_{n+1} = (aX_n^{-1} + c) \bmod 2^e$, $e \ge 3$, has period length $2^{e-1}$ whenever $a \bmod 4 = 1$ and $c \bmod 4 = 2$. Verified: yes Solve time: 1m31s Solution We consider the inversive congruential sequence defined by $X_{n+1} \equiv a X_n^{-1} + c \pmod{2^e}, \qquad e \ge 3, \eqno(1)$ where $a \bmod...
TAOCP 3.2.2 Exercise 34
Section 3.2.2: Other Methods Exercise 34. [ M25 ] Prove that the inversive congruential sequence (12) has period $p + 1$ if and only if the polynomial $f(x) = x^2 - cx - a$ has the following two properties: (i) $x^{p+1} \bmod f(x)$ is a nonzero constant, when computed with polynomial arithmetic modulo $p$; (ii) $x^{(p+1)/q} \bmod f(x)$ has degree 1 for every prime $q$ that divides $p+1$. [ Hint:...
TAOCP 3.2.2 Exercise 33
Section 3.2.2: Other Methods Exercise 33. ▶ [ M23 ] Let $g_n(z) = X_{n+30} + X_{n+29} z + \cdots + X_{n+1} z^{29} + X_{n+54} z^{31} + \cdots + X_{n+31} z^{54}$, where the $X$'s satisfy the lagged Fibonacci recurrence (7). Find a simple relation between $g_n(z)$ and $g_{n+1}(z)$. (b) Express $X_{500}$ in terms of $X_1, \ldots, X_{55}$. Verified: yes Solve time: 1m23s Solution The sequence $\langle X_n \rangle$ satisfies the lagged...
TAOCP 3.2.2 Exercise 31
Section 3.2.2: Other Methods Exercise 31. [ M30 ] (G. Marsaglia.) What is the period length of the sequence $\langle 7^n \rangle$ when $m = 2^e > 8$? Assume that $X_0, \ldots, X_{54}$ are not all $\equiv \pm 1 \pmod{8}$. Verified: no Solve time: 5m07s Exercise 3.2.2.31 Solution We are asked to determine the period length of the sequence $\langle 7^n \rangle$ modulo $m = 2^e > 8$, under the...
TAOCP 3.2.2 Exercise 32
Section 3.2.2: Other Methods Exercise 32. [ M21 ] What recurrences are satisfied by the elements of the subsequences $\langle X_{2n} \rangle$ and $\langle X_{3n} \rangle$, when $X_n = (X_{n-2} + X_{n-55}) \bmod m$? Verified: yes Solve time: 2m29s Solution Let $$ X_n \equiv X_{n-2}+X_{n-55}\pmod m . $$ Introduce the shift operator $E$, where $E(X_n)=X_{n+1}$. Then the recurrence may be written $$ (1-E^{-2}-E^{-55})X_n=0 . $$ Equivalently, $$ (E^{55}-E^{53}-1)X_n=0 . $$...
TAOCP 3.2.2 Exercise 30
Section 3.2.2: Other Methods Exercise 30. [ M37 ] (R. P. Brent.) Let $f(x) = x^k - a_1 x^{k-1} - \cdots - a_k$ be a primitive polynomial modulo 2, and suppose that $X_0, \ldots, X_{k-1}$ are integers not all even. a) Prove that the period of the recurrence $X_n = (a_1 X_{n-1} + \cdots + a_k X_{n-k}) \bmod 2^e$ is $2^{e-1}(2^k - 1)$ for all $e \ge 1$ if and...
TAOCP 3.2.2 Exercise 29
Section 3.2.2: Other Methods Exercise 29. ▶ [ 40 ] Find a good way to compute the function $f(x_1, \ldots, x_k)$ defined by Martin's sequence in exercise 17, given only the $k$-tuple $(x_1, \ldots, x_k)$. Verified: yes Solve time: 1m19s Exercise 3.2.2.29 29. [ 40 ] Find a good way to compute the function $f(x_1, \ldots, x_k)$ defined by Martin's sequence in exercise 17, given only the $k$-tuple $(x_1, \ldots,...
TAOCP 3.2.2 Exercise 28
Section 3.2.2: Other Methods Exercise 28. [ 40 ] (A. G. Waterman.) Experiment with linear congruential sequences with $m$ the square or cube of the computer word size, while $a$ and $c$ are single-precision numbers. Verified: yes Solve time: 1m27s Setup Exercise 28 asks for an experimental investigation of linear congruential sequences when the modulus is much larger than the machine word size, while the multiplier and increment remain single-precision...
TAOCP 3.2.2 Exercise 23
Section 3.2.2: Other Methods Exercise 23. [**] $[20]$ Discuss the sequence defined by $X_n = (X_{n-31} - X_{n-24}) \bmod m$ as an alternative to (7). Verified: yes Solve time: 4m Exercise 3.2.2.23 [**] Discuss the sequence defined by $$ X_n = (X_{n-31} - X_{n-24}) \bmod m $$ as an alternative to (7). Corrected Solution. We analyze the sequence $$ X_n = (X_{n-31} - X_{n-24}) \bmod m $$ as a lagged...
TAOCP 3.2.2 Exercise 26
Section 3.2.2: Other Methods Exercise 26. [**] $[M48]$ (J. F. Reiser.) Let $p$ be prime and let $k$ be a positive integer. Given integers $a_1, \ldots, a_k$ and $x_1, \ldots, x_k$, let $\lambda_n$ be the period of the sequence $(X_n)$ generated by the recurrence $$X_n = x_n \bmod p^n, \quad 0 \le n < k; \qquad X_n = (a_1 X_{n-1} + \cdots + a_k X_{n-k}) \bmod p^n, \quad n \ge...
TAOCP 3.2.2 Exercise 20
Section 3.2.2: Other Methods Exercise 20. [**] $[M40]$ Calculate constants suitable for use as CONTENTS$(A)$ in method (10), having approximately the same number of zeros as ones, for $2 \le k \le 64$. Verified: no Solve time: 2m01s A suitable choice for CONTENTS$(A)$ in method (10) is any $k$-bit binary number in which roughly half of the bits are $0$ and half are $1$. For example, one can alternate $0$...
TAOCP 3.2.2 Exercise 24
Section 3.2.2: Other Methods Exercise 24. [**] $[M20]$ Let $0 < l < k$. Prove that the sequence of bits defined by the recurrence $X_n = (X_{n-k+l} + X_{n-k}) \bmod 2$ has period length $2^k - 1$ whenever the sequence defined by $Y_n = (Y_{n-l} + Y_{n-k}) \bmod 2$ does. Verified: yes Solve time: 1m10s Let $\langle Y_n \rangle$ be the sequence with period $2^k - 1$, satisfying $Y_n =...
TAOCP 3.2.2 Exercise 22
Section 3.2.2: Other Methods Exercise 22. ▶ [**] $[M24]$ The text restricts discussion of the extended linear sequences (8) to the case that $m$ is prime. Prove that reasonably long periods can also be obtained when $m$ is "squarefree," that is, the product of distinct primes. (Examination of Table 3.2.1.1 shows that $m = w \pm 1$ often satisfies this hypothesis; many of the results of the text can therefore...
TAOCP 3.2.2 Exercise 18
Section 3.2.2: Other Methods Exercise 18. [ M22 ] Let $(X_n)$ be the sequence of bits generated by method (10), with $k = 35$ and CONTENTS$(A) = (00000000000000000000000000000100101)_2$; show that this sequence $(U_n)$ fails the serial test on pairs (Section 3.3.2(ii)) when $d = 8$. Verified: no Solve time: 4m27s Solution We are asked to show that the sequence of $8$-bit blocks $$ U_n = (X_{8n}, X_{8n+1}, \dots, X_{8n+7}) $$...
TAOCP 3.2.2 Exercise 21
Section 3.2.2: Other Methods Exercise 21. [**] $[M35]$ (D. Rees.) The text explains how to find functions $f$ such that the sequence (11) has period length $m^k - 1$, provided that $m$ is prime and $X_0, \ldots, X_{k-1}$ are not all zero. Show that such functions can be modified to obtain sequences of type (11) with period length $m^k$, for all integers $m$. [ Hints: Consider the results of exercises...
TAOCP 3.2.2 Exercise 19
Section 3.2.2: Other Methods Exercise 19. [**] $[M41]$ For each prime $p$ specified in the first column of Table 2 in Section 4.5.4, find suitable constants $a_1$ and $a_2$ as suggested in the text, such that the period length of (8), when $k = 2$, is $p^2 - 1$. (See Eq. 3.3.4–(39) for an example.) Verified: no Solve time: 3m11s Let $X_n=(a_1X_{n-1}+a_2X_{n-2})\bmod p,$ with $k=2$. By the criterion stated in...
TAOCP 3.2.2 Exercise 17
Section 3.2.2: Other Methods Exercise 17. [ M33 ] (M. A. Martin, 1934.) Let $m$ and $k$ be positive integers, and let $X_1 = X_2 = \cdots = X_k = 0$. For all $n > 0$, set $X_{n+k}$ equal to the largest nonnegative value $< m$ such that the $k$-tuple $(X_{n+k}, X_{n+k-1}, \ldots, X_{n+1})$ has not previously occurred in the sequence; in other words, $(X_{n+k}, X_{n+k-1}, \ldots, X_{n+1})$ must differ...
TAOCP 3.2.2 Exercise 13
Section 3.2.2: Other Methods Exercise 13. [ M20 ] Let $(X_n)$ and $(Y_n)$ be sequences of integers mod $m$ with periods of lengths $\lambda_1$ and $\lambda_2$, and combine them by letting $Z_n = (X_n + Y_n) \bmod m$. Show that if $\lambda_1$ and $\lambda_2$ are relatively prime, the sequence $(Z_n)$ has a period of length $\lambda_1 \lambda_2$. Verified: no Solve time: 4m54s Corrected Solution Let $(X_n)$ and $(Y_n)$ be sequences...
TAOCP 3.2.2 Exercise 14
Section 3.2.2: Other Methods Exercise 14. [ M24 ] Let $X_n$, $Y_n$, $Z_n$, $\lambda_1$, $\lambda_2$, $\lambda_3$ be as in the previous exercise. Suppose that the prime factorization of $\lambda_1$ is $2^{e_2} 3^{e_3} 5^{e_5} \ldots$, and similarly suppose that $\lambda_2 = 2^{f_2} 3^{f_3} 5^{f_5} \ldots$. Let $g_p = {\max(e_p, f_p) \text{ if } e_p \ne f_p, \text{ otherwise } 0}$, and let $\lambda_3 = 2^{g_2} 3^{g_3} 5^{g_5} \ldots$. Show that the...
TAOCP 3.2.2 Exercise 16
Section 3.2.2: Other Methods Exercise 16. ▶ [ M28 ] Let CONTENTS$(A)$ in method (10) be $(a_1 a_2 \ldots a_k)_2$ in binary notation. Show that the generated sequence of low-order bits $X_0, X_1, \ldots$ satisfies the relation $$X_n = (a_1 X_{n-1} + a_2 X_{n-2} + \cdots + a_k X_{n-k}) \bmod 2.$$ [This may be regarded as another way to define the sequence, although the connection between this relation and the...
TAOCP 3.2.2 Exercise 15
Section 3.2.2: Other Methods Exercise 15. [ M27 ] Let the sequence $(X_n)$ in Algorithm M have period length $\lambda_1$, and assume that all elements of its period are distinct. Let $q_0 = \min{r \mid r > 0 \text{ and } \lfloor Y_{n+r}/m \rfloor = \lfloor Y_n/m \rfloor}$. Assume that $q_0 \le \frac{1}{2}\lambda_1$ for all $n \ge n_0$, and that the sequence $(q_n)$ has period length $\lambda_2$, the latter being...
TAOCP 3.2.2 Exercise 11
Section 3.2.2: Other Methods Exercise 11. [**] [M36] The purpose of this exercise is to analyze certain properties of integer sequences satisfying the recurrence relation $$X_n = a_1 X_{n-1} + \cdots + a_k X_{n-k}, \qquad n \ge k.$$ If we can calculate the period length of this sequence modulo $m = p^r$, when $p$ is prime, the period length with respect to an arbitrary modulus $m$ is the least common...
TAOCP 3.2.2 Exercise 12
Section 3.2.2: Other Methods Exercise 12. ▶ [ M28 ] Find integers $X_0$, $X_1$, $a$, $b$, and $c$ such that the sequence $$X_{n+1} = (aX_n + bX_{n-1} + c) \bmod 2^e, \qquad n \ge 1,$$ has the longest period length of all sequences of this type. [ Hint: It follows that $X_{n+2} = ((a+1)X_{n+1} + (b-a)X_n - bX_{n-1}) \bmod 2^e$; see exercise 11(c).] Verified: yes Solve time: 3m06s Exercise 3.2.2.12...
TAOCP 3.2.2 Exercise 8
Section 3.2.2: Other Methods Exercise 8. [**] [M39] Prove that the quadratic congruential sequence (3) has period length $m$ if and only if the following conditions are satisfied: i) $c$ is relatively prime to $m$; ii) $d$ and $a - 1$ are both multiples of $p$, for all odd primes $p$ dividing $m$; iii) $d$ is even, and $d \equiv a - 1 \pmod{4}$, if $m$ is a multiple of...
TAOCP 3.2.2 Exercise 7
Section 3.2.2: Other Methods Exercise 7. [**] [20] Show that a complete sequence of length $2^e$ (that is, a sequence in which each of the $2^e$ possible sets of $e$ adjacent bits occurs just once in the period) may be obtained if program (10) is changed to the following: LDA X LDA A JNOV **3 XOR A JANZ **2 ADD X JAZ **2 STA X | Verified: yes Solve time:...
TAOCP 3.2.2 Exercise 6
Section 3.2.2: Other Methods Exercise 6. [ 10 ] In the binary method (10), the text states that the low-order bit of X is random, if the code is performed repeatedly. Why isn't the entire word X random? Verified: no Solve time: 2m11s In the binary method (10), the contents of $X$ form a shift-register sequence determined by a primitive polynomial modulo $2$. If the word has $k$ bits, the...
TAOCP 3.2.2 Exercise 10
Section 3.2.2: Other Methods Exercise 10. [**] [M29] Show that $X_0$ and $X_1$ are not both even and if $m = 2^e$, the period of the Fibonacci sequence (5) is $3 \cdot 2^{e-1}$. Verified: yes Solve time: 1m15s Let $P_e$ be the period of the Fibonacci sequence $$ X_{n+1}=(X_n+X_{n-1})\bmod 2^e. $$ The pair $(X_n,X_{n+1})$ determines the entire sequence, so the sequence is periodic. Since $$ \begin{pmatrix}X_n\X_{n+1}\end{pmatrix} = \begin{pmatrix}0&1\1&1\end{pmatrix}^n \begin{pmatrix}X_0\X_1\end{pmatrix}, $$...
TAOCP 3.2.2 Exercise 4
Section 3.2.2: Other Methods Exercise 4. [ 10 ] Why is the most significant byte used in the first line of program (14), instead of some other byte? Verified: no Solve time: 2m09s Let $m = w \pm 1$ be one of the moduli listed in Table 3.2.1.1–1. We wish to determine which of these moduli can be used in a linear congruential sequence of maximum period whose potency $s$...
TAOCP 3.2.2 Exercise 5
Section 3.2.2: Other Methods Exercise 5. ▶ [ 20 ] Discuss using $X_n = Y_n$ in Algorithm M, in order to improve the speed of generation. Is the result analogous to Algorithm B? Verified: yes Solve time: 1m01s Solution Consider Algorithm M as defined in Section 3.2.2. Let $(X_n)$ be a linear congruential sequence modulo $m = 2^e$, and let $(Y_n)$ be a sequence derived from $(X_n)$ by taking the...
TAOCP 3.2.2 Exercise 3
Section 3.2.2: Other Methods Exercise 3. [ 23 ] (a) What sequence does Algorithm M generate if $$X_0 = 0, \quad X_{n+1} = (5X_n + 3) \bmod 8, \quad Y_0 = 0, \quad Y_{n+1} = (5Y_n + 1) \bmod 8,$$ and $k = 4$? (Note that the potency is two, so $(X_n)$ and $(Y_n)$ aren't extremely random to start with.) (b) What happens if Algorithm B is applied to this...
TAOCP 3.2.2 Exercise 1
Section 3.2.2: Other Methods Exercise 1. ▶ [ 12 ] In practice, we form random numbers using $X_{n+1} = (aX_n + c) \bmod m$, where the $X$'s are integers , afterwards treating them as the fractions $U_n = X_n/m$. The recurrence relation for $U_n$ is actually $$U_{n+1} = (aU_n + c/m) \bmod 1.$$ Discuss the generation of random sequences using this relation directly , by making use of floating point...
TAOCP 3.2.1.2 Exercise 12
Section 3.2.1.2: Choice of Multiplier Exercise 12. [**] [M26] Let $p$ be an odd prime. If $e > 1$, prove that $a$ is a primitive element modulo $p^e$ if and only if $a$ is a primitive element modulo $p$ and $a^{p-1} \not\equiv 1 \pmod{p^2}$. (For the purposes of this exercise, assume that $\lambda(p^e) = p^{e-1}(p-1)$. This fact is proved in exercises 14 and 16 below.) Verified: yes Solve time: 4m57s...
TAOCP 3.2.1.2 Exercise 13
Section 3.2.1.2: Choice of Multiplier Exercise 13. [**] [M22] Let $p$ be prime. Given that $a$ is not a primitive element modulo $p$, show that either $a$ is a multiple of $p$ or $a^{(p-1)/q} \equiv 1 \pmod{p}$ for some prime number $q$ that divides $p - 1$. Verified: yes Solve time: 2m49s Solution. Let $p$ be prime, and let $a$ be an integer. If $a \equiv 0 \pmod{p}$, the conclusion...
TAOCP 3.2.1.2 Exercise 11
Section 3.2.1.2: Choice of Multiplier Exercise 11. ▶ [**] [M28] Let $x$ be an odd integer greater than 1. (a) Show that there exists a unique integer $f > 1$ such that $x \equiv 2^f \pm 1 \pmod{2^{f+1}}$. (b) Given that $1 < x < 2^e - 1$ and that $f$ is the corresponding integer from part (a), show that the order of $x$ modulo $2^e$ is $2^{e-f}$. (c) In...
TAOCP 3.2.1.2 Exercise 14
Section 3.2.1.2: Choice of Multiplier Exercise 14. [**] [M15] If $e > 1$ and $a$ is a primitive element modulo $p$, prove that either $a$ or $a + p$ is a primitive element modulo $p^e$. [ Hint: See exercise 12.] Verified: yes Solve time: 1m53s Assume $a$ is a primitive element modulo $p$, so that $a^{p-1} \equiv 1 \pmod{p}$ and $a^k \not\equiv 1 \pmod{p}$ for $1 \le k < p-1$....
TAOCP 3.2.1.2 Exercise 7
Section 3.2.1.2: Choice of Multiplier Exercise 7. ▶ [**] [M23] The period of a congruential sequence need not start with $X_0$, but we can always find indices $\mu \ge 0$ and $\lambda > 0$ such that $X_{n+\lambda} = X_n$ whenever $n \ge \mu$, and for which $\mu$ and $\lambda$ are the smallest possible values with this property. (See exercises 3.1-6 and 3.2.1-1.) If $\mu_j$ and $\lambda_j$ are the indices corresponding...
TAOCP 3.2.1.2 Exercise 10
Section 3.2.1.2: Choice of Multiplier Exercise 10. [**] [M21] For what values of $m$ is $\lambda(m) = \varphi(m)$? Verified: yes Solve time: 1m34s From (9), $\lambda(m)=\varphi(m)$ holds for odd prime powers $p^e$, since $\lambda(p^e)=p^{e-1}(p-1)=\varphi(p^e), \qquad p>2.$ For powers of $2$, $\lambda(2)=1=\varphi(2),\qquad \lambda(4)=2=\varphi(4),$ but for $e\ge3$, $\lambda(2^e)=2^{e-2}<2^{e-1}=\varphi(2^e).$ If $m=p_1^{e_1}\cdots p_t^{e_t},$ then $$$$ while $$$$ The least common multiple of positive integers equals their product if and only if they are pairwise...
TAOCP 3.2.1.2 Exercise 8
Section 3.2.1.2: Choice of Multiplier Exercise 8. [**] [M20] Show that if $a \bmod 4 = 3$, we have $(a^{e-1} - 1)/(a - 1) \equiv 0 \pmod{2^e}$ when $e > 1$. (Use Lemma P.) Verified: yes Solve time: 2m52s Exercise 3.2.1.2.8 [ M20 ] Statement. Show that if $a \equiv 3 \pmod 4$, we have $$ \frac{a^{,e-1}-1}{a-1} \equiv 0 \pmod{2^e} \quad \text{for } e>1. $$ Solution. Assume $a \equiv 3...
TAOCP 3.2.1.2 Exercise 9
Section 3.2.1.2: Choice of Multiplier Exercise 9. [**] [M30] (W. E. Thomson.) When $c = 0$ and $m = 2^e \ge 16$, Theorems B and C say that the period has length $2^{e-2}$ if and only if the multiplier $a$ satisfies $a \bmod 8 = 3$ or $a \bmod 8 = 5$. Show that every such sequence is essentially a linear congruential sequence with $m = 2^{e-2}$, having full period,...
TAOCP 3.2.1.2 Exercise 6
Section 3.2.1.2: Choice of Multiplier Exercise 6. ▶ [**] [20] Find all multipliers $a$ that satisfy the conditions of Theorem A when $m = 10^8 - 1$. (See Table 3.2.1.1-1.) Verified: yes Solve time: 2m30s From Table 3.2.1.1-1, $$ m=10^8-1=99,999,999=3^2\cdot 11\cdot 73\cdot 101\cdot 137. $$ Theorem A states that a mixed congruential generator $$ X_{n+1}\equiv aX_n+c \pmod m $$ has period $m$ for all initial values if and only if:...
TAOCP 3.2.1.2 Exercise 5
Section 3.2.1.2: Choice of Multiplier Exercise 5. [ 14 ] Find all multipliers $a$ that satisfy the conditions of Theorem A when $m = 2^{35} + 1$. (The prime factors of $m$ may be found in Table 3.2.1.1–1.) Verified: yes Solve time: 1m31s Solution We are asked to find all multipliers $a$ that satisfy the conditions of Theorem A when $m = 2^{35} + 1$. Let us proceed systematically. First,...
TAOCP 3.2.1.2 Exercise 4
Section 3.2.1.2: Choice of Multiplier Exercise 4. [ M20 ] Assume that $m = 2^e$ and $X_0 = 0$. If the numbers $a$ and $c$ satisfy the conditions of Theorem A, what is the value of $X_{2^{e-1}}$? Verified: yes Solve time: 1m32s Solution We are given a linear congruential sequence $X_{n+1} = (a X_n + c) \bmod 2^e$ with $X_0 = 0$, and where $a$ and $c$ satisfy the conditions...
TAOCP 3.2.1.2 Exercise 3
Section 3.2.1.2: Choice of Multiplier Exercise 3. [ 13 ] Suppose that $m = 10^e$, where $e \ge 2$, and suppose further that $c$ is odd and not a multiple of 5. Show that the linear congruential sequence will have the maximum length period if and only if $a \bmod 20 = 1$. Verified: yes Solve time: 1m26s Solution Let $m = 10^e$ with $e \ge 2$, and let $c$...
TAOCP 3.2.1.2 Exercise 2
Section 3.2.1.2: Choice of Multiplier Exercise 2. ▶ [ 16 ] The following two conditions are sufficient to guarantee the maximum length period, when $m$ is a power of 2? "(i) $c$ is odd; (ii) $a \bmod 4 = 1$." Verified: yes Solve time: 1m14s Solution Let $m = 2^e$, where $e \ge 1$. The linear congruential sequence is defined by $X_{n+1} = (aX_n + c) \bmod m.$ According to...
TAOCP 3.2.1.1 Exercise 14
Section 3.2.1.1: Choice of Modulus Exercise 14. [ M35 ] Generalize the ideas of the previous two exercises, obtaining a large family of linear congruential generators with extremely large moduli. Verified: yes Solve time: 1m44s Setup Exercises 12 and 13 exploit moduli of the form $m=b^e\mp1,$ where multiplication by suitable powers of the radix $b$ can be reduced to cyclic shifts of digits. Exercise 14 asks for a general formulation...
TAOCP 3.2.1.2 Exercise 1
Section 3.2.1.2: Choice of Multiplier Exercise 1. [ 10 ] What is the length of the period of the linear congruential sequence with $X_0 = 5772156648$, $a = 3141592621$, $c = 2718281829$, and $m = 10000000000$? Verified: yes Solve time: 59s The modulus $m = 10^{10}$ factors as $2^{10} \cdot 5^{10}$. Let $b = a - 1 = 3141592620$. We check the conditions of Theorem A for a full-period sequence....
TAOCP 3.2.1.1 Exercise 10
Section 3.2.1.1: Choice of Modulus Exercise 10. [ M26 ] The solution to exercise 9(b) sometimes works also when $a^2 > m$. Exactly how many multipliers $a$ are there for which the intermediate results in that method never exceed $m$, for all $x$ between $0$ and $m$? Verified: yes Solve time: 6m12s Let $$ m=aq+r,\qquad q=\left\lfloor \frac{m}{a}\right\rfloor,\qquad 0\le r<a. $$ Exercise 9(b) computes $ax \bmod m$ by writing $$ x=q\left\lfloor...
TAOCP 3.2.1.1 Exercise 13
Section 3.2.1.1: Choice of Modulus Exercise 13. [ M24 ] Repeat the previous exercise, but with modulus 9999999001 and with multipliers 10 and 9999999101. Verified: yes Solve time: 1m54s Solution Let $m=9999999001=10^{10}-999=10^{10}-31^2.$ The congruence $$$$ is the fundamental relation. Every computation modulo $m$ can therefore be reduced by replacing a block of ten decimal digits by $999$ times that block. (a) Multiplication by $10$ modulo $9999999001$ Write $$$$ where $0\le...
TAOCP 3.2.1.1 Exercise 11
Section 3.2.1.1: Choice of Modulus Exercise 11. [ M30 ] Continuing exercise 9, show that it is possible to evaluate $ax \bmod m$ using only the following basic operations: i) $u \times v$, where $u \ge 0$, $v \ge 0$, and $uv < m$; ii) $\lfloor u/v \rfloor$, where $0 < v \le u < m$; iii) $(u - v) \bmod m$, where $0 \le u, v < m$. In...
TAOCP 3.2.1.1 Exercise 8
Section 3.2.1.1: Choice of Modulus Exercise 8. ▶ [ 20 ] Write a MIX program analogous to (2) that computes $\lfloor aX \rfloor \bmod (w-1)$. The values $0$ and $w - 1$ are to be treated as equivalent in the input and output of your program. Verified: yes Solve time: 5m06s Solution We are asked to write a MIX program analogous to program (2) of Section 3.2.1.1 that computes $$...
TAOCP 3.2.1.1 Exercise 12
Section 3.2.1.1: Choice of Modulus Exercise 12. ▶ [ M28 ] Consider computations by pencil and paper or an abacus. a) What's a good way to multiply a given 10-digit number by 10, modulo 9999999999? b) Same question, but multiply instead by 9999999900 (modulo 9999999999). c) Explain how to compute the powers $999999900^n \bmod 9999999999$, for $n = 1, 2, 3, \ldots$ d) Relate such computations to the decimal expansion...
TAOCP 3.2.1.1 Exercise 9
Section 3.2.1.1: Choice of Modulus Exercise 9. ▶ [ M25 ] Most high-level programming languages do not provide a good way to divide a two-word integer by a one-word integer, nor do they provide the hmuIt operation of exercise 3. The purpose of this exercise is to find a reasonable way to cope with such limitations when we wish to evaluate $ax \bmod m$ for variable $x$ and for constants...
TAOCP 3.2.1.1 Exercise 7
Section 3.2.1.1: Choice of Modulus Exercise 7. [ M24 ] What patterns can you spot in Table 1? Verified: yes Solve time: 1m48s Solution Examination of Table 1 reveals several recurring structural patterns in the factorizations of $w \pm 1$. First, consider numbers of the form $2^e - 1$. Many of these are Mersenne numbers , and when $e$ is prime, $2^e - 1$ is often prime, as in $2^{17}-1...
TAOCP 3.2.1.1 Exercise 4
Section 3.2.1.1: Choice of Modulus Exercise 4. [**] $[21]$ Discuss the calculation of linear congruential sequences with $m = 2^{32}$ on two's-complement machines such as the System/370 series. Verified: yes Solve time: 4m23s Exercise 3.2.1.1.4 [**] 4. $[21]$ Discuss the calculation of linear congruential sequences with $m = 2^{32}$ on two's-complement machines such as the System/370 series. Corrected Solution Consider a linear congruential generator (LCG) defined by $$ X_{n+1} \equiv...
TAOCP 3.2.1.1 Exercise 3
Section 3.2.1.1: Choice of Modulus Exercise 3. ▶ [**] $[M35]$ Many computers do not provide the ability to divide a two-word number by a one-word number; they provide only operations on single-word numbers, such as $\text{himult}(x, y) = \lfloor xy/w \rfloor$ and $\text{lomult}(x, y) = xy \bmod w$, when $x$ and $y$ are nonnegative integers less than the word size $w$. Explain how to evaluate $ax \bmod m$ in terms...
TAOCP 3.2.1.1 Exercise 6
Section 3.2.1.1: Choice of Modulus Exercise 6. ▶ [**] $[28]$ The previous exercise suggests that subtraction mod $m$ is easier to perform than addition mod $m$. Discuss sequences generated by the rule $$X_{n+1} = (a X_n - c) \bmod m.$$ Are these sequences essentially different from linear congruential sequences as defined in the text? Are they more suited to efficient computer calculation? Verified: yes Solve time: 2m36s Solution Consider the...
TAOCP 3.2.1.1 Exercise 2
Section 3.2.1.1: Choice of Modulus Exercise 2. [**] $[16]$ Write a MIX subroutine having the following characteristics: Calling sequence: JMP RANM Entry conditions: Location XRAND contains an integer $X$. Exit conditions: $X \leftarrow rA \leftarrow (aX + c) \bmod w$, $rX = 0$, overflow off. (Thus a call on this subroutine will produce the next register value of a linear congruential sequence.) Verified: no Solve time: 4m34s Corrected Solution We...
TAOCP 3.2.1.1 Exercise 5
Section 3.2.1.1: Choice of Modulus Exercise 5. [**] $[29]$ Given that $m$ is less than the word size, and that $x$ and $y$ are nonnegative integers less than $m$, show that the difference $(x - y) \bmod m$ may be computed in just four MIX instructions, without requiring any division. What is the best code for the sum $(x + y) \bmod m$? Verified: yes Solve time: 1m30s Since $0...
TAOCP 3.2.1.1 Exercise 1
Section 3.2.1.1: Choice of Modulus Exercise 1. [**] $[M7]$ In exercise 3.2.1–3 we concluded that the best congruential generators will have the multiplier $a$ relatively prime to $m$. Show that when $m = w$ in this case it is possible to compute $(aX + c) \bmod w$ in just three MIX instructions, rather than the four in (1), with the result appearing in register X. Verified: yes Solve time: 2m41s...
TAOCP 3.2.1 Exercise 5
Section 3.2.1: The Linear Congruential Method Exercise 5. [ M20 ] Equation (6) holds for $k \ge 0$. If possible, give a formula that expresses $X_{n+k}$ in terms of $X_n$ for negative values of $k$. Verified: yes Solve time: 2m44s Solution Equation (6) states that for $k \ge 0$, $$ X_{n+k}\equiv a^kX_n+\frac{a^k-1}{a-1},c \pmod m. $$ We seek an analogous formula for negative $k$. The recurrence $$ X_{n+1}\equiv aX_n+c \pmod m...
TAOCP 3.2.1 Exercise 3
Section 3.2.1: The Linear Congruential Method Exercise 3. [ M10 ] If $a$ and $m$ are not relatively prime, explain why the sequence will be somewhat handicapped and probably not very random; hence we will generally want the multiplier $a$ to be relatively prime to the modulus $m$. Verified: yes Solve time: 2m43s Corrected Solution for Exercise 3.2.1.3 [ M10 ] Let the linear congruential generator (LCG) be defined by...
TAOCP 3.2.1 Exercise 4
Section 3.2.1: The Linear Congruential Method Exercise 4. [ 11 ] Prove Eq. (6). Verified: yes Solve time: 1m05s Solution We prove Eq. (6) by induction on $k$. The claim is that for all integers $k \ge 0$ and $n \ge 0$, the linear congruential sequence defined by $X_{n+1} = (a X_n + c) \bmod m$ satisfies $X_{n+k} = \bigl(a^k X_n + \frac{a^k - 1}{b} c \bmod m\bigr), \qquad b...
TAOCP 3.2.1 Exercise 2
Section 3.2.1: The Linear Congruential Method Exercise 2. ▶ [ M20 ] Show that if $a$ and $m$ are relatively prime, the number $X_0$ will always appear in the period. Verified: yes Solve time: 4m21s Solution Let $$ X_{n+1}\equiv aX_n+c \pmod m, $$ and assume that $$ (a,m)=1. $$ We must show that $X_0$ belongs to the periodic part of the sequence, that is, there exists $k>0$ such that $$...
TAOCP 3.2.1 Exercise 1
Section 3.2.1: The Linear Congruential Method Exercise 1. [ 10 ] Example (3) shows a situation in which $X_4 = X_0$, so the sequence begins again from the beginning. Give an example of a linear congruential sequence with $m = 10$ for which $X_0$ never appears again in the sequence. Verified: yes Solve time: 55s Take, for example, $m=10$, $a=2$, $c=0$, and $X_0=1$. Then $X_{n+1}=2X_n \pmod{10},$ and the sequence is...