arXiv:1701.00052v2 [math.PR] 5 Dec 2017 Optimal selection of the k-th best candidate Yi-Shen Lin, Shoou-Ren Hsiau, and Yi-Ching Yao December 4, 2017 Abstract In the subject of optimal stopping, the classical secretary problem is concerned with optimally selecting the best of n candidates when their relative ranks are observed sequentially. This problem has been extended to optimally selecting the k-th best candidate for k 2. While the optimal stopping rule for k = 1, 2 (and all n 2) is known to be of threshold type (involving one threshold), we solve the case k = 3 (and all n 3) by deriving an explicit optimal stopping rule that involves two thresholds. We also prove several inequalities for p(k, n), the maximum probability of selecting the k-th best of n candidates. It is shown that (i) p(1, n) = p(n, n) > p(k, n) for 1 < k < n, (ii) p(k, n) p(k, n + 1), (iii) p(k, n) p(k + 1, n + 1), and (iv) p(k, ) := limn p(k, n) is decreasing in k. Keywords: secretary problem; best choice; backward induction; optimal stopping. 2010 Mathematics Subject Classification: Primary 60G40 Secondary 62L15 1 Introduction The classical secretary problem (also known as the best choice problem) has been extensively studied in the literature on optimal stopping, which is usually described as follows. Institute of Statistical Science, Academia Sinica, Taipei 115, Taiwan, R.O.C. Email address: yslin@stat.sinica.edu.tw Department of Mathematics, National Changhua University of Education, No. 1, Jin-De Rd., Changhua 500, Taiwan, R.O.C. Email address: srhsiau@cc.ncue.edu.tw Institute of Statistical Science, Academia Sinica, Taipei 115, Taiwan, R.O.C. Email address: yao@stat.sinica.edu.tw 1 There are n (fixed) candidates to be interviewed sequentially in random order for one sec- retarial position. It is assumed that these candidates can be ranked linearly without ties by a manager (rank 1 being the best). Upon interviewing a candidate, the manager is only able to observe the candidate's (relative) rank among those that have been interviewed so far. The manager then must decide whether to accept the present candidate (and stop in- terviewing) or to reject the candidate (and continue interviewing). No recall is allowed. The object is to maximize the probability of selecting the best candidate. More precisely, let Rj, j = 1, 2, . . . , n, be the absolute rank of the j-th candidate such that (R1, . . . , Rn) = n with probability 1/n! for every permutation n of (1, 2, . . . , n). Define Xj = |{1 i j : Ri Rj}|, the relative rank of the j-th candidate among the first j candidates. It is desired to find a stopping rule 1,n Mn such that P (R1,n = 1) = supMn P (R = 1) where Mn denotes the set of all stopping rules adapted to the filtration {Fj}, Fj being the -algebra generated by X1, X2, . . . , Xj. It is well known (cf. Lindley [6]) that the optimal stopping rule 1,n is of threshold type given by 1,n = min{rn j n : Xj = 1} where min := n and the threshold rn := min{j 1 : n i=j+1 1 i-1 1}. Moreover, the maximum probability of selecting the best candidate (under 1,n) is p(1, n) := rn-1 n n i=rn 1 i-1 , which converges as n to p(1, ) := 1/e = limn rn/n. A great many interesting variants of the secretary problem have been formulated and solved in the literature (cf. the review papers by Ferguson [2] and Freeman [4] and Samuels [9]), most of which are concerned with optimally selecting the best candidate or one of the k best candidates. In contrast, only a few papers (cf. Rose [7], Szajowski [11] and Vanderbei [12]) considered and solved the problem of optimally selecting the second best candidate. (According to Vanderbei [12], in 1980, E.B. Dynkin proposed this problem to him with the motivating story that "We are trying to hire a postdoc and we are confident that the best candidate will receive and accept an offer from Harvard." Thus Vanderbei [12] refers to the problem as the postdoc variant of the secretary problem.) These authors showed that the optimal stopping rule 2,n is also of threshold type given by 2,n = min{rn j n : Xj = 2} with rn = n+1 2 (the smallest integer not less than n+1 2 ), which attains the maximum probability of selecting the second best candidate p(2, n) := P (R2,n = 2) = sup Mn P (R = 2) = (rn - 1)(n - rn n(n - 1) + 1) . Note that p(2, ) = limn p(2, n) = 1/4 < 1/e = p(1, ). In this paper, we consider the problem of optimally selecting the k-th best candidate for 2 general k. Let p(k, n) := supMn P (R = k), the maximum probability of selecting the k-th best of n candidates. Szajowski [11] derived the asymptotic solutions as n for k = 3, 4, 5. Rose [8] dealt with the case k = (n + 1)/2 for odd n, which was called the median problem and suggested by M. DeGroot with the motivation of selecting a candidate representative of the entire sequence. (The candidate of rank k = (n + 1)/2 is, in some sense, representative of all candidates.) In the next section, we solve the case k = 3 for all finite n 3 by showing (cf. Theorem 2.1) that the stopping rule 3,n = min{an j n : Xj = 2} min{bn j n : Xj = 3} attains the maximum probability P (R3,n = 3) = p(3, n) for n 3, where x y := min{x, y} and the two thresholds an < bn are given in (2.8) and (2.5), respectively. In Section 3, we prove (cf. Theorems 3.1 and 3.2) that (i) p(1, n) = p(n, n) > p(k, n) for 1 < k < n, (ii) p(k, n) p(k, n + 1), (iii) p(k, n) p(k + 1, n + 1), and (iv) p(k, ) := limn p(k, n) is decreasing in k. It is also noted (cf. Remark 3.1) that the inequality p(k, n) p(k + 1, n) occasionally fails to hold for k close to (but less than) n 2 . Furthermore, we extend the result p(1, n) = p(n, n) > p(k, n) for 1 < k < n to the setting where the goal is to select a candidate whose absolute rank belongs to a prescribed subset of {1, . . . , n} with || = c (1 c < n) (cf. Suchwalko and Szajowski [10]). It is shown (cf. Theorem 3.3) that the probability of optimally selecting a candidate whose absolute rank belongs to is maximized when = {1, . . . , c} or = {n - c + 1, . . . , n}. The proofs of several technical lemmas are relegated to Section 4. Section 5 contains a computer program in Mathematica for verification of Theorem 2.1 for 3 n 31. It should be remarked that the optimal stopping rule is not necessarily unique. For example, a slight modification 2,n of the optimal stopping rule 2,n also attains the maximum probability p(2, n) where 2,n rn - 1 is given by 2,n = rn - 1 if Xrn -1 = 1 and 2,n = 2,n otherwise. The uniqueness issue of the optimal stopping rule is not addressed in this paper. 2 Maximizing the probability of selecting the k-th best candidate with k = 3 We adopt the setup and notations in Ferguson [3, Chapter 2]. As defined in Section 1, Xj is the relative rank of the j-th candidate among the first j candidates and Rj is the absolute rank. Given X1 = x1, X2 = x2, . . . , Xj = xj, 1 j n, let yj(x1, x2, . . . , xj) be the return for stopping at stage j (i.e. accepting the j-th candidate) and Vj(x1, x2, . . . , xj) the maximum return by optimally stopping from stage j onwards. In other words, yj(x1, x2, . . . , xj) is the 3 conditional probability of Rj = k (given Xi = xi, 1 i j), which defines the reward function for the stopping problem of optimally selecting the k-th best candidate. Given Xi = xi, 1 i j, Vj(x1, x2, . . . , xj) is the (maximum) expected reward by optimally stopping from stage j onwards. Then Vn(x1, x2, . . . , xn) = yn(x1, x2, . . . , xn), and Vj (x1, . . . , xj) = max yj(x1, . . . , xj), E Vj+1 (x1, . . . , xj, Xj+1) X1 = x1, . . . , Xj = xj , (2.1) for j = n - 1, n - 2, . . . , 1. Given Xi = xi, i = 1, . . . , j, it is optimal to stop at stage j if Vj (x1, x2, . . . , xj) = yj(x1, x2, . . . , xj) and to continue otherwise. The (optimal) value of the stopping problem is V1(1), i.e. V1(1) = supMn P (R = k). This formalizes the method of backward induction. See also Chow, Robbins and Siegmund [1]. It is well known that X1, X2, . . . , Xn are independent and Xj has a uniform distribution over {1, 2, . . . , j}. Given Xi = xi, i = 1, . . . , j, the conditional probability of Rj = k is the same as the probability that a random sample of size j contains the k-th best candidate whose (relative) rank in the sample is xj; thus k-1 n-k P (Rj = k|X1 = x1, X2 = x2, . . . , Xj = xj) = , xj -1 j-xj n j (2.2) where we adopt the usual convention that m = 0 for m < . From the independence of X1, X2, . . . , Xn, the conditional expectation on the right hand side of (2.1) reduces to E(Vj+1(x1, x2, . . . , xj, Xj+1)). Note also that yj(x1, . . . , xj) depends only on xj (cf. (2.2)), and so does Vj(x1, . . . , xj). Hence, we have Vn(xn) = yn(xn) and Vj(xj) = max yj (xj ), j 1 + 1 j+1 Vj+1(i) i=1 Thus, it is optimal to stop at the first j with for j = n - 1, n - 2, . . . , 1. (2.3) yj (xj ) j 1 + 1 j+1 Vj+1(i). i=1 For the problem of optimally selecting the k-th best candidate with k = 3, we have 4 yj(xj) = P (Rj = 3|X1 = x1, . . . , Xj = xj), which equals (cf. (2.2)) j(n - n(n j - 1)(n - j) - 1)(n - 2) , yj (xj ) = 2j(j n(n - 1)(n - j) - 1)(n - 2) , j(j n(n - - 1)(j - 2) 1)(n - 2) , 0, if xj = 1; if xj = 2; if xj = 3; otherwise. Setting m i= ci := 0 whenever > m, define for n 3, bn = min j = 2, 3, . . . , n : n i 1 - 2 1 2 , i=j+1 un = (bn - 2)(2n - 4) n i 1 - 2 , i=bn fn(x) = 3x2 - (1 + 4n)x + (n - 2)bn + 2(n + 1) + un, an = min {j = 2, 3, . . . , n : fn(j) 0} . (2.4) (2.5) (2.6) (2.7) (2.8) Remark 2.1. Note that 3 bn bn+1 bn + 1 for n 3, implying that fn(1) > 0 for all n 3. In order for an in (2.8) to be well defined, we need to show that the second-order polynomial equation fn(x) = 0 has two real roots x0 < y0 with x0 y0 (so that an = x0). For 3 n 31, this can be verified by numerical computations. For n 32, we have bn < 2n-1 3 and un (n - 2)bn (cf . (4.2) and (4.5)), implying that fn ( 2n-1 3 ) < 0 and fn( 2n+2 3 ) < 0. So, x0 < 2n-1 3 , implying that x0 < 2n+2 3 < y0. With a little effort, it can be shown that 2 an an+1 an + 1 for n 3. The next theorem is our main result. Theorem 2.1. For n 3, we have an < bn. Furthermore, the stopping rule 3,n = min{an j n : Xj = 2} min{bn j n : Xj = 3} maximizes the probability of selecting the 3rd best candidate. Figure 1 illustrates the optimality of 3,n for the case n = 13 with a13 = 7 and b13 = 9. With the help of a computer program in Mathematica, we have verified Theorem 2.1 for 3 n 31 by numerically evaluating Vj(xj), j = n, n - 1, . . . , 1. (For completeness, the 5 Figure 1: The optimality of 3,13. computer program is provided in Section 5.) While it seems intuitively reasonable for the optimal stopping rule 3,n to involve two thresholds for general n, the exact expressions for the thresholds an and bn in (2.8) and (2.5) were found by some guesswork and tedious analysis. To prove Theorem 2.1 for n 32, we need the following lemmas whose proofs are relegated to Section 4. Lemma 2.1. Let y0 be the larger root of the second-order polynomial equation fn(x) = 0. Then for n 32, we have (i) an < bn; (ii) bn < y0; (iii) an > (n + 4)/3. Lemma 2.2. Given X1 = x1, X2 = x2, . . . , Xj = xj, let hj(xj) = hj(x1, x2, . . . , xj) be the conditional probability of selecting the 3rd best candidate when 3,n is used for stages j, j + 1, . . . , n. Then for n 32, 6 (i) (an - 1) [a2n - (1 + 2n)an n(n - + (n 1)(n - - 2)bn 2) + 2(n + 1) + un] , yj (2), hj (xj ) = j [j2 + (1 - 2n)j + (n - 2)bn n(n - 1)(n - 2) + 2 + un] , yj (3), j(j - 1) n(n - 1)(n - 2) (2n - 4) n i 1 - 2 - (n - j) i=j+1 , if j < an; if j an and xj = 2; if an j bn - 1 and xj = 2; if j bn and xj = 3; if j bn and xj = 2, 3. (ii) (an - 1) [a2n - (1 + 2n)an n(n - + (n 1)(n - - 2)bn 2) + 2(n + 1) + un] , j 1 + 1 j+1 i=1 hj+1(i) = j [j2 + (1 - 2n)j + (n - 2)bn n(n - 1)(n - 2) + 2 + un] , j(j - 1) n(n - 1)(n - 2) (2n - 4) n i 1 - 2 - (n - j) , i=j+1 if j < an; if an j bn - 1; if bn j n - 1. Lemma 2.3. For n 32, 1 j < an and 1 xj j, we have yj (xj ) < j 1 + 1 j+1 hj+1(i). i=1 Lemma 2.4. For n 32 and an j < bn, we have (i) yj (2) 1 j+1 yj (1) < 1 j+1 j+1 i=1 hj+1(i); (iii) yj (3) < 1 j+1 j+1 i=1 hj+1(i). j+1 i=1 hj +1(i); (ii) Lemma 2.5. For n 32 and bn j n - 1, we have (i) yj (1) < 1 j+1 yj (2) 1 j+1 j+1 i=1 hj +1(i); (iii) yj (3) 1 j+1 j+1 i=1 hj+1(i). j+1 i=1 hj+1 (i); (ii) Proof of Theorem 2.1. As remarked before, the theorem has been verified for 3 n 31 by numerical computations. For n 32, we need to show that hj satisfies hj(xj) = max yj (xj ), j 1 + 1 j+1 hj+1(i) i=1 for 1 j < n. (2.9) Since hj(xj) is the conditional probability of selecting the 3rd best candidate when 3,n is used for stages j, . . . , n, we have hj (xj ) = 1 j+1 j+1 i=1 hj+1(i) if either (j < an) or (an j < bn and xj = 2) or (bn j < n and xj = 2, 3), which together with Lemmas 2.3 ­ 2.5 establishes (2.9). 7 Remark 2.2. Let d1 = limn an/n and d2 = limn bn/n. It is shown in Section 4 that d1 = 2e + 2 4e - 6e 0.466 and d2 = 1 e 0.606. (2.10) It is also shown in Section 4 that as n , h1(1) = p(3, n), the maximum probability of selecting the 3rd best candidate, tends to 8 2 e - 2 + 4e - 6 e p(3, ) = 2d21(1 - d1) = 2 e+ 4e - 6e 3 . (2.11) Note that p(3, ) 0.232 < 0.25 = p(2, ). These limiting results agree with the asymptotic solution for k = 3 in Szajowski [11]. 3 Some results on p(k, n) and p(k, ) In this section, we present several inequalities for p(k, n) and p(k, ) := limn p(k, n). Theorem 3.1. For n 3 and 1 < k < n, we have p(1, n) = p(n, n) > p(k, n). Proof. By symmetry, p(1, n) = p(n, n). (More generally, p(k, n) = p(n - k + 1, n).) For the problem of selecting the k-th best candidate (1 < k < n), a (non-randomized) optimal stopping rule is determined by a sequence of subsets {Sj} such that Sj {1, 2, . . . , j} (j = 1, . . . , n) and = min{j : Xj Sj}. Since stopping at n is enforced (if > n - 1), we may assume that Sn = {1, 2, . . . , n}. Thus, P (R = k) = p(k, n). (3.1) Define, for j = 1, . . . , n - 1, Sj = , {1}, if Sj = ; if Sj = ; and Sn = {1, 2, . . . , n}. Let = min{j : Xj Sj }, which, as a stopping rule, may be applied to selecting the best candidate. Thus P (R = 1) sup P (R = 1) = p(1, n). Mn Note that for j = 1, . . . , n, P (Rj = 1, Xj = 1) = 1 n = P (Rj = k) P (Rj = k, Xj Sj). (3.2) (3.3) 8 By (2.2), given X1 = x1, . . . , Xj = xj, the conditional distribution of Rj depends only on xj, implying that X1, . . . , Xj-1 and (Xj, Rj) are independent. So if Sj = , P ( = j, Rj = k) = P (Xi / Si, i = 1, . . . , j - 1, Xj Sj, Rj = k) j-1 = P (Xi / Si) P (Xj Sj, Rj = k) i=1 j-1 P (Xi / Si) P (Xj = 1, Rj = 1) i=1 = P ( = j, Rj = 1), (3.4) where the inequality follows from (3.3) and |Si| |Si| for all i. (If Sj = , then P ( = j, Rj = k) = P ( = j, Rj = 1) = 0.) By (3.1), (3.2) and (3.4), we have n p(k, n) = P (R = k) = P ( = j, Rj = k) j=1 n P ( = j, Rj = 1) = P (R = 1) p(1, n). j=1 (3.5) It remains to show that (at least) one of the two inequalities in (3.5) is strict (so that p(k, n) < p(1, n)). If the stopping rule is not optimal for selecting the best candidate, then the second inequality in (3.5) is strict. Suppose is optimal for selecting the best candidate, which implies, in view of n 3, that S1 = and Sn -1 = {1}, which in turn implies that |Sn-1| 1. If |Sn-1| 2, then the inequality in (3.4) is strict for j = n, implying that the first inequality in (3.5) is strict. Suppose Sn-1 = {} for some . Then we have n-k n(n-1) , P (Rn-1 = k, Xn-1 = ) = k-1 n(n-1) , 0, if k = ; if k = + 1; if k - = 0, 1; implying, in view of 1 < k < n, that the inequality in (3.3) is strict for j = n - 1, which in turn implies that the inequality in (3.4) is strict for j = n - 1. It follows that the first inequality in (3.5) is strict. The proof is complete. Theorem 3.2. For 1 k n, we have p(k, n) p(k, n + 1) (i.e. p(k, n) is decreasing in n) and p(k, n) p(k + 1, n + 1). Furthermore, p(k, ) := limn p(k, n) is well defined, and p(k, ) p(k + 1, ). 9 Proof. (i) To show p(k, n) p(k, n + 1), consider the case of selecting the k-th best of n + 1 candidates. Let the random variable I {1, . . . , n + 1} be such that RI = n + 1 (i.e. the worst candidate is the I-th person to be interviewed). If I is known to the manager (or more precisely, the manager knows the position of the worst candidate before the interview process begins), then the problem of optimally selecting the k-th best of the n + 1 candidates is equivalent to that of optimally selecting the k-th best of the n candidates (excluding the worst one). (Indeed, let Xi = Xi for 1 i < I and Xi = Xi+1 for I i n. Given I, X1 , . . . , Xn are (conditionally) independent with each Xi being uniform over {1, . . . , i}.) Thus, when I is known to the manager, the maximum probability of selecting the k-th best candidate equals p(k, n), which must be at least as large as p(k, n + 1), the maximum probability of selecting the k-th best of the n + 1 candidates when I is unavailable. This proves that p(k, n) p(k, n + 1). (ii) To show p(k, n) p(k + 1, n + 1), note that p(k, n) = p(n - k + 1, n) p(n + 1 - k, n + 1) = p(k + 1, n + 1), (3.6) where the two equalities follow from the symmetry property p(k, n) = p(n - k + 1, n) and the inequality follows from the decreasing property of p(k, n) in n. (iii) Since p(k, n) is decreasing in n, p(k, ) := limn p(k, n) is well defined. By (3.6), we have p(k, ) = lim p(k, n) lim p(k + 1, n + 1) = p(k + 1, ). n n The proof is complete. Remark 3.1. We conjecture that the three inequalities in Theorem 3.2 are all strict. While p(k, n) is decreasing in n, in view of p(1, n) > p(k, n) for 1 < k < n and p(k, ) p(k + 1, ), it may be tempting to conjecture that p(k, n) p(k + 1, n) for 1 k < n 2 . However, this inequality occasionally fails to hold for k close to (but less than) n 2 . Our numerical results show that the set {(k, n) : 1 k < n 2 , n 50, p(k, n) < p(k + 1, n)} consists of (2, 5), (2, 7), (7, 15), (9, 19), (10, 21), (12, 25), (21, 43), (22, 47), (24, 49) and (24, 50). Moreover, it can be shown that p(2, n) > p(3, n) for all n 8. Let = lim infn K(n)/n where K (n) = max{1 k n 2 : p(1, n) p(2, n) ··· p(k, n)}. While 0 1/2, it appears to be a challenging task to find the exact value of . Our limited numerical results suggest that may be equal to 1/2. 10 Remark 3.2. It may be of interest to see how fast p(k, ) tends to 0 as k increases. By considering some suboptimal rules, we have derived a crude lower bound k -k k-1 for p(k, ). The details are omitted. The next theorem extends Theorem 3.1 to the setting where the goal is to select a candidate whose rank belongs to a prescribed subset of {1, . . . , n} (cf. Suchwalko and Szajowski [10]). Let p(, n) = sup P (R ). Mn Theorem 3.3. For any subset of {1, 2, . . . , n} with || = c (1 c < n), we have p(, n) p({1, 2, . . . , c}, n) = p({n - c + 1, . . . , n}, n). In the proof below, it is convenient to take the convention that 0 0 := 1 and n k := 0 if n < k or n < 0 or k < 0, so that n k = n-1 k + n-1 k-1 for (k, n) Z × Z\{(0, 0)}, (3.7) and n k n-1 k + n-1 k-1 where Z is the set of all integers. for (k, n) Z × Z, (3.8) Proof of Theorem 3.3. As in the proof of Theorem 3.1, let be a (non-randomized) optimal stopping rule determined by a sequence of subsets {Sj} of {1, . . . , n} such that Sj {1, . . . , j}, = min{j : Xj Sj} and P (R ) = p(, n). Again, as stopping at n is enforced (if > n - 1), we may assume that Sn = {1, 2, . . . , n}. Let Sj = {1, 2, . . . , |Sj|}, so |Sj | = |Sj| (in particular, Sj = if Sj = ). Let = min{j : Xj Sj }. Claim P (Rj {t1, t2, . . . , tc}, Xj {s1, s2, . . . , sd}) P (Rj {1, 2, . . . , c}, Xj {1, 2, . . . , d}) (3.9) for 1 d j n, 1 c n, 1 t1 < t2 < · · · < tc n, and 1 s1 < s2 < · · · < sd j. If 11 the claim (3.9) is true, then for j = 1, . . . , n, P ( = j, Rj ) = P (Xi / Si, i = 1, . . . , j - 1, Xj Sj, Rj ) j-1 = P (Xi / Si) P (Rj , Xj Sj) i=1 j-1 P (Xi / Si) P (Rj {1, . . . , c}, Xj Sj ) (by (3.9)) i=1 = P (Xi / Si, i = 1, . . . , j - 1, Xj Sj , Rj {1, . . . , c}) = P ( = j, Rj {1, . . . , c}), implying that p(, n) = P (R ) P (R {1, . . . , c}) p({1, . . . , c}, n). It remains to establish (3.9). Note that P (Rj {t1, . . . , tc}, Xj {s1, . . . , sd}) P (Rj {t1, . . . , tc}) = c n = P (Rj {1, . . . , c}) = P (Rj {1, . . . , c}, Xj {1, . . . , d}) (if d c), showing that (3.9) holds for d c. Since a-1 n-a P (Rj = a, Xj = b) = b-1 j-b n n-1 j-1 for all integers a > 0, b > 0, (3.9) is equivalent to dc i=1 =1 t - 1 si - 1 n - t j - si dc i=1 =1 -1 i-1 n- j-i , (3.10) for 1 d j n, 1 c n, 1 t1 < · · · < tc n and 1 s1 < · · · < sd j. Note that (3.10) holds for d c (since (3.9) does for d c). Also, from n-t j -si = 0 for t > n or si > j, it follows easily that for fixed n, if (3.10) holds for all (j, c, d, t1, . . . , tc, s1, . . . , sd) with 1 d j n, 1 c n, 1 t1 < · · · < tc n and 1 s1 < · · · < sd j, then (3.10) holds for all (j, c, d, t1, . . . , tc, s1, . . . , sd) with 1 j n, 1 t1 < · · · < tc and 1 s1 < · · · < sd. This (trivial) observation is needed later. To prove (3.10), we proceed by induction on n. For n = 1, necessarily j = 1 and c = d = 1 (since 1 d j n and 1 c n). So (3.10) holds for n = 1. 12 Suppose (3.10) holds for (fixed) n 1 and for all (j, c, d, t1, . . . , tc, s1, . . . , sd) with 1 d j n, 1 c n, 1 t1 < · · · < tc n and 1 s1 < · · · < sd j (and hence for all (j, c, d, t1, . . . , tc, s1, . . . , sd) with 1 j n, 1 t1 < · · · < tc and 1 s1 < · · · < sd). We need to show that (3.10) holds for n + 1 (with 1 d < c), i.e. dc i=1 =1 t - 1 si - 1 n - t + 1 j - si dc i=1 =1 -1 i-1 n-+1 j-i , (3.11) for 1 d j n + 1, 1 d < c n + 1, 1 t1 < t2 < · · · < tc n + 1 and 1 s1 < s2 < · · · < sd j. If j = 1, then necessarily d = 1 and s1 = 1, so that both sides of (3.11) equal c, implying that (3.11) holds for j = 1. For j = n + 1, the left hand side of (3.11) equals dc i=1 =1 t - 1 si - 1 n - t + 1 n - si + 1 d, since the two inequalities t - 1 si - 1 and n - t + 1 n - si + 1 hold simultaneously if and only if t = si. The right hand side of (3.11) equals dc i=1 =1 -1 i-1 n-+1 n-i+1 = d, since -1 i-1 n-+1 n-i+1 = 1 or 0 according to whether i = or i = . Thus, (3.11) holds for j = n + 1. We now consider 2 j n. Suppose n - tc + 1 = j - sd = 0. Then the left hand side of (3.11) equals d c-1 i=1 =1 d c-1 = i=1 =1 d c-1 = i=1 =1 t - 1 si - 1 t - 1 si - 1 t - 1 si - 1 n - t + 1 j - si + n j-1 n - t j - si + n - t j - si - 1 + n j-1 (by (3.7)) n - t j - si d-1 c-1 + i=1 =1 t - 1 si - 1 n - t (j - 1) - si + n j-1 . (3.12) By the induction hypothesis (applied to each of the two double sums), (3.12) is less than or equal to d c-1 i=1 =1 d-1 c-1 = i=1 =1 -1 i-1 -1 i-1 n- j-i d-1 c-1 + i=1 =1 -1 i-1 n- (j - 1) - i n- j-i + n- j-i-1 c-1 + -1 d-1 =1 + n j-1 n- j-d + n j-1 , 13 which by (3.7) is equal to d-1 c-1 i=1 =1 -1 i-1 n-+1 j-i c-1 + -1 d-1 =d We need the following identity n- j-d + n j-1 . (3.13) c c-1 i-1 i=d+1 n-c+1 j-i c-1 = -1 d-1 =d n- j -d-1 , (3.14) which holds by observing that the left hand side is the total number of subsets of {1, . . . , n} with j -1 elements and with the d-th smallest element less than c while the term -1 d-1 n- j-d-1 on the right hand side is the number of subsets of {1, . . . , n} with j - 1 elements and with the d-th smallest element being . In view of (3.14), n j-1 d = i=1 d = i=1 c-1 i-1 c-1 i-1 n-c+1 j-i n-c+1 j-i c + c-1 i-1 i=d+1 c-1 + -1 d-1 =d n-c+1 j-i n- j-d-1 . (3.15) We have shown that the left hand side of (3.11) is less than or equal to (3.13), which by (3.15) equals d-1 c-1 i=1 =1 d-1 c-1 = i=1 =1 dc = i=1 =1 -1 i-1 -1 i-1 -1 i-1 n-+1 j -i n-+1 j -i c-1 + =d c-1 + =d n-+1 j -i , -1 d-1 -1 d-1 n- j-d + n- j-d-1 d + c-1 i-1 i=1 n-c+1 j-i n-+1 j-d d + c-1 i-1 i=1 n-c+1 j-i (by (3.7)) establishing (3.11) for the case that 2 j n and n - tc + 1 = j - sd = 0. It remains to deal with the case that 2 j n and (n - tc + 1, j - sd) = (0, 0) (implying 14 that (n - t + 1, j - si) = (0, 0) for all i, ). By (3.7), the left hand side of (3.11) equals dc i=1 =1 dc i=1 =1 dc = i=1 =1 dc i=1 =1 t - 1 si - 1 -1 i-1 -1 i-1 -1 i-1 n - t j - si dc + i=1 =1 t - 1 si - 1 n - t (j - 1) - si n- j-i d + c -1 i-1 i=1 =1 n- (j - 1) - i (by the induction hypothesis) n- j-i + n- j-i-1 n-+1 j-i (by (3.8)). Note that the first inequality follows from the induction hypothesis applied to each of the two double sums where tc > n or sd > j -1 is possible. (Recall that the induction hypothesis applies to all (j, c, d, t1, . . . , tc, s1, . . . , sd) with 1 j n, 1 t1 < · · · < tc and 1 s1 < · · · < sd.) The proof is complete. Remark 3.3. As pointed out by a referee, the identities (3.14) and (3.15) are variants of Chu-Vandermonde convolution formula. (See the first identity in Table 169 of Graham et al. [5].) 4 Proofs of Lemmas 2.1­2.5 and (2.10)­(2.11) To prove Lemmas 2.1­2.5, we need the following lemma. Lemma 4.1. For n 32, we have n- 1 e +1 < bn < n- 3 2 e + 5 2 . In particular, n + 2 5 < bn < 2n - 3 1 . Proof. By (2.5), we have 1 2 < n 1 i-2 = n-2 1 i < i=bn i=bn -2 n- 3 2 bn - 5 2 dx x = log n - 3 2 bn - 5 2 and 1 2 n 1 i-2 = n-2 1 i > i=bn+1 i=bn -1 n-1 bn-1 dx x = log n-1 bn - 1 . 15 (4.1) (4.2) (4.3) (4.4) By (4.3), we have bn < n- 3 2 e + 5 2 ; and from (4.4), bn > n-1 e + 1, establishing (4.1). Since n- 3 2 e + 5 2 < 2n-1 3 and n-e1 + 1 > n+5 2 (for n 32), we have n+5 2 < bn < 2n-1 3 . The proof is complete. From (2.5) and (2.6), we have (bn - 2)(n - 2) = (bn - 2)(2n 2 - 4) < un = 2n - 4 + (bn - 2)(2n - 4) n 1 i-2 i=bn+1 2n - 4 + (bn - 2)(2n 2 - 4) = bn(n - 2), i.e. (bn - 2)(n - 2) < un bn(n - 2). (4.5) Remark 4.1. The assumption of n 32 is needed for Lemmas 2.1­2.5 since the following proofs of the lemmas rely on (4.2). Proof of Lemma 2.1. (i) Note (cf. Remark 2.1) that an = x0 < x0 + 1 where x0 is the smaller root of fn(x) = 0. We now show fn(bn - 1) < 0 (which implies that an < x0 + 1 < (bn - 1) + 1 = bn). We have fn(bn - 1) = 3(bn - 1)2 - (1 + 4n)(bn - 1) + (n - 2)bn + 2(n + 1) + un 3(bn - 1)2 - (1 + 4n)(bn - 1) + (n - 2)bn + 2(n + 1) + bn(n - 2) (by (4.5)) = (bn - 3) [3bn - (2n + 2)] < 0 (by (4.2)). This proves (i). (ii) Note that fn(bn) 3b2n - (1 + 4n)bn + (n - 2)bn + 2(n + 1) + bn(n - 2) = (bn - 1) [3bn - (2n + 2)] < 0 (by (4.2)). This proves that bn < y0. (iii) By (4.2) and (ii), y0 > bn > n+5 2 > n+4 3 . We now show fn n+4 3 > 0 (which implies that n+4 3 < x0 x0 = an). By (4.5), fn n+4 3 = -n2 - 3n + 4 + (n - 2)bn + 2(n + 1) + un > -n2 - 3n + 4 + (n - 2)bn + 2(n + 1) + (bn - 2)(n - 2) (by (4.5)) = (n - 2) (2bn - (n + 5)) > 0 (by (4.2)). 16 The proof is complete. Proof of Lemma 2.2. By Lemma 2.1, an < bn. (i) Let Qi = {X = 2 for an i - 1, Xi = 2}, an i bn - 1; Qi = {X = 2 for an bn - 1, X = 2, 3 for bn i - 1, Xi = 2}, i bn; and Qi = {X = 2 for an bn - 1, X = 2, 3 for bn i - 1, Xi = 3}, i bn. Since X is uniformly distributed over {1, 2, . . . , }, the Xs are independent and Ri is conditionally independent of X1, . . . , Xi-1 given Xi, we have P (Qi) = (an i(i - - 1) 1) , P (Ri = 3|Qi) = yi(2) for an i bn - 1, P (Qi) = P (Qi) = (an i(i - - 1)(bn - 2) 1)(i - 2) , P (Ri = 3|Qi) = yi(2), P (Ri = 3|Qi) = yi(3), for i bn. Thus, by (2.4) and (2.6), for j < an, n hj(xj) = P (Ri = 3 and the i-th candidate is selected under 3,n) i=an bn -1 n = P (Qi)P (Ri = 3|Qi) + P (Qi)P (Ri = 3|Qi) + P (Qi)P (Ri = 3|Qi) i=an i=bn = bn -1 (an i(i - - 1) 1) yi(2) + n (an i(i - - 1)(bn - 2) 1)(i - 2) (yi(2) + yi(3)) i=an i=bn = n(n an - 1 - 1)(n - 2) bn -1 2(n - i) + (bn - 2) n 2n - i - 2 i-2 i=an i=bn = n(n an - 1 - 1)(n - 2) (2n - an - bn + 1)(bn - an) - (bn - 2)(n - bn + 1) +(bn - 2)(2n - 4) n 1 i-2 i=bn = (an - 1) [a2n - (1 + 2n)an + (n n(n - 1)(n - - 2)bn 2) + 2(n + 1) + un] =: cn. (4.6) This proves (i) for j < an. The other cases can be treated similarly. (ii) By (i), for j < an - 1, hj+1(i) does not depend on i, so that 1 j+1 j+1 i=1 hj+1(i) = cn. To establish the identity for j = an - 1, we have by (i) that han(2) = yan(2) and han (i) = an (a2n + (1 - 2n)an + (n - 2)bn n(n - 1)(n - 2) + 2 + un) for i = 2 with 1 i an. 17 So, 1 an an han(i) = i=1 1 an yan(2) + (an - 1) an (a2n + (1 - 2n)an + (n - 2)bn + 2 + un) n(n - 1)(n - 2) = 1 an 2an(an - 1)(n - an) n(n - 1)(n - 2) + (an - 1) an (a2n + (1 - 2n)an + (n - 2)bn + 2 + un) n(n - 1)(n - 2) = (an - 1) [a2n - (1 + 2n)an + (n - 2)bn n(n - 1)(n - 2) + 2(n + 1) + un] = cn. This proves (ii) for the case j < an. The other cases can be treated similarly. Proof of Lemma 2.3. Since, by Lemma 2.2(ii), 1 j+1 j+1 i=1 hj+1(i) = cn for j < an where cn is defined in (4.6), we need to show max{yj(i) : i = 1, 2, 3, j < an} < cn, (4.7) where yj (i) is given in (2.4). Since yj (2) > yj (3) if and only if 2(n - j) > j-2 (i.e. j < 2n+2 3 ) and, since by Lemma 2.1(i) and (4.2), an < bn < 2n-1 3 , we have yj (2) > yj (3) for j < an, implying that max yj(2) > max yj(3). j n+4 3 > n-2 3 + 1. So, max 1jn yj (1) = y n-2 3 (1) y n-2 3 +1 (2) max j 0, which is equivalent to fn(an - 1) > 0. This holds by (2.8). The proof is complete. 18 Proof of Lemma 2.4. (i) Note that n(n - 1)(n - 2) j yj (2) - j 1 + 1 j+1 hj+1(i) i=1 = 2(j - 1)(n - j) - j2 - (1 - 2n)j - (n - 2)bn - 2 - un = -3j2 + (1 + 4n)j - (n - 2)bn - 2(n + 1) - un = -fn(j) 0, where the inequality holds since fn(j) 0 for x0 an j < bn < y0 where x0 and y0 denote the two roots of fn(x) = 0. (ii) Note that n(n - 1)(n - 2) j yj (1) - j 1 + 1 j+1 hj+1(i) i=1 =(n - j - 1)(n - j) - j2 - (1 - 2n)j - (n - 2)bn - 2 - un =n2 - n - (n - 2)bn - 2 - un n+5 2 by (4.2)). This proves (i). (ii) By (2.4) and Lemma 2.2(ii), for bn j n - 1, n(n - 1)(n - 2) j(j - 1) yj (2) - j 1 + 1 j+1 hj+1(i) i=1 = 3(n - j) - (2n - 4) n 1 i-2 i=j+1 = (n - j) 3 - 2n - 4 n-j n 1 i-2 i=j+1 (n - j) 3 - 2n - 4 n - bn n i=bn+1 i 1 - 2 (n - j) 3 - n-2 n - bn (by (2.5)) (by (4.12)) > 0 (since bn < (2n - 1)/3 by (4.2)). This proves (ii). 20 (iii) By (2.4) and Lemma 2.2(ii), for bn j n - 1, n(n - 1)(n - 2) j(j - 1) yj (3) - j 1 + 1 j+1 hj+1(i) i=1 = n - 2 - (2n - 4) n 1 i-2 i=j+1 = (n - 2) 1 - 2 n 1 i-2 i=j+1 (n - 2) 1 - 2 n 1 i-2 i=bn+1 0 (by (2.5)). The proof is complete. Proof of (2.10)­(2.11). It follows immediately from Lemma 4.1 that d2 = 1/ e. Let x0 be the smaller root of fn(x) = 0, i.e. x0 : = (1 + 4n) - (1 + 4n)2 - 12[(n - 2)bn + 2(n + 1) + un] 6 = 2[(n - 2)bn + 2(n + 1) + un] . 1 + 4n + (1 + 4n)2 - 12[(n - 2)bn + 2(n + 1) + un] Since bn n d2 = 1/e and n i=bn 1 i-2 11/e dx x = 1 2 as n , un n2 = (bn - 2)(2n n2 - 4) n i 1 - 2 d2 as n . i=bn By (4.13), (4.14) and an = x0, we have d1 = lim n an n = lim n x0 n = 2d2 2 + 4 - 6d2 = 2e + 2 4e - 6e , (4.13) (4.14) proving (2.10). By Lemma 2.2(i), p(3, n) = h1(1) = (an - 1)[a2n - (1 + 2n)an + (n - 2)bn n(n - 1)(n - 2) + 2(n + 1) + un] , which together with (2.11) and (4.14) yields 8 2 e - 2 + 4e - 6 e p(3, ) = lim n p(3, n) = d1(d21 - 2d1 + 2d2) = 2d21(1 - d1) = 3 , 2 e + 4e - 6 e proving (2.11). 21 5 A computer program in Mathematica for verification of Theorem 2.1 for 3 n 31 Clear[f, u, n, j, x]; For[n = 3, n < 32, n++, u[n , j , x ]:=Which x == 1, (n-j+1)(j-2)(j-1) n(n-1)(n-2) , x == 2, 2(n-j+1)(n-j)(j-1) n(n-1)(n-2) , , x == 3, (n-j+1)(n-j)(n-j-1) n(n-1)(n-2) , True, 0 ; For[j = 1, j n, j++, For[x = 1, x n, x++, f [n, j, x] = If j > 1, Max u[n, j, x], 1 n-j+2 ] (*This sets the values backwards*) n-j+2 i=1 f [n, j - 1, i] , Which[x == 3, 1, x = 3, 0] ] ] Clear[y, v, b, n]; y[n , j , x ]:=u[n, n + 1 - j, x]; (*Define the conditional probability y*) v[n , j , x ]:=f [n, n + 1 - j, x]; (*Define the value function*) b[3] = 3; (*Define the threshold bn*) For[n = 4, n < 32, n++, For i = 2, i < n, i++, If b[n] = i n k=i+1 1 k-2 1 2 , i && Break[] ; ] Clear[a, n, j]; a[n ]:=Ceiling ( ) 1+4n- (1+4n)22-12 (n-2)b[n]+2(n+1)+(b[n]-2)(2n-4) nn 1 jj==bb[[nn]] j-2 6 ; (*Define the threshold an*) For[n = 3, n < 32, n++, If[a[n] - b[n] > 0, Print[n] && Break[]] (*This verifies that an < bn for 3 n 31*) ] 22 Clear[i, j, n, x]; For[n = 3, n < 32, n++, For[j = 1, j < a[n], j++, For[x = 1, x j, x++, If y[n, j, x] 1 j+1 j+1 i=1 v[n, j + 1, i], Print[{n, j, x}] && Break[] ] ] ] (*This verifies Lemma 2.3 for 3 n 31*) Clear[i, j, n]; For[n = 3, n < 32, n++, For[j = a[n], j < b[n], j++, If y[n, j, 2] < 1 j+1 j+1 i=1 v[n, j + 1, i] y[n, j, 1] 1 j+1 j+1 i=1 v[n, j + 1, i] y[n, j, 3] 1 j+1 j+1 i=1 v[n, j + 1, i], Print[{n, j, x}] && Break[]] ] ] (*This verifies Lemma 2.4 for 3 n 31*) Clear[i, j, n]; For[n = 3, n < 32, n++, For[j = b[n], j < n, j++, If y[n, j, 1] 1 j+1 j+1 i=1 v[n, j + 1, i] y[n, j, 2] < 1 j+1 j+1 i=1 v[n, j + 1, i] y[n, j, 3] < 1 j+1 j+1 i=1 v[n, j + 1, i], Print[{n, j}] && Break[]] ] ] (*This verifies Lemma 2.5 for 3 n 31*) Acknowledgements The authors gratefully acknowledge support from the Ministry of Science and Technology of Taiwan, ROC. References [1] Chow, Y.-S., Robbins, H. and Siegmund, D. (1971). Great Expectations: the Theory of Optimal 23 Stopping. Houghton Mifflin, Boston, MA. [2] Ferguson, T.S. (1989). Who solved the secretary problem? Statistical Science 4, 282­296. [3] Ferguson, T.S. Optimal Stopping and Applications. Mathematics Department, UCLA. http://www.math.ucla.edu/tom/Stopping/Contents.html. [4] Freeman, P. R. (1983). The secretary problem and its extensions: a review. Int. Statist. Rev. 51, 189­206. [5] Graham, R., Knuth, D., and Patashnik, O. (1994). Concrete Mathematics: A Foundation for Computer Science. Addison-Wesley Professional. [6] Lindley, D.V. (1961). Dynamic programming and decision theory. Appl. Statist. 10, 39­51. [7] Rose, J.S. (1982). A problem of optimal choice and assignment. Oper. Res. 30, 172­181 [8] Rose, J.S. (1982). Selection of nonextremal candidates from a sequence. J. Optimization Theory Appl. 38, 207­219. [9] Samuels, S.M. (1991). Secretary problems. In Handbook of Sequential Analysis (Statist. Textbooks Monogr. 118), eds B. K. Ghosh and P.K. Sen, Marcel Dekker, New York, pp. 381­405. [10] Suchwalko, A. and Szajowski, K. (2002). Non standard, no information secretary problems. Sci. Math. Jpn. 56, 443­456. [11] Szajowski, K. (1982). Optimal choice problem of a-th object. Mat. Stos. 19, 51­65 (in Polish). [12] Vanderbei, R.J. (2012). The postdoc variant of the secretary problem. Tech. Report. http://www.princeton.edu/rvdb/tex/PostdocProblem/PostdocProb.pdf 24